Information processing device, information processing method, and program

Abstract

To provide an information processing device capable of correcting a dynamic range of a video signal.SOLUTION: An information processing device includes a first processing unit for specifying a first histogram being a histogram of luminance in one image with the whole shown by a plurality of pixels as the image, a second processing unit for setting an adjustment value in the first histogram, and a third processing unit for generating a second histogram by subtracting a part near a peak of the first histogram from the first histogram on the basis of the adjustment amount and moving a histogram corresponding to the part to an area in which a peak of the first histogram is at least relatively low.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Due to the spread of the Internet, video shooting has become popular. Patent Documents 1 and 2 disclose related techniques relating to video processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-069525 A [Patent Document 2] JP 2008-048188 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, even if an ISP (Image Signal Processor) performs processing with a bit depth of, for example, 20 to 24 bits as in the information processing system 1a shown in Fig. 16, if the bit depth of the display is lower than the bit depth of the processing (for example, 8 to 10 bits), the image processing is also performed with a bit depth that matches the bit depth of the display. Therefore, the image displayed on the display may include blackout and whiteout, which may degrade visibility. Therefore, there is a demand for a technique that can correct the dynamic range of a video signal.

[0005] An object of each aspect of the present disclosure is to provide an information processing device, an information processing method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] In order to achieve the above object, according to one aspect of the present disclosure, an information processing device includes a first processing unit that specifies a first histogram, which is a histogram of luminance in an image represented by a plurality of pixels; Clip value A second processing unit that sets Clip value of the first histogram based on Part exceeding the Clip value and a third processing unit that generates a second histogram in which the luminance histogram of the image is flattened by subtracting a portion of the first histogram from the first histogram and shifting a histogram corresponding to the portion to a region where the peaks of the first histogram are at least relatively low.

[0007] In order to achieve the above object, according to another aspect of the present disclosure, an information processing method includes: identifying a first histogram, which is a histogram of luminance in an image represented by a plurality of pixels; Clip value and setting the Clip value of the first histogram based on Part exceeding the Clip value and generating a second histogram in which the histogram of luminance in the image is flattened by subtracting a portion of the first histogram from the first histogram and shifting a histogram corresponding to the portion to a region where the peaks of the first histogram are at least relatively low.

[0008] In order to achieve the above object, according to another aspect of the present disclosure, a program is provided that causes a computer to: identify a first histogram, which is a histogram of luminance in an image represented by a plurality of pixels; set an adjustment amount in the first histogram; and adjust the first histogram based on the adjustment amount. Part exceeding the Clip value and generating a second histogram in which the histogram of luminance in the image is flattened by subtracting a portion of the first histogram from the first histogram and shifting a histogram corresponding to the portion to a region where the peaks of the first histogram are at least relatively low. Effect of the Invention

[0009] According to each aspect of the present disclosure, the dynamic range of a video signal can be corrected. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device according to an embodiment of the present disclosure. [Diagram 3] 11 is a diagram for explaining an example of processing by a third processing unit according to an embodiment of the present disclosure. FIG. [Figure 4] FIG. 11 is a diagram illustrating an example of teacher data. [Diagram 5] FIG. 5 is a diagram showing an example of teacher data different from that shown in FIG. 4. [Figure 6] FIG. 2 is a diagram illustrating an example of a processing flow of an entire process performed by an information processing device according to an embodiment of the present disclosure. [Figure 7] FIG. 11 is a diagram illustrating an example of a processing flow of the processing of step S2 performed by the information processing device according to an embodiment of the present disclosure. [Figure 8] FIG. 11 is a diagram illustrating an example of a processing flow of the processing of step S4 performed by the information processing device according to an embodiment of the present disclosure. [Figure 9] FIG. 11 is a diagram illustrating an example of a processing flow of the processing of step S6 performed by the information processing device according to an embodiment of the present disclosure. [Figure 10] FIG. 13 is a diagram illustrating an example of a configuration of an information processing device according to another embodiment of the present disclosure. [Figure 11] FIG. 13 is a diagram illustrating an example of a configuration of an information processing device according to another embodiment of the present disclosure. [Figure 12] FIG. 11 is a first diagram for explaining the effect of HDR processing performed by a seventh processing unit according to another embodiment of the present disclosure. [Figure 13] FIG. 13 is a second diagram for explaining the effect of HDR processing performed by the seventh processing unit according to another embodiment of the present disclosure. [Figure 14] FIG. 1 is a schematic block diagram illustrating a configuration of a computer according to at least one embodiment. [Figure 15] FIG. 1 is a diagram illustrating an example of implementation of an information processing device according to an embodiment of the present disclosure. [Figure 16] FIG. 2 is a diagram illustrating an example of a configuration of an information processing system to be compared. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. <Embodiment> (Configuration of Information Processing System) Fig. 1 is a diagram illustrating an example of a configuration of an information processing system 1 according to an embodiment of the present disclosure. As shown in Fig. 1, the information processing system 1 includes a sensor 10, an ISP (Image Signal Processor) 20, a storage device 30, a video processing device 40, an information processing device 50, an AI (Artificial Intelligence) / CV (Computer Vision) 60, and a display 70.

[0012] The sensor 10 acquires an image by converting an optical signal into an electrical signal. Examples of the sensor 10 include image sensors such as a Complementary Metal Oxide Semiconductor (CMOS) sensor and a Charge Coupled Device (CCD) sensor.

[0013] The ISP 20 performs signal processing on the electrical signal after conversion by the sensor 10. Examples of the signal processing include black level adjustment processing, HDR (High Dynamic Range) processing, gain / exposure adjustment processing, defect correction processing, shading correction processing, gamma correction processing, sharpness processing, and the like.

[0014] The black level adjustment process will now be described. If the sensor 10 is a sensor capable of expressing data of 12 bits per pixel, for example, the pixel value of each pixel is expressed by a quantization level of 4096 stages (0 to 4095). This quantization level represents luminance, and the black level is the luminance reference. The ISP 20 achieves the black level adjustment process by adjusting the luminance reference so that the black level remains the same even when the environment changes.

[0015] HDR processing is explained below. HDR processing is processing to expand the dynamic range. Dynamic range is an index that indicates the allowable range of brightness difference between black and white in an image. The ISP20 realizes a wider dynamic range than can be achieved with a single exposure time by synthesizing images captured with two or more different exposure times.

[0016] The gain / exposure adjustment process will be described. The gain / exposure adjustment process is a process for adjusting the light taken in by the sensor 10. For example, the ISP 20 adjusts the light taken in by the sensor 10 by adjusting the exposure time of the sensor 10. Also, for example, when the amount of light taken in by the sensor 10 is small, the ISP 20 increases the gain and amplifies the charge stored in the sensor 10, thereby adjusting so that a large amount of charge is stored. This makes it possible to obtain the same effect as when light is taken in even when the exposure time is short.

[0017] The defect correction process will be described. The defect correction process is a process for correcting defects in pixels. Some sensors 10 have defective pixels, and include pixels that emit red, blue, or green light even when the screen is completely dark. In such cases, for example, the coordinates of the defective pixel are stored in memory, and the ISP20 corrects the coordinates by complementing them using surrounding pixels. Also, for example, the ISP20 identifies the defective coordinates in an image captured in real time, and performs defect correction by complementing the identified coordinates using surrounding pixels.

[0018] The shading correction process will be described. Shading is the ratio between the amount of light at the center of the screen and the amount of light at the periphery of the screen. The shading correction process is a process for reducing that ratio. For example, if the amount of light is large at the center of the screen and decreases toward the periphery of the screen, the ISP20 performs shading correction by adjusting the gain so that it is gradually higher toward the periphery of the screen than at the center of the screen.

[0019] The gamma correction process is described below. The gamma correction process changes the brightness of the input and output, and makes the image appear to have a changed contrast. For example, the ISP20 performs gamma correction by changing the output brightness relative to the input brightness based on a gamma curve, whose curve is determined by the gamma value.

[0020] Sharpness processing will now be described. Sharpness processing is processing for enhancing edges. The ISP 20 identifies edges in a captured image and performs processing to enhance the edges.

[0021] The storage device 30 stores image data and video data. An example of image data is a JPEG (Joint Photographic Experts Group) file. An example of video data is an MP4 file, which is a part of the MPEG-4 standard.

[0022] The video processing device 40 processes the video data output by the ISP 20 and the image data and video data stored in the storage device 30 at a bit depth (e.g., 8 to 10 bits) lower than the bit depth (e.g., 20 to 24 bits) of the ISP 20. The video processing device 40 outputs the processed data to the information processing device 50 as an RGB signal.

[0023] The information processing device 50 performs signal processing on the RGB signals. Details of the signal processing performed by the information processing device 50 will be described later. The information processing device 50 outputs data after the signal processing to the AI / CV 60. The information processing device 50 is realized by a computer such as a personal computer, for example.

[0024] The AI / CV60 executes a predetermined process for extracting useful information from an image using AI technology. Examples of useful information include information on a recognized face, information on a recognized object, and information on a recognized flow line. Examples of the predetermined process include a process for recognizing a face, a process for recognizing an object, and a process for recognizing a flow line. The AI / CV60 may also perform a process for an application that does not use AI technology. The AI / CV60 outputs data received from the information processing device 50 to the display 70.

[0025] The display 70 displays the data output by the AI / CV 60 (that is, the data after the signal processing by the information processing device 50).

[0026] (Configuration of information processing device) Here, the information processing device 50 will be described. FIG. 2 is a diagram showing an example of the configuration of the information processing device 50 according to an embodiment of the present disclosure. As shown in FIG. 2, the information processing device 50 includes a first processing unit 501, a second processing unit 502, a third processing unit 503, a fourth processing unit 504, and a fifth processing unit 505. Hereinafter, the Y signal (an example of a first signal) is a signal indicating luminance Y. The R signal (an example of a second signal) is a signal related to red (an example of a first color). The B signal (an example of a third signal) is a signal related to blue (an example of a second color). The G signal (an example of a fourth signal) is a signal related to green (an example of a third color). The information processing device 50 performs signal processing on the Y signal, the R signal, and the B signal, or the Y signal, the R signal, the B signal, and the G signal. The information processing device 50 is a device that can correct the dynamic range of a video signal input to the display 70.

[0027] The first processing unit 501 converts the RGB signal into a Y signal, a color difference signal Xr, and a color difference signal Xb. The first processing unit 501 divides an image represented by a plurality of pixels into M×N (for example, divides the image into 1×1 to make the whole into one image), and specifies a first histogram Y which is a histogram of luminance (i.e., Y signal) in each divided image.

[0028] The second processing unit 502 sets a Clip value (an example of an adjustment amount) in the first histogram Y specified by the first processing unit 501. The Clip value is an amount that specifies the amount by which a portion y near the peak of the first histogram Y is subtracted from the first histogram Y at one time. This Clip value may be a value determined in advance. Also, this Clip value may be determined by the user each time processing is performed.

[0029] The third processing unit 503 subtracts a portion y of the first histogram Y near the peak from the first histogram Y based on the Clip value. Then, the third processing unit 503 moves a histogram y1 corresponding to the subtracted portion y of the first histogram Y to a region where the value of the first histogram Y is at least relatively low. Note that the region where the value of the first histogram Y is at least relatively low may include a region where the portion y of the first histogram Y near the peak is subtracted. An example of the process of moving the histogram y1 performed by the third processing unit 503 is an adaptive histogram equalization method (Contrast Limited Adaptive Histogram Equalizer; CLAHE) or the like. By repeating this process as many times as necessary, the third processing unit 503 generates a second histogram Y1, which is a histogram of luminance.

[0030] FIG. 3 is a diagram showing an example of a change in a histogram due to processing by the third processing unit 503 according to an embodiment of the present disclosure. Part (a) in FIG. 3 shows the first histogram Y identified by the first processing unit 501. Part (b) in FIG. 3 shows the second histogram Y1, which is a histogram of luminance after the third processing unit 503 moves the histogram y1 corresponding to a part y near the peak of the first histogram Y to a region where the value of the first histogram Y is at least relatively low. Note that the region where the value of the first histogram Y is relatively low includes a region where the first histogram Y does not exist. As can be seen from part (b) in FIG. 3, the third processing unit 503 reduces the value near the peak of the first histogram Y and increases the value in the region where the value of the first histogram Y is at least relatively low. That is, in the flattening process by the third processing unit 503, the degree of flattening can be adjusted by the Clip value and the number of times the process is repeated (i.e., the above-mentioned "necessary number of times"). When a general histogram equalization method that does not allow adjustment of the degree of equalization is used, excessive equalization may occur, and the entire image may become too white. However, the third processing unit 503 can suppress such excessive equalization, and prevent the entire image from becoming too white.

[0031] The number of times the process is repeated may be determined by stopping the process when the user checks the image and determines that the image is a desired one each time the process is performed, or the number of times the process is repeated may be determined in advance.

[0032] The number of times the process is repeated may be determined together with the Clip value using AI technology. For example, the third processing unit 503 predicts the number of times the process is repeated and the Clip value by using a trained model (for example, a convolutional neural network) whose parameters are determined using teacher data, which is one of the machine learning techniques. Here, the trained models used by the third processing unit 503 for each prediction will be described.

[0033] (Pre-trained model) A trained model will be described. The third processing unit 503 predicts the number of times the process is repeated and the Clip value based on the image data before the histogram is flattened and the image data after the histogram is flattened. Here, a trained model will be described in which the third processing unit 503 predicts the number of times the process is repeated and the Clip value based on the image data before the histogram is flattened and the image data after the histogram is flattened.

[0034] In this case, the image data before flattening is one of the inputs. Also, the Clip value actually set for the image data, the number of repetitions of the process actually performed on the image data, and the image data after the flattening of the histogram obtained by actually performing the process on the image data are one of the output data. Then, a combination of the input data and the output data corresponding to the input data is one of the teacher data. For example, before the third processing unit 503 predicts the number of repetitions of the process and the Clip value, output data (i.e., the Clip value actually set for the image data before flattening, the number of repetitions of the process actually performed on the image data, and the image data after the flattening of the histogram obtained by actually performing the process on the image data) is specified for the input data used by another device to predict the number of repetitions of the process and the Clip value. Alternatively, for example, the output data is specified for the input data by performing an experiment, a simulation, or the like. In this way, teacher data consisting of a plurality of data combining the input data and the output data can be prepared. Note that the teacher data is data used to determine the value of a parameter in a learning model in which the value of the parameter has not been determined.

[0035] Fig. 4 is a diagram showing an example of training data. One set of data is input data, which is image data before flattening, and output data for the input data (i.e., the Clip value actually set for the image data before flattening, the number of times the processing is actually performed on the image data, and the image data after flattening the histogram obtained by actually performing the processing on the image data). In the example shown in Fig. 4, the training data includes 10,000 sets of data.

[0036] For example, consider a case where parameters in a learning model are determined using teacher data consisting of 10,000 sets of data shown in FIG. 4. In this case, the teacher data is divided into, for example, training data, evaluation data, and test data. Examples of the ratios of the training data, evaluation data, and test data include 70%, 15%, 15%, 95%, 2.5%, and 2.5%. For example, it is assumed that teacher data of data #1 to #10,000 is divided into data #1 to #7000 as training data, data #7001 to #8500 as evaluation data, and data #8501 to #10,000 as 15% test data. In this case, data #1, which is training data, is input to a convolutional neural network, which is a learning model. The convolutional neural network outputs the Clip value actually set for the image data before flattening, the number of repetitions of the process actually performed on the image data, and the image data after the flattening of the histogram obtained by actually performing the process on the image data. The input data of the training data is input to the convolutional neural network, and each time the Clip value actually set for the image data before flattening, the number of times the processing was actually performed on that image data, and the image data after flattening the histogram obtained by actually processing that image data are output from the convolutional neural network (in this case, each time each of the data #1 to #7000 is input to the convolutional neural network), a parameter indicating the weighting of the data connections between nodes is changed by performing, for example, backpropagation according to the output (i.e., the model of the convolutional neural network is changed).In this way, the training data is input to the neural network to adjust the parameters.

[0037] Next, the input data of the evaluation data (data #7001 to #8500) is input in order to the convolutional neural network whose parameters have been changed by the training data. The convolutional neural network outputs the Clip value actually set for the image data before flattening, the number of repetitions of the process actually performed on the image data, and the image data after flattening the histogram obtained by actually performing the process on the image data, according to the input evaluation data. Here, if the data output by the convolutional neural network is different from the output data associated with the input data in FIG. 4, the parameters are changed so that the output of the convolutional neural network becomes the output data associated with the input data in FIG. 4. The convolutional neural network (i.e., the learning model) whose parameters have been determined in this way is the learned model.

[0038] Next, as a final check, input data of test data (data #8501 to #10000) is input in order to the convolutional neural network of the trained model. The convolutional neural network of the trained model outputs the Clip value actually set for the image data before flattening, the number of times the process was actually performed on the image data, and the image data after the histogram was flattened, which is obtained by actually performing the process on the image data, according to the input test data. If the output data output by the convolutional neural network of the trained model for all the test data matches the output data associated with the input data in FIG. 4, the convolutional neural network of the trained model is the desired model. Also, if the output data output by the convolutional neural network of the trained model for even one of the test data does not match the output data associated with the input data in FIG. 4, the parameters of the trained model are determined using new teacher data. The above-mentioned determination of the parameters of the trained model is repeated until a trained model having the desired parameters is obtained. If a trained model having the desired parameters is obtained, the trained model is recorded in the third processing unit 503.

[0039] In the trained model in which the third processing unit 503 predicts the number of times the process is repeated and the Clip value, the image data before flattening is the input data, and the Clip value actually set for the image data before flattening, the number of times the process is repeated and the image data after the flattening of the histogram obtained by actually processing the image data are the output data. However, in the trained model in which the third processing unit 503 predicts the number of times the process is repeated and the Clip value, the first histogram Y may be used as input data in addition to or instead of the image data before flattening, the second histogram Y1 may be used as output data in addition to or instead of the image data after the flattening of the histogram obtained by actually processing the image data before flattening, and the Clip value actually set for the image data before flattening and the number of times the process is repeated and the image data before flattening are the output data. Then, these input data and output data are used as one set of data, and for example, 10,000 sets of data are prepared. FIG. 5 is a diagram showing an example of teacher data different from that in FIG. 4. Using these 10,000 sets of data, parameters of a learning model are determined and recorded as a learned model in the third processing unit 503. Then, the third processing unit 503 may use this learned model to predict the number of times the process is repeated and the Clip value.

[0040] The fourth processing unit 504 generates a Y1 signal (an example of a fifth signal) by changing the luminance indicated by the Y signal based on the second histogram Y1. Specifically, the fourth processing unit 504 generates the luminance of the second histogram Y1 corresponding to each pixel as the Y1 signal. Note that the fourth processing unit 504 may increase the luminance of the Y1 signal as the luminance of the Y signal decreases, and decrease the luminance of the Y1 signal as the luminance of the Y signal increases.

[0041] When the information processing device 50 performs signal processing on the Y, R, and B signals, the fifth processing unit 505 changes the first color difference signal Xr including the value obtained by subtracting the Y signal from the R signal to a first color difference signal Xr1 including the value obtained by subtracting the Y1 signal from the R signal in response to the change from luminance Y to luminance Y1 by the fourth processing unit 504. Furthermore, when the information processing device 50 performs signal processing on the Y, R, and B signals, the fifth processing unit 505 changes the second color difference signal Xb obtained by subtracting the Y signal from the B signal to a second color difference signal Xb1 including the value obtained by subtracting the Y1 signal from the B signal in response to the change from luminance Y to luminance Y1 by the fourth processing unit 504.

[0042] Specifically, when the information processing device 50 processes the Y, R, and B signals, the fifth processing unit 505 changes the first color difference signal Xr including the value obtained by subtracting the Y signal from the R signal to a first color difference signal Xr1 including the value obtained by subtracting the Y1 signal from the R signal for each pixel. Also, specifically, when the information processing device 50 processes the Y, R, and B signals, the fifth processing unit 505 changes the second color difference signal Xb including the value obtained by subtracting the Y signal from the B signal to a second color difference signal Xb1 including the value obtained by subtracting the Y1 signal from the B signal for each pixel. Note that when the video signal handled by the information processing device 50 is an SD (Standard Definition) video, the first color difference signal Xr is a color difference signal Cr, and the second color difference signal Xb is a color difference signal Cb. Furthermore, when the video signal handled by the information processing device 50 is an HD (High Definition) video, the first color difference signal Xr is the color difference signal Pr, and the second color difference signal Xb is the color difference signal Pb.

[0043] More specifically, when the information processing device 50 performs signal processing on the Y signal, R signal, and B signal that are SD video, the fifth processing unit 505 changes the first color difference signal Xr based on a signal (an example of a ninth signal) obtained by subtracting the Y1 signal from a signal obtained by multiplying the R signal by a predetermined number when the fourth processing unit 504 generates the Y1 signal by multiplying the luminance of the Y signal by a predetermined number. The fifth processing unit 505 also changes the second color difference signal Xb based on a signal (an example of a tenth signal) obtained by subtracting the Y1 signal from a signal obtained by multiplying the B signal by a predetermined number. For example, the fifth processing unit 505 changes the first color difference signal Cr expressed by equation (1) for each pixel to the first color difference signal Cr1 expressed by equation (2). The fifth processing unit 505 also changes the second color difference signal Cb expressed by equation (3) for each pixel to the second color difference signal Cb1 expressed by equation (4).

[0044]

number

[0045]

number

[0046]

number

[0047]

number

[0048] In addition, when the information processing device 50 performs signal processing on the Y signal, R signal, and B signal that are HD video, the fifth processing unit 505 generates a first color difference signal Xr1 (an example of an eleventh signal) and a second color difference signal Xb1 (an example of a twelfth signal) by multiplying the first color difference signal Xr and the second color difference signal Xb generated based on the R signal, the B signal, and the G signal by a predetermined number when the fourth processing unit 504 generates a Y1 signal by multiplying the luminance of the Y signal by a predetermined number. For example, the fifth processing unit 505 changes the first color difference signal Pr expressed by equation (5) for each pixel to the first color difference signal Pr1 expressed by equation (6). In addition, the fifth processing unit 505 changes the second color difference signal Pb expressed by equation (7) for each pixel to the second color difference signal Pb1 expressed by equation (8).

[0049]

number

[0050]

number

[0051]

number

[0052]

number

[0053] The value α (an example of a predetermined number) is obtained by dividing the luminance Y1 by the luminance Y.

[0054] Furthermore, when the information processing device 50 processes the Y signal, the R signal, the B signal, and the G signal, the fifth processing unit 505 generates a first color difference signal Xr and a second color difference signal Xb based on the R signal, the B signal, and the G signal. Then, the fifth processing unit 505 changes the generated first color difference signal Xr to a first color difference signal Xr1 based on the change from luminance Y to luminance Y1 by the fourth processing unit 504. Also, the fifth processing unit 505 changes the generated second color difference signal Xb to a second color difference signal Xb1 based on the change from luminance Y to luminance Y1 by the fourth processing unit 504. Note that, when the video signal handled by the information processing device 50 is an SD video, the first color difference signal Xr is a color difference signal Cr, and the second color difference signal Xb is a color difference signal Cb. Also, when the video signal handled by the information processing device 50 is an HD video, the first color difference signal Xr is a color difference signal Pr, and the second color difference signal Xb is a color difference signal Pb.

[0055] Specifically, when the information processing device 50 processes the Y, R, B, and G signals of an SD video, the fifth processing unit 505 changes, for each pixel, the first color difference signal Cr expressed by equation (9) to the first color difference signal Cr1 expressed by equation (10). Also, the fifth processing unit 505 changes, for each pixel, the second color difference signal Cb expressed by equation (11) to the second color difference signal Cb1 expressed by equation (12).

[0056]

number

[0057]

number

[0058]

number

[0059]

number

[0060] Furthermore, when the information processing device 50 processes the Y, R, B, and G signals that are HD video, the fifth processing unit 505 changes, for each pixel, the first color difference signal Pr expressed by equation (13) to the first color difference signal Pr1 expressed by equation (14). Furthermore, the fifth processing unit 505 changes, for each pixel, the second color difference signal Pb expressed by equation (15) to the second color difference signal Pb1 expressed by equation (16).

[0061]

number

[0062]

number

[0063]

number

[0064]

number

[0065] The information processing device 50 may also execute processes other than the processes described above. The processes performed by the information processing device 50, including processes other than the processes described above that the information processing device 50 may execute, will be described in detail in the following description of the processing flow.

[0066] Next, the processing performed by the information processing device 50 will be described with reference to the processing flows shown in FIGS.

[0067] (Processing performed by information processing device) FIG. 6 is a diagram showing an example of a processing flow of the entire processing performed by the information processing device 50 according to an embodiment of the present disclosure. When the information processing device 50 receives an RGB signal, the first processing unit 501 executes setting of signal processing described later, setting or changing of parameters necessary for the signal processing described later, and the like (step S1). For example, it is set whether to execute the processing for correcting low illuminance for the RGB signal, for the Y signal, or to pass without executing the processing for correcting low illuminance. When pass is set, the processing for correcting low illuminance is not performed. An example of the processing for correcting low illuminance is processing using a single scale Retinex (SSR) method that changes an image using the Retinex theory. In the processing using the Retinex method, for example, a filter such as a Center / Surround model, that is, log(Y1 / Gaussian filter(Y1)) filter is used. The process using the Retinex technique is a process in which the luminance of the Y1 signal is increased as the luminance of the Y signal decreases, and the luminance of the Y1 signal is decreased as the luminance of the Y signal increases, that is, the luminance is corrected to be higher as the luminance in the image decreases, and the luminance is corrected to be lower as the luminance in the image increases. Note that the process using the Retinex technique is not limited to a process using a Center / Surround model, that is, a log(Y1 / Gaussian filter(Y1)) filter, etc., as long as it is a process in which the luminance in the image decreases as the luminance decreases, and the luminance is corrected to be lower as the luminance in the image increases. Also, for example, when a user specifies a Clip value, which is one of the parameters, the first processing unit 501 receives an input specifying the Clip value by the user, and sets the received Clip value. Also, when the value of the parameter specified by the user is different from the value already set, the first processing unit 501 changes the value of the parameter that is set to the value of the parameter specified by the user. By performing processing using the Retinex technique, an image that is dark overall and makes it difficult to see the subject can be changed into an image in which the subject is easily seen. For example, no image quality adjustment is required even for images taken during the day and at night.

[0068] When the process of step S1 ends, the fourth processing unit 504 executes low luminance correction for the RGB signal (if the first processing unit 501 performs signal processing on the RGB signal, the RGB signal after the signal processing) (step S2).

[0069] 7 is a diagram showing an example of a processing flow of the processing of step S2 performed by the information processing device 50 according to an embodiment of the present disclosure. Here, with reference to FIG. 7, a process of correcting low luminance performed by the fourth processing unit 504 in the processing of step S2 will be described.

[0070] In the process of step S2, the fourth processing unit 504 judges whether or not to perform low luminance correction on the RGB signal (step S201). If the fourth processing unit 504 judges that low luminance correction is to be performed on the RGB signal (YES in step S201), it executes processing using the SSR method (step S202). Then, the fourth processing unit 504 executes processing to emphasize V (luminance) / S (chroma), and sets variables of luminance and chroma after the processing is executed (step S203). If the fourth processing unit 504 judges that low luminance correction is not to be performed on the RGB signal (NO in step S201), it proceeds to the process of step S203.

[0071] After the process of step S2 is completed, the first processing unit 501 converts the RGB signals into a Y signal, a color difference signal Xr, and a color difference signal Xb (step S3).

[0072] After completing the process of step S3, the information processing device 50 executes a process of correcting the dynamic range of luminance (step S4).

[0073] Specifically, the second processing unit 502 sets a Clip value in the first histogram Y specified by the first processing unit 501. The first processing unit 501 divides an image represented by a plurality of pixels into M×N (for example, divides the image into 1×1 to make the whole into one image), and specifies a first histogram Y which is a histogram of luminance (i.e., Y signal) in each divided image. The third processing unit 503 subtracts a part y near the peak of the first histogram Y from the first histogram Y based on the Clip value. Then, the third processing unit 503 moves a histogram y1 corresponding to the part y of the first histogram Y which has been subtracted to an area where the value of the first histogram Y is at least relatively low. By repeating this process as many times as necessary, the third processing unit 503 generates a second histogram Y1 which is a histogram of luminance. Note that the number of times the process is repeated may be determined by stopping the process when the user checks the image and determines that the image has become the desired image each time the process is performed. The number of times the process is repeated may be determined in advance. The number of times the process is repeated may be determined, together with the Clip value, using AI technology. For example, the third processing unit 503 may predict the number of times the process is repeated and the Clip value by using a trained model (e.g., a convolutional neural network) whose parameters are determined using teacher data, which is one of the machine learning methods.

[0074] 8 is a diagram showing an example of a processing flow of the processing of step S4 performed by the information processing device 50 according to an embodiment of the present disclosure. Here, with reference to FIG. 8, the processing of correcting low luminance performed by the information processing device 50 in the processing of step S4 will be described in more detail.

[0075] In the process of step S4, the first processing unit 501 sets the number of divisions M×N into which an image represented by a plurality of pixels is divided (step S401). The second processing unit 502 sets a Clip value (step S401). The third processing unit 503 sets the number of times the process is repeated L (step S401). The first processing unit 501 divides an image represented by a plurality of pixels into M×N (for example, divides the image into 1×1 and treats the whole as one image) to perform histogram flattening for luminance (i.e., Y signal), and in a pre-stage process described later or a post-stage process described later, divides the image into a number relatively larger than the number of divisions of M×N described above (for example, divides the image into 8×8 and treats the whole as 64 images) to perform histogram flattening for luminance, thereby specifying a first histogram Y, which is a histogram of luminance in each image after division.

[0076] The third processing unit 503 judges whether or not there is a previous stage process for the image after division into M×N (i.e., a process for flattening the histogram for the luminance of the image with a large number of divisions) (step S402). When the third processing unit 503 judges that there is a previous stage process (YES in step S402), it executes a process of flattening the histogram for the luminance of the image divided into M×N (for example, 64 images divided into 8×8) using the Clip value (step S403). That is, the process of step S403 is a process for finely flattening the histogram for the entire image by this previous stage process before performing the processes of steps S404 to S407 described later. Then, the third processing unit 503 sets the initial value of a value IN used to judge the end of the number of times the process is repeated to zero (step S404). When the third processing unit 503 judges that there is no previous stage process (NO in step S402), it proceeds to the process of step S404.

[0077] The third processing unit 503 moves the histogram y1 corresponding to the part y of the subtracted first histogram Y to an area where the value of the first histogram Y is at least relatively low for each of the images numbered 1 to M×N of the M×N divided images (for example, an image divided into 1×1 and treated as one image as a whole) (step S405). The third processing unit 503 adds 1 to the value IN (step S406).

[0078] The third processing unit 503 judges whether or not the value IN determined by the process of step S406 is equal to the value obtained by subtracting 1 from the number of times the process is repeated L (step S407). When the third processing unit 503 judges that the value IN is different from the value obtained by subtracting 1 from the number of times the process is repeated L (NO in step S407), the process returns to the process of step S405. When the third processing unit 503 judges that the value IN is equal to the value obtained by subtracting 1 from the number of times the process is repeated L (YES in step S407), the fourth processing unit 504 generates a Y1 signal in which the luminance indicated by the Y signal is changed based on the second histogram Y1. The fourth processing unit 504 judges whether or not there is a subsequent process (i.e., a process for flattening the histogram for the luminance of an image with a large number of divisions) (step S408). When the fourth processing unit 504 determines that there is a post-stage process (YES in step S408), it executes a process of flattening the histogram using the Clip value for the luminance of the image divided into M×N (for example, 64 images divided into 8×8) (step S409). That is, the process of step S409 is a process of finely flattening the histogram for the entire image by this post-stage process after the processes of steps S404 to S407 are performed. Then, the fourth processing unit 504 determines whether or not readjustment is necessary (step S410). For example, the fourth processing unit 504 determines whether or not a preset number of times of processing L-1 has been reached. Then, when the fourth processing unit 504 determines that the preset number of times of processing L-1 has not been reached, it determines that readjustment is necessary. Also, when the fourth processing unit 504 determines that the preset number of times of processing L-1 has been reached, it determines that readjustment is unnecessary. The fourth processing unit 504 may use a predetermined determiner to determine whether or not readjustment is necessary based on the distribution of the histogram. If the fourth processing unit 504 determines that readjustment is necessary (YES in step S410), the process returns to step S401. If the fourth processing unit 504 determines that readjustment is not necessary (NO in step S410), the fourth processing unit 504 outputs a Y1 signal indicating the luminance after processing.

[0079] As described above, by reducing the number of M×N divisions of the image (for example, dividing the image into 1×1) and repeating the process of flattening the luminance histogram L-1 times, and by increasing the number of M×N divisions of the image by pre-processing or post-processing (for example, dividing the image into 8×8) and performing the process of flattening the luminance histogram once, it is possible to increase the contrast ratio compared to the case where the number of M×N divisions of the image is increased (for example, dividing the image into 8×8) and the process of flattening the luminance histogram is repeated L times.

[0080] In another embodiment, both the pre-processing and the post-processing may be performed. In this case, the brightness is appropriately corrected by performing processing using the Retinex method, and the contrast ratio can be increased. In another embodiment, both the pre-processing and the post-processing may not be performed. In this case, the contrast ratio can be increased by increasing the number of times the processing is performed to flatten the brightness histogram of an image with a relatively small number of divisions (in the above example, the image divided into 1×1).

[0081] If there is other luminance-related processing, the fourth processing unit 504 executes that processing (step S5). Examples of other luminance-related processing include NR (Noise Reduction), Sharpness, and weighted average processing.

[0082] After the process of step S5 is completed, the fourth processing unit 504 performs low luminance correction on the Y1 signal after the process of step S5 (step S6). Since the process of correcting low luminance in step S6 is performed on the Y1 signal, the number of calculations can be reduced compared to the process of correcting low luminance on the RGB signals in step S2.

[0083] 9 is a diagram showing an example of a processing flow of the processing of step S6 performed by the information processing device 50 according to an embodiment of the present disclosure. Here, with reference to FIG. 9, a process of correcting low luminance performed by the fourth processing unit 504 in the processing of step S6 will be described.

[0084] In the process of step S6, the fourth processing unit 504 judges whether or not to perform low luminance correction on the Y1 signal (step S601). If the fourth processing unit 504 judges that low luminance correction is to be performed on the Y1 signal (YES in step S601), it executes processing using the SSR method (step S602). If the fourth processing unit 504 judges that low luminance correction is not to be performed on the Y1 signal (NO in step S601), it proceeds to the process of step S7.

[0085] The fifth processing unit 505 determines the value α by dividing the Y1 signal that has been subjected to the process of step S6 by the Y signal obtained by the process of step S3 (step S7).

[0086] After completing the process of step S7, the information processing device 50 executes a process of correcting the dynamic range of color differences (step S8).

[0087] Specifically, the fifth processing unit 505 uses the value α obtained in step S7 for the color difference signals Xr and Xb obtained in step S3 to specify the color difference signals Cr1 and Cb1 by using equations (2) and (4) when the information processing device 50 processes an SD video signal. Also, the fifth processing unit 505 uses the value α obtained in step S7 for the color difference signals Xr and Xb obtained in step S3 to specify the color difference signals Pr1 and Pb1 by using equations (6) and (8) when the information processing device 50 processes an HD video signal.

[0088] Specifically, the fifth processing unit 505 may specify the color difference signal Cr1 and the color difference signal Cb1 by using the value α obtained in the process of step S7 for the RGB signal received by the information processing device 50, and the color difference signal Pr1 and the color difference signal Pb1 by using equations (10) and (12) when the information processing device 50 processes an SD video signal. The fifth processing unit 505 may specify the color difference signal Pr1 and the color difference signal Pb1 by using the value α obtained in the process of step S7 for the RGB signal received by the information processing device 50, and the color difference signal Pb1 by using equations (14) and (16) when the information processing device 50 processes an HD video signal.

[0089] The fifth processing unit 505 converts the Y1 signal processed in step S6 and the color difference signals Xr1 and Xb1 identified in step S8 into an RGB signal (step S9). Then, the fifth processing unit 505 judges whether or not readjustment is necessary (step S10). For example, the fifth processing unit 505 judges whether or not a preset number of processes has been reached. Then, when the fifth processing unit 505 judges that the preset number of processes has not been reached, it judges that readjustment is necessary. Also, when the fifth processing unit 505 judges that the preset number of processes has been reached, it judges that readjustment is unnecessary. Note that the fifth processing unit 505 may use a predetermined judger to judge whether or not readjustment is necessary based on the RGB signal. When the fifth processing unit 505 judges that readjustment is necessary (YES in step S10), it returns to the processing of step S1. Also, when the fifth processing unit 505 judges that readjustment is unnecessary (NO in step S10), it outputs the converted RGB signal.

[0090] The information processing system 1 according to an embodiment of the present disclosure has been described above. The information processing device 50 in the information processing system 1 includes a first processing unit 501, a second processing unit 502, and a third processing unit 503. The first processing unit 501 regards an entire image represented by a plurality of pixels as one image, and specifies a first histogram which is a histogram of luminance in the image. The second processing unit 502 sets an adjustment amount for the first histogram. The third processing unit 503 subtracts a part near the peak of the first histogram from the first histogram based on the adjustment amount, and moves the histogram corresponding to the part to a region where the peak of the first histogram is at least relatively low, thereby generating a second histogram.

[0091] In this way, the information processing device 50 can correct the dynamic range of the video signal.

[0092] In addition, by doing so, the information processing device 50 can perform a complex operation on one of the luminance signals and apply only the operation result to the color difference signals, without performing a complex operation on each of the three RGB signals. As a result, compared to performing a complex operation on each of the three RGB signals, by performing a complex operation on one of the luminance signals and applying only the operation result to the color difference signals, it is possible to reduce the amount of operation of the information processing device 50, that is, the operation time and the power consumption required for the operation.

[0093] Furthermore, by doing so, the information processing device 50 can be realized by a general-purpose computer such as a personal computer that processes the three RGB signals using software, rather than by expensive dedicated hardware that processes the signals using hardware.

[0094] Furthermore, by doing so, the information processing device 50 can be realized by a general-purpose computer that performs relatively low-bit processing (for example, 8-bit processing) by performing processing by software.

[0095] In another embodiment of the present disclosure, the information processing device 50 may be mounted on a camera that captures still and moving images. FIG. 10 is a diagram showing an example of the configuration of the information processing device 50 according to another embodiment of the present disclosure. For example, as shown in FIG. 10, the information processing device 50 includes a sixth processing unit 506. The sixth processing unit 506 performs contrast AF (Autofocus) processing of the camera. The contrast AF (Autofocus) processing is a process of searching for a place with high contrast and adjusting the focus. The information processing device 50 can increase the contrast ratio. Therefore, by using this image as an image in the contrast AF processing of the camera, the sixth processing unit 506 can easily adjust the focus on the subject compared to the case where the image before the contrast ratio is increased by the information processing device 50 is used.

[0096] In another embodiment of the present disclosure, a part or all of the processing performed by the ISP 20 may be performed by the information processing device 50. FIG. 11 is a diagram showing an example of the configuration of the information processing device 50 according to another embodiment of the present disclosure. For example, as shown in FIG. 11, the information processing device 50 includes a seventh processing unit 507. The seventh processing unit 507 may execute a part or all of the processing performed by the ISP 20. In this way, the ISP 20 can be realized simply and inexpensively or the ISP 20 becomes unnecessary, and the information processing device 50 can realize a part or all of the processing performed by the ISP 20 by changing the software executed by a general-purpose computer.

[0097] In addition, in another embodiment of the present disclosure, the information processing device 50 may perform HDR processing. For example, as shown in FIG. 11, the information processing device 50 includes a seventh processing unit 507. The seventh processing unit 507 may perform HDR processing. In this case, it is possible to avoid problems that occur when the ISP 20 performs HDR processing. FIG. 12 is a first diagram for explaining the effect of HDR processing performed by the seventh processing unit 507 according to another embodiment of the present disclosure. FIG. 12 shows, for example, an example of conventional HDR processing performed by the ISP 20. FIG. 13 is a second diagram for explaining the effect of HDR processing performed by the seventh processing unit 507 according to another embodiment of the present disclosure. FIG. 13 shows an example of HDR processing performed by the seventh processing unit 507.

[0098] In the conventional HDR processing performed by the ISP20, as shown in part (a) of FIG. 12, two frames of images taken with two or more different exposure times are synthesized. Therefore, if the subject moves during the long exposure, it does not match the image exposed for a short time, and as a result, as shown in part (b) of FIG. 12, the image becomes one in which two subjects overlap. In contrast, in the HDR processing performed by the seventh processing unit 507, as shown in part (a) of FIG. 13, one frame of image processing is performed with a short exposure for each subject. Therefore, as shown in part (b) of FIG. 13, two subjects do not overlap as in the conventional HDR processing. In addition, since the HDR processing performed by the seventh processing unit 507 is one frame processing with a short exposure, it is possible to realize processing at a higher frame rate than the conventional HDR processing.

[0099] In the embodiment of the present disclosure, the fifth processing unit 505 has been described as converting the Y1 signal processed in step S6 and the color difference signals Xr1 and Xb1 identified in the process of step S8 into RGB signals. However, in another embodiment of the present disclosure, the fifth processing unit 505 may convert each of the RGB signals (i.e., each of the R signal, G signal, and B signal) received by the information processing device 50 into an RGB signal after changing the luminance (after changing the contrast) by multiplying each of the RGB signals (i.e., each of the R signal, G signal, and B signal) by the value α obtained in the process of step S7.

[0100] It should be noted that the order of the processes in the embodiments of the present disclosure may be changed as long as appropriate processes are performed.

[0101] Although the embodiment of the present disclosure has been described, the above-mentioned information processing system 1, information processing device 50, and other control devices may have a computer system inside. The above-mentioned process steps are stored in a computer-readable recording medium in the form of a program, and the above-mentioned process is performed by the computer reading and executing the program. Specific examples of computers are shown below.

[0102] 14 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in FIG. 14, the computer 5 includes a CPU 6, a main memory 7, a storage 8, and an interface 9.

[0103] For example, the above-mentioned information processing system 1, information processing device 50, and other control devices are each implemented in a computer 5. The operations of the above-mentioned processing units are stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8, loads it in the main memory 7, and executes the above-mentioned processes according to the program. The CPU 6 also secures storage areas in the main memory 7 corresponding to the above-mentioned storage units according to the program.

[0104] Examples of the storage 8 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), and a semiconductor memory. The storage 8 may be an internal medium directly connected to the bus of the computer 5, or an external medium connected to the computer 5 via an interface 9 or a communication line. In addition, when the program is distributed to the computer 5 via a communication line, the computer 5 that receives the program may load the program into the main memory 7 and execute the above-mentioned process. In at least one embodiment, the storage 8 is a non-transitory tangible storage medium.

[0105] The program may also realize some of the functions described above. Furthermore, the program may be a file that can realize the functions described above in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0106] FIG. 15 is a diagram showing an example of implementation of an information processing device 50 according to an embodiment of the present disclosure. The information processing device 50 is implemented by combining a computer 5 with an external GPU (Graphics Processing Unit) 50a. Most recent CPUs 6 have multiple cores and are equipped with a GPU. When the information processing device 50 is implemented in this way, the processing is highly parallel, and can be handled by parallel programming processing using a GPU. In addition, if the specifications can be handled by an iGPU, a dGPU is not required (i.e., an external GPU is not required), which allows for cost reduction.

[0107] Although several embodiments of the present disclosure have been described, these embodiments are merely examples and do not limit the scope of the invention. Various additions, omissions, substitutions, and modifications may be made to these embodiments without departing from the spirit of the invention.

[0108] Note that some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0109] (Appendix 1) a first processing unit that defines an image as a whole represented by a plurality of pixels and identifies a first histogram that is a histogram of luminance in the image; A second processing unit that sets an adjustment amount in the first histogram; a third processing unit that generates a second histogram by subtracting a portion of the first histogram near a peak based on the adjustment amount and shifting a histogram corresponding to the portion to a region where the peak of the first histogram is at least relatively low; An information processing device comprising:

[0110] (Appendix 2) 1. An information processing device that processes a first signal indicating luminance, a second signal related to a first color, and a third signal related to a second color, or the first signal, the second signal, the third signal, and a fourth signal related to a third color, comprising: a fourth processing unit that generates a fifth signal by modifying a luminance indicated by the first signal based on the second histogram; a fifth processing unit that, when performing signal processing on the first signal, the second signal, and the third signal, changes a first color difference signal including a value obtained by subtracting the first signal from the second signal to a first color difference signal including a value obtained by subtracting the fifth signal from the second signal, and changes a second color difference signal including a value obtained by subtracting the first signal from the third signal to a second color difference signal including a value obtained by subtracting the fifth signal from the third signal, in accordance with the change in luminance by the fourth processing unit; and, when performing signal processing on the first signal, the second signal, the third signal, and the fourth signal, generates a first color difference signal and a second color difference signal based on the second signal, the third signal, and the fourth signal, and changes the generated first color difference signal and the generated second color difference signal based on the change in luminance by the fourth processing unit. 2. The information processing device according to claim 1, comprising:

[0111] (Appendix 3) When performing signal processing on the first signal, the second signal, and the third signal, The fifth processing unit is when the fourth processing unit generates the fifth signal by multiplying the luminance of the first signal by a predetermined number, the fourth processing unit modifies the first color difference signal based on a ninth signal obtained by subtracting the fifth signal from a signal obtained by multiplying the second signal by the predetermined number, and modifies the second color difference signal based on a tenth signal obtained by subtracting the fifth signal from a signal obtained by multiplying the third signal by the predetermined number; When performing signal processing on the first signal, the second signal, the third signal, and the fourth signal, The fifth processing unit is when the fourth processing unit generates the fifth signal by multiplying the luminance of the first signal by a predetermined number, the fourth processing unit generates an eleventh signal and a twelfth signal by multiplying, by the predetermined number, a first color difference signal and a second color difference signal generated based on the second signal, the third signal, and the fourth signal, respectively. 3. An information processing device according to claim 2.

[0112] (Appendix 4) The fourth processing unit is As the luminance of the first signal decreases, the luminance of the fifth signal is increased, and as the luminance of the first signal increases, the luminance of the fifth signal is decreased. 4. The information processing device according to claim 2 or 3.

[0113] (Appendix 5) The fourth processing unit is a calculation of a process of increasing the luminance of the fifth signal as the luminance of the first signal decreases, and decreasing the luminance of the fifth signal as the luminance of the first signal increases, is performed with double precision. 5. The information processing device according to claim 4.

[0114] (Appendix 6) By using a Retinex technique for modifying an image using Retinex theory, the luminance of the fifth signal is increased as the luminance of the first signal decreases, and the luminance of the fifth signal is decreased as the luminance of the first signal increases. 6. The information processing device according to claim 4 or 5.

[0115] (Appendix 7) a sixth processing unit that performs a contrast autofocus process using an image whose luminance has been corrected based on the second histogram generated by the third processing unit; Equipped with 7. An information processing device according to any one of claims 1 to 6.

[0116] (Appendix 8) a seventh processing unit that performs HDR processing based on an image whose luminance has been corrected based on the second histogram generated by the third processing unit; Equipped with 8. An information processing device according to any one of claims 1 to 7.

[0117] (Appendix 9) determining a first histogram which is a histogram of luminance in an image represented by a plurality of pixels; Setting an adjustment amount for the first histogram; generating a second histogram by subtracting a portion of the first histogram near a peak based on the adjustment amount and shifting a histogram corresponding to the portion to a region where the peak of the first histogram is at least relatively low; An information processing method comprising:

[0118] (Appendix 10) On the computer, determining a first histogram which is a histogram of luminance in an image represented by a plurality of pixels; Setting an adjustment amount for the first histogram; generating a second histogram by subtracting a portion of the first histogram near a peak based on the adjustment amount and shifting a histogram corresponding to the portion to a region where the peak of the first histogram is at least relatively low; A program that executes the following. [Explanation of symbols]

[0119] 1. Information Processing System 5. Computers 6. CPU 7. Main memory 8. Storage 9. Interface 10. Sensor 20. ISP 30...Storage device 40....Image processing device 50....Information processing device 50a...External GPU 60···AI / CV 70. Display 501 First processing section 502 Second processing section 503 Third processing section 504 Fourth processing section 505 5th processing section

Claims

1. a first processing unit that identifies a first histogram that is a histogram of luminance in an image represented by a plurality of pixels; a second processing unit that sets a clip value in the first histogram; a third processing unit that subtracts a portion of the first histogram from the first histogram based on the Clip value and moves a histogram corresponding to the portion to a region where the values ​​of the first histogram are at least relatively low, thereby generating a second histogram in which a luminance histogram of the image is flattened; An information processing device comprising:

2. 1. An information processing device that processes a Y signal indicating luminance, an R signal relating to red, and a B signal relating to blue, or the Y signal, the R signal, the B signal, and a G signal relating to green, comprising: a fourth processing unit that generates a Y1 signal by changing a luminance indicated by the Y signal based on the second histogram; a fifth processing unit that, when performing signal processing on the Y signal, the R signal, and the B signal, changes a first color difference signal obtained by subtracting the Y signal from the R signal to a first color difference signal obtained by subtracting the Y1 signal from the R signal, and changes a second color difference signal obtained by subtracting the Y signal from the B signal to a second color difference signal obtained by subtracting the Y1 signal from the B signal, in accordance with the change in luminance by the fourth processing unit; and, when performing signal processing on the Y signal, the R signal, the B signal, and the G signal, generates a first color difference signal and a second color difference signal based on the R signal, the B signal, and the G signal, and changes the first color difference signal and the second color difference signal generated based on the R signal, the B signal, and the G signal, in accordance with the change in luminance by the fourth processing unit; The information processing device according to claim 1 .

3. When performing signal processing on the Y signal, the R signal, and the B signal, The fifth processing unit is when the fourth processing unit generates the Y1 signal by multiplying the luminance of the Y signal by a predetermined number, the fourth processing unit modifies the first color difference signal based on a signal obtained by subtracting the Y1 signal from a signal obtained by multiplying the R signal by the predetermined number, and modifies the second color difference signal based on a signal obtained by subtracting the Y1 signal from a signal obtained by multiplying the B signal by the predetermined number, When performing signal processing on the Y signal, the R signal, the B signal, and the G signal, the fifth processing unit when the fourth processing unit generates the Y1 signal by multiplying the luminance of the Y signal by a predetermined number, a new first color difference signal and a new second color difference signal are generated by multiplying a first color difference signal and a second color difference signal generated based on the R signal, the B signal, and the G signal by the predetermined number, respectively. The information processing device according to claim 2 .

4. The fourth processing unit is As the luminance of the Y signal decreases, the luminance of the Y1 signal is increased, and as the luminance of the Y signal increases, the luminance of the Y1 signal is decreased.

4. The information processing device according to claim 2.

5. The fourth processing unit is a process of increasing the luminance of the Y1 signal as the luminance of the Y signal decreases, and decreasing the luminance of the Y1 signal as the luminance of the Y signal increases, the process being performed with double precision; The information processing device according to claim 4.

6. By using a Retinex technique for modifying an image using Retinex theory, the luminance of the Y1 signal is increased as the luminance of the Y signal is decreased, and the luminance of the Y1 signal is decreased as the luminance of the Y signal is increased. The information processing device according to claim 4.

7. determining a first histogram, the first histogram being a histogram of luminance in an image represented by a plurality of pixels; Setting a clip value in the first histogram; generating a second histogram in which a histogram of luminance in the image is flattened by subtracting a portion of the first histogram from the first histogram based on the clip value and shifting a histogram corresponding to the portion to a region in which values ​​of the first histogram are at least relatively low; An information processing method comprising:

8. On the computer, determining a first histogram, the first histogram being a histogram of luminance in an image represented by a plurality of pixels; Setting a clip value in the first histogram; generating a second histogram in which a histogram of luminance in the image is flattened by subtracting a portion of the first histogram from the first histogram based on the clip value and shifting a histogram corresponding to the portion to a region in which values ​​of the first histogram are at least relatively low; A program that executes the following.

Images