Image processing device, image pickup device, image processing method, and program

Abstract

To provide a technology for improving image quality of an image to which haze correction has been applied.SOLUTION: Provided is an image processing device including: evaluation image generation means for generating an evaluation image for haze correction based on an image; transmission map generation means for generating a transmission map for the haze correction based on the evaluation image; first correction means for applying the haze correction based on the transmission map to the evaluation image; second correction means for applying the haze correction based on the transmission map to the image; and third correction means for correcting brightness of the image to which the haze correction has been applied based on an amount of change in the evaluation image before and after application of the haze correction.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an imaging apparatus, an image processing method, and a program.

Background Art

[0002] There is known a technique of haze correction that estimates the amount of haze in an image as a transmission map based on a known method called the DarkChannelPrior method using a haze model, and realizes an effect of removing haze from the image based on the estimated transmission map. Haze correction is used for improving the visibility of surveillance camera images and adjusting the appearance of images captured by a camera.

[0003] However, when haze correction is performed by a known technique, the image becomes dark due to the principle of the technique. Therefore, when haze correction is performed on an image captured with appropriate brightness by exposure control with a camera, depending on the scene, the image may appear underexposed and the impression may be poor. In addition, not only does the image become dark, but the saturation of the image also increases, so depending on the scene, the image may appear unnatural in color.

[0004] Patent Document 1 discloses a technique of calculating an emphasis suppression region that is at least one of an empty region and a backlight region estimated to be a backlight region, and suppressing the intensity of haze correction in the emphasis suppression region. Patent Document 2 discloses a technique of converting color signals before and after haze correction into an HSV color space, calculating a saturation value difference based on the pixel saturation values before and after haze correction, and correcting the saturation of the image after haze correction based on the saturation value difference.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, in the technique of Patent Document 1, even when the change in brightness in the emphasis suppression region is small, the effect of blurring removal in the emphasis suppression region deteriorates. Further, Patent Document 2 does not sufficiently consider the relationship between color components of the image to be subjected to chroma correction.

[0007] The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for improving the image quality of an image to which blurring correction is applied.

Means for Solving the Problems

[0008] In order to solve the above problems, the present invention includes an evaluation image generation means for generating an evaluation image for blurring correction based on an image, a transmission map generation means for generating a transmission map for the blurring correction based on the evaluation image, a first correction means for applying the blurring correction based on the transmission map to the evaluation image, a second correction means for applying the blurring correction based on the transmission map to the image, and a third correction means for correcting the brightness of the image to which the blurring correction has been applied based on the amount of change in the evaluation image before and after application of the blurring correction. An image processing apparatus is provided.

Effects of the Invention

[0009] According to the present invention, it is possible to improve the image quality of an image to which blurring correction is applied.

[0010] Other features and advantages of the present invention will become more apparent from the accompanying drawings and the description in the following mode for carrying out the invention.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0013] [First Embodiment] ● Configuration of the imaging device 100 FIG. 1 is a block diagram showing the configuration of an imaging device 100 including an image processing device. In the present embodiment, when performing blurring correction on an image, the imaging device 100 performs processing to correct (adjust) the brightness of the image so that the image does not become too dark. In the blurring correction of the present embodiment, for example, based on a known method called the DarkChannelPrior method using a blurring model, the amount of blurring of the image is estimated as a transmission map, and the image is corrected based on the estimated transmission map.

[0014] The imaging unit 101 includes a lens, an image sensor, an A / D conversion processing unit, and a development processing unit. The imaging unit 101 captures a subject image and generates an image based on a control signal output from the system control unit 103 according to a user instruction via the operation unit 107.

[0015] The image processing unit 102 performs blurring correction and brightness correction on an image input from the imaging unit 101, the recording unit 105, or the network processing unit 106. Details of the image processing unit 102 will be described later.

[0016] The system control unit 103 includes a ROM in which a control program is stored and a RAM used as a work memory, and controls the overall operation of the imaging device 100 according to the control program. Further, the system control unit 103 performs drive control of the imaging unit 101 based on a control signal input from the network processing unit 106 and the operation unit 107.

[0017] The display unit 104 is a display device including a liquid crystal display or an organic Electro Luminescence (EL) display, and displays an image output from the image processing unit 102.

[0018] The recording unit 105 has a function of recording data such as images. For example, the recording unit 105 may include an information recording medium using a memory card equipped with a semiconductor memory, or a package containing a rotating recording medium such as a magneto-optical disk. This information recording medium may be configured to be detachable from the imaging device 100.

[0019] The network processing unit 106 performs communication processing with external devices. For example, the network processing unit 106 may be configured to acquire an image from an external input device via a network. Also, the network processing unit 106 may be configured to send the image output from the image processing unit 102 to an external display device or an image processing apparatus (e.g., a personal computer (PC)) via a network.

[0020] The operation unit 107 includes operation members such as buttons and a touch panel, and is configured to receive input operations by the user. The operation unit 107 outputs a control signal corresponding to the input operation by the user to the system control unit 103. The user can give a user instruction to the system control unit 103 through the input operation on the operation unit 107.

[0021] The bus 108 is used for exchanging data such as images among the imaging unit 101, the image processing unit 102, the system control unit 103, the display unit 104, the recording unit 105, and the network processing unit 106.

[0022] ● Configuration of the image processing unit 102 Next, with reference to FIG. 2, the configuration of the image processing unit 102 will be described. As shown in FIG. 2, the image processing unit 102 includes an image input unit 201, an evaluation image generation unit 202, a transmission map generation unit 203, a correction processing unit 204, and an image output unit 205. The image input to the image processing unit 102 via the image input unit 201 is an image to be subjected to blurring correction, and is composed of signals of three channels of red (R), green (G), and blue (B). The image output from the image processing unit 102 via the image output unit 205 is an image subjected to blurring correction. The operations of each part of the image processing unit 102 will be described later.

[0023] ● Operation of the image processing unit 102 FIG. 5 is a flowchart showing the operation of the image processing unit 102. In S501, the image input unit 201 receives the input of an image (RGB image) to be subjected to blurring correction.

[0024] In S502, the evaluation image generation unit 202 generates an evaluation image for blurring correction based on the image input in S501. Details of the process of S502 will be described later.

[0025] In S503, the transparency map generation unit 203 generates a transparency map for blurring correction based on the evaluation image generated in S502. The transparency map is an image having the amount of blurring at each pixel of the image to be subjected to blurring correction as a pixel value. Details of the process of S503 will be described later.

[0026] In S504, the correction processing unit 204 performs correction processing including blurring correction and brightness correction on the image input in S501 based on the transparency map generated in S503.

[0027] In S505, the image output unit 205 outputs an image (RGB image) subjected to correction processing including blurring correction and brightness correction in S504.

[0028] ● Details of the process of S502 Next, with reference to FIGS. 3 and 6, details of the process of S502 (process of generating an evaluation image) by the evaluation image generation unit 202 will be described.

[0029] FIG. 3 is a block diagram showing the configuration of the evaluation image generation unit 202. The evaluation image generation unit 202 includes a minimum value image generation unit 301, a hierarchical image generation unit 302, and a hierarchical image integration unit 303. The image input to the minimum value image generation unit 301 is the RGB image input to the image input unit 201. The image output from the hierarchical image integration unit 303 is the evaluation image necessary for generating the transparency map for blurring correction.

[0030] FIG. 6 is a flowchart showing details of the process of S502. In S601, the minimum value image generation unit 301 generates a minimum value image from the input RGB image according to the following equation (1). In equation (1), (x, y) indicates the coordinates (pixel position) of a pixel, pix(x, y) indicates the minimum value image, and R(x, y), G(x, y), and B(x, y) indicate the R signal value, G signal value, and B signal value of the RGB image, respectively. As can be understood from equation (1), the minimum value image is an image having the minimum value of the RGB signals at each pixel position of the RGB image as the pixel value.

[0031]

Number

[0032] In S602, the hierarchical image generation unit 302 performs a process of generating a hierarchical image. The hierarchical image refers to a plurality of images with different frequencies. FIG. 8 is a conceptual diagram of the hierarchical image. In FIG. 8, the horizontal axis indicates the pixel position, and the vertical axis indicates the signal value (pixel value). In the example of FIG. 8, the hierarchical image includes three types of images: the minimum value image 801, the first low-frequency image 802, and the second low-frequency image 803. The minimum value image 801 is the image generated by the minimum value image generation unit 301 in S601. The first low-frequency image 802 is an image obtained by applying a first low-pass filter to the minimum value image 801. The second low-frequency image 803 is an image obtained by applying a second low-pass filter having characteristics different from those of the first low-pass filter to the minimum value image 801. Note that, as the hierarchical image, an image different from the example of FIG. 8 may be used. For example, a plurality of images with different resolutions generated by reduction processing or the like may be used as the hierarchical image.

[0033] In S603, the hierarchical image integration unit 303 performs a process of integrating the hierarchical images generated in S602 to generate a single evaluation image. Specifically, the hierarchical image integration unit 303 generates an evaluation image by obtaining the minimum value of each layer according to the following formula (2) based on a known method called the DarkChannelPrior method. In formula (2), pix(x, y) represents the pixel value of the minimum value image 801. Also, lpf1(x, y) represents the pixel value of the first low-frequency image 802, lpf2(x, y) represents the pixel value of the second low-frequency image 803, and eva(x, y) represents the pixel value of the evaluation image. As can be understood from formula (2), in this embodiment, at each pixel position, a local minimum value image having the minimum value among the minimum value image and the two low-frequency images as the pixel value is generated as the evaluation image.

[0034]

Number

[0035] ●Details of the process of S503 Next, the details of the process of S503 will be described. The transmission map generation unit 203 generates transmission maps for each of the RGB signals based on the evaluation image. The generation of the transmission map is performed according to the following formula (3).

[0036]

Number

[0037] In formula (3), t R (x, y), t G (x, y), and t B (x, y) represent the transmission map for the R signal, the transmission map for the G signal, and the transmission map for the B signal, respectively. eva(x, y) represents the evaluation image obtained according to the aforementioned formula (2). A R , A G , and A Gis the signal value for each RGB showing the atmospheric image in the haze model, and any value can be used, such as the signal value for each RGB of the sky, the signal value for each RGB of a light source such as the sun or a light, or the maximum value that the image signal can take (for example, 4095 in the case of a 12-bit image signal). K is an arbitrary parameter, and by adjusting the value of K, the intensity of the transmission map for haze correction can be adjusted.

[0038] ●Details of the processing in S504 Next, with reference to FIGS. 4, 7, and 9, the details of the processing in S504 by the correction processing unit 204 will be described.

[0039] FIG. 4 is a block diagram showing the configuration of the correction processing unit 204. The correction processing unit 204 includes a first haze correction unit 401, a brightness correction amount calculation unit 402, a second haze correction unit 403, and a brightness correction unit 404. Input to the correction processing unit 204 are the RGB image input to the image input unit 201, the evaluation image generated by the evaluation image generation unit 202, and the transmission map for each of the RGB signals generated by the transmission map generation unit 203. Output from the correction processing unit 204 is an RGB image to which haze correction and brightness correction are applied.

[0040] FIG. 7 is a flowchart showing the details of the processing in S504 according to the first embodiment. In S701, the first haze correction unit 401 applies haze correction to the evaluation image by performing gain processing based on the transmission map on the evaluation image. The following formula (4) based on the haze model is used for the gain processing of haze correction.

[0041]

Equation

[0042] In Equation (4), eva(x, y) represents the evaluation image input from the evaluation image generation unit 202, and out_eva(x, y) represents the evaluation image to which the haze correction has been applied. t(x, y) represents the transmission map, and A is the signal value representing the atmospheric image. However, in Equation (3), the transmission map for each RGB signal is generated, and the signal value representing the atmospheric image is determined for each RGB. On the other hand, the evaluation image corrected by Equation (4) is an image having a single-channel signal generated according to Equation (2). Therefore, the first haze correction unit 401 uses, as t(x, y) and A in Equation (4), for example, the values (t G (x, y) and A G ) for the G signal. Alternatively, the first haze correction unit 401 may use the values for the R signal or the B signal as t(x, y) and A in Equation (4), or may use the value obtained by weighted-averaging the values for the R signal, the G signal, and the B signal.

[0043] In S702, the brightness correction amount calculation unit 402 calculates (generates) a brightness correction amount (brightness correction map) based on the evaluation image to which the haze correction has not been applied and the evaluation image to which the haze correction has been applied in S701. Specifically, as shown in the following Equation (5), the brightness correction amount calculation unit 402 generates a brightness correction map based on the change amount of the evaluation image before and after the application of the haze correction. In Equation (5), adj_map(x, y) represents the brightness correction map, eva(x, y) represents the evaluation image to which the haze correction has not been applied, and out_eva(x, y) represents the evaluation image to which the haze correction has been applied. k is a parameter for adjusting the intensity of the brightness correction, and takes a value greater than 0 and less than or equal to 1 (the greater the value of k, the greater the intensity).

[0044]

Equation

[0045] Note that in Equation (5), the change amount of the evaluation image before and after applying the fog correction is the difference (the pixel-by-pixel difference between the evaluation image without applying the fog correction and the evaluation image with the fog correction applied). However, as shown in Equation (6) below, the brightness correction amount calculation unit 402 may use the ratio (the pixel-by-pixel ratio between the evaluation image without applying the fog correction and the evaluation image with the fog correction applied) as the change amount of the evaluation image before and after applying the fog correction.

[0046]

Equation

[0047] In S703, the second fog correction unit 403 applies fog correction to the RGB image by performing gain processing based on the transmission map on the RGB image (the image to be fog-corrected) input from the image input unit 201.

[0048] The following Equation (7) based on the fog model is used for the gain processing of the fog correction. In Equation (7), R(x, y), G(x, y), and B(x, y) represent the RGB signals of the target image, and Ra(x, y), Ga(x, y), and Ba(x, y) represent the RGB signals of the target image after the fog correction has been applied. t R (x, y), t G (x, y), and t B (x, y) represents the transmission map for each of the RGB signals generated in S503, and A R , A G , and A G represents the signal value for each RGB showing the atmospheric image in the fog model.

[0049]

Equation

[0050] In S704, the brightness correction unit 404 performs brightness correction on the RGB image with the fog correction applied in S703 based on the brightness correction map generated in S702.

[0051] When the brightness correction map is generated according to Equation (5) (when the difference is used as the amount of change in the evaluation image before and after applying the blurring correction), the brightness correction map includes a correction value for each pixel. In this case, brightness correction is performed by adding the correction value for each pixel to each pixel of the target image to which the blurring correction has been applied, as shown in Equation (8) below. In Equation (8), Ra(x, y), Ga(x, y), and Ba(x, y) represent the RGB signals of the target image to which the blurring correction has been applied according to Equation (7). adj_map(x, y) represents the brightness correction map generated according to Equation (5). Ro(x, y), Go(x, y), and Bo(x, y) represent the RGB signals of the target image to which both the blurring correction and the brightness correction have been applied.

[0052]

Number

[0053] When the brightness correction map is generated according to Equation (6) (when the ratio is used as the amount of change in the evaluation image before and after applying the blurring correction), the brightness correction map includes a gain value for each pixel. In this case, brightness correction is performed by multiplying the gain value for each pixel by each pixel of the target image to which the blurring correction has been applied, as shown in Equation (9) below. Different from Equation (8), in Equation (9), adj_map(x, y) represents the brightness correction map generated according to Equation (6).

[0054]

Number

[0055] FIG. 9 is a conceptual diagram of correction processing including blurring correction and brightness correction according to the first embodiment. As shown in FIG. 9, by applying the brightness correction map generated based on the amount of change 901 in the evaluation image before and after applying the blurring correction to the RGB image 902 to which the blurring correction has been applied, an RGB image 903 to which the correction processing including the blurring correction and the brightness correction has been applied is generated.

[0056] ● Summary of the First Embodiment As described above, according to the first embodiment, the image processing unit 102 generates an evaluation image for blurring correction based on the input target image, and generates a transparency map for blurring correction based on the evaluation image. Then, the image processing unit 102 applies blurring correction based on the transparency map to the evaluation image. Also, the image processing unit 102 applies blurring correction based on the transparency map to the target image, and corrects the brightness of the target image to which the blurring correction has been applied based on the change amount of the evaluation image before and after the application of the blurring correction.

[0057] In this way, according to the present embodiment, since the brightness of the target image to which the blurring correction has been applied is corrected based on the change amount of the evaluation image before and after the application of the blurring correction, it is possible to suppress the possibility that the image to which the blurring correction has been applied becomes too dark. Therefore, according to the present embodiment, it is possible to improve the image quality of the image to which the blurring correction has been applied.

[0058] [Second Embodiment] In the first embodiment, a configuration for improving the image quality by correcting the brightness of the image to which the blurring correction has been applied has been described. On the other hand, in the second embodiment, a configuration for improving the image quality by correcting the saturation of the image to which the blurring correction has been applied will be described. Note that in the second embodiment, the basic configuration of the imaging device 100 is the same as that of the first embodiment. Hereinafter, mainly the differences from the first embodiment will be described.

[0059] In the present embodiment, the image sensor of the imaging unit 101 is configured to generate an image including a plurality of color components. In the following description, as an example, the image sensor of the imaging unit 101 is assumed to generate an image (RGB image) including a red component (R component), a green component (G component), and a blue component (B component).

[0060] FIG. 10 is a block diagram showing the configuration of the image processing unit 102 according to the second embodiment. As shown in FIG. 2, the image processing unit 102 includes an image input unit 201, an evaluation image generation unit 202, a transmission map generation unit 203, a correction processing unit 1004, and an image output unit 205. The image input to the image processing unit 102 via the image input unit 201 is a target image for blurring correction and is composed of signals of three channels of R, G, and B. The image output from the image processing unit 102 via the image output unit 205 is an image subjected to blurring correction. As can be understood from FIG. 10, the image processing unit 102 according to the second embodiment is different from the image processing unit 102 according to the first embodiment in that the correction processing unit 204 is replaced by the correction processing unit 1004, and the evaluation image generated by the evaluation image generation unit 202 is not input to the correction processing unit 1004.

[0061] Similar to the first embodiment, the image processing unit 102 performs operations according to the flowchart of FIG. 5, but the content of the correction processing in S504 is different from that of the first embodiment. Hereinafter, with reference to FIGS. 11 to 13, the details of the processing of S504 by the correction processing unit 1004 will be described.

[0062] FIG. 11 is a block diagram showing the configuration of the correction processing unit 1004. The correction processing unit 1004 includes a blurring correction unit 1101 and a chroma correction unit 1102. The correction processing unit 1004 is input with the RGB image input to the image input unit 201 and the transmission maps for each of RGB generated by the transmission map generation unit 203. The correction processing unit 1004 outputs an RGB image to which blurring correction and chroma correction are applied.

[0063] FIG. 12 is a flowchart showing the details of the processing of S504 according to the second embodiment. In S1201, similar to S703 in FIG. 7, the blurring correction unit 1101 applies blurring correction to the RGB image by performing gain processing based on the transmission map on the RGB image input from the image input unit 201.

[0064] In S1202, the saturation correction unit 1102 corrects the saturation of the RGB image to which the blurring correction has been applied based on the RGB image to which the blurring correction has not been applied and the RGB image to which the blurring correction has been applied in S1201. In the present embodiment, the saturation correction unit 1102 performs saturation correction based on the amount of change in the color difference between the RGB images before and after the application of the blurring correction.

[0065] Specifically, first, the saturation correction unit 1102 calculates the color difference after saturation correction according to the following formula (10). In formula (10), R(x, y), G(x, y), and B(x, y) represent the RGB signals of the target image to which the blurring correction has not been applied, and Ra(x, y), Ga(x, y), and Ba(x, y) represent the RGB signals of the target image to which the blurring correction has been applied. RG_adj(x, y) and BG_adj(x, y) represent the color difference signals after saturation correction. k is a parameter for adjusting the intensity of the saturation correction, and takes a value greater than 0 and less than or equal to 1 (the larger the value of k, the greater the intensity).

[0066]

Equation

[0067] In formula (10), (Ra(x, y) - Ga(x, y)) - (R(x, y) - G(x, y)) and (Ba(x, y) - Ga(x, y)) - (B(x, y) - G(x, y)) represent the amount of change in the color difference between the RGB images (target images) before and after the application of the blurring correction. Therefore, the color difference signals RG_adj(x, y) and BG_adj(x, y) after saturation correction are based on the amount of change in the color difference between the target images before and after the application of the blurring correction.

[0068] Note that in formula (10), the difference (the pixel-by-pixel difference between the color difference of the target image to which the blurring correction has not been applied and the color difference of the target image to which the blurring correction has been applied) is used as the amount of change in the color difference between the target images before and after the application of the blurring correction. However, as shown in the following formula (11), the saturation correction unit 1102 may use the ratio (the pixel-by-pixel ratio between the color difference of the target image to which the blurring correction has not been applied and the color difference of the target image to which the blurring correction has been applied) as the amount of change in the color difference between the target images before and after the application of the blurring correction.

[0069]

Number

[0070] Next, the saturation correction unit 1102 performs saturation correction using the saturation-corrected color difference calculated by Equation (10) or Equation (11) according to the following Equation (12). In Equation (12), Ro(x, y), Go(x, y), and Bo(x, y) represent the RGB signals of the target image to which saturation correction has been applied in addition to blurring correction.

[0071]

Number

[0072] As described above, the color difference signals RG_adj(x, y) and BG_adj(x, y) after saturation correction are based on the change amount of the color difference of the target image before and after the application of blurring correction. Therefore, the saturation correction according to Equation (12) is a process of correcting the saturation of the RGB image to which blurring correction has been applied based on the change amount of the color difference of the target image (RGB image) before and after the application of blurring correction.

[0073] According to Equation (12), without changing the G signal among the RGB signals (Ra(x, y), Ga(x, y), Ba(x, y)) to which blurring correction has been applied, the color difference can be corrected from (Ra(x, y) - Ga(x, y)), (Ba(x, y) - Ga(x, y)) to RG_adj(x, y), BG_adj(x, y). The G signal among the RGB signals is a signal in the medium wavelength range that most contributes to the luminance signal indicating brightness. Therefore, By correcting the color difference with the G signal as a fixed reference color signal, it is possible to correct the saturation of the RGB image to which the blurring correction has been applied without significantly changing the impression of brightness. Further, when the saturation correction is performed without changing the reference color signal, even if the intensity of the saturation correction changes, it is not necessary to change the color matrix correction according to the RGB spectral characteristics of the image sensor of the imaging unit 101, and the parameters of general signal processing such as color correction and gamma correction according to the saturation level of the image sensor are not changed significantly.

[0074] In the above description, the reference color signal (predetermined color signal) is assumed to be the G signal, but the R signal or the B signal may be used as the reference color signal. In this case, equations (10) to (12) are appropriately changed to calculate the color difference with respect to the reference color signal (R signal or B signal). Even when the reference color signal is the R signal or the B signal, the effect of being able to correct the saturation of the target image so as not to give the user a strong sense of discomfort can be obtained.

[0075] FIG. 13 is a conceptual diagram of correction processing including blurring correction and saturation correction according to the second embodiment. As shown in FIG. 13, based on the change amount 1301 of the color difference of the RGB image before and after the application of the blurring correction, the color difference 1302 after the saturation correction is generated. Then, by applying the color difference 1302 after the saturation correction to the G signal 1304 which is the reference color signal in the RGB image 1303 to which the blurring correction has been applied, an RGB image 1305 to which the correction processing including the blurring correction and the saturation correction has been applied is generated.

[0076] ● Summary of the Second Embodiment As described above, according to the second embodiment, the image processing unit 102 applies blurring correction to a target image including a plurality of color components (in the example described above, the red component, the green component, and the blue component). Then, the image processing unit 102 corrects the color difference of the target image to which the blurring correction has been applied without changing a predetermined color component (in the example described above, the green component) among the plurality of color components based on the change amount of the color difference of the target image before and after the application of the blurring correction, thereby correcting the saturation of the target image to which the blurring correction has been applied.

[0077] Thus, according to this embodiment, without changing a predetermined color component among a plurality of color components, the color difference of the target image to which the fog correction has been applied is corrected, so that it is possible to correct the saturation of the target image so as not to give much discomfort to the user. Therefore, according to this embodiment, it is possible to improve the image quality of the image to which the fog correction has been applied.

[0078] [Other Embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0079] [Summary] The above-described embodiments disclose the inventions shown in at least the following items, but are not limited to these inventions. [Item 1] An evaluation image generation means for generating an evaluation image for fog correction based on an image; A transmission map generation means for generating a transmission map for the fog correction based on the evaluation image; A first correction means for applying the fog correction based on the transmission map to the evaluation image; A second correction means for applying the fog correction based on the transmission map to the image; A third correction means for correcting the brightness of the image to which the fog correction has been applied based on a change amount of the evaluation image before and after application of the fog correction; An image processing apparatus comprising the same. [Item 2] The third correction means adds a per-pixel correction value based on the per-pixel difference between the evaluation image before application of the fog correction and the evaluation image after application of the fog correction to each pixel of the image to which the fog correction has been applied. The image processing apparatus according to item 1, characterized in that... [Item 3] For each pixel of the image to which the blurring correction has been applied, the third correction means multiplies a per-pixel gain value based on the per-pixel ratio between the evaluation image to which the blurring correction has not been applied and the evaluation image to which the blurring correction has been applied. The image processing apparatus according to item 1, characterized in that... [Item 4] The evaluation image generation means generates the evaluation image based on the image and a low-frequency image generated from the image. The image processing apparatus according to any one of items 1 to 3, characterized in that... [Item 5] An image processing apparatus according to any one of items 1 to 4, imaging means for generating the image, An imaging apparatus characterized by comprising the same. [Item 6] First correction means for applying blurring correction to an image including a plurality of color components, Second correction means for correcting the chromatic aberration of the image to which the blurring correction has been applied by correcting the chromatic aberration of the image to which the blurring correction has been applied without changing a predetermined color component among the plurality of color components based on the amount of change in chromatic aberration of the image before and after application of the blurring correction, thereby correcting the saturation of the image to which the blurring correction has been applied. An image processing apparatus characterized by comprising the same. [Item 7] The plurality of color components include a red component, a green component, and a blue component. The image processing apparatus according to item 6, characterized in that... [Item 8] The predetermined color component is the green component. The image processing apparatus according to item 7, characterized in that... [Item 9] The second correction means corrects the chromatic aberration of the image to which the blurring correction has been applied for each pixel based on the per-pixel difference between the chromatic aberration of the image to which the blurring correction has not been applied and the chromatic aberration of the image to which the blurring correction has been applied. The image processing apparatus according to any one of items 6 to 8, characterized in that... [Item 10] The second correction means corrects, for each pixel, the color difference of the image to which the blurring correction has been applied, based on the ratio of the color difference of the image before the blurring correction is applied to the color difference of the image after the blurring correction is applied. The image processing apparatus according to any one of items 6 to 8, characterized in that... [Item 11] The first correction means... generates an evaluation image for the blurring correction based on the image, generates a transmission map for the blurring correction based on the evaluation image, and applies the blurring correction to the image based on the transmission map. The image processing apparatus according to any one of items 6 to 10, characterized in that... [Item 12] An image processing apparatus according to any one of items 6 to 11, imaging means for generating the image, An imaging apparatus characterized by comprising the same. [Item 13] An image processing method executed by an image processing apparatus, an evaluation image generation step of generating an evaluation image for blurring correction based on the image, a transmission map generation step of generating a transmission map for the blurring correction based on the evaluation image, a first correction step of applying the blurring correction based on the transmission map to the evaluation image, a second correction step of applying the blurring correction based on the transmission map to the image, a third correction step of correcting the brightness of the image to which the blurring correction has been applied based on the amount of change in the evaluation image before and after the application of the blurring correction, An image processing method characterized by comprising the same. [Item 14] An image processing method executed by an image processing apparatus, A first correction step of applying blurring correction to an image including a plurality of color components; Based on the amount of change in the color difference of the image before and after application of the blurring correction, by correcting the color difference of the image to which the blurring correction has been applied without changing a predetermined color component among the plurality of color components, a second correction step of correcting the saturation of the image to which the blurring correction has been applied; An image processing method characterized by comprising the above. [Item 15] A program for causing a computer to function as each means of the image processing apparatus according to any one of Items 1 to 4. [Item 16] A program for causing a computer to function as each means of the image processing apparatus according to any one of Items 6 to 11.

[0080] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0081] 201... Image input unit, 202... Evaluation image generation unit, 203... Transmission map generation unit, 204... Correction processing unit, 205... Image output unit

Claims

1. Evaluation image generation means for generating an evaluation image for blurring correction based on an image; Transparency map generation means for generating a transparency map for the blurring correction based on the evaluation image; First correction means for applying the blurring correction based on the transparency map to the evaluation image; Second correction means for applying the blurring correction based on the transparency map to the image; Third correction means for correcting the brightness of the image to which the blurring correction has been applied based on the amount of change in the evaluation image before and after application of the blurring correction; An image processing apparatus comprising the same.

2. The third correction means adds, to each pixel of the image to which the blurring correction has been applied, a per-pixel correction value based on the per-pixel difference between the evaluation image before application of the blurring correction and the evaluation image after application of the blurring correction. The image processing apparatus according to claim 1, characterized by the above.

3. The third correction means multiplies, to each pixel of the image to which the blurring correction has been applied, a per-pixel gain value based on the per-pixel ratio between the evaluation image before application of the blurring correction and the evaluation image after application of the blurring correction. The image processing apparatus according to claim 1, characterized by the above.

4. The evaluation image generation means generates the evaluation image based on the image and a low-frequency image generated from the image. The image processing apparatus according to claim 1, characterized by the above.

5. An imaging apparatus comprising: the image processing apparatus according to any one of claims 1 to 4; and Imaging means for generating the image. characterized by the above.

6. First correction means for applying blurring correction to an image including a plurality of color components; Second correction means for correcting the chroma of the image to which the blurring correction has been applied by correcting the color difference of the image to which the blurring correction has been applied based on the amount of change in the color difference of the image before and after application of the blurring correction without changing a predetermined color component among the plurality of color components. An image processing apparatus comprising the same.

7. The plurality of color components include a red component, a green component, and a blue component. The image processing apparatus according to claim 6, characterized by the above.

8. The predetermined color component is the green component. The image processing apparatus according to claim 7, characterized by the above.

9. The second correction means corrects the color difference of each pixel of the image to which the blurring correction has been applied based on the per-pixel difference between the color difference of the image to which the blurring correction has not been applied and the color difference of the image to which the blurring correction has been applied. The image processing apparatus according to claim 6, characterized in that.

10. The second correction means corrects the color difference of each pixel of the image to which the blurring correction has been applied based on the per-pixel ratio between the color difference of the image to which the blurring correction has not been applied and the color difference of the image to which the blurring correction has been applied. The image processing apparatus according to claim 6, characterized in that.

11. The first correction means generates an evaluation image for the blurring correction based on the image, generates a transmission map for the blurring correction based on the evaluation image, applies the blurring correction to the image based on the transmission map. The image processing apparatus according to claim 6, characterized in that.

12. An image processing apparatus according to any one of claims 6 to 11, imaging means for generating the image, An imaging apparatus characterized by comprising.

13. An image processing method executed by an image processing apparatus, an evaluation image generation step of generating an evaluation image for blurring correction based on the image, a transmission map generation step of generating a transmission map for the blurring correction based on the evaluation image, a first correction step of applying the blurring correction based on the transmission map to the evaluation image, a second correction step of applying the blurring correction based on the transmission map to the image, a third correction step of correcting the brightness of the image to which the blurring correction has been applied based on the amount of change in the evaluation image before and after application of the blurring correction. An image processing method characterized by comprising.

14. An image processing method executed by an image processing apparatus, a first correction step of applying blurring correction to an image including a plurality of color components, a second correction step of correcting the chroma of the image to which the blurring correction has been applied by correcting the color difference of the image to which the blurring correction has been applied without changing a predetermined color component among the plurality of color components based on the amount of change in the color difference of the image before and after application of the blurring correction. An image processing method characterized by comprising.

15. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 4.

16. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 6 to 11.

Images