Image processing device, imaging device, image processing method, program, and storage medium

Abstract

To provide an image processing device, an imaging device, an image processing method, a program and a storage medium which suppress a phase shift and brightness reduction of a synthetic image synthesized from a plurality of images by adjusting at least one of brightness and a color of a saturation region in the plurality of images.SOLUTION: In an image processing device, a synthesizing unit 200 in an image processing unit being detection means for detecting saturation regions of a plurality of images comprises: a saturation corrected integrated image generation part 203 being correction means which corrects at least one of a color and brightness of the saturation region and generates a plurality of saturation corrected images, and first integration means which generates a first integrated image by using the plurality of saturation corrected images; second integration means which generates a second integrated image by using the plurality of images; and an integrated image synthesizing part 205 being synthesizing means which generates a synthetic image by synthesizing the first integrated image and the second integrated image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, and particularly relates to an image processing apparatus that performs image composition.

Background Art

[0002] Patent Document 1 discloses a technique that achieves suppression of overexposure and the like in the same manner as a physical ND filter by composing a plurality of images captured by divided exposure without mounting a physical ND filter.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] However, if pixel values in a certain region of an image captured by divided exposure are saturated, a phenomenon may occur in which the hue of a portion corresponding to the region in a composite image after averaging addition processing shifts and the brightness becomes darker.

[0005] In view of the above problem, an object of the present invention is to provide an image processing apparatus capable of suppressing hue shift and brightness reduction that occur in a composite image when composing a plurality of images captured by divided exposure.

Means for Solving the Problem

[0006] In order to solve the above problem, the present invention comprises: detection means for detecting saturated regions in a plurality of images; and the saturated regions The saturation will be reduced.The present invention provides an image processing apparatus comprising: a correction means for correcting and generating a plurality of saturation-corrected images; a first integration means for generating a first integrated image using the plurality of saturation-corrected images; a second integration means for generating a second integrated image using the plurality of images; and a synthesis means for combining the first integrated image and the second integrated image to generate a composite image. [Effects of the Invention]

[0007] According to the present invention, by adjusting at least one of the brightness and color of the saturated region in multiple images, phase shift and brightness reduction in a composite image synthesized from multiple images can be suppressed. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram illustrating the imaging device in an embodiment of the present invention. [Figure 2] This figure illustrates an example of the configuration of the synthesis section in an embodiment of the present invention. [Figure 3] This is a flowchart illustrating the processing flow of the synthesis section in an embodiment of the present invention. [Figure 4] This figure illustrates an example of imaging conditions in an embodiment of the present invention. [Figure 5] This figure illustrates an example of the configuration of the color correction unit in an embodiment of the present invention. [Figure 6] This is a flowchart illustrating the processing flow of the color correction unit in an embodiment of the present invention. [Figure 7] This figure illustrates an example of a low-saturation gain calculation curve in an embodiment of the present invention. [Figure 8] This figure illustrates an example of the configuration of the brightness correction unit in an embodiment of the present invention. [Figure 9] This is a flowchart illustrating the processing flow of the brightness correction unit in an embodiment of the present invention. [Figure 10]It is a diagram for describing an example of a brightness enhancement gain calculation curve in an embodiment of the present invention. [Figure 11] It is a diagram for describing a change in pixel values before and after image processing in an embodiment of the present invention. [Figure 12] It is a diagram for describing a configuration example of an integrated image synthesis unit in an embodiment of the present invention. [Figure 13] It is a flowchart for describing a processing flow of an integrated image synthesis unit in an embodiment of the present invention. [Figure 14] It is a diagram for describing an example of a composition ratio calculation curve in an embodiment of the present invention. Mode for Carrying Out the Invention

[0009] [First Embodiment] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention according to the claims, and not all combinations of features described in the present embodiment are necessarily essential to the solution of the present invention.

[0010] In the present embodiment, a composite image in which blown-out highlights are suppressed is generated by combining a plurality of divided exposure images captured without mounting a physical ND filter.

[0011] FIG. 1 is a block diagram for describing an image pickup apparatus according to the present embodiment. Hereinafter, a configuration example of the present embodiment will be described with reference to FIG. 1.

[0012] The control unit 101 is, for example, a CPU, reads a control program for each block included in the image pickup apparatus 100 from a ROM 102 described later, develops the program in a RAM 103 described later, and executes the program.

[0013] The ROM 102 is an electrically erasable and recordable non-volatile memory, and stores parameters necessary for the operation of each block in addition to an operation program for each block included in the image pickup apparatus 100.

[0014] The RAM 103 is a rewritable volatile memory, and is used for expanding programs executed by the control unit 101 and the like, and for temporary storage of data generated through operations of each block included in the image capturing apparatus 100, and the like.

[0015] The optical system 104 includes a lens group including a zoom lens and a focus lens, and an aperture mechanism, and forms an image of a subject on the image capturing surface of an image capturing unit 105 described later.

[0016] The image capturing unit 105 is, for example, an image sensor such as a CCD or a CMOS sensor. The image capturing unit 105 photoelectrically converts an optical image formed on the image capturing surface thereof by the optical system 104, and outputs the obtained analog image signal to an A / D conversion unit 106.

[0017] The A / D conversion unit 106 converts an input analog image signal into digital image data and outputs the converted digital image data. The digital image data output from the A / D conversion unit 106 is temporarily stored in the RAM 103.

[0018] An image processing unit 107 performs various types of image processing on image data stored in the RAM 103. Specifically, for example, the image processing unit 107 performs various image processing for developing, displaying and recording digital image data, such as demosaicing processing, noise reduction processing, white balance correction processing, and gamma processing. Furthermore, the image processing unit 107 includes a combining unit 200, and performs combining processing that is a feature of the present embodiment. Details of the combining processing will be described later.

[0019] A recording unit 108 records data including image data on an internal recording medium.

[0020] A display unit 109 includes a display device such as an LCD, and displays images stored in the RAM 103 and images recorded in the recording unit 108 on the display device. The display unit 109 also displays an operation user interface for receiving instructions from a user, and the like.

[0021] Next, an example of the configuration of the synthesis unit 200, which is included in the image processing unit 107 that is a feature of this embodiment, will be described with reference to Figure 2.

[0022] Figure 2 is a diagram illustrating an example of the configuration of the synthesis unit 200 in this embodiment. The synthesis unit 200 consists of a color correction unit 201, a brightness correction unit 202, a saturation correction integrated image generation unit 203, a simple integrated image generation unit 204, an integrated image synthesis unit 205, and a gain multiplication unit 206.

[0023] Next, the processing flow by which the synthesis unit 200 synthesizes multiple segmented exposure images will be explained in detail with reference to the flowchart in Figure 3.

[0024] Figure 3 is a flowchart illustrating the processing flow of the multi-image compositing unit in this embodiment. In step S301, the control unit 101 sets the imaging conditions and causes the imaging unit 105 to take images under the set imaging conditions.

[0025] Here, the setting of imaging conditions by the control unit 101 will be explained in detail with reference to Figure 4. Figure 4 is a diagram illustrating an example of imaging conditions in this embodiment. Figure 4 schematically shows an example of imaging processing when the user sets the exposure conditions to a shutter speed of 4 seconds, an aperture of F4.0, an ISO sensitivity of ISO100, and an ND density of ND4. Here, ND density indicates the degree to which the subject light is reduced. For example, ND2 means a reduction of 1 stop, ND4 means a reduction of 2 stops, and ND8 means a reduction of 3 stops. In this invention, imaging processing is performed without attaching a physical ND filter, so the exposure due to the reduction in light is reduced by shortening the shutter speed. Therefore, the control unit 101 sets the aperture and ISO sensitivity to the aperture and ISO sensitivity set by the user as imaging conditions, and shortens the shutter speed set by the amount of light reduction due to the ND density. Specifically, as shown in Figure 4, when the shutter speed is 4 seconds and the ND density is ND4, a reduction of 2 stops is required, so the shutter speed is shortened by 2 stops to 1 second and set as the imaging condition for one split exposure image. Furthermore, the control unit 101 sets the number of images to be captured such that the combined exposure time of the multiple segmented exposure images is equal to the shutter speed set by the user. In the example in Figure 4, the exposure time of one segmented exposure image is 1 second, and the shutter speed set by the user is 4 seconds, so the control unit 101 sets the number of images to 4.

[0026] In this invention, the exposure of all multiple segmented exposure images is assumed to be the same.

[0027] Furthermore, by minimizing the unexposed time (the time between the end of exposure for one segmented exposure image and the start of exposure for the next segmented exposure image), the blur of moving objects can be smoothed out.

[0028] When a shutter button (not shown) is pressed by the user, the control unit 101 begins processing for imaging. Based on the imaging conditions set in step S301, the control unit 101 controls the optical system 104 and the imaging unit 105, and repeatedly captures the subject light for the set number of images, generating segmented exposure image data. In this invention, one example of the format of the segmented exposure image processed by the synthesis unit 200 is RGB (BAYER-RAW) in the BAYER array. In this case, the segmented exposure image consists of R pixels, G pixels, and B pixels with linear signal characteristics. However, the format of the segmented exposure image is not limited to this. For example, image formats such as RGB444, YUV444, YUV422, and ICtCp may also be used.

[0029] In step S302, the image processing unit 107 detects the saturation region of the captured image. For example, the image processing unit 107 detects the region in the captured image where the pixel value is greater than or equal to a predetermined value as the saturation region.

[0030] In step S303, the color correction unit 201 corrects the multiple segmented exposure images captured in step S301 by reducing the saturation of the saturated colors in order to reduce hue shift. Details of the color correction unit 201 will be described later. Here, hue shift refers to a state in which the original hue of the subject and the hue in the image data are different.

[0031] In step S304, the brightness correction unit 202 corrects the brightness of the saturated region in the color-corrected image data from step S303 to reduce the decrease in brightness. Details of the brightness correction unit 202 will be described later.

[0032] In step S305, the saturation-corrected integrated image generation unit 203 sequentially and cumulatively adds the brightness-corrected image data (referred to as saturation-corrected images) from step S304 to generate a saturation-corrected integrated image.

[0033] In step S306, the simple integrated image generation unit 203 sequentially and cumulatively adds the segmented exposure images captured in step S301 to generate a simple integrated image.

[0034] In step S307, the processes in steps S303 to S306 are repeated until the integration process in steps S305 and S306 is performed for the number of images.

[0035] In step S308, the integrated image synthesis unit 205 synthesizes the saturation-corrected integrated image and the simple integrated image. Details of the integrated image synthesis unit 205 will be described later.

[0036] In step S309, the integrated image data synthesized in step S308 is multiplied by a gain of "1 per image" calculated by the control unit 101. In other words, the integrated image data, which is obtained by integrating and synthesizing the segmented exposure images for the number of images captured, is multiplied by a gain of 1 per image captured to perform an averaging process.

[0037] As described above, by combining multiple segmented exposure images using the compositing unit 200, it is possible to reduce the exposure by the set ND density by combining multiple segmented exposure images taken with shorter shutter speeds, without having to attach a physical ND filter. In the example in Figure 4, by combining four segmented exposure images taken with exposure conditions of 1 second, F4.0, ISO100, which is two stops lower in exposure equivalent to ND4, from exposure conditions of 4 seconds, F4.0, ISO100, using the compositing unit 200, a composite image with suppressed overexposure can be generated.

[0038] Next, the details of the color correction unit 201 provided in the synthesis unit 200 will be described with reference to Figures 5, 6, 7, and 11.

[0039] Figure 5 is a diagram illustrating an example of the configuration of the color correction unit 201 in this embodiment. The color correction unit 201 consists of a debayering processing unit 501, a desaturation gain calculation unit 502, a WB coefficient multiplication unit 503, a desaturation processing unit 504, and an inverse WB coefficient multiplication unit 505.

[0040] Next, the processing flow by which the color correction unit 201 reduces the saturation of the saturated region of the segmented exposure image will be explained in detail with reference to the flowchart in Figure 6.

[0041] Figure 6 is a flowchart illustrating the processing flow of the color correction unit in this embodiment.

[0042] In step S601, the debayering processing unit 501 performs debayering on the BAYER-RAW format split exposure image and outputs image data in RGB444 format. The debayering process of the present invention uses a known method of interpolation processing using a smoothing filter, and a detailed explanation is omitted.

[0043] In step S602, the desaturation gain calculation unit 502 calculates the desaturation gain based on the maximum pixel value among the R pixel value, G pixel value, and B pixel value at the corresponding position in the RGB444 format image data. The method for calculating the desaturation gain will be explained in detail with reference to Figure 7. Figure 7 is a diagram illustrating an example of the desaturation gain calculation curve in this embodiment. As shown in Figure 7, the desaturation gain calculation unit 502 calculates a desaturation gain such that it increases as the maximum pixel value approaches the saturation level when it exceeds a certain value.

[0044] In step S603, the WB coefficient multiplication unit 503 multiplies the segmented exposure image by a white balance coefficient. Figure 11 is a diagram illustrating the change in pixel values ​​before and after image processing in this embodiment. Figure 11(a) shows a segmented exposure image, and Figure 11(b) shows image data obtained by multiplying the segmented exposure image by a white balance coefficient. Figure 11(b) also shows an example where the white balance coefficient is 1.8 times for the R pixel and 1.5 times for the B pixel. The WB coefficient multiplication unit 503 multiplies the segmented exposure image as shown in Figure 11(a) by a white balance coefficient for each color and outputs a WB coefficient multiplied image as shown in Figure 11(b).

[0045] In step S604, the desaturation processing unit 504 performs desaturation processing on the image data to which the white balance coefficient was multiplied in step S603, based on the desaturation gain calculated in step S602. Specifically, the desaturation processing unit 504 performs desaturation processing based on the following equations (Equation 1) and (Equation 2). SAT_Rwb=(Rwb-G)×(1.0-SAT_GAIN)+G (Formula 1) SAT_Bwb=(Bwb-G)×(1.0-SAT_GAIN)+G (Formula 2) SAT_Rwb represents the R pixel value after desaturation processing, and SAT_Bwb represents the B pixel value after desaturation processing. Rwb represents the R pixel value of the WB coefficient multiplied image, G represents the G pixel value of the WB coefficient multiplied image, and Bwb represents the B pixel value of the WB coefficient multiplied image. SAT_GAIN represents the desaturation gain. In this embodiment, the desaturation processing unit 504 does not process the G pixel value and outputs it as is.

[0046] Figure 11(c) shows the pixel values ​​after desaturation processing of the WB coefficient multiplied image.

[0047] According to (Equation 1) and (Equation 2), when SAT_GAIN is at its maximum of 1.0, SAT_Rwb and SAT_Bwb and G have the same pixel value as shown in Figure 11(c), and the area where the desaturation process has been performed becomes achromatic. In other words, (Equation 1) and (Equation 2) control how close the R pixel value and B pixel value are to the G pixel value by using the desaturation gain, thereby performing a desaturation process.

[0048] In step S605, the inverse white balance coefficient multiplication unit 505 multiplies the white-balanced, desaturated image by an inverse white balance coefficient. Figure 11(d) shows image data obtained by multiplying the white-balanced, desaturated image by an inverse white balance coefficient. Figure 11(d) also shows an example where the inverse white balance coefficient is 1 / 1.8 for the R pixel and 1 / 1.5 for the B pixel. The inverse white balance coefficient multiplication unit 505 multiplies the white-balanced, desaturated image in Figure 11(c) by an inverse white balance coefficient for each color, generating an inverse white balance coefficient multiplied image like the one in Figure 11(d), and outputs it as a desaturated image without white balance processing.

[0049] As described above, by reducing the color saturation of areas with pixel values ​​close to the saturation level in the segmented exposure image, it is possible to reduce the hue shift that occurs in the composite image of multiple images.

[0050] In this invention, we have described an example of inputting and outputting image data in BAYER-RAW format that has not been multiplied by a white balance coefficient. However, the input and output image data may also be data that has been multiplied by a white balance coefficient. In such cases, the WB coefficient multiplication process and the inverse WB coefficient multiplication process performed in steps S603 and S604 of the flowchart shown in Figure 6 do not need to be performed.

[0051] Next, the details of the brightness correction unit 202 provided in the synthesis unit 200 will be described with reference to Figures 8, 9, 10, and 11.

[0052] Figure 8 is a diagram illustrating an example of the configuration of the brightness correction unit 202 in this embodiment. The brightness correction unit 202 consists of a debayer processing unit 801, a brightness enhancement gain calculation unit 802, and a brightness enhancement processing unit 803.

[0053] Next, the processing flow in which the brightness correction unit 202 brightens the low-saturation image area corresponding to the saturation area of ​​the segmented exposure image will be explained in detail with reference to the flowchart in Figure 9.

[0054] Figure 9 is a flowchart illustrating the processing flow of the brightness correction unit 202 in this embodiment. In step S901, the debayer processing unit 801 performs debayer processing on the segmented exposure image in BAYER-RAW format and outputs image data in RGB444 format. The debayer processing in this embodiment uses a known method of interpolation processing with a smoothing filter, and a detailed explanation is omitted.

[0055] In step S902, the brightness enhancement gain calculation unit 802 calculates the brightness enhancement gain based on the maximum pixel value among the R pixel value, G pixel value, and B pixel value at the corresponding position in the RGB444 format image data. The method for calculating the brightness enhancement gain will be explained in detail with reference to Figure 10. Figure 10 is a diagram illustrating an example of the brightness enhancement gain calculation curve in this embodiment. As shown in Figure 10, the brightness enhancement gain calculation unit 802 calculates a brightness enhancement gain that increases as the maximum pixel value approaches the saturation level. Note that the brightness enhancement gain is 1.0 (1x) when the maximum pixel value is small.

[0056] In step S903 of Figure 9, the brightness enhancement processing unit 803 multiplies the desaturated image by the brightness enhancement gain calculated in step S902. Specifically, it multiplies the image by the brightness enhancement gain based on (Equation 3), (Equation 4), and (Equation 5). BRT_R=SAT_R×BRT_GAIN (Formula 3) BRT_G=SAT_G×BRT_GAIN (Formula 4) BRT_B=SAT_B×BRT_GAIN (Formula 5) BRT_R represents the R pixel value multiplied by the brightness enhancement gain, BRT_G represents the G pixel value multiplied by the brightness enhancement gain, and BRT_B represents the B pixel value multiplied by the brightness enhancement gain. SAT_R represents the R pixel value of the desaturated image, SAT_G represents the G pixel value of the desaturated image, and SAT_B represents the B pixel value of the desaturated image. BRT_GAIN represents the brightness enhancement gain.

[0057] Figure 11(e) shows the pixel values ​​of a high-luminance image obtained by multiplying the inverse WB coefficient multiplied image data (low-saturation image) of Figure 11(d) by a 2x high-luminance gain.

[0058] The brightness enhancement processing unit 803 brightens the low-saturation image by multiplying it by a brightness enhancement gain, as shown in (Equations 3), (4), and (5), and outputs a brightness enhancement image.

[0059] As described above, by brightening the brightness of the low-saturation image region corresponding to the region with pixel values ​​close to the saturation level in the segmented exposure image, it is possible to reduce the brightness reduction that occurs in the composite image of multiple images.

[0060] Next, the details of the integrated image synthesis unit 205 provided in the synthesis unit 200 will be described with reference to Figures 12, 13, and 14.

[0061] Figure 12 is a diagram illustrating an example of the configuration of the integrated image synthesis unit 205 in this embodiment. The integrated image synthesis unit 205 consists of a debayer processing unit 1201, a synthesis ratio calculation unit 1202, and a synthesis processing unit 1203.

[0062] Next, the processing flow by which the integrated image synthesis unit 205 synthesizes the saturation-corrected integrated image and the simple integrated image will be explained in detail with reference to the flowchart in Figure 13.

[0063] Figure 13 is a flowchart illustrating the processing flow of the integrated image synthesis unit 205 in this embodiment.

[0064] In step S1301, the debayering processing unit 1201 performs debayering on the simple integrated image in BAYER-RAW format and outputs image data in RGB444 format. The debayering in this embodiment uses a known method that performs interpolation using a smoothing filter, and a detailed explanation is omitted.

[0065] In step S1302, the composite ratio calculation unit 1202 calculates the composite ratio based on the maximum pixel value among the R pixel value, G pixel value, and B pixel value at the corresponding position in the RGB444 format image data. The method for calculating the composite ratio will be explained in detail with reference to Figure 14. Figure 14 is a diagram illustrating an example of the composite ratio calculation curve in this embodiment. As shown in Figure 14, when the maximum pixel value exceeds a certain value, the composite ratio calculation unit 1202 calculates the composite ratio such that the composite ratio of the simple integrated image increases as it approaches the saturation level of the integrated image. Here, the saturation level of the integrated image is the value obtained by multiplying the saturation level of the segmented exposure image by the number of integrated images.

[0066] Here, we will explain why the composite ratio calculation unit 1202 calculates the composite ratio such that the composite ratio of the simple integrated image increases as the maximum pixel value approaches the saturation level of the integrated image. In the segmented exposure image, the portion of the saturation-corrected integrated image corresponding to the area where the saturated subject is stationary has reduced color saturation and increased brightness. On the other hand, the portion of the simple integrated image corresponding to the area where the saturated subject is stationary in the segmented exposure image has image quality that is almost equivalent to that of an image with a physical ND filter attached. Therefore, the composite ratio calculation unit 1202 increases the composite ratio of the simple integrated image for the area where the saturated subject is stationary in the segmented exposure image.

[0067] In the segmented exposure images, the pixel values ​​in all the parts of the segmented exposure images corresponding to areas where the saturated subject is stationary are close to the saturation level. Therefore, the pixel values ​​in the corresponding areas of the simple integrated image are close to the saturation level of the integrated image. On the other hand, in areas where the saturated subject is moving in the segmented exposure images, the movement results in a mixture of images where the pixel values ​​in the segmented exposure images are close to the saturation level and images where they are not. Therefore, the pixel values ​​in the areas of the simple integrated image corresponding to areas where the saturated subject is moving in the segmented exposure images are dark values ​​that are far from the saturation level of the integrated image.

[0068] Therefore, by calculating a larger composite ratio for the simple integrated image as the maximum pixel value of the simple integrated image approaches the saturation level of the integrated image, the composite ratio of the simple integrated image becomes larger in areas where the saturated subject is stationary in the segmented exposure image.

[0069] In step S1303, the synthesis processing unit 1203 synthesizes the saturation-corrected integrated image and the simple integrated image based on the synthesis ratio calculated in step S1302, and outputs it as an integrated composite image.

[0070] As described above, by combining a saturation-corrected integrated image and a simple integrated image, it is possible to combine a region of the saturation-corrected integrated image corresponding to a region of the simple integrated image with pixel values ​​close to the saturation level with a higher proportion of the simple integrated image.

[0071] In this embodiment, an example was described in which the synthesis unit 200 processes both color correction and brightness correction for the saturated region of the segmented exposure image. However, it is also possible to process only color correction or only brightness correction.

[0072] In this embodiment, an example was described in which the integrated image synthesis unit 205 calculates the synthesis ratio based on a simple integrated image, but the synthesis ratio may also be calculated based on a saturation-corrected integrated image. When calculating the synthesis ratio based on a saturation-corrected integrated image, the pixel values ​​of the saturation-corrected integrated image change due to color correction and brightness correction, so it is necessary to change the saturation level of the integrated image to match the amount of change in the pixel values ​​of the saturation-corrected integrated image.

[0073] Furthermore, if only color correction is performed on the saturated region of the segmented exposure image, and no brightness correction is performed, the G pixel values ​​of the simple integrated image and the saturation-corrected integrated image will be the same. Therefore, the composite ratio can be calculated based on the G pixel values ​​of the saturation-corrected integrated image.

[0074] In this embodiment, the gain multiplication unit 206 has been described as performing gain multiplication on a composite integrated image, but the image to be multiplied by gain is not limited to this. For example, it may also be a process of combining images in which a saturation-corrected integrated image and a simple integrated image have each been multiplied by gain.

[0075] (Other embodiments) Although the above embodiment was described using a personal digital camera as an example, it can also be applied to portable devices, smartphones, or network cameras connected to a server, as long as they are equipped with a synthesis function.

[0076] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and operate the program. It can also be realized by a circuit (for example, an ASIC) that implements one or more functions. [Explanation of symbols]

[0077] 100 Imaging device 101 Control Unit 102 ROM 103 RAM 104 Optical system 105 Imaging Unit 106 A / D Conversion Unit 107 Image Processing Unit 108 Records Section 109 Display section

Claims

1. A detection means for detecting saturation regions in multiple images, Correction means that corrects the saturation of the saturation region to be lower and generates multiple saturation-corrected images, A first integration means that generates a first integrated image using the plurality of saturation-corrected images, A second integration means for generating a second integrated image using the aforementioned plurality of images, An image processing apparatus characterized by comprising: a synthesis means for combining the first integrated image and the second integrated image to generate a composite image.

2. The image processing apparatus according to claim 1, characterized in that the combining means calculates a combining ratio based on at least one of the pixel values ​​of the first integrated image and the second integrated image, combines the first integrated image and the second integrated image based on the combining ratio, and generates a combined image.

3. The image processing apparatus according to claim 2, characterized in that the synthesis means calculates the synthesis ratio for generating the composite image based on the pixel values ​​of the second integrated image.

4. The image processing apparatus according to claim 2 or 3, characterized in that the synthesis means calculates the synthesis ratio for generating the composite image based on the pixel values ​​of the first integrated image.

5. The image processing apparatus according to any one of claims 1 to 4, characterized in that the first integration means generates the first integrated image by sequentially adding to the plurality of saturation-corrected images.

6. The image processing apparatus according to any one of claims 1 to 5, characterized in that the second integration means generates the second integrated image by sequentially adding to the plurality of images.

7. Furthermore, the image processing apparatus according to any one of claims 1 to 6 is characterized by having a gain multiplication means for performing gain multiplication on the composite image.

8. Furthermore, it has gain multiplication means for performing gain multiplication on the first integrated image and the second integrated image, The image processing apparatus according to claim 7, wherein the combining means combines the first integrated image and the second integrated image after the gain multiplication means has performed the gain multiplication, and generates the combined image.

9. The image processing apparatus according to claim 1, characterized in that the correction by the correction means is a process of multiplying the pixel value of the early saturation region by a predetermined coefficient.

10. The image processing apparatus according to claim 9, characterized in that the correction means sets a second value as a predetermined coefficient to approximate achromatic when the pixel value of the saturated region is greater than a first value, and sets a fourth value as a predetermined coefficient that is greater than the second value when the pixel value of the saturated region is greater than the first value.

11. The image processing apparatus according to any one of claims 1 to 10, characterized in that the correction means corrects the brightness of the saturation region to make it brighter and generates the plurality of saturation-corrected images.

12. The correction means sets a sixth value as a predetermined coefficient for increasing the brightness of the saturation region if the pixel value of the saturation region is greater than the fifth value, and sets an eighth value greater than the sixth value as the predetermined coefficient if the pixel value of the saturation region is greater than the fifth value and is a seventh value. The image processing apparatus according to claim 11, characterized in that the pixel values ​​of the saturation region are multiplied by the predetermined coefficient to generate the plurality of saturation-corrected images.

13. The image processing apparatus according to any one of claims 1 to 12, characterized in that the detection means detects the saturation region based on the pixel values ​​of the plurality of images.

14. The image processing apparatus according to claim 13, characterized in that the detection means detects an area where the pixel value is greater than or equal to a predetermined value as the saturation area.

15. The image processing apparatus according to any one of claims 1 to 14, characterized in that the plurality of images are composed of R pixels, G pixels, and B pixels having linear signal characteristics.

16. The image processing apparatus according to any one of claims 1 to 15, characterized in that the exposure of the plurality of images is the same.

17. An imaging means for capturing multiple images, A detection means for detecting the saturated region of the plurality of images, Correction means that corrects the saturation of the saturation region to be lower and generates multiple saturation-corrected images, A first integration means that generates a first integrated image using the plurality of saturation-corrected images, A second integration means for generating a second integrated image using the aforementioned plurality of images, An imaging apparatus characterized by having a combining means for combining the first integrated image and the second integrated image to generate a composite image.

18. The imaging apparatus according to claim 17, characterized in that the imaging means captures the plurality of images without attaching an ND filter.

19. A detection step for detecting saturation regions in multiple images, A correction step that corrects the saturation of the saturated region to reduce its saturation and generates multiple saturation-corrected images, A first integration step of generating a first integrated image using the plurality of saturation-corrected images, A second integration step in which a second integrated image is generated using the aforementioned plurality of images, An image processing method characterized by comprising a synthesis step of combining the first integrated image and the second integrated image to generate a composite image.

20. A program that allows a computer to operate an image processing device. A detection step for detecting saturation regions in multiple images, A correction step that corrects the saturation of the saturated region to reduce its saturation and generates multiple saturation-corrected images, A first integration step of generating a first integrated image using the plurality of saturation-corrected images, A second integration step in which a second integrated image is generated using the aforementioned plurality of images, A program characterized by performing a synthesis step of combining the first integrated image and the second integrated image to generate a composite image.

21. A storage medium that is readable by a computer storing the program described in claim 20.

Images