Imaging device, imaging method, program, and recording medium

Abstract

To solve a problem in which coloring and saturation may not occur when the exposure is changed and the exposure is corrected in omnifocal imaging.SOLUTION: An imaging device according to the present invention includes determination means for determining a predetermined value of exposure in advance, imaging means for imaging a plurality of images with different focus positions, control means for controlling such that when the imaging means captures the plurality of images, the images are captured with an exposure lower than the predetermined value, and correction means for correcting the brightness of at least a part of the plurality of images on the basis of the predetermined value determined by the determination means.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an imaging device capable of performing all-focus imaging for imaging an image at a plurality of focus positions.

Background Art

[0002] Conventionally, an imaging device that performs all-focus imaging has been known. All-focus imaging is to image a plurality of images with the focus position changed over the entire range of subject distances that can be focused by the imaging device (Patent Document 1). A technique of generating an image with a deeper depth of field than the original image, which is in focus over the entire imaging area by extracting only the in-focus areas from the plurality of images obtained by all-focus imaging and synthesizing them into one image, is called depth synthesis.

[0003] Also, in all-focus imaging, it is known that the brightness changes between the acquired images due to various factors. For example, Patent Document 2 discloses that the brightness of an image changes due to a change in the effective F value in all-focus imaging. FIG. 7 is a diagram for explaining an example of the relationship between the focus position and the change in the effective F value in all-focus imaging. Also, FIG. 8 is a diagram for explaining an example of the relationship between the focus position and the change in the luminance value in all-focus imaging. As shown in FIGS. 7 and 8, there may be a difference in brightness between images in all-focus imaging.

[0004] In addition, it is known that brightness changes also occur in all-focus imaging using a strobe or all-focus imaging using a high-speed shutter.

[0005] When performing synthesis on a plurality of images, if there is a change in brightness between the plurality of images, unevenness in brightness may occur in the synthesized image. In order to avoid unevenness in brightness of the synthesized image, it is desirable to perform correction so as to equalize the brightness after each image used for synthesis is acquired.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Japanese Patent Application Publication No. 10-290389 [Patent Document 2] Japanese Patent Publication No. 2020-107956 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, when adjusting brightness, if the scene is being darkened, problems can occur where the high-luminance parts of the image become color-enhanced, or where saturation is lost.

[0008] The above issues will be explained using a diagram. Figure 12 shows the gamma curve when converting RAW to YUV. When the RAW brightness value exceeds 1201, as shown on the horizontal axis, the YUV brightness value becomes the upper limit of 1202. For example, when correcting to darken the brightness, if the maximum RAW value drops to a brightness value of 1203, the upper limit of the YUV brightness becomes a brightness value of 1204, and saturation will not occur. Also, if the brightness is reduced in the part of the RAW brightness value that is close to saturation, the ratio of red or blue pixels to green pixels in the RAW changes, causing the high-brightness areas to become colored.

[0009] As mentioned above, in scenes where the brightness is corrected to be darker, even if the brightness is corrected to be uniform, there is a problem in that color fringing and brightness differences appear in the full-focus image. [Means for solving the problem]

[0010] To solve the above problems, the present invention provides an imaging device comprising: a determination means for determining a predetermined value of exposure in advance; an imaging means for capturing a plurality of images with different focus positions; a control means for controlling the imaging means to capture the plurality of images at an exposure lower than the predetermined value; and a correction means for correcting the brightness of at least some of the plurality of images based on the predetermined value determined by the determination means. [Effects of the Invention]

[0011] According to the configuration of the present invention, it is possible to generate a good composite image with reduced differences in color and brightness. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the structure of a digital camera according to an embodiment of the present invention. [Figure 2] This is a flowchart illustrating the generation of a composite image in the first embodiment of the present invention. [Figure 3] This is a flowchart illustrating the imaging process in the first embodiment of the present invention. [Figure 4] This is a flowchart illustrating the brightness correction in the first embodiment of the present invention. [Figure 5] This is a flowchart illustrating the alignment process in the first embodiment of the present invention. [Figure 6] This is a flowchart illustrating the image synthesis in the first embodiment of the present invention. [Figure 7] This diagram illustrates the change in brightness when the aperture is driven, as the brightness value changes with a change in focus position. [Figure 8] This figure illustrates the change in brightness when the aperture is driven in the first embodiment of the present invention. [Figure 9] This is a flowchart illustrating the synthesis of composite images in a second embodiment of the present invention. [Figure 10] This figure illustrates an example of the relationship between the focus position and the change in effective F-number during full-focus imaging. [Figure 11] This figure illustrates an example of the relationship between focus position and brightness value changes in full-focus imaging. [Figure 12] This diagram shows the gamma curve when converting RAW to YUV. [Figure 13]This is a diagram for explaining that the luminance changes due to light emission unevenness or exposure unevenness when imaging a plurality of images in the present invention.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0014] (First Embodiment) FIG. 1 is an example of a block diagram showing the structure of a digital camera as an image processing apparatus according to this embodiment. The digital camera 100 can capture a plurality of still images while changing the focus position.

[0015] The control unit 101 is a signal processor such as a CPU or MPU, and controls each part of the digital camera 100 while reading out a program stored in a ROM 105 described later in advance. For example, as will be described later, the control unit 101 issues commands regarding the start and end of imaging to the imaging unit 104 described later. Or, it issues an image processing command to the image processing unit 107 described later based on a program stored in the ROM 105. Commands from the user are input to the digital camera 100 by an operation unit 110 described later and reach each part of the digital camera 100 through the control unit 101.

[0016] The drive unit 102 is composed of a motor or the like, and mechanically operates the optical system 103 described later under the command of the control unit 101. For example, based on the command of the control unit 101, the drive unit 102 moves the position of the focus lens included in the optical system 103 and adjusts the focal length of the optical system 103.

[0017] The optical system 103 is composed of a zoom lens, a focus lens, an aperture, and the like. The aperture is a mechanism for adjusting the amount of light transmitted. By changing the position of the lens, the focus position can be changed.

[0018] The imaging unit 104 is a photoelectric conversion element that performs photoelectric conversion, converting incident light signals into electrical signals. For example, a CCD sensor or a CMOS sensor can be applied to the imaging unit 104. The imaging unit 104 is equipped with a video imaging mode, which allows it to capture multiple temporally consecutive images as individual frames of a video.

[0019] ROM105 is a read-only non-volatile memory used as a recording medium, storing the operating programs for each block of the digital camera 100, as well as parameters necessary for the operation of each block. RAM106 is a rewritable volatile memory used as a temporary storage area for data output during the operation of each block of the digital camera 100.

[0020] The image processing unit 107 performs various image processing operations, such as white balance adjustment, color interpolation, and filtering, on the image output from the imaging unit 104 or on the image signal data recorded in the built-in memory 109 (described later). It also compresses the image signal data captured by the imaging unit 104 using a standard such as JPEG.

[0021] The image processing unit 107 is composed of an integrated circuit (ASIC) that combines circuits for specific processing. Alternatively, the control unit 101 may perform some or all of the functions of the image processing unit 107 by processing according to a program read from the ROM 105. If the control unit 101 performs all of the functions of the image processing unit 107, there is no need to have the image processing unit 107 as hardware.

[0022] The display unit 108 is a liquid crystal display or an organic EL display for displaying images temporarily stored in RAM 106, images stored in the built-in memory 109 (described later), or the settings screen of the digital camera 100.

[0023] The internal memory 109 is a location for recording images captured by the imaging unit 104, images processed by the image processing unit 107, and information about the focus position at the time of image capture. A memory card or the like may be used instead of the internal memory.

[0024] The operation unit 110 may include, for example, buttons, switches, keys, or mode dials attached to the digital camera 100, or a touch panel that also functions as a display unit 108. User commands reach the control unit 101 via the operation unit 110.

[0025] Figure 2 is a flowchart illustrating the generation of a composite image in this embodiment.

[0026] In step S201, the imaging unit 104 captures multiple images with different focus positions. In this embodiment, the imaging unit 104 performs imaging while changing the focus position from near to infinity.

[0027] In step S202, the control unit 101 and the image processing unit 107 correct the brightness of the multiple images captured by the imaging unit 104 in step S201.

[0028] In step S203, the control unit 101 and the image processing unit 107 perform alignment on the multiple images that have been brightness corrected in step S202.

[0029] In step S204, the control unit 101 and the image processing unit 107 perform synthesis on the image after alignment in step S203 to generate a composite image. Each step will be described in detail below.

[0030] Figure 3 is a flowchart illustrating the imaging in step S201 in this embodiment.

[0031] In step S301, the control unit 101 sets the focus position. For example, the user specifies the focus position through the touch panel, which is also used by the display unit 108, and specifies multiple focus positions at equal intervals before and after the focus position corresponding to the specified focus position in the optical axis direction. At the same time, the control unit 101 determines the imaging order in order of distance at the set focus positions.

[0032] In step S302, the control unit 101 determines the exposure conditions for imaging. The control unit 101 determines the shutter speed, F-number, and ISO sensitivity. The exposure conditions may be set manually by the user, or the control unit 101 may set them automatically by having the imaging unit 104 measure the light from the subject.

[0033] In step S303, the control unit 101 calculates the amount of exposure reduction (amount of exposure reduction) that will result in a darker exposure than the exposure condition (predetermined exposure value) defined in step S301. The amount of exposure reduction may be determined, for example, by the performance of the lens mounted in the optical system 103. Since the extent to which the effective F-number changes with the amount of focus position variation depends on the lens performance, for example, the amount of change in the effective F-number when the focus position is moved from the closest to infinity may be treated as the amount of exposure reduction. Furthermore, when using a strobe, it is recommended to determine the amount of exposure reduction considering the performance of the strobe, including uneven flash emission, and when using a high-speed shutter, it is recommended to determine the amount of exposure reduction considering the shutter performance, including uneven exposure. In addition, the amount of exposure reduction may be determined using at least one of the shutter speed, F-number, or ISO sensitivity.

[0034] In step S304, the control unit 101 moves the focus to an uncaptured focus position among the focus positions set in step S301, according to the imaging order determined in step S301.

[0035] In step S305, the imaging unit 104 takes an image with the exposure darkened by the amount determined in step S303 compared to the exposure determined in step S302.

[0036] In step S306, the control unit 101 determines whether imaging has been performed at all focus positions set in step S301. If imaging has been performed at all focus positions, the process in this flowchart ends. If there are still focus positions that have not been imaged, the process returns to step S304.

[0037] In the flowchart shown in Figure 3, the control unit 101 prevents the camera from changing settings other than the focus position, including exposure, while the imaging unit 104 is capturing multiple images. As mentioned above, even a change in the focus position can alter the brightness of the image.

[0038] Figure 4 is a flowchart illustrating the brightness correction in step S202 in this embodiment.

[0039] In step S401, the control unit 101 acquires the image to be corrected for brightness from among the images captured by the imaging unit 104 in step S201. The image to be corrected for brightness is the image that was captured earliest among the images that have not yet been designated as images to be corrected for brightness, excluding the first image that was captured initially.

[0040] In step S402, the control unit 101 acquires a reference image for brightness correction from among the images captured by the imaging unit 104 in step S201. The reference image for brightness correction is an image captured before the target image.

[0041] In step S403, the control unit 101 calculates the brightness ratio between the reference image and the target image. Preferably, this is calculated by comparing the integral values ​​of the green pixels in the RAW images of the reference image and the target image, but it may also be done by comparing them in the state of YUV or RGB images, or by looking at the photometric amount at the time of imaging.

[0042] In step S404, the control unit 101 calculates a cumulative gain amount by accumulating the brightness ratios obtained in step S403. The cumulative gain amount is a value that brings the brightness of the target image closer to the exposure determined in step S302 by multiplying the gain amounts obtained before the current reference image and the target image by the ratio obtained in step S403.

[0043] In step S405, the control unit 101 associates the cumulative gain amount obtained in step S404 with the target image. It is desirable to store the combination of cumulative gain amount and target image in RAM 106 in an array-like format.

[0044] In step S406, the control unit 101 determines whether all images have been processed. If all images have been processed, the process proceeds to step S407; otherwise, it returns to step S401.

[0045] In step S407, the image processing unit 107 performs brightness correction uniformly across the image plane for all images except the first image captured, by the amount of cumulative gain associated in step S405. The brightness correction method can be performed in the RAW state after sensor output, when converting from RAW to YUV, or in the RGB state.

[0046] Figure 5 is a flowchart illustrating the alignment in step S203 of this embodiment.

[0047] In step S501, the control unit 101 acquires a reference image for alignment from among the images captured by the imaging unit 104 in step S201. The reference image for alignment is the one captured earliest in the image acquisition order. Alternatively, since the field of view changes slightly between captured images due to the change in focus position, the reference image with the narrowest field of view among the captured images may be used.

[0048] In step S502, the control unit 101 acquires the image to be processed for alignment. The target image is an image other than the reference image acquired in step S501 that has not yet undergone alignment processing. If the control unit 101 uses the image acquired earliest as the reference image, then it should acquire the target images sequentially in the order they were acquired.

[0049] In step S503, the control unit 101 calculates the amount of positional shift between the reference image and the target image. An example of the calculation method is described below. First, the control unit 101 sets multiple blocks in the reference image. It is preferable that the control unit 101 sets each block to be the same size. Next, the control unit 101 sets a search range in the target image that is wider than the block in the reference image, at the same position as each block in the reference image. Finally, the control unit 101 calculates the corresponding point in each search range of the target image that minimizes the sum of absolute differences in brightness (SAD) between the target image and the block in the reference image. The control unit 101 calculates the positional shift referred to in step S503 as a vector from the center of the block in the reference image and the aforementioned corresponding point. In calculating the corresponding point as described above, the control unit 101 may use methods other than SAD, such as the sum of squared differences (SSD) or normalized cross-correlation (NCC).

[0050] In step S504, the control unit 101 calculates a transformation coefficient from the positional shift between the reference image and the target image. The control unit 101 uses, for example, a projection transformation coefficient as the transformation coefficient. However, the transformation coefficient is not limited to only projection transformation coefficients; an affine transformation coefficient or a simplified transformation coefficient consisting only of horizontal and vertical shift may also be used.

[0051] In step S505, the image processing unit 107 performs a transformation on the target image using the transformation coefficient calculated in step S504.

[0052] For example, the control unit 101 can be modified using the equation shown in (Equation 1).

[0053]

number

[0054] In equation (1), (x',y') represents the coordinates after the deformation, and (x,y) represents the coordinates before the deformation. Matrix A shows the deformation coefficients calculated by the control unit 101 in step S404.

[0055] In step S506, the control unit 101 determines whether alignment has been performed on all images except the reference image. If alignment has been performed on all images except the reference image, the process shown in the flowchart in Figure 5 is terminated. If there are still images that have not been processed, the process returns to step S502.

[0056] Furthermore, when aligning multiple images captured by the multi-lens camera described above, the amount of parallax resulting from the difference in the position of the optical system 103 can be calculated in step S503, allowing for alignment using a similar process.

[0057] Figure 6 is a flowchart illustrating the image synthesis in step S204 of this embodiment.

[0058] In step 601, the image processing unit 107 calculates a contrast value for each image (including the reference image) after alignment. As an example of how to calculate the contrast value, first, the image processing unit 107 calculates the luminance Y from the color signals Sr, Sg, and Sb of each pixel using the following equation (2). Y=0.299Sr+0.587Sg+0.114Sb (Formula 2)

[0059] Next, the contrast value I is calculated using a Sobel filter on the luminance matrix L of the 3x3 pixels, as shown in (Equations 3) to (5) below.

[0060]

number

[0061]

number

[0062]

number

[0063] Furthermore, the above method for calculating contrast values ​​is merely one example; for instance, it is also possible to use edge detection filters such as Laplacian filters or bandpass filters that pass through a predetermined bandwidth.

[0064] In step S602, the image processing unit 107 generates a composite map. The method for generating the composite map involves the image processing unit 107 comparing the contrast values ​​of pixels at the same position in each image and calculating a composite ratio according to the magnitude of the contrast values.

[0065] An example of a specific calculation method is shown below.

[0066] A composite map Am(x,y) is generated using the contrast value Cm(x,y) calculated in step S601. Here, m is the m-th image among multiple images with different focus positions, x is the horizontal coordinate of the image, and y is the vertical coordinate. In generating the composite map, the image processing unit 107 compares the contrast values ​​of pixels at the same position in each image and calculates a composite ratio according to the magnitude of the contrast values. Specifically, a composite ratio of 100% is given to the pixel with the largest contrast value among the images at the same position, and a composite ratio of 0% is given to the other pixels at the same position. In other words, the following (Equation 6) holds true.

[0067]

number

[0068] However, in step S602, it is necessary to adjust the blending ratio appropriately so that the boundaries do not look unnatural. As a result, the blending ratio of the blended map in a single image will not be a binary system of 0% and 100%, but will change continuously.

[0069] In step S603, the image processing unit 107 generates a full-focus image O(x,y) by combining the captured images according to the composite map calculated in step 602. If the original captured image is Im(x,y), the image is generated by the following equation (7).

[0070]

number

[0071] In this embodiment, when attempting to compensate for changes in brightness values ​​due to changes in focus position, it is conceivable to cancel out brightness steps caused by aperture drive.

[0072] Figure 7 illustrates the change in brightness when the aperture is driven as the brightness value changes with a change in focus position. As the brightness increases with the movement of the focus position, a system can be conceived that maintains the brightness at the start of imaging by stopping down the lens aperture when a specific change in brightness is detected, as shown in Figure 7. However, even with a system like the one shown in Figure 7, if brightness correction is performed to match the brightness of the captured image to the brightness at the start of imaging, the image will be corrected in the direction of darkening, resulting in problems such as color fringing in the bright parts of the image or the loss of saturation.

[0073] Figure 10 is a diagram illustrating the change in brightness when the aperture is driven in the first embodiment. In the first embodiment, by specifying the size of the exposure step when the aperture is driven in Figure 10 during the calculation of the amount of exposure reduction in step S303 in Figure 3, it is possible to take an image without exceeding a predetermined exposure as shown in Figure 8. If an image can be taken as shown in Figure 8, the image can be corrected toward the brighter side in image processing in step S407 in Figure 4, and a composite image with reduced color fringing and exposure step can be generated.

[0074] (Second embodiment) The second embodiment of the present invention will be described below with reference to the figures. In the second embodiment, unlike the first embodiment, the method of determining the exposure when capturing an image is changed depending on whether or not image synthesis is performed. Details will be explained below.

[0075] Figure 9 is a flowchart illustrating the generation of a composite image in this embodiment. In the second embodiment, the digital camera 100 is equipped with multiple imaging modes, and it is assumed that one of the multiple imaging modes is a depth-of-field stacking mode.

[0076] In step S901, the control unit 101 determines whether the imaging mode of the digital camera 100 is set to depth composition mode. If it is in depth composition mode, the process proceeds to step S902, and the subsequent processing is as described in the first embodiment. If it is not in depth composition mode, the flow transitions to step S906.

[0077] In step S906, the control unit 101 determines the exposure conditions for imaging. Here, similar to step S302 in the first embodiment, the control unit 101 determines the shutter speed, F-number, and ISO sensitivity. The exposure conditions may be set manually by the user, or the control unit 101 may set them automatically by having the imaging unit 104 measure light from the subject.

[0078] In step S907, the imaging unit 104 takes an image according to the exposure conditions determined in step S906. In step S907, even if the imaging unit 104 takes multiple images, the exposure conditions used for taking multiple images remain the same as those determined in step S906.

[0079] According to this embodiment, in the case of depth stacking, images are captured with a darker exposure than predetermined and then corrected to be brighter through image processing, thereby generating a composite image with reduced color fringing and exposure differences. In cases other than depth stacking, images can be captured without changing the exposure.

[0080] (Other embodiments) The above embodiments are described based on depth stacking, but are not limited to this. When capturing multiple images, even if the focus position is not changed, the exposure may change due to uneven light emission when using a strobe or uneven exposure when using a high-speed shutter. Figure 13 is a diagram illustrating how brightness changes due to uneven light emission or uneven exposure when capturing multiple images. When some kind of stacking is performed on the multiple images shown in Figure 13, uneven brightness occurs in the combined image, similar to the first embodiment. To solve this problem, a method of capturing images with the exposure lower than a predetermined value, as described in the first embodiment, is effective.

[0081] The above embodiments have been described based on implementation with a digital camera, but are not limited to digital cameras. For example, they may be implemented with a portable device that has a built-in image sensor, or with a network camera that can capture images.

[0082] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and operate the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of Symbols]

[0083] 100 Digital Cameras 101 Control Unit 102 Drive unit 103 Optical system 104 Imaging Unit 105 ROM 106 RAM 107 Image Processing Unit 108 Display section 109 Internal memory 110 Operation section

Claims

1. A determination means for determining a predetermined value of exposure in advance; An imaging means for capturing a plurality of images with different focus positions; a control means for controlling the imaging means to capture the plurality of images at an exposure lower than the predetermined value; a correction unit that corrects brightness of at least some of the plurality of images based on the predetermined value determined by the determination unit.

2. 2. The imaging device according to claim 1, wherein the correction means corrects brightness of at least some of the images so that exposure of the some of the images coincides with the predetermined value.

3. 3. The imaging apparatus according to claim 1, wherein the imaging means captures the plurality of images while changing the focus position.

4. 4. The imaging apparatus according to claim 3, wherein the imaging means captures the plurality of images while changing the focus position from a close distance side to an infinity side.

5. 3. The imaging apparatus according to claim 1, wherein the determining unit determines the predetermined value based on a photometric amount when a first image of the plurality of images is captured.

6. 3. The imaging apparatus according to claim 1, wherein the determining unit determines the predetermined value based on a setting by a user.

7. 3. The imaging apparatus according to claim 1, wherein the control means controls at least one of a shutter speed, an F-number, and an ISO sensitivity, thereby controlling exposure when the imaging means captures the plurality of images.

8. an effective F-number when the imaging means captures the plurality of images is changed, 3. The image pickup apparatus according to claim 1, wherein the control means controls the exposure when the image pickup means captures the plurality of images based on an amount of change in the effective F-number.

9. the imaging means captures the images using a strobe; 3. The imaging apparatus according to claim 1, wherein the control means controls the exposure when the imaging means captures the plurality of images based on the performance of the strobe.

10. 10. The imaging apparatus according to claim 9, wherein the performance of the strobe includes unevenness in light emission of the strobe.

11. the imaging means captures the plurality of images using a shutter; 3. The imaging apparatus according to claim 1, wherein the control means controls the exposure when the imaging means captures the plurality of images based on the performance of a high-speed shutter.

12. The imaging device according to claim 11, characterized in that the performance of the shutter includes exposure unevenness caused by the shutter.

13. 3. The imaging apparatus according to claim 1, wherein the correction means corrects the RAW images, YUV images, or RGB images of the plurality of images.

14. 3. The image pickup apparatus according to claim 1, wherein the correction means performs correction based on an integral value of green pixels of RAW of the plurality of images.

15. 3. The imaging apparatus according to claim 1, wherein the correction means performs correction based on a photometric amount when the imaging means captures the plurality of images.

16. a synthesis means for synthesizing the plurality of images to generate a synthetic image, 3. The imaging device according to claim 1, wherein the depth of field of the composite image is deeper than the depth of field of any of the plurality of images.

17. 17. The imaging apparatus according to claim 16, wherein the synthesis means generates the synthetic image by extracting in-focus areas of the respective images.

18. A determination step of determining a predetermined value of exposure in advance; An imaging step of capturing a plurality of images having different focus positions; a control step of controlling the imaging step so that the images are captured at an exposure lower than the predetermined value when the images are captured in the imaging step; and a correcting step of correcting brightness of at least some of the plurality of images based on the predetermined value determined by the determining means.

19. A program for causing a computer to operate an imaging device, A determination step of determining a predetermined value of exposure in advance; An imaging step of capturing a plurality of images having different focus positions; a control step of controlling the imaging step so that the images are captured at an exposure lower than the predetermined value when the images are captured in the imaging step; a correcting step of correcting brightness of at least some of the plurality of images based on the predetermined value determined by the determining means.

20. A determination means for determining a predetermined value of exposure in advance; An imaging means for capturing a plurality of different images; a control means for controlling exposure when the imaging means captures the plurality of images; a correction means for correcting brightness of at least some of the images based on the predetermined value determined by the determination means; A synthesis unit for synthesizing at least a part of the plurality of images, In the first mode in which the composition is performed, the control means controls the imaging means to capture the plurality of images at an exposure lower than the predetermined value, and the correction means performs the correction; In the second mode in which no synthesis is performed, the control means controls the imaging means to capture the plurality of images at the predetermined value.

21. 21. The image pickup apparatus according to claim 20, wherein in the second mode, the correction means does not perform the correction.

22. 22. The imaging apparatus according to claim 20, wherein the plurality of images have different focus positions.

23. 22. The imaging apparatus according to claim 20, wherein the synthesis means generates a synthetic image having a depth of field deeper than a depth of field of at least some of the plurality of images by performing the synthesis.

24. A determination step of determining a predetermined value of exposure in advance; An imaging step of capturing a plurality of different images; a control step of controlling exposure when capturing the plurality of images in the imaging step; a correction step of correcting brightness of at least some of the images based on the predetermined value determined in the determination step; A synthesis step of synthesizing at least some of the images of the plurality of images, In the first mode in which the composition is performed, in the control step, when the plurality of images are captured in the imaging step, control is performed so that the images are captured with an exposure lower than the predetermined value, and in the correction step, the correction is performed; In the second mode in which no synthesis is performed, the control step controls the imaging step to capture the plurality of images at the imaging step using the predetermined value.

25. A program for causing a computer to operate an imaging device, A determination step of determining a predetermined value of exposure in advance; An imaging step of capturing a plurality of different images; a control step of controlling exposure when capturing the plurality of images in the imaging step; a correction step of correcting brightness of at least some of the images based on the predetermined value determined in the determination step; A synthesis step of synthesizing at least some of the images of the plurality of images; In the first mode in which the composition is performed, in the control step, when the plurality of images are captured in the imaging step, control is performed so that the images are captured with an exposure lower than the predetermined value, and in the correction step, the correction is performed; A computer program product, comprising: in a second mode in which the synthesis is not performed, the control step controls the imaging step to capture the plurality of images at the predetermined value.

26. A computer-readable recording medium having the program according to claim 19 or 25 recorded thereon.

Images