Imaging apparatus, method for controlling imaging apparatus, and program

Abstract

To provide an imaging apparatus enabling efficient generation of a depth composite image having a wide dynamic range, a method for controlling the imaging apparatus, and a program.SOLUTION: An imaging apparatus 100 comprises: an optical system 101 in which focus positions A and B are changed; an imaging device 102 which acquires for each of the focus positions A and B, a plurality of images by single exposure at different exposure; and an image processing device 105 which uses each image acquired by the imaging device 102 to thereby generate a depth composite image having a wider dynamic range and a deeper depth of field than each image acquired by the imaging device 102.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Conventionally, a technique called depth stacking has been known as a technique for obtaining an image with a deep depth of field. This technique generates a pan-focus image with a deep depth of field by extracting and synthesizing the in-focus parts from a plurality of images obtained by repeatedly shooting while changing the focus position. Since depth stacking is characterized by synthesizing a plurality of images, it is necessary to calculate a movement vector between each image and perform alignment. In addition, when a plurality of images are obtained, image magnification fluctuations often occur due to changes in the focus position. Therefore, processing equivalent to alignment is required every time the focus position is changed. Incidentally, there are cases where depth stacking is performed with an HDR (High Dynamic Range) image. In this regard, Patent Document 1 describes a technique for obtaining a plurality of HDR images using a plurality of images taken with different exposures, and further synthesizing the plurality of HDR images to generate a depth stacked image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-57626 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, the technology described in Patent Document 1 generates a depth stacking image by synthesizing a plurality of images taken with different exposures. In this case, it is known that the images of each exposure must be developed twice. One development is for generating an HDR image, and the other development is for performing alignment. In other words, in the case of the technology described in Patent Document 1, it is necessary to perform two developments for one RAW image. In addition, since the exposure of each RAW image is different, alignment may not be performed well in the first place in a scene where the brightness of the subject is biased.

[0005] The present invention has been made in view of the above problems, and has an object to provide an imaging device, an imaging device control method, and a program that can efficiently generate a depth stacking image with a wide dynamic range. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the imaging device of the present invention is characterized by comprising a change means for changing a focus position, an imaging means for acquiring multiple images with different exposures for each focus position with a single exposure, and a synthesis means for generating a depth synthesis image having a wider dynamic range and a deeper depth of field than each image acquired by the imaging means, using each image acquired by the imaging means. Effect of the Invention

[0007] According to the present invention, a depth stacking image with a wide dynamic range can be efficiently generated. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a basic configuration of an imaging apparatus. [Diagram 2] 1A and 1B are schematic diagrams illustrating a case where a depth stacking image with a wide dynamic range is generated. [Diagram 3]FIG. 3 is a diagram showing the relationship between the images used for image synthesis and the images used for image magnification correction and position adjustment in the case of FIG. 2(d). [Figure 4] 1 is a flowchart showing the flow of focus stacking. [Diagram 5] 1A and 1B are schematic diagrams illustrating a case where a depth stacking image with a wide dynamic range is generated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. In addition, any configuration may be added. In addition, any two or more configurations (features) of each embodiment can be combined.

[0010] First Embodiment Hereinafter, the first embodiment will be described with reference to Figs. 1 to 4. Fig. 1 is a block diagram showing the basic configuration of an image capturing device 100 according to the first embodiment. The image capturing device 100 may be any electronic device having a camera function, such as a digital camera, a digital video camera, a mobile phone with a camera function, or a computer with a camera. The image capturing device 100 has an optical system 101, an image capturing element 102, a CPU 103, a primary storage device 104, an image processing device 105, a recording medium 106, a secondary storage device 107, a display unit 108, and an operation unit 109.

[0011] The optical system 101 is composed of a lens, a shutter, and an aperture, and is controlled by the CPU 103 to cause light from a subject to be imaged on the image sensor 102. The CPU 103 adjusts the optical system 101 so that the light from the subject is imaged on the image sensor 102. For example, the CPU 103 can adjust the optical system 101 by moving a part of the lens of the optical system 101 in the optical axis direction. Alternatively, the CPU 103 can adjust the optical system 101 by changing the distance between the optical system 101 and the image sensor 102. By such adjustment, the optical system 101 (changing means) can change the focus position.

[0012] The image sensor 102 is a CCD image sensor, a CMOS image sensor, or the like, and converts the light imaged through the optical system 101 into an image signal. The image sensor 102 (imaging means) has two column circuits for the output signal from the unit pixel, and further has separate gains for the amplifiers in the column circuits, making it possible to output two images with different gains (Dual Gain Output). Hereinafter, Dual Gain Output is abbreviated as "DGO." The DGO can output two images with different gains (high gain and low gain images) with a single exposure. Comparing the synthesis of two images obtained by a single shooting with the DGO and the synthesis of two images obtained by time-shared exposure, the DGO has the advantages of not requiring alignment processing and being resistant to moving objects. Therefore, the DGO is compatible with means for obtaining images with an expanded dynamic range.

[0013] The CPU 103 realizes the functions of the imaging device 100 by controlling each component of the imaging device 100 according to an input signal or a program stored in advance. The primary storage device 104 is a volatile device such as a RAM, which stores temporary data and is used for the work of the CPU 103. The information stored in the primary storage device 104 is used by the image processing device 105 and is also recorded in a recording medium 106. The recording medium 106 records data of images obtained by shooting stored in the primary storage device 104. The recording medium 106 is removable from the imaging device 100, for example, like a semiconductor memory card. The data recorded in the recording medium 106 can be loaded into a personal computer or the like and read out. That is, the imaging device 100 has a mechanism for attaching and detaching the recording medium 106 and a read / write function.

[0014] The secondary storage device 107 is a non-volatile storage device such as an EEPROM, and stores a program (firmware) for controlling the imaging device 100 and various setting information, and is used by the CPU 103. The display unit 108 displays a viewfinder image during shooting, a shot image, and a GUI image for interactive operation. The operation unit 109 is a group of input devices that accept user operations and transmit input information to the CPU 103, and may be, for example, a button, a lever, a touch panel, or an input device using voice or line of sight. The imaging device 100 has multiple image processing patterns that the image processing device 105 applies to the shot image. Regarding the multiple patterns, any of them can be set by the user as the shooting mode on the operation unit 109. The image processing device 105 (compositing means) performs image processing called so-called development processing, image composition processing using vector calculation and geometric transformation to detect deviations between multiple images, and color tone adjustment according to the shooting mode. At least a part of the functions of the image processing device 105 may be realized by the CPU 103 as software.

[0015] FIG. 2 is a schematic diagram showing a case where a depth stacking image with a wide dynamic range is generated. Four cases will be described below with reference to FIG. 2. FIG. 2 shows that an image capture device captures a plurality of images of a subject while changing the focus position, and generates a depth stacking image by performing image magnification correction and position alignment on each image and then synthesizing the strong contrast parts. Details of this will be described later with reference to FIG. 2. For such a process, it is set at which focus position, at which exposure, and how to capture the image.

[0016] In the following, for convenience, a case where a properly exposed image, an overexposed image, and an underexposed image are acquired as images acquired to generate a depth stacking image with a wide dynamic range will be described, but the exposure, the number of images, and the method are not limited. In FIG. 2, focus positions A and B used when the imaging device performs depth stacking are alternately provided. The codes of each focus position A and B are assigned numbers that increase by one in the order from the imaging device to the subject. Each focus position A1, A2, A3, and A4 is provided at equal intervals. Each focus position B1, B2, B3, and B4 is also provided at equal intervals. In addition, focus position B1 is located in the middle of focus positions A1 and A2, focus position B2 is located in the middle of focus positions A2 and A3, and focus position B3 is located in the middle of focus positions A3 and A4. Therefore, each focus position A1, B1, A2, B2, A3, B3, A4, and B4 is provided at equal intervals.

[0017] FIG. 2(a) shows a case where the imaging device captures three images at each focus position A. That is, FIG. 2(a) shows a case where a properly exposed image, an overexposed image, and an underexposed image are captured at each focus position A. In this case, the imaging device needs to capture three RAW images at each focus position A. In addition, since the imaging device needs to perform development for image synthesis and another development for making the images the same brightness for image magnification correction and position alignment, it is known that one RAW image needs to be developed two or more times. FIG. 2(b) shows a case where the imaging device captures one image at each focus position. That is, FIG. 2(b) shows a case where one of a properly exposed image, an overexposed image, and an underexposed image is captured at each focus position in the order of description. In this case, as in the case of FIG. 2(a), it is known that one RAW image needs to be developed two or more times.

[0018] FIG. 2(c) shows a case where the imaging device captures four images at each focus position A and the DGO. That is, FIG. 2(c) shows a case where two properly exposed images, one overexposed image, and one underexposed image are captured at each focus position A by the DGO. When the imaging device captures images with the DGO, two images with different exposures can be obtained in one capture. Of the two images with different exposures, one image is an overexposed image or an underexposed image for expanding the dynamic range of the depth stacking image, and the other image is a properly exposed image with the same exposure in each capture. The properly exposed image is also used for image magnification correction and alignment, but is captured with the same exposure in each capture. Therefore, in the case of FIG. 2(c), development is not necessary only for image magnification correction and alignment, and development is sufficient for the number of images. That is, each image acquired by capture (properly exposed image, overexposed image, underexposed image) is developed only once. This reduces the number of developments. In addition, the number of times images are taken can be reduced by using a DGO that takes two images simultaneously. Furthermore, in the case of Figure 2(c), the image used for image magnification correction and alignment is an image with a proper exposure, so image magnification correction and alignment can be performed with high accuracy.

[0019] However, as shown in Fig. 2(c), if a properly exposed image, an overexposed image, and an underexposed image are taken at each focus position A, at least two DGO acquisitions are required, even if two images are taken simultaneously with the DGO. Furthermore, two DGO acquisitions result in four images, of which two properly exposed images with the same exposure are almost identical. Therefore, while the overexposed image, the underexposed image, and one properly exposed image are useful, the remaining properly exposed image is often of relatively little value.

[0020] In the first embodiment, in view of the above, as shown in FIG. 2(d), the imaging device captures two images at each focus position A and B and with the DGO. That is, FIG. 2(d) of the first embodiment shows a case where a properly exposed image and an overexposed image are captured with the DGO at each focus position A, and a properly exposed image and an underexposed image are captured with the DGO at each focus position B. The properly exposed image is an identically exposed image with the same exposure in each capture. In addition, the first embodiment may be a case where a properly exposed image and an underexposed image are captured with the DGO at each focus position A, and a properly exposed image and an overexposed image are captured with the DGO at each focus position B, which is the opposite of FIG. 2(d). In the case of FIG. 2(d) of the first embodiment, the advantages of the case of FIG. 2(c) can be enjoyed as they are, that is, the accuracy of image magnification correction and alignment, and the reduction in the number of captures and developments. Therefore, each image (properly exposed image, overexposed image, underexposed image) acquired by capture is developed only once.

[0021] Furthermore, of the four images acquired at adjacent focus positions A and B, the overexposed image, the underexposed image, and one properly exposed image are used as images for synthesis, and the remaining properly exposed image is used only for image magnification correction and alignment. In this regard, in the case of FIG. 2(c) above, the focus positions at which properly exposed images used for image magnification correction and alignment are acquired are A1, A2, A3, etc. Therefore, in the case of FIG. 2(c) above, for example, image magnification correction and alignment are performed by comparing the properly exposed image at focus position A2 with the properly exposed images at focus positions A1 and A3 adjacent to focus position A2.

[0022] In contrast, in the case of FIG. 2(d) of the first embodiment, a properly exposed image with the same exposure is acquired every time the focus positions A and B are changed. Among the focus positions A and B, the focus positions at which properly exposed images used only for image magnification correction and alignment are acquired are B1, B2, B3, etc. Therefore, in the case of FIG. 2(d) of the first embodiment, for example, the properly exposed image at the focus position B1 is compared with the properly exposed images at the focus positions A1 and A2 adjacent to the focus position B1, thereby performing image magnification correction and alignment. This point is indicated by the white arrow in FIG. 3, which will be described later. Furthermore, in the case of FIG. 2(d) of the first embodiment, the interval between the focus positions at which properly exposed images are acquired is half that of the case of FIG. 2(c), so that the accuracy of image magnification correction and alignment can be improved.

[0023] In the case of FIG. 2(d) of the first embodiment, as shown in FIG. 3, the step width between images with a common exposure (properly exposed images, overexposed images, and underexposed images) used as a composite image is constant. That is, in the case of FIG. 2(d) of the first embodiment, the interval between focus positions at which properly exposed images and overexposed images are acquired by simultaneous shooting with the DGO is equal to the interval between focus positions at which properly exposed images and underexposed images are acquired by simultaneous shooting with the DGO. In addition, both intervals are equal to the interval at which properly exposed images used as a composite image are acquired by simultaneous shooting with the DGO. In this way, in the case of FIG. 2(d) of the first embodiment, the interval between focus positions at which images with a common exposure (properly exposed images, overexposed images, and underexposed images) are acquired are constant. Therefore, a depth stacking image with a desired step width (focus interval) can be obtained.

[0024] In the case of Fig. 2(d) of the first embodiment, an underexposed image and an overexposed image are alternately acquired each time the focus position is changed. Therefore, in the case of Fig. 2(d) of the first embodiment, the underexposed image and the overexposed image used for dynamic range extension can be acquired with the same number of shots as in the case of Fig. 2(c) above. Note that, as shown in Figs. 2(c) and 2(d), the two identically exposed images obtained when two shots are taken with the DGO have been described as properly exposed images, but this is not limited to properly exposed images, and there is no problem as long as the exposure of the two images is the same.

[0025] Fig. 4 is a flowchart showing the flow of focus stacking. The process of Fig. 4 (a control method of the imaging device) is realized by the CPU 103 (computer) expanding a program stored in the secondary storage device 107 into the primary storage device 104, executing it, and controlling each unit of the imaging device 100. The flow of the process of Fig. 4 will be explained below. In step S401, the CPU 103 starts a focus stacking shooting mode. This process is performed based on a user's operation on the operation unit 109.

[0026] In step S402, the CPU 103 determines the number of shots and the focus swing range for depth stacking. This process is performed based on the user's operation on the operation unit 109. Therefore, the number of shots and the focus swing range are described here as the number of shots required for depth stacking based on the user's settings for depth stacking and the swing range to the focus position at each shot, but are not limited to this. In addition, the user settings do not necessarily match the actual number of shots and the swing range of the focus. For example, even if the user's setting of the number of shots is two, if two shots are required by the DGO to obtain images with different exposures to generate a depth stacked image with a wide dynamic range, the number of shots required for depth stacking will be four.

[0027] In step S403, the CPU 103 determines the order of shooting. As described above, when two images are shot simultaneously by the DGO, one image is assumed to be used for dynamic range extension, while the other image is assumed to be shot with a common exposure for use in alignment. In the first embodiment, for convenience, the discussion will be centered on the case where a properly exposed image and an overexposed image are shot simultaneously, and a properly exposed image and an underexposed image are shot simultaneously, as shown in FIG. 2(d), but this is not limiting. In the above case, the CPU 103 determines which of the properly exposed image and the overexposed image and the properly exposed image and the underexposed image should be shot first. In other words, the CPU 103 determines which of the properly exposed image and the overexposed image and the properly exposed image and the underexposed image should be shot first.

[0028] In this regard, the CPU 103 (determination means) measures the brightness of the subject at the first focus position, and captures the images in an order that can best express the brightness of the subject. For example, in the case of a bright subject, the CPU 103 uses an image captured with underexposure rather than an image captured with overexposure in order to best express the brightness of the subject. Furthermore, if the CPU 103 uses the properly exposed image of two images captured simultaneously with a combination of proper and underexposure in image synthesis, the properly exposed image that is closer to the brightness of the subject can be used in the image synthesis. Therefore, the CPU 103 determines the shooting order so that the simultaneous capture of a properly exposed image and an underexposed image comes first.

[0029] On the other hand, in the case of a dark subject, the CPU 103 uses an image taken with an overexposure rather than an image taken with an underexposure in order to express the brightness of the subject more clearly. Furthermore, when the CPU 103 uses the properly exposed image of two images taken simultaneously with a combination of proper and overexposure in image synthesis, the properly exposed image closer to the brightness of the subject can be used in image synthesis. Therefore, the CPU 103 determines the shooting order so that the simultaneous shooting of the properly exposed image and the overexposed image comes first. In this way, the CPU 103 determines the combination order of different exposures according to the brightness of the subject, so that the properly exposed image closer to the brightness of the subject can be used in image synthesis. The method of determining the shooting order may be selected by the user through the operation unit 109 (selection means).

[0030] In step S404, the CPU 103 (determination means) determines the exposure for shooting. This process is performed based on the shooting order determined in step S403. This means that the exposure for each of the two images captured simultaneously by the DGO is determined by determining the shooting order in step S403. In step S405, the CPU 103 performs the first shooting of the depth stacking (imaging process). In this process, the CPU 103 controls the optical system 101 and the image sensor 102 to simultaneously capture two images by the DGO at the initial focus position set by the user, with the respective exposures determined in step S404. In step S406, the CPU 103 controls the image processing device 105 to perform optical correction. The optical correction here refers to correction that is completed by the single capture of one image, and includes peripheral illumination correction, chromatic aberration correction, distortion aberration correction, etc. This point is also the same in step S411 described later.

[0031] In step S407, the CPU 103 controls the image processing device 105 to develop the image optically corrected in step S406. Here, two images simultaneously captured by the DGO in step S405 are developed. In step S408, the CPU 103 controls the optical system 101 to change the focus position (changing step). This change of the focus position is performed based on the focus swing determined in step S402. In step S402, the CPU 103 sets the focus position B in the middle of the focus swing set by the user (i.e., the interval between each focus position A), as shown in FIG. 2(d). As a result, the focus positions at which the properly exposed images and the overexposed images are acquired by simultaneous capture by the DGO are set at equal intervals, and the focus positions at which the properly exposed images and the underexposed images are acquired by simultaneous capture by the DGO are set at equal intervals. As a result, the focus positions at which the properly exposed images are acquired by simultaneous capture by the DGO are also set at equal intervals. In this way, in the first embodiment, as described above, image magnification correction and alignment accuracy are improved compared to the case of FIG. 2(c) in which four images are captured at the same focus position.

[0032] In step S409, the CPU 103 determines the exposure. In this process, the exposure of each of the two images captured simultaneously by the DGO is determined in the same manner as in step S404. However, since this process is performed based on the shooting order determined in step S403, an exposure different from the exposure in step S404 is determined. In step S410, the CPU 103 performs the Nth shooting of the focus stacking (imaging process). In this process, the CPU 103 controls the image sensor 102 to simultaneously capture two images by the DGO at the respective exposures determined in step S409. In step S411, the CPU 103 performs optical correction in the same manner as in step S406. In step S412, the CPU 103 controls the image processing device 105 to develop the images optically corrected in step S411. Here, development is performed for the two images captured simultaneously by the DGO in step S410. In step S413, the CPU 103 controls the image processing device 105 to perform image magnification correction for depth stacking (compositing step). In step S414, the CPU 103 controls the image processing device 105 to perform alignment for depth stacking (compositing step). Note that although the image magnification correction in step S413 and the alignment in step S414 are divided into two in the flowchart of FIG. 4, they both involve geometric transformation and may be performed in a single process.

[0033] In step S415, the CPU 103 controls the image processing device 105 to extract a synthesis position for the depth stacking (synthesis step). Depth stacking generates a depth stacked image by synthesizing only high-contrast parts of the captured images, so it is necessary to extract a synthesis position. In step S416, the CPU 103 controls the image processing device 105 to perform image synthesis (synthesis step). This process is performed based on the synthesis position extracted in step S415. In step S417, the CPU 103 determines whether the number of times of shooting for depth stacking has reached the maximum number of times of shooting. Note that the maximum number of times of shooting is the number of times of shooting determined in step S402. If the CPU 103 determines that the number of times of shooting for depth stacking has not reached the maximum number of times of shooting, the process returns to step S408. In this way, steps S408 to S416 are repeated to generate a depth stacked image. On the other hand, if the CPU 103 determines that the number of times of shooting for depth stacking has reached the maximum number of times of shooting, the flowchart in FIG. 4 ends. This completes the generation of the depth stacking image.

[0034] As described above, the imaging device 100 according to the first embodiment can efficiently generate a depth stacked image having a wider dynamic range and a deeper depth of field than the images acquired by simultaneous imaging with the DGO by using the images acquired by simultaneous imaging with the DGO. In addition, in the first embodiment, the combination of exposures of two images acquired by simultaneous imaging with the DGO is changed every time the focus position is changed. Furthermore, in the first embodiment, the distance between the focus positions at which two images are acquired by simultaneous imaging with the DGO is half that in the case of FIG. 2(c). As a result, the imaging device 100 according to the first embodiment can reduce the number of times of imaging and development compared to the cases of FIG. 2(a) and (b), and can improve the accuracy of image magnification correction and alignment compared to the case of FIG. 2(c).

[0035] <Second embodiment> The second embodiment will be described below with reference to Fig. 5. The differences from the first embodiment will be mainly described. The second embodiment has the advantage that the present invention can be realized even when the focus positions at which two images are acquired by simultaneous imaging with the DGO are not equidistant.

[0036] In the above first embodiment, as shown in FIG. 5(a), the case where the focus positions at which two images are acquired by simultaneous imaging with the DGO are equidistant has been described (see FIG. 2(d)). In FIG. 5(a), the open arrows indicate the interval between the focus positions at which properly exposed images to be used for image synthesis are acquired, and this interval is equal to the interval between the focus positions at which properly exposed images to be used only for image magnification correction and alignment are acquired. The solid arrows indicate the interval between the focus positions at which underexposed images to be used for image synthesis are acquired, and this interval is equal to the interval between the focus positions at which overexposed images to be used for image synthesis are acquired. Note that these points are also the same in FIG. 5(b) described later.

[0037] On the other hand, in the second embodiment, as shown in FIG. 5(b), the intervals between the focus positions at which two images are acquired by simultaneous imaging with the DGO are not all equal. Even in this case, as shown by the open arrow and the filled arrow in FIG. 5(b), it is sufficient that the step width between images with a common exposure (properly exposed images, overexposed images, and underexposed images) among the images used as the composite image is kept constant. In other words, it is sufficient that the intervals between the focus positions at which images with a common exposure (properly exposed images, overexposed images, and underexposed images) among the images used as the composite image are acquired are kept constant. In this way, the accuracy of image magnification correction and alignment is improved compared to FIG. 2(c) in which four images are acquired at the same focus position, as explained in FIG. 2.

[0038] <Third embodiment> The third embodiment will be described below. Here, the differences from the first embodiment will be mainly described. The third embodiment relates to the shooting by the DGO at the first focus position and the shooting by the DGO at the last focus position. When depth stacking is performed, the shooting by the DGO at the first focus position is performed at a focus position based on the user's intention. Therefore, at the first focus position, depending on the brightness of the subject, the focus may be slightly shifted from the subject, so the user may want to shoot both an underexposed image and an overexposed image. Similarly, at the last focus position, if the subject is a background with a wide dynamic range, the focus may be slightly shifted from the subject, so the user may want to shoot both an underexposed image and an overexposed image.

[0039] However, in the case of Fig. 2(d) of the first embodiment, at each focus position A, B, the DGO can only simultaneously capture either a properly exposed image and an overexposed image, or a properly exposed image and an underexposed image. In other words, in the first embodiment, at the first focus position, it is not possible to obtain both an underexposed image and an overexposed image, contrary to the user's intention. The same is true for the last focus position.

[0040] Therefore, the imaging device 100 according to the third embodiment simultaneously captures both a properly exposed image and an overexposed image by the DGO and simultaneously captures a properly exposed image and an underexposed image by the DGO at the initial focus position. Similarly, the imaging device 100 according to the third embodiment simultaneously captures both a properly exposed image and an overexposed image by the DGO and simultaneously captures a properly exposed image and an underexposed image by the DGO at the final focus position. As a result, the imaging device 100 according to the third embodiment acquires both an underexposed image and an overexposed image at the initial focus position as intended by the user. Similarly, the imaging device 100 according to the third embodiment acquires both an underexposed image and an overexposed image at the final focus position as intended by the user.

[0041] That is, the imaging device 100 according to the third embodiment acquires, at the initial focus position, an image with an exposure other than that determined for the initial focus position in addition to an image with an exposure determined for the initial focus position. Also, at the final focus position, the imaging device 100 according to the third embodiment acquires, at the initial focus position, an image with an exposure other than that determined for the final focus position in addition to an image with an exposure determined for the final focus position. In this way, the imaging device 100 according to the third embodiment ensures that images in which the focus is not shifted from the subject are acquired at the initial focus position and the final focus position.

[0042] Incidentally, an exposure other than the exposure determined for the first focus position may be automatically set from an exposure determined for each focus position after the first focus position, or may be set by the user using the operation unit 109. Similarly, an exposure other than the exposure determined for the last focus position may be automatically set from an exposure determined for each focus position before the last focus position, or may be set by the user using the operation unit 109.

[0043] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) A change means for changing a focus position; an imaging means for acquiring a plurality of images at different exposures for each focus position by a single exposure; An imaging device comprising: a synthesis means for generating a depth synthesis image having a wider dynamic range and a deeper depth of field than each image acquired by the imaging means, using each image acquired by the imaging means. (Configuration 2) The imaging device according to configuration 1, wherein the change means changes the focus position so that focus positions at which images of a common exposure are acquired among the plurality of images are equally spaced. (Configuration 3) The imaging device according to configuration 2, wherein the change means changes the focus positions so that focus positions common to all of the different exposures are equally spaced from one another. (Configuration 4) The imaging device according to configuration 2, wherein the change means changes the focus position so that focus positions having one of the different exposures in common are equally spaced from one another. (Configuration 5) The imaging device according to any one of configurations 1 to 4, wherein the imaging means obtains an identically exposed image every time the focus position is changed. (Configuration 6) The imaging device according to configuration 5, wherein the synthesizing means performs position alignment using the identically exposed images. (Configuration 7) The imaging device according to configuration 6, wherein the identically exposed image is a properly exposed image. (Configuration 8) The imaging device according to any one of Configurations 1 to 7, wherein the imaging means alternately obtains an underexposed image and an overexposed image each time the focus position is changed. (Configuration 9) A determination means for determining the combination order of the different exposures in accordance with the brightness of a subject; 9. The imaging apparatus according to any one of configurations 1 to 8, further comprising: a determination unit that determines the different exposure for each of the focus positions according to the combination order. (Configuration 10) A selection means for allowing a user to select a combination order of the different exposures; 9. The imaging apparatus according to any one of configurations 1 to 8, further comprising: a determination unit that determines the different exposure for each of the focus positions according to the combination order. (Configuration 11) An imaging device described in any one of configurations 1 to 10, characterized in that the imaging means acquires, at the initial focus position, an image with an exposure other than that determined for the initial focus position in addition to an image with an exposure determined for the initial focus position. (Configuration 12) An imaging device described in any one of configurations 1 to 11, characterized in that the imaging means acquires, at the last focus position, an image with an exposure other than the exposure determined for the last focus position in addition to an image with an exposure determined for the last focus position. (Method 1) A change step of changing a focus position; an imaging step of acquiring a plurality of images at different exposures for each focus position by a single exposure; A control method for an imaging device, comprising: a synthesis process for generating a depth synthesis image having a wider dynamic range and a deeper depth of field than each image acquired in the imaging process, using each image acquired in the imaging process. (Program 1) A program for causing a computer to execute each means of the imaging device according to any one of configurations 1 to 12.

[0044] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0045] Furthermore, the functions of the image processing device 105 may be realized by a personal computer, a smartphone, a tablet terminal, or the like separate from the imaging device 100. In this case, data used by the image processing device 105 is sent to the personal computer, the smartphone, the tablet terminal, or the like from the imaging device 100 via a wired connection, a wireless connection, a recording medium 106, or the like. [Explanation of symbols]

[0046] 100 Imaging device 101 Optical system (change means) 102 Imaging element (imaging means) 105 Image processing device (composite means) A,B focus position

Claims

1. An imaging means; A change means for changing the focus position; a synthesis means for generating a depth synthesis image, the imaging means acquires a plurality of images with different gains in a single exposure for each focus position set by the change means, The imaging device is characterized in that the synthesis means generates a depth synthesis image having a wider dynamic range and a deeper depth of field than each image acquired by the imaging means, using each image acquired by the imaging means.

2. The imaging device according to claim 1 , wherein the change unit changes the focus position so that focus positions at which images with a common exposure are acquired among the plurality of images are spaced at equal intervals.

3. 3. The imaging apparatus according to claim 2, wherein the change unit changes the focus positions so that the focus positions common to all of the different exposures are spaced at equal intervals.

4. 3. The imaging apparatus according to claim 2, wherein the change unit changes the focus positions so that focus positions that share one of the different exposures are spaced equally apart.

5. 2. The imaging apparatus according to claim 1, wherein the imaging means acquires an image with the same exposure each time the focus position is changed.

6. 6. The imaging apparatus according to claim 5, wherein the synthesizing means performs position alignment using the identically exposed images.

7. 7. The imaging apparatus according to claim 6, wherein the identically exposed image is a properly exposed image.

8. 2. The imaging apparatus according to claim 1, wherein the imaging means alternately captures an underexposed image and an overexposed image each time the focus position is changed.

9. a determination means for determining the order of combinations of the different exposures in accordance with the brightness of an object; 2. The imaging apparatus according to claim 1, further comprising: a determination unit that determines the different exposures for each of the focus positions in accordance with the combination order.

10. a selection means for allowing a user to select a combination order of the different exposures; 2. The imaging apparatus according to claim 1, further comprising: a determination unit that determines the different exposures for each of the focus positions in accordance with the combination order.

11. 2. The imaging device according to claim 1, wherein the imaging means acquires, at the initial focus position, an image with an exposure other than that determined for the initial focus position in addition to an image with an exposure determined for the initial focus position.

12. 2. The imaging device according to claim 1, wherein the imaging means acquires, at the last focus position, an image with an exposure other than that determined for the last focus position in addition to an image with an exposure determined for the last focus position.

13. An imaging step; a changing step of changing a focus position; a synthesis step of generating a depth synthesis image, the imaging step acquires a plurality of images with different gains by one exposure for each focus position set by the changing step, A control method for an imaging device, characterized in that the synthesis process generates a depth synthesis image having a wider dynamic range and a deeper depth of field than each image acquired in the imaging process using each image acquired in the imaging process.

14. A program for causing a computer to execute each means of the imaging apparatus according to claim 1.

Images