Imaging device, imaging method, program, and recording medium

Abstract

To indicate parameters, such as aperture value to which a depth of field of an image of depth composition corresponds, in advance, when performing the depth composition.SOLUTION: In order to solve the problem, an imaging device according to the invention of the present application includes: setting means for setting a first aperture value; image capturing means for capturing on the basis of the aperture value set by the setting means; display means for displaying a second aperture value; and control means. When the first aperture value does not satisfy a predetermined condition, the image capturing means performs the capturing using the first aperture value. When the first aperture value satisfies the predetermined condition, the control means calculates, as the second aperture value, an aperture value of composite image to be generated when the image capturing means captures a plurality of images with different focus positions and performs a composite on the plurality of images. The aperture value of the composite image satisfies the predetermined condition.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an imaging device that captures a plurality of images with different focus positions and performs depth stacking. [Background technology]

[0002] When capturing images of multiple subjects at significantly different distances from a digital camera or a subject that is long in the depth direction, the depth of field may be insufficient to focus on only a portion of the subject. To solve this problem, Patent Document 1 discloses a so-called depth stacking technique that captures multiple images with different focus positions, extracts only the in-focus areas from each image, and combines them into a single image to generate a composite image in which the entire captured area is in focus.

[0003] Furthermore, Patent Document 2 discloses a method of calculating an aperture value required to capture an image having the same depth of field as the depth stacking image, and recording the calculated aperture value as additional information of the depth stacking image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-216532 A [Patent Document 2] JP 2019-36885 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when using the method described in Patent Document 2, additional information of a focus stacking image cannot be obtained unless a focus stacking image is generated.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an imaging device that, when performing depth stacking imaging, can indicate an aperture value corresponding to the depth of field of a depth stacking image by using parameters of the depth stacking imaging. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a control system that includes a setting means for setting a first aperture value, an imaging means for capturing an image based on the aperture value set by the setting means, a display means for displaying a second aperture value, and a control means, wherein if the first aperture value does not satisfy a predetermined condition, the imaging means captures the image at the first aperture value, and if the first aperture value satisfies the predetermined condition, the control means calculates, as the second aperture value, an aperture value of a composite image to be generated when the imaging means captures a plurality of images with different focus positions and composites the plurality of images, and the aperture value of the composite image satisfies the predetermined condition. The F-number corresponding to the aperture value is greater than the largest F-number that the imaging means can capture. The present invention provides an imaging device that satisfies the above requirements. Effect of the Invention

[0008] According to the configuration of the present invention, when capturing an image using depth stacking, it is possible to calculate in advance the aperture value that corresponds to the depth of field of the image using depth stacking. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing the exterior of a digital camera as an image capturing apparatus according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a hardware configuration of a digital camera according to an embodiment of the present invention. [Diagram 3] 4 is a flowchart for explaining generation of a composite image in the first embodiment. [Figure 4] 4 is a flowchart illustrating imaging in the first embodiment. [Diagram 5]4 is a flowchart illustrating alignment in the first embodiment. [Figure 6] 4 is a flowchart for explaining synthesis in the first embodiment. [Figure 7] 4A and 4B are diagrams illustrating a state in which a subject image is formed on an imaging surface in the first embodiment. [Figure 8] 4 is a flowchart for explaining setting of an imaging mode in the first embodiment. [Figure 9] FIG. 4 is a diagram for explaining the setting of an aperture value in the first embodiment. [Figure 10] 5 is a diagram for explaining calculation of a virtual F-number in the first embodiment. FIG. [Figure 11] 10 is a flowchart illustrating setting of an imaging mode in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention.

[0011] (First embodiment) FIG. 1 is a diagram showing the external appearance of a digital camera 100 as an image capturing apparatus according to the present invention.

[0012] 1, the digital camera 100 includes a display unit 28, a mode change switch 60, a shutter button 61, an operation unit 70, a power switch 72, a controller wheel 73, a connector 112, a recording medium 200, a recording medium slot 201, and a cover 202.

[0013] The display unit 28 displays images and various types of information.

[0014] The mode changeover switch 60 is an operation unit for switching between various modes.

[0015] The shutter button 61 is an operation unit for issuing an instruction to capture an image.

[0016] The operation unit 70 is an operation unit including various operation members such as various switches, buttons, a touch panel, etc., which accept various operations from the user.

[0017] A power switch 72 switches the power of the digital camera 100 on and off.

[0018] The controller wheel 73 is a rotatable operating member included in the operating unit 70.

[0019] The connector 112 connects the connection cable 111 to the digital camera 100 .

[0020] The recording medium 200 is a recording medium such as a memory card or a hard disk.

[0021] The recording medium slot 201 is a slot for storing the recording medium 200. The recording medium 200 stored in the recording medium slot 201 is capable of communicating with the digital camera 100.

[0022] The cover 202 is a cover for the recording medium slot 201 .

[0023] FIG. 2 is a block diagram showing the hardware configuration of the digital camera 100 of FIG.

[0024] In FIG. 2, the digital camera 100 includes an imaging system including a barrier 102 , a focus lens 103 , a shutter 104 , and an imaging unit 22 .

[0025] The barrier 102 covers the imaging system of the digital camera 100 to prevent the imaging system from becoming dirty or damaged.

[0026] The focus lens 103 is a lens included in a lens group (not shown) that is disposed between the shutter 104 and the barrier 102. Note that this lens group also includes other lenses such as a zoom lens.

[0027] The shutter 104 is a shutter having an aperture function.

[0028] The imaging unit 22 includes an imaging element such as a CCD or CMOS element that converts an optical image into an electric signal, and an A / D conversion function. Output data (captured images) from the imaging unit 22 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. When focus bracketing imaging is performed using a driving release, which will be described later, all of the captured images up to the set number are written to the memory 32.

[0029] The digital camera 100 further includes an AF evaluation value detection unit 23 , a strobe 90 , an image processing unit 24 , a memory 32 , a D / A converter 13 , a display unit 28 , a non-volatile memory 56 , a system control unit 50 , and a system timer 53 .

[0030] The AF evaluation value detection unit 23 is disposed within the imaging unit 22 and calculates an AF evaluation value from contrast information and the like obtained from a digital image signal, and outputs the obtained AF evaluation value from the imaging unit 22 to the system control unit 50 .

[0031] The strobe 90 can compensate for the lack of illumination when capturing an image in a low-illumination scene or a backlit scene by emitting light during capturing an image.

[0032] The image processing unit 24 performs predetermined pixel interpolation, resizing such as reduction, and color conversion processing on the image data output from the imaging unit 22 or image data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained arithmetic results. This allows TTL (through-the-lens) type AE (automatic exposure) processing and EF (automatic flash light adjustment) processing to be performed. The image processing unit 24 also performs AF (autofocus) processing, and at this time, the output of the AF evaluation value detection unit 23 provided in the imaging unit 22 may be used. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and also performs TTL type AWB (auto white balance) processing based on the obtained arithmetic results.

[0033] The memory 32 stores image data acquired and A / D converted by the imaging unit 22, and image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 32 also serves as a memory for displaying images (video memory).

[0034] The D / A converter 13 converts the image display data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the display image data written in the memory 32 is displayed by the display unit 28 via the D / A converter 13.

[0035] The display unit 28 performs display on a display device such as an LCD in response to the analog signal from the D / A converter 13. The digital signal that has been A / D converted once by the imaging unit 22 and stored in the memory 32 is converted to analog by the D / A converter 13 and sequentially transferred to and displayed on the display unit 28, thereby functioning as an electronic viewfinder and capable of displaying a through image.

[0036] The non-volatile memory 56 is an electrically erasable and recordable memory, such as a flash memory. Constants, programs, and the like for operating the system control unit 50 are stored in the non-volatile memory 56. The programs referred to here are programs for executing various flowcharts described later in Examples 1 and 2.

[0037] The system control unit 50 controls the entire digital camera 100. Specifically, by executing the program recorded in the non-volatile memory 56 described above, focus bracketing imaging using a driving release in the following embodiments 1 and 2 based on subject information, subject distance, and image contrast information is realized. That is, during such imaging, the system control unit 50 performs driving control of the focus lens 103 and the shutter 104, so that a plurality of images with different focus positions are sequentially captured. The amount of change in focus position (focus step) between adjacent captured images obtained by such imaging processing is set by the user via the operation unit 70 by selecting from a plurality of values ​​set in advance.

[0038] The system memory 52 is composed of a RAM etc., and develops constants and variables for the operation of the system control unit 50, programs read from the non-volatile memory 56, etc. The system control unit also performs display control by controlling the memory 32, the D / A converter 13, the display unit 28 etc.

[0039] The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0040] The digital camera 100 also includes operation means, consisting of a mode change switch 60, a first shutter switch 64, a second shutter switch 62, and an operation unit 70, for inputting various operational instructions to the system control unit 50.

[0041] The mode changeover switch 60 changes the operation mode of the system control unit 50 to one of a still image recording mode, a moving image recording mode, a playback mode, etc. Modes included in the still image recording mode include an auto imaging mode, an auto scene determination mode, a manual mode, various scene modes in which imaging settings are made for each imaging scene, a program AE mode, a custom mode, etc. The mode changeover switch 60 can directly change to one of these modes included in the still image imaging mode. Alternatively, after switching to the still image imaging mode once with the mode changeover switch 60, another operating member can be used to switch to one of these modes included in the still image imaging mode. Similarly, the moving image imaging mode may also include a plurality of modes.

[0042] The first shutter switch 64 is turned on by half-pressing (image capture preparation instruction) during operation of the shutter button 61 provided on the digital camera 100, and generates a first shutter switch signal SW1. The first shutter switch signal SW1 starts operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (auto flash) processing. Note that the user can select central one-point AF processing or face AF processing as the AF processing started by the image capture preparation instruction. Here, central one-point AF processing refers to processing that performs AF on a central point in the image capture screen, and face AF processing refers to processing that performs AF on a face in the image capture screen detected by a face detection function.

[0043] The second shutter switch 62 is turned on when the shutter button 61 is fully pressed (image capture instruction) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of image capture processing operations from reading out a signal from the image capture unit 22 to writing image data to the recording medium 200 in response to the second shutter switch signal SW2.

[0044] Each operating member of the operating unit 70 is assigned a function appropriate for each situation by selecting and operating various functional icons displayed on the display unit 28, and acts as various functional buttons. The functional buttons include, for example, an end button, a back button, an image forward button, a jump button, a filter button, an attribute change button, and the like. For example, when the menu button is pressed, a menu screen in which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four directional buttons (up, down, left, right) and the SET button.

[0045] The controller wheel 73 is a rotatable operation member included in the operation unit 70, and is used together with the direction button to indicate a selection item. When the controller wheel 73 is rotated, an electric pulse signal is generated according to the amount of operation, and the system control unit 50 controls each unit of the digital camera 100 based on this pulse signal. This pulse signal makes it possible to determine the angle at which the controller wheel 73 is rotated and how many times it has rotated. The controller wheel 73 may be any operation member that can detect a rotation operation. For example, the controller wheel 73 may be a dial operation member that rotates itself in response to a rotation operation by the user to generate a pulse signal. Alternatively, the controller wheel 73 may be a so-called touch wheel, which is an operation member made of a touch sensor, and does not rotate itself but detects the rotation of the user's finger on the controller wheel 73.

[0046] The digital camera 100 further includes a power supply control unit 80, a power supply unit 40, and a recording medium I / F 18.

[0047] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, and detects whether a battery is attached, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each unit including the recording medium 200 for the necessary period.

[0048] The power supply unit 40 is made up of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like.

[0049] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0050] The digital camera 100 described above is capable of capturing images using central one-point AF or face AF. Central one-point AF is performing AF on a single point at the center of the image capture screen. Face AF is performing AF on a face in the image capture screen detected by a face detection function.

[0051] In this embodiment, the system control unit 50 determines an imaging sequence for focus bracketing imaging based on subject information, subject distance, and image contrast information. Then, based on the conditions of the determined imaging sequence, the system control unit 50 drives the imaging lens 103 and the aperture / shutter 104 to sequentially capture multiple images with different focus positions. The user can select the amount of change in focus position between adjacent captured images (focus step) from multiple preset values ​​via the operation unit 70.

[0052] Next, the focus stacking process will be described.

[0053] 3 is a flowchart for explaining the generation of a composite image in this embodiment. In step S301, the imaging unit 22 captures a plurality of images with different focus positions. In step S302, the system control unit 50 performs alignment on the plurality of images captured by the imaging unit 22 in step S801. In step S303, the image processing unit 24 performs compositing on the images after alignment to generate a composite image. Each step will be described in detail below.

[0054] FIG. 4 is a flowchart for explaining the imaging in step S301 in this embodiment.

[0055] In step S401, the system control unit 50 sets a focus position. For example, the user specifies a focus position through a touch panel provided on the operation unit 70, and specifies multiple focus positions at equal intervals before and after the focus position corresponding to the specified focus position in the optical axis direction. At the same time, the system control unit 50 determines the imaging order in order of distance at the set focus positions.

[0056] In step S402, the imaging unit 22 captures an image at the focus position that has the earliest imaging order among the focus positions set in step S401 that have not yet been imaged.

[0057] In step S403, the system control unit 50 determines whether or not imaging has been performed at all focus positions set in step S401. If imaging has been performed at all focus positions, the process in the flowchart shown in Fig. 4 is terminated, and if there is a focus position at which imaging has not yet been performed, the process returns to step S402.

[0058] FIG. 5 is a flowchart for explaining the alignment in step S302 in this embodiment.

[0059] In step S501, the system control unit 50 acquires a reference image for alignment from among the images captured by the imaging unit 22 in step S301. The reference image for alignment is the image captured first. Alternatively, since capturing images while changing the focus position causes a slight change in the angle of view between captured images, the image with the narrowest angle of view may be used.

[0060] In step S502, the system control unit 50 acquires a target image for alignment processing. The target image is an image other than the reference image acquired in step S501, and is assumed to have not yet been subjected to alignment processing. If the system control unit 50 uses the image captured earliest as the reference image, it is sufficient to acquire the target images sequentially in the order in which they were captured.

[0061] In step S503, the system control unit 50 calculates the amount of positional deviation between the reference image and the target image. An example of the calculation method is described below. First, the system control unit 50 sets a plurality of blocks in the reference image. It is preferable that the system control unit 50 sets each block to have the same size. Next, the system control unit 50 sets a search range in the target image at the same position as each block of the reference image, which is wider than the block of the reference image. Finally, the system control unit 50 calculates a corresponding point in each search range of the target image that minimizes the sum of absolute difference (Sum of Absolute Difference, hereinafter referred to as SAD) of brightness with the block of the reference image. The system control unit 50 calculates the positional deviation referred to in step S503 as a vector from the center of the block of the reference image and the corresponding point described above. In calculating the corresponding point described above, the system control unit 50 may use the sum of squared difference (SSD, hereinafter referred to as SSD) or normalized cross correlation (NCC, hereinafter referred to as NCC) in addition to SAD.

[0062] In step S504, the system control unit 50 calculates a conversion coefficient from the amount of positional deviation between the reference image and the target image. The system control unit 50 uses, for example, a projection transformation coefficient as the conversion coefficient. However, the conversion coefficient is not limited to the projection transformation coefficient, and an affine transformation coefficient or a simplified conversion coefficient including only a horizontal and vertical shift may be used.

[0063] In step S505, the image processing unit 24 performs conversion on the target image using the conversion coefficients calculated in step S504.

[0064] For example, the system control unit 50 can perform the transformation using the equation shown in equation (1).

[0065]

number

[0066] In step S506, the system control unit 50 determines whether or not alignment has been performed for all images other than the reference image. If alignment has been performed for all images other than the reference image, the process in the flowchart shown in Fig. 5 ends, and if there are images that have not yet been processed, the process returns to step S502.

[0067] FIG. 6 is a flowchart for explaining the image synthesis in step S303 in this embodiment.

[0068] In step S601, the image processing unit 24 calculates a contrast value for each image (including the reference image) after the alignment. As an example of a method for calculating the contrast value, the image processing unit 24 first calculates a luminance Y from the color signals Sr, Sg, and Sb of each pixel using the following formula (2).

[0069] Y=0.299Sr+0.587Sg+0.114Sb...Formula (2) Next, a contrast value I is calculated for a matrix L of luminance Y of 3×3 pixels using a Sobel filter as shown in the following equations (3) to (5).

[0070]

number

[0071]

number

[0072]

number

[0073] In step S602, the image processing unit 24 generates a composite map by comparing the contrast values ​​of pixels at the same positions in the respective images and calculating a composite ratio according to the magnitude of the contrast value.

[0074] An example of a specific calculation method is shown below.

[0075] To generate the composite map, the image processor 24 compares the contrast values ​​of pixels at the same position in each image and calculates a composite ratio according to the magnitude of the contrast value. Specifically, a composite ratio of 100% is assigned to the pixel with the largest contrast value among the images at the same position, and a composite ratio of 0% is assigned to other pixels at the same position. In other words, the following (Equation 6) is established.

[0076]

number

[0077] In (Equation 6), Cm(x, y) represents the contrast value calculated in step S601, Am(x, y) represents the ratio of the composite map, and generates: where m is the mth image among the multiple images with different focus positions, x is the horizontal coordinate of the image, and y is the vertical coordinate.

[0078] However, in step S602, it is necessary to adjust the synthesis ratio appropriately so that the boundary does not become unnatural. As a result, the synthesis ratio of the synthesis map for one image is not binarized between 0% and 100%, but changes continuously.

[0079] In step S603, the image processing unit 24 generates a composite image in accordance with the composite map generated in step S602.

[0080] The above is an explanation of the process for creating a composite image for depth stacking.

[0081] The necessity for depth stacking will be explained below.

[0082] FIG. 7 is a diagram showing how a subject image is formed on an imaging plane in this embodiment.

[0083] 7(a) shows a state in which a subject 701 is imaged as an image 704 on a surface 703a by an optical lens 702. That is, if the surface 703a and the image sensor surface of the imaging unit 104 coincide with each other, the subject 701 is imaged as a "point" on the surface 703a and recorded as a focused image.

[0084] FIG. 7(b) shows a case where the image forming plane and the image sensor plane do not coincide. When the image sensor plane 703b is located at a position different from the plane 703a shown in FIG. 7A, the subject 701 imaged by the optical lens 702 is reflected on the image sensor plane 703b as a circle of confusion 705. In this case, if the circle of confusion 705 is smaller than the allowable circle of confusion of the image sensor, the circle of confusion 705 can be regarded as equivalent to a "point" when in focus, and an image equivalent to a focused image is obtained. On the other hand, if the circle of confusion 705 is larger than the allowable circle of confusion, a blurred image is obtained on the image sensor plane 703b.

[0085] FIG. 7(c) is a side view of the above state. Here, when an image of the subject is formed at the focal point 710 and the image sensor surface is located at the position of the surface 711a, a circle of confusion diameter 712a is obtained. The circle of confusion diameter 712a at this time is smaller than the allowable circle of confusion diameter 713 of the image sensor. Therefore, the image 717 recorded by the image sensor is a focused image without blur. On the other hand, when the image sensor surface is located at the position of the surface 714a, the circle of confusion diameter 715a at this time is larger than the allowable circle of confusion diameter 713. Therefore, the image 718a on the image sensor surface 714a is a blurred image. The area indicated by the diagonal lines where the circle of confusion diameter 712a is smaller than the allowable circle of confusion diameter 713 is the depth of field 716a, and the depth of field is obtained by converting this to the subject side.

[0086] FIG. 7(d) is a diagram showing a state where the aperture is narrowed compared to FIG. 7(c). In the narrowed aperture state, the diameter of the incident light varies depending on the depth, and changes to a circle of confusion diameter 712b for surface 711b and a circle of confusion diameter 715b for surface 714b. At this time, the circle of confusion diameter 715b in FIG. 7(d) is smaller than the circle of confusion diameter 715a in FIG. 7(c). Therefore, the image 718b obtained at this time has a smaller amount of blur than the image 718a. Also, the depth of field 716b at this time is deeper than the depth of field 716a.

[0087] However, narrowing the aperture increases the depth of field, but there is a limit due to the optical system of the imaging device. Also, narrowing the aperture increases the depth of field, but the sense of resolution decreases. Therefore, when taking an image, the photographer may want to limit the aperture depending on the scene he or she wants to capture. In either case, it can be said that there is an upper limit to the aperture value that can be used in imaging. If you want to obtain an image with a deeper depth of field than the depth of field when the aperture value reaches the upper limit, the depth of field function becomes useful.

[0088] Next, the setting of the image capture mode in this embodiment will be described.

[0089] FIG. 8 is a flowchart for explaining the setting of the image capture mode in this embodiment.

[0090] First, in step S801, before an image capturing operation, the user sets an aperture value. For example, the user rotates the controller wheel 73 to sequentially change the aperture value, while looking at the subject image displayed in live view on the display unit 28, and determines an aperture value suitable for capturing an image.

[0091] Next, in step S802, the system control unit 50 compares the F-number corresponding to the aperture value set in step S801 with a threshold value and determines the imaging mode. The imaging mode here is a focus stacking mode or a mode for capturing one image.

[0092] FIG. 9 is a diagram for explaining the setting of the aperture value in this embodiment. The letters at the bottom of FIG. 9(a) to (d) respectively indicate the imaging mode, shutter speed, F-number, and ISO sensitivity. It is assumed that a user fixes the ISO sensitivity to 100 and captures images while changing the aperture value. First, in FIG. 9(a), the user sets the F-number to F16. The digital camera 100 automatically sets the shutter speed to 1 / 100. Next, in FIG. 9(b), the user sets the F-number to F22. The digital camera 100 automatically sets the shutter speed to 1 / 50.

[0093] Here, the upper limit (threshold) of the F-number of the digital camera 100 is set to 22. The upper limit (threshold) of the F-number here may be determined by the performance of the optical system, or may be preset by the user for resolution or the like. Although it is assumed that the threshold is set to an aperture value that can be realized by the aperture of the lens 103, the threshold may be determined under different conditions. For example, in order to avoid diffraction blur at a small aperture, the threshold may be set to an aperture value at a stage where the influence of diffraction blur is small. In the case of an interchangeable lens digital camera, the information of the threshold is stored in the interchangeable lens itself, and the threshold is obtained and set from the lens. Alternatively, it is also possible to store the lens and threshold conditions on a Web server, and obtain and set information of the attached interchangeable lens from the Web server.

[0094] Even if the user tries to set the F-number to a value narrower than F22, capturing a single image will not result in an image with an F-number narrower than F22. Therefore, if you want to capture an image with an F-number narrower than F22, you need to perform focus stacking here.

[0095] FIG. 9(c) shows a live view image displayed when the user narrows the aperture further than in FIG. 9(b). The characters indicating the imaging mode change from Dep to Dep+, indicating that the focus stacking mode has been entered. The characters indicating the F-number become underlined F27.6, indicating that this is a virtual F-number obtained by calculation. The F-number used when the imaging unit 22 actually captures an image is not F27.6. FIG. 9(d) shows a live view image when the aperture is narrowed further than in FIG. 9(c).

[0096] 8, if the F-number set in step S801 is greater than the threshold, the system control unit 50 proceeds to step S803 to calculate a virtual F-number, and if not, proceeds to step S806. In step S806, the display unit 28 displays the F-number set in step S801, and in step S807, the imaging unit 22 captures one image with the F-number set in step S801.

[0097] FIG. 10 is a diagram for explaining calculation of the virtual F value in this embodiment.

[0098] In the scene shown in FIG. 10, the relationship in (Equation 7) below holds.

[0099]

number

[0100] In step S805, the system control unit 50 performs a process of depth stacking. The process of depth stacking is the process of the flow shown in FIG. 3 described above. The composite image generated by depth stacking has the same depth of field as the image obtained if it could be captured with the aperture value set in step S801. Or, even if it is difficult to achieve a completely identical depth of field, it is desirable that the depth of field of the composite image includes the depth of field of the image obtained if it could be captured with the aperture value set.

[0101] However, the above implementation method is merely an example, and can be implemented in various modifications. For example, in the above description, the images used for depth stacking have the same depth of field, but this is not limited to this. For example, if the aperture value set by the user in step S801 corresponds to F25 and the F-number threshold is 22, the digital camera 100 can perform depth stacking as follows. First, the digital camera 100 captures an image at F22. Next, the digital camera 100 opens the aperture and captures another image at, for example, F5.6. Next, the digital camera 100 composites the two captured images to generate a composite image with the same depth of field as when captured at F27.6.

[0102] According to this embodiment, when digital camera 100 is unable to obtain an image with the aperture value set by the user in a single imaging operation, it automatically enters focus stacking mode and can obtain a composite image with the aperture value set by the user.

[0103] Second Embodiment In the second embodiment, unlike the first embodiment, the imaging mode is selected taking into consideration the shutter speed as well.

[0104] Generally, when the shutter speed Tv becomes longer than (1 / focal length f), there is a concern that image quality may be degraded due to camera shake. When the aperture is operated to increase the depth of field, the shutter speed Tv increases in order to maintain the amount of light, and as a result, the contrast of the captured image decreases due to the effects of camera shake. In this embodiment, in addition to the first embodiment, a process of switching between image capture modes will be described, taking into consideration the effects of camera shake.

[0105] FIG. 11 is a flowchart for explaining the setting of the image capture mode in this embodiment.

[0106] The processes in steps S1101 to S1107 are similar to those in steps S701 to S707 in the first embodiment.

[0107] In step S1110, the system control unit 50 compares the shutter speed with 1 / focal length f. If the shutter speed is equal to or greater than 1 / focal length f, the system control unit 50 determines that focus stacking is appropriate, and proceeds to step S1103. Conversely, if the shutter speed is smaller than 1 / focal length f, the system control unit 50 does not set the focus stacking mode even if the set F-number is greater than the threshold value.

[0108] According to this embodiment, when a deterioration in image quality due to camera shake is expected, taking into consideration the shutter speed, it is possible to avoid setting the focus stacking mode and to prevent unnecessary imaging.

[0109] The present invention can also be realized by a process in which a program for realizing one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and run the program. Also, the present invention can be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0110] The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main characteristics. [Explanation of symbols]

[0111] 15 Memory control section 18 Interfaces 22 Imaging unit 23 A / D converter 24 Image Processing Section 28 Display section 30 Power supply section 32 Memory 40 Exposure control section 42 Focus control section 50 System control section 51 Arithmetic section 52 System Memory 53 System Timer 54 Image synthesis unit 56 Non-volatile memory 70 Operation section 80 Power supply control unit 100 Imaging device 101 Shutter 102 Barrier 103 Lens 200 Recording media

Claims

1. A setting means for setting a first aperture value; an imaging means for capturing an image based on the aperture value set by the setting means; a display means for displaying a second aperture value; A control means, When the first aperture value does not satisfy a predetermined condition, the imaging means performs the imaging at the first aperture value, When the first aperture value satisfies the predetermined condition, the control means calculates, as the second aperture value, an aperture value of a composite image to be generated when the imaging means captures a plurality of images having different focus positions and composites the plurality of images, and An imaging device, characterized in that the aperture value of the composite image satisfies the predetermined condition that the F-number value corresponding to the aperture value is greater than the largest F-number value that the imaging means can capture.

2. 2. The imaging apparatus according to claim 1, wherein the display means displays the first aperture value as the second aperture value when the first aperture value does not satisfy a predetermined condition.

3. 3. The imaging apparatus according to claim 1, wherein the display means displays the second aperture value before the imaging means captures the image.

4. 4. The imaging apparatus according to claim 1, wherein the display unit displays the second aperture value together with a live view image.

5. When the first aperture value satisfies the predetermined condition, the control means controls the imaging means to capture a predetermined number of the plurality of images; 5. The imaging device according to claim 1, wherein the predetermined number is set to the smallest number that satisfies the predetermined condition when the plurality of images are combined to generate a composite image.

6. 6. The imaging apparatus according to claim 1, wherein the control means generates the composite image by extracting in-focus areas from the plurality of images.

7. 7. The imaging device according to claim 1, wherein the composite image has a depth of field that would be achieved if the image were captured with the first aperture value.

8. 7. The imaging device according to claim 1, wherein the depth of field of the composite image includes a depth of field that would be achieved if the image were captured with the first aperture value.

9. 9. The imaging device according to claim 1, wherein, when the first aperture value does not satisfy a predetermined condition, the imaging means captures one image at the first aperture value.

10. Even if the first aperture value satisfies the predetermined condition, when the shutter speed when the image capturing means captures the image is slower than a predetermined speed, 10. The imaging apparatus according to claim 1, wherein the imaging means performs the imaging operation at the first aperture value.

11. 11. The imaging apparatus according to claim 1, wherein the plurality of images have different focus positions in the optical axis direction.

12. a setting step of setting a first aperture value; an imaging step of capturing an image based on the aperture value set in the setting step; a display step of displaying a second aperture value; A control step, When the first aperture value does not satisfy a predetermined condition, the imaging step performs the imaging at the first aperture value; When the first aperture value satisfies the predetermined condition, in the control step, a plurality of images having different focus positions are captured in the imaging step, and an aperture value of a composite image to be generated when the plurality of images are composited is calculated as the second aperture value; An imaging method, characterized in that the aperture value of the composite image satisfies the predetermined condition that the F-number value corresponding to the aperture value is greater than the largest F-number value that can be captured in the imaging step.

13. A computer program for causing a computer of an image processing device to operate, a setting step of setting a first aperture value; an imaging step of capturing an image based on the aperture value set in the setting step; a display step of displaying a second aperture value; A control step; When the first aperture value does not satisfy a predetermined condition, the imaging step performs the imaging at the first aperture value; When the first aperture value satisfies the predetermined condition, in the control step, a plurality of images having different focus positions are captured in the imaging step, and an aperture value of a composite image to be generated when the plurality of images are composited is calculated as the second aperture value; The program is characterized in that the aperture value of the composite image satisfies the predetermined condition that the F-number value corresponding to the aperture value is greater than the largest F-number value that can be captured in the imaging step.

14. A computer-readable recording medium having the program according to claim 13 recorded thereon.

Images