Image pickup apparatus, control method thereof, storage medium, and computer program product

Abstract

Provided are an imaging apparatus capable of obtaining a captured image focused on a desired portion, a control method therefor, a storage medium, and a computer program product. The camera device comprises at least one processor and / or circuit. And an imaging unit configured to serve as a unit for detecting predetermined subjects from an image obtained by the imaging unit, a unit for hierarchically detecting feature regions of each of the predetermined subjects, a unit for selecting two subjects having different distances from the imaging unit in a depth direction, a unit for determining a focus region, and a unit for determining the focus region. A means for maintaining an aperture value and a focus position of a portion corresponding to a focus region within a depth-of-field range is provided, and a means for predicting the positions of the two subjects after a predetermined period of time has elapsed on the basis of the subject image surface position obtained by past focus detection and the defocus amount obtained by the nearest focus detection.

Description

Technical Field

[0001] This disclosure relates to camera equipment, control methods for camera equipment, and storage media. Background Technology

[0002] Typically, when photographing multiple subjects at different positions along the depth direction, or subjects that are long along the depth direction, the aperture of the lens device is usually narrowed (the aperture value (F-number) is usually increased) to widen (increase) the depth of field. While this method increases the depth of field, if the aperture is narrowed too much, it will impair the out-of-focus effect. Furthermore, when attempting to capture a moving subject clearly, it may be impossible to narrow the aperture to the desired value because a faster shutter speed (shorter shutter time) is needed to achieve the required exposure.

[0003] In view of this, Japanese Patent No. 6253454 discloses a camera device equipped with a scene mode suitable for capturing multiple moving subjects. Specifically, in the technology (camera device) disclosed in Japanese Patent No. 6253454, firstly, feature portions of multiple subjects detected in an image obtained from a camera optical system are detected, and the positions of the multiple feature portions after a predetermined time period are predicted by comparing the changes in the positions of the multiple feature portions in the depth direction between the latest image and an image immediately preceding the latest image. Then, in the technology disclosed in Japanese Patent No. 6253454, a depth of field is set that enables the multiple feature portions to be in focus within the image, based on the predicted position information for the multiple feature portions and the focal length of the camera optical system.

[0004] In the technology disclosed in Japanese Patent No. 6253454, depth of field has been set relative to the subject at different positions in the depth direction. In this case, in the technology disclosed in Japanese Patent No. 6253454, since multiple parts of the subject are not identified in layers, it may be impossible to obtain a photographic image that is reliably focused on the part desired by the photographer. Summary of the Invention

[0005] This disclosure provides a camera device capable of obtaining a captured image focused on the portion desired by the photographer when capturing multiple subjects at different positions in the depth direction, a control method for the camera device, and a storage medium.

[0006] Therefore, a first aspect of this disclosure provides a camera device including at least one processor and / or circuitry configured to function as: a subject detection unit that detects a predetermined subject from an image obtained by a camera unit including a camera optical system; a region detection unit that detects characteristic regions of each of the predetermined subjects in a layered manner; a selection unit that selects two subjects from the predetermined subjects that are at different distances from the camera unit in the depth direction; a determination unit that determines a focus region from the layered regions detected for each of the two subjects; a control unit that sets an aperture value and a focus position that keep portions of the two subjects corresponding to the focus region selected by the determination unit within the depth of field; and a prediction unit that predicts the position of the two subjects in the depth direction after a predetermined time period based on the subject image plane position obtained by past focus detection for the two subjects and the amount of defocus obtained by most recent focus detection for the two subjects.

[0007] Therefore, a second aspect of this disclosure provides a camera device including at least one processor and / or circuitry configured to function as: a subject detection unit that detects a predetermined subject from an image obtained by a camera unit including a camera optical system; a region detection unit that detects characteristic regions of each of the predetermined subjects in a layered manner; a selection unit that selects two subjects from the predetermined subjects that are at different distances from the camera unit in the depth direction; a determination unit that determines a focus region from the regions detected layered for each of the two subjects; and a control unit that shifts the focal position when focusing on the focus region of the closer subject of the two subjects toward the farther subject side and obtains the minimum aperture value of the two subjects within the depth of field.

[0008] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is given by way of example. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating a schematic structure of a camera device according to an embodiment.

[0010] Figure 2A and Figure 2B This is a plan view used to explain the structure of the camera device included in the camera equipment. Figure 2C It is a cross-sectional diagram used to explain the structure of the camera device.

[0011] Figure 3 This diagram is used to explain the relationship between the camera device and the exit pupil surface.

[0012] Figure 4 This is a flowchart of the shooting operation performed by the camera equipment according to the first embodiment.

[0013] Figure 5 Is Figure 4 The flowchart of the subject determination process performed in S102.

[0014] Figure 6A and Figure 6B This is an example in Figure 5 A schematic diagram of the layered detection of feature regions performed in S201.

[0015] Figure 7 Is Figure 4 The flowchart shows the aperture and focus position adjustment process performed in S105.

[0016] Figure 8A and Figure 8B This is an example image illustrating a shooting scene.

[0017] Figure 9 This is an example Figure 8A and Figure 8B The diagram shows the relationship between F-number, focus position, and depth of field in the shooting scene shown.

[0018] Figure 10 This is a flowchart of the shooting operation performed by the camera equipment according to the second embodiment.

[0019] Figure 11A and Figure 11B This is another example of a shooting scene.

[0020] Figure 12A , Figure 12B and Figure 12C This is an example in Figure 10 The graph shows the moving body prediction processing performed in S605.

[0021] Figure 13 Is Figure 10 The flowchart shows the moving body prediction process performed in S605.

[0022] Figure 14 Is Figure 10 The flowchart shows the aperture and focus position adjustment process performed in S607.

[0023] Figure 15 This is an example Figure 11A and Figure 11B The graph shows the relationship between the F-number and depth of field in the shooting scene shown. Detailed Implementation

[0024] The present disclosure will now be described in detail below with reference to the accompanying drawings, which illustrate embodiments of the present disclosure.

[0025] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating a schematic structure of a camera device 100 according to an embodiment.

[0026] The video recording device 100 is a so-called digital camera, and it has still image capture and moving image capture functions. For example... Figure 1 As shown, the camera device 100 includes a camera optical system, which includes a first lens group 101, an aperture 102, a second lens group 103, a third lens group 104, an optical low-pass filter 106, a zoom actuator 111, an aperture actuator 112, and a focus actuator 114.

[0027] The first lens group 101 is positioned closest to the subject in the imaging optical system (at the front of the imaging device 100) so that it can move along the imaging optical axis. The aperture (stop) 102 adjusts the amount of light passing through the imaging optical system by adjusting its diameter. The second lens group 103 is integrally configured with the aperture 102 so that it can be moved in the direction of the imaging optical axis by an actuator (not shown), and cooperates with the first lens group 101 to achieve a zoom function. The third lens group 104 is a so-called focusing lens, and its focus is adjusted by moving in the direction of the imaging optical axis. The optical low-pass filter 106 is an optical element used to reduce false colors and moiré patterns in the captured image (image). The zoom actuator 111 is a drive unit that moves the first lens group 101 in the direction of the imaging optical axis. The aperture actuator 112 drives the aperture 102 to adjust the aperture value (F-number). The focusing actuator 114 is a driving component (driving unit) that moves the third lens group 104 in the direction of the imaging optical axis.

[0028] The camera device 100 includes a camera (image sensor) 107, a shutter 108, a system control unit 121, a flash control unit 122, an auxiliary light drive unit 123, a camera drive unit 124, an image processing unit 125, a focus drive unit 126, an aperture drive unit 128, and a zoom drive unit 129. Additionally, the camera device 100 includes a flash unit 115, an AF auxiliary light emitting unit 116, a display unit 131, an operation unit 132, a storage medium 133, a subject detection unit 140, a dictionary data storage unit 141, and a focus area determination unit 142.

[0029] The imaging device 107 is, for example, composed of a two-dimensional CMOS sensor and its peripheral circuitry. The imaging device 107 converts the optical image, which passes through the imaging optical system and is formed on the imaging surface of the imaging device 107, into an analog electrical signal, and outputs the generated analog electrical signal to the system control unit 121 via the imaging device drive unit 124. The system control unit 121 converts the analog electrical signal into a digital image signal and provides the digital image signal to one or more functional units that require the digital image signal. The shutter 108 controls the exposure time of the imaging device 107 to incident light.

[0030] The system control unit 121 includes a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), an A / D converter, a D / A converter, communication interface circuitry, etc., all of which are not contained within... Figure 1 As shown, the system control unit 121 comprehensively controls the various units of the camera device 100 by loading a predetermined program already stored in the ROM into the RAM. It should be noted that some functions of the system control unit 121 can be implemented as hardware circuitry, and some circuitry can use reconfigurable circuitry, such as a field-programmable gate array (FPGA). Furthermore, in this embodiment, the system control unit 121 performs calculations for focus detection, which will be described below; however, to shorten the calculation time, a structure in which some calculations are performed by dedicated hardware circuitry can be employed.

[0031] The camera device drive unit 124 controls the operation of the camera device 107 according to commands from the system control unit 121. It should be noted that an A / D conversion function for converting the analog electrical signals output from the camera device 107 into digital image signals can be provided in the camera device drive unit 124, rather than in the system control unit 121. The image processing unit 125 obtains the digital image signals from the system control unit 121 and performs various types of image processing (such as gamma conversion, color interpolation, and compression / expansion) to generate image data.

[0032] Display unit 131 includes a liquid crystal display, an organic EL display, etc., and displays various images and information, such as information about the shooting mode, live view video images, confirmation images after shooting, and focus status during focus detection. Operation unit 132 includes a power button, a release button, a zoom operation button, a mode selection button, and a menu button. It should be noted that when the release button is half-pressed, switch SW1 (not shown) is turned on, and shooting preparation processes such as automatic exposure processing (AE processing) and automatic focus processing (AF processing) are performed. Then, when the release button is fully pressed, switch SW2 (not shown) is turned on, and a series of shooting operations from exposure of the imaging device 107 to storage of image data of the captured image are performed.

[0033] For example, storage medium 133 is a memory card such as flash memory that can be attached to and removed from camera device 100, and stores and saves image data of captured images. Focusing drive unit 126 drives focusing actuator 114 in response to a focusing drive command from system control unit 121 to move third lens group 104 in the imaging optical axis direction. Aperture drive unit 128 drives aperture actuator 112 in response to an aperture drive command from system control unit 121 to adjust the aperture value of aperture 102. Zoom drive unit 129 drives zoom actuator 111 in response to a zoom operation performed by the photographer (user) to move first lens group 101 in the imaging optical axis direction.

[0034] The subject detection unit 140 performs subject detection processing and feature region detection processing based on subject detection dictionary data (subject detection dictionary data) set by the photographer of the camera device 100 for detecting specific subjects. The subject detection dictionary data defines the features of subjects for various types of subjects (e.g., "people," "vehicles," "animals," etc.) and is stored in the dictionary data storage unit 141. In the subject detection processing, the type of subject is detected.

[0035] The user of the camera device 100 is not limited to setting one subject detection dictionary data, but can set multiple subject detection dictionary data to detect various types of subjects. When multiple subject detection dictionary data have been set, the user of the camera device 100 can set the order in which the multiple set subject detection dictionary data are used to detect subjects via the operation unit 132.

[0036] The subject detection dictionary data also defines a higher-level hierarchy corresponding to the entire area of ​​the subject and lower-level hierarchy corresponding to parts of the subject, and the number of lower-level hierarchy is not limited to one. For example, in the subject detection dictionary data "human (human body)," "whole body" is defined as the higher-level hierarchy, while multiple parts such as "upper body," "face (or head)," and "eyes" are defined as lower-level hierarchy. Furthermore, in the subject detection dictionary data "vehicle," "body" is defined as the higher-level hierarchy, while "front grille," "headlights," "tires," etc., are defined as lower-level hierarchy.

[0037] Feature region detection processing is a process of hierarchically detecting the feature portions of a detected subject to use those feature portions as the region to be focused (focus region). Specifically, in feature region detection processing, corresponding portions of the upper and lower levels already defined in the subject detection dictionary data are detected as feature regions. It should be noted that feature regions are detected as rectangular regions that include the feature portions of the subject, and preferably as rectangular regions with the smallest possible area.

[0038] It should be noted that the detection of subject type and feature regions can be performed by first detecting a portion of the higher-level region, and then detecting the lower-level region from the already detected higher-level region, and vice versa. For example, in the case of detecting a person (assuming a subject detection dictionary of "person" has already been set), it is easy to detect "face" as a lower-level feature. Therefore, after detecting "face", the entire person can be detected (estimated) from the location of "face", and features such as "eyes" and "mouth" can be detected from the region of "face".

[0039] The focus region determination unit 142 determines a focus region for each subject from the feature regions detected by the feature region detection process for each subject that has been detected in the subject detection process.

[0040] Next, the pixel structure of the camera device 107 will be described. Figure 2A This is a plan view illustrating the pixel array (pixel arrangement) of the camera device 107. Figure 2A The z-axis shown is the same as the camera optical axis (refer to). Figure 1 Parallel. The camera device 107 has a generally rectangular camera surface on which a large number of 2-column × 2-row pixel groups 200 are arranged in the x and y directions, which are perpendicular to each other and also perpendicular to the z-axis, and each pixel outputs a camera signal. It should be noted that Figure 2A Only a range of 4 columns × 4 rows is shown. Figure 2B yes Figure 2A The diagram shown is a 200G pixel plan view. Figure 2C It is along Figure 2B The cross-sectional view shown by arrow AA.

[0041] like Figure 2A As shown, the 2-column × 2-row pixel group 200 includes a pixel 200R with R (red) spectral sensitivity, two pixels 200G with G (green) spectral sensitivity, and a pixel 200B with B (blue) spectral sensitivity, arranged in a Bayer array. Each pixel in pixels 200R, 200G, and 200B has two focus detection pixels 201 and 202 arranged in 2 columns and 1 row.

[0042] In this embodiment, a pixel consisting of two focus detection pixels arranged in the x-direction is discussed; however, this disclosure is not limited thereto. The two focus detection pixels may be arranged in the y-direction, and the number of focus detection pixels in a pixel is not limited to two. The imaging device 107 may have a structure that combines pixels, each having multiple focus detection pixels, and pixels for image generation (image generation pixels), each having one pixel.

[0043] Since pixels 200G, 200B, and 200R have the same structure except for their light-splitting characteristics, the structure of pixel 200G will be described in detail below as an example, while the descriptions of pixels 200B and 200R will be omitted.

[0044] like Figure 2C As shown, a microlens 305 for collecting incident light is disposed on the light-receiving surface of a semiconductor substrate (such as a silicon substrate), on which a photodiode (PD) of a pixel 200G is formed. A microlens 305 is arranged for each pixel at a predetermined distance from the light-receiving surface in the z-direction. Furthermore, within a pixel 200G, two PDs (i.e., photoelectric conversion units 301 and 302) are formed, divided into two parts in the x-direction. The two photoelectric conversion units 301 and 302 are respectively connected to… Figure 2A The focus detection pixels 201 and 202 shown correspond to each other. It should be noted that when each pixel is divided into Nx in the x-direction and Ny in the y-direction, the number of photoelectric conversion units to be formed is Nx × Ny = NLF (the number of divisions).

[0045] Each photoelectric conversion unit in photoelectric conversion units 301 and 302 is formed as a pn junction PD consisting of a p-type layer and an n-type layer. It should be noted that each photoelectric conversion unit in photoelectric conversion units 301 and 302 can be configured as a PD with a pin structure, in which an intrinsic layer is disposed between the p-type layer and the n-type layer as needed.

[0046] In pixel 200G, a color filter 306 is disposed between the microlens 305 and the photoelectric conversion units 301 and 302. If necessary, the spectral transmittance of the color filter 306 can be changed for each pixel or each photoelectric conversion unit, and a structure without a color filter can also be used.

[0047] Light incident on pixel 200G is collected by microlens 305, split by color filter 306, and then incident on photoelectric conversion units 301 and 302. In photoelectric conversion units 301 and 302, electron-hole pairs are generated according to the amount of incident light (the amount of light received), and after the electron-hole pairs separate in the depletion layer, electrons (negative charges) accumulate in the n-type layer. On the other hand, holes (positive charges) are discharged to the outside of the imaging device 107 through a p-type layer connected to a constant voltage source (not shown). Electrons accumulated in the n-type layers of photoelectric conversion units 301 and 302 are transferred to an electrostatic capacitor unit (FD) via a transfer gate, where they are converted into a voltage signal.

[0048] Figure 3 When viewed from the +y side Figure 2B The diagram shows a cross-sectional view of the arrow AA section of pixel 200G, and also a view showing the pupil surface at a predetermined distance from the imaging surface of the imaging device 107 in the z-direction. It should be noted that... Figure 3 In order to correspond to the coordinate axes of the exit pupil plane, the x-axis and y-axis of the cross-sectional view are relative to... Figure 2A , Figure 2B and Figure 2C Reverse.

[0049] The imaging surface of the imaging device 107 is disposed on the imaging surface of the imaging optical system. In other words, incident light forms an image on the imaging surface. When photoelectric conversion units 301 and 302 are combined, the two pupil segmentation regions constituting the pupil region 500 capable of receiving light through the entire pixel 200G are respectively referred to as the first pupil segmentation region 501 and the second pupil segmentation region 502. The first pupil segmentation region 501 and the light-receiving surface of the photoelectric conversion unit 301, whose centroid position is off-center in the -x direction, are approximately conjugate via microlens 305. Therefore, the first pupil segmentation region 501 corresponds to the pupil region capable of receiving light through the focus detection pixel 201, and the centroid position of the first pupil segmentation region 501 is off-center towards the +x side on the pupil surface. Similarly, the second pupil segmentation region 502 and the light-receiving surface of the photoelectric conversion unit 302, whose centroid position is off-center in the +x direction, are approximately conjugate via microlens 305. Therefore, the second pupil segmentation region 502 corresponds to the pupil region that can receive light through the focus detection pixel 202, and the centroid of the second pupil segmentation region 502 is off-center to the -x side on the pupil surface.

[0050] Therefore, phase difference information is obtained by performing correlation calculations on the output signals of focus detection pixels 201 and 202, and the obtained phase difference information is converted into defocus amount by using known techniques, thereby enabling phase difference AF of the camera plane to detect the focus position.

[0051] It should be noted that the imaging signal output from the imaging device 107 is used not only as a focus detection signal for performing phase difference AF on the imaging plane, but also for generating the imaging image and capturing the image. It should be noted that, although... Figure 2A , Figure 2B , Figure 2C and Figure 3 The diagram illustrates a structure where one pixel is divided into two by the pupil in the x-direction, but the number of pupil segments and the pupil segmentation direction are not limited to these. For example, the pupil segmentation direction can be only the y-direction, or the pupil can be segmented in both the x-direction and the y-direction (2×2=4 pupil segments).

[0052] The first embodiment will be described below. Figure 4 This is a flowchart of the shooting operation performed by the camera device 100 according to the first embodiment. The shooting operation is achieved by the CPU, included in the system control unit 121, loading a predetermined program stored in ROM into RAM and comprehensively controlling the operation of each unit of the camera device 100. Figure 4 The flowchart is indicated by the number S for each process (step). When the photographer turns on the power switch (power button) of the camera device 100, the camera operation performed by the camera device 107 begins, the live view video image is displayed on the display unit 131, and the processing of S101 begins.

[0053] In S101, the system control unit 121 determines whether the switch SW1 is on (whether the release button is half-pressed). If the system control unit 121 determines that the switch SW1 is not on (no in S101), the system control unit 121 continues to execute the process of S101. On the other hand, if the system control unit 121 determines that the switch SW1 is on (yes in S101), the system control unit 121 executes the process of S102.

[0054] In S102, the system control unit 121 performs a subject determination process. This subject determination process detects the subject from the real-time viewfinder video image (frame image) through the subject detection unit 140 and determines the subject to be focused on. Here, reference will be made to... Figure 5 The subject identification process performed in S102 is described in detail.

[0055] Figure 5 Is Figure 4 The flowchart for the subject determination process performed in S102 is shown. Figure 5 As shown, in S201, the system control unit 121 causes the subject detection unit 140 to perform subject detection processing to detect subjects from the frame image. The subject detection unit 140 uses pre-set subject detection dictionary data to detect subjects from the frame image that match the features specified (defined) in the subject detection dictionary data.

[0056] The subject detection unit 140 also performs the aforementioned feature region detection processing. Figure 6A and Figure 6B This is an example in Figure 5 A schematic diagram of the layered detection of feature regions performed in S201. Figure 6A This shows a case where the subject is a human being. Figure 6B This shows a case where the subject is a vehicle (car). Figure 6A The illustration schematically shows a person 1001 being detected as a subject, with feature regions 1002 corresponding to the "whole body" at a higher level of the person 1001, and feature regions 1003 and 1004 corresponding to the "face" and "eyes" at a lower level of the person 1001. Figure 6B The illustration schematically shows a car 1005 being detected as the subject, with feature region 1006 corresponding to the "body" at a higher level than the detected car 1005, and feature region 1007 corresponding to the "front grille (front part of the body)" at a lower level than the detected car 1005. It should be noted that when photographing a person from behind, features such as the "head" can be detected at a lower level, while when photographing a vehicle (car) from the side, features such as the "side window" and "tire" can be detected at a lower level.

[0057] Since this disclosure relates to the setting of shooting parameters when multiple subjects have already been detected in the depth direction (in the depth of field direction) from the camera device 100 toward the subject, the following description will be given under the assumption that multiple subjects have already been detected in the depth direction. It should be noted that when multiple subjects have not yet been detected in the depth direction, but only one subject is detected in the depth direction, the photographer simply sets the desired aperture value and takes the picture.

[0058] In S202, the system control unit 121 selects the closest subject (the subject closest to the camera device 100), the subject located at the center of the image, or the subject with the largest area in the image from the multiple subjects detected in S201 as the first subject. It should be noted that, for example, if multiple people have been detected, the first subject can be automatically determined based on preset settings, or it can be determined by the selection made by the photographer during subject detection. For ease of description, it is assumed here that a close-range subject has already been selected as the first subject. The subject detection unit 140 may have functions for selecting the first subject and a second subject, which will be described below.

[0059] In S203, the system control unit 121 causes the focus area determination unit 142 to determine a region (focus area) to be focused from the feature regions that have been detected layer by layer for the first subject. Figure 6A In the case where feature regions 1002, 1003, and 1004 are identified as the focus region, while... Figure 6B In this case, one of the feature regions 1006 and 1007 is determined as the focus region. The determined focus region is superimposed on the live view video image being displayed on the display unit 131.

[0060] It should be noted that when multiple feature regions have been detected, any one of the multiple feature regions can be selected as the focus region, but rules can be predetermined, such as prioritizing lower-level feature regions, regardless of the type of subject.

[0061] In S204, the system control unit 121 selects a subject that is farther away than the first subject (a distant subject) from the multiple subjects detected in S201 as the second subject. Inevitably, the positional relationship between the first and second subjects becomes that of a near-field subject and a far-field subject. It should be noted that even if the subject located at the center of the image or the subject with the largest area on the image plane is selected as the first subject in S202, the second subject still needs to be a subject farther away in the depth direction than the first subject. In other words, the selection of the first subject is based on the assumption that the second subject is located further away.

[0062] In S205, the system control unit 121 causes the focus area determination unit 142 to determine the focus area from the feature areas that have been detected layer by layer for the second subject. It should be noted that the processing in S205 is performed in the same manner as the processing in S203, therefore its description will be omitted here.

[0063] In S206, the system control unit 121 calculates the depth difference (the distance in the depth direction between the first and second subjects) between the first and second subjects and ends the subject determination process. The method used to calculate the depth difference between the first and second subjects can be a known technique. As a result, the process from... Figure 4 The process in the flowchart proceeds from S102 to S103.

[0064] Return to Figure 4The flowchart is described below. In S103, the system control unit 121 controls the focusing drive unit 126 and the aperture drive unit 128 to perform AF operation relative to the first subject by using the focus area already determined in S203, while the aperture 102 is opened to its maximum value (set to its minimum F value).

[0065] In S104, the system control unit 121 determines whether a focus state relative to the first subject has been achieved through the AF operation performed in S103. Specifically, the system control unit 121 calculates the defocus amount based on the corresponding output signals of the focus detection pixels 201 and 202 (photoelectric conversion units 301 and 302), and determines that a focus state has been achieved if the calculated defocus amount is within a preset value range. If the system control unit 121 determines that a focus state relative to the first subject has not yet been achieved (no in S104), the system control unit 121 executes the process of S102; on the other hand, if the system control unit 121 determines that a focus state relative to the first subject has been achieved (yes in S104), the system control unit 121 executes the process of S105.

[0066] In S105, the system control unit 121 adjusts the aperture and focus position using the previously calculated defocus amount, thereby setting an appropriate depth of field. Here, reference will be made to... Figure 7 Describe the processing of S105. Figure 7 Is Figure 4 The flowchart shows the aperture and focus position adjustment process performed in S105.

[0067] In S103, aperture 102 is opened (F-number is set to minimum), therefore in S301, system control unit 121 narrows aperture 102 to a pre-set specified F-number (specified aperture value) (increases F-number). It should be noted that the "specified aperture value" is set according to the structure of the camera optical system and can be, for example, the upper limit of the F-number range desired by the photographer, and can be set by the photographer via operation unit 132.

[0068] In S302, the system control unit 121 determines whether the first subject and the second subject are within the depth of field. If the system control unit 121 determines that the first subject and the second subject are within the depth of field (yes in S302), the system control unit 121 executes the processing in S303.

[0069] In S303, the system control unit 121 opens the aperture 102 by a fixed amount (decreases the F-value by a fixed value). In S304, the system control unit 121 moves the focus position towards the distance side (the second subject side). In S305, the system control unit 121 determines whether the first subject and the second subject are within the depth of field. In other words, in S303 to S305, it is determined whether the state of the first subject and the second subject being within the depth of field has been maintained when the focus position is shifted towards the second subject side and the aperture 102 is gradually driven towards the open side. If the system control unit 121 determines that the first subject and the second subject are within the depth of field (yes in S305), the system control unit 121 executes the process of S303; on the other hand, if the system control unit 121 determines that the first subject and the second subject are not within the depth of field (no in S305), the system control unit 121 executes the process of S306.

[0070] In S306, the system control unit 121 sets the aperture value and focus position in the shooting conditions when the judgment result immediately preceding the judgment result of S305 was "yes" and changed to "no". If the aperture value and focus position have changed to be such that the first and second subjects are no longer within the depth of field, by returning to the previous aperture value and previous focus position, the minimum F-value for the first and second subjects within the depth of field can be determined.

[0071] In S307, the system control unit 121 confirms (determines) whether the first and second subjects are in focus. For example, if the positional relationship between the subjects and the photographer remains unchanged, since the first and second subjects should be within the depth of field in S306, the determination result of S307 becomes "yes". On the other hand, for example, if the first or second subject moves, it cannot be guaranteed that the first or second subject will be within the depth of field when processing in S306. If the system control unit 121 determines that the first and second subjects are in focus (yes in S307), the system control unit 121 terminates. Figure 7 The aperture and focus position adjustment processes shown in the flowchart are executed, and the process in S106 is performed. On the other hand, if the system control unit 121 determines that the first or second subject is not in focus (not in S307), the system control unit 121 terminates the process. Figure 7 The aperture and focus position adjustment process shown in the flowchart is executed, and the process S101 is performed.

[0072] If the system control unit 121 determines in the previous S302 that the first subject and the second subject are not within the depth of field (no in S302), the system control unit 121 executes the processing of S308.

[0073] In S308, the system control unit 121 returns the aperture 102 to the open aperture value (open value) that was the aperture value before the processing in S301, refocuses on the first subject, and then executes the processing in S106. It should be noted that if the judgment result in S302 becomes "no," and the specified aperture value already set in S301 is not the maximum value—that is, if the aperture 102 can be further narrowed—the F-value can first be changed to the maximum value, and then the processing in S302 can be executed again. This method can be used without imposing any restrictions on the F-value during shooting. Therefore, if the re-judgment result in S302 becomes "no," the processing in S308 can be executed; conversely, if the re-judgment result in S302 becomes "yes," the processing in S303 can be executed.

[0074] Furthermore, in the processing of S105, the minimum F-number for the first and second subjects within the depth of field can be determined by gradually narrowing the aperture 102 from a fully open state in S301, while simultaneously shifting the focus position towards the second subject. However, it is easier to control and determine the minimum F-number for the first and second subjects within the depth of field from the state where the aperture 102 has already been narrowed.

[0075] Return to Figure 4 The flowchart describes the process. In S106, the system control unit 121 sets the shutter speed Tv and ISO sensitivity to provide appropriate exposure for the aperture value already set in S105.

[0076] In S107, the system control unit 121 determines whether switch SW2 is on (whether the release button is fully pressed). If the system control unit 121 determines that switch SW2 is on (yes in S107), the system control unit 121 executes the process in S108. On the other hand, if the system control unit 121 determines that switch SW2 is not on (no in S107), the system control unit 121 executes the process in S109.

[0077] In S108, the system control unit 121 performs the image capture process (a series of processes from the exposure of the camera device 107 to the storage of image data), and then ends according to... Figure 4 The processing of flowcharts.

[0078] In S109, the system control unit 121 determines whether switch SW1 is on (whether the release button is half-pressed). If the system control unit 121 determines that switch SW1 is not on (no in S109), the system control unit 121 terminates the operation. Figure 4 On the one hand, when the system control unit 121 determines that switch SW1 is turned on (yes in S109), the system control unit 121 executes the process of S107.

[0079] Figure 8A This is an example diagram illustrating the shooting scene in the first embodiment. Figure 8B It shows Figure 8A The positional relationship between the camera equipment 100 and the subject in the shooting scene. Figure 8A In the shooting scene, the subject detection unit 140 has detected three people. The system control unit 121 or the photographer has selected the closest person 1101 as the first subject, and the system control unit 121 or the photographer has selected the furthest person 1102 as the second subject. The focus area determination unit 142 determines the focus area from the feature areas that have been detected layer by layer relative to the first subject 1101 and the second subject 1102. Here, since the pupils (eyes) of both person 1101 and person 1102 have been detected, the focus can be on the "pupil (eye)" which is at a lower level relative to the first and second subjects. In this case, the AF operation in S103 is performed on the feature area 1105 relative to the first subject (person 1101). Then, in S302 and S305, it is determined whether the feature area 1105 of person 1101 and the feature area 1106 of person 1102 are within the depth of field.

[0080] Figure 9 It is used for explanation Figure 8A and Figure 8B The diagram illustrates the relationship between F-number, focus position, and depth of field in the shooting scene shown. Foreground depth of field Lf and background depth of field Lr are represented by the following expressions 1 and 2, respectively, using the distance L from the camera plane to the first subject, the focal length f of the camera optical system, the circle of confusion δ, and the aperture value F. Since the focus range extends from "L-Lf" to "L+Lr", the depth of field range ΔL is represented by the following expression 3.

[0081] [Expression 1]

[0082]

[0083] [Expression 2]

[0084]

[0085] [Expression 3]

[0086]

[0087] exist Figure 9 In this configuration, the pupil (focus area) of the first subject, i.e., the eye of the first subject, is located at "0cm," while the pupil (eye) of the second subject, i.e., the eye of the second subject, is located at "70cm." When focusing on the eye of the first subject, with a focal length f of 50mm (f=50mm), a distance L from the imaging surface of the imaging device 107 to the eye of the first subject being 3m (L=3m), and an aperture value F=8, the depth of field Lr will be approximately 71cm. It should be noted that the allowable circle of confusion δ is a predetermined value based on the size of the imaging device 107. Therefore, in this configuration, it is possible to achieve a state where the eyes of the first and second subjects are focused only within the depth of field Lr.

[0088] With an aperture of f / 5.6 (F=5.6), the depth of field (Lr) is approximately 53cm, making it impossible to achieve focus between the eyes of the first and second subjects within the range of Lr alone. Therefore, a depth of field (Lf) of approximately 40cm is utilized (Lf=approximately 40cm). For example, by shifting the focus point 18cm to the far side, it is possible to achieve focus between the eyes of the first and second subjects, even with an aperture of f / 5.6 (F=5.6).

[0089] As described above, in the shooting operation according to the first embodiment, when multiple subjects are detected at different positions in the depth direction, the detected subjects are kept within the depth of field with the aperture opened as wide as possible, and then the image is taken. In this case, hierarchical feature regions are detected for each of the multiple subjects, and the focus area is determined from the detected hierarchical feature regions. As a result, a high-quality image can be obtained that is focused on the part desired by the photographer and has an appropriate defocus effect.

[0090] The second embodiment will be described below. In the first embodiment, a subject with minimal movement has been discussed, but in the second embodiment, the case where the subject is a moving object will be described.

[0091] Figure 10 This is a flowchart of the shooting operation performed by the camera device 100 according to the second embodiment. The shooting operation is achieved by the CPU, included in the system control unit 121, loading a predetermined program stored in ROM into RAM and comprehensively controlling the operation of each unit of the camera device 100. Figure 10The flowchart is indicated by the number S for each process (step). After turning on the power switch of the camera device 100, the photographer sets the still image shooting mode and also sets the AF mode to servo AF (subject tracking AF). After these settings are made, the camera operation performed by the camera device 107 begins, the live view video image is displayed on the display unit 131, and the processing of S601 begins.

[0092] The processes S601 to S603 are the same as those S101 to S103 in the first embodiment, so their description will be omitted. Figure 11A This is an example diagram illustrating the shooting scene in the second embodiment. Figure 11B It shows Figure 11A The positional relationship between the camera equipment 100 and the subject in the shooting scene. Figure 11A In the shooting scenario, during the subject determination process in S602, the subject detection unit 140 detects two cars 1301 and 1302. The system control unit 121 selects the closer car 1301 as the first subject and the farther car 1302 as the second subject. Here, it is assumed that the focus area determination unit 142 has already determined the feature area 1311 as the focus area. This feature area 1311 is the front end portion and is a lower level relative to the closer car 1301. On the other hand, since the front end portion of the farther car 1302 cannot be continuously detected because it is obscured by car 1301, it is assumed that the focus area determination unit 142 has already determined the feature area 1312, which is the entire area, as the focus area.

[0093] In S604, the system control unit 121 begins subject tracking using servo AF. The subject tracking mentioned here is performed through a known template matching process, which uses the focus region already set in S602 as a template. Specifically, it involves searching for image regions similar to the template from the continuously acquired frame images.

[0094] In S605, the system control unit 121 performs a moving object prediction calculation. This calculation predicts the subject's position in the depth direction (subject position) after a predetermined time period, based on the subject image plane position obtained in the previous S603 (through past focus detection) and the defocus amount obtained through the most recent focus detection. Focus control is then performed based on the result of the moving object prediction calculation. The processing in S605 (the moving object prediction processing performed in S605) will be described in detail here.

[0095] Figure 12A , Figure 12B and Figure 12C This is an example in Figure 10 The graph shows the moving body prediction processing performed in S605. Figure 13 Is Figure 10 The flowchart shows the moving body prediction process performed in S605. (Example) Figure 13 As shown, in S701, the system control unit 121 references historical data on the corresponding image plane positions of the first and second subjects. Here, the image plane position refers to the position of the focal point when the camera optical system (lens group) is located at the subject's focus position. Figure 12A Historical data on the image plane position of the subject is shown. It should be noted that, for reference... Figure 12A , Figure 12B and Figure 12C The description will be given in a general manner and will not limit the subject to a first subject or a second subject. Figure 12A In the text, the historical data of the subject's image plane position represents the shift of the subject's image plane position 1201 at various predetermined times obtained in the past.

[0096] In S702, the system control unit 121 references the current image plane positions of the first and second subjects. In S703, the system control unit 121 predicts the corresponding focus positions of the first and second subjects after a predetermined time based on historical data. In S704, the system control unit 121 updates the prediction curves of the corresponding focus positions of the first and second subjects. Figure 12A The predicted curve 1202 for the focal position is shown based on historical data of the subject's image plane position 1201.

[0097] A pre-prepared function is used to predict curve 1202. For example, a quadratic function f(t) represented by the following expression 4 can be used, where time t is a variable. When updating the predicted curve in S704, the values ​​of the corresponding coefficients a, b, and c of the quadratic function f(t) are updated.

[0098] [Expression 4]

[0099]

[0100] In S705, the system control unit 121 determines whether the prediction accuracy is within a predetermined range of a specified value. Prediction accuracy is a value that indicates the degree of consistency between the focal position and the prediction curve, and values ​​such as the coefficient of determination (known technique) or mean square error (known technique) can be used as prediction accuracy. Figure 12B This is a graph illustrating historical data under the condition of high prediction accuracy. Figure 12C This is a graph illustrating historical data under conditions of low prediction accuracy.

[0101] When the system control unit 121 determines that the prediction accuracy is within the specified range (e.g., ...), Figure 12B In the case of the relationship shown (Yes in S705), the system control unit 121 executes the processing of S706. In S706, the system control unit 121 updates the predicted positions of the corresponding focus positions of the first and second subjects, and ends the process. Figure 13 The flowchart shows the moving body prediction process, which then advances the process to S606.

[0102] In the judgment process of S705, the system control unit 121 determines that the prediction accuracy is not within the specified range (e.g., Figure 12C If the relationship shown in the diagram is not specified in S705, the system control unit 121 executes the processing of S707. In S707, the system control unit 121 focuses on the first subject and ends the process. Figure 13 The flowchart shows the moving body prediction process, and then the process is advanced to S608.

[0103] Return to Figure 10 The flowchart describes the process. In S606, the system control unit 121 determines whether a focus state has been achieved relative to the first subject. If the system control unit 121 determines that a focus state has been achieved relative to the first subject (yes in S606), the system control unit 121 executes the process in S607. On the other hand, if the system control unit 121 determines that a focus state has not yet been achieved relative to the first subject (no in S606), the system control unit 121 executes the process in S602.

[0104] In S607, the system control unit 121 performs processing for adjusting the aperture 102 and the focus position (aperture and focus position adjustment processing). Figure 14 Is Figure 10 The flowchart shows the aperture and focus position adjustment process performed in S607. The processes in S801 to S806 are the same as those described in the first embodiment. Figure 7 The processes S301 to S306 in the flowchart are the same, so their description will be omitted here.

[0105] In S807, the system control unit 121 determines whether the predicted positions of the first and second subjects are within the focal movement range of the imaging optical system (from the shortest shooting distance to infinity). As a result, it can be confirmed whether the first and second subjects have not yet become unfocusable. If the system control unit 121 determines that the predicted positions of the first and second subjects are within the focal movement range (yes in S807), the system control unit 121 executes the process in S808. On the other hand, if the system control unit 121 determines that the predicted positions of the first and second subjects are not within the focal movement range (no in S807), the system control unit 121 executes the process in S602.

[0106] In S808, the system control unit 121 controls the application of the defocus amount predicted by the moving body to the focus position set in S806, thereby obtaining the focus position for shooting, and then ends. Figure 14 The flowchart shows the aperture and focus position adjustment process, followed by the execution of process S608. It should be noted that "the amount of defocus obtained through prediction by the moving body" is the difference between the focus position at time point t and the focus position at time point t+Δt after a time Δt elapsed from time point t. (Refer to...) Figure 12A In this context, it would be expressed as the difference between the focal position at the "predicted" time point and the latest subject image plane position in the historical data.

[0107] Return to Figure 10 The flowchart is described below. The processes of S608 to S611 are the same as those of S106 to S109 in the first embodiment, so their description will be omitted here.

[0108] Figure 15 This is an example Figure 11A and Figure 11B The diagram illustrates the relationship between F-number and depth of field in the shooting scene shown. Assuming two cars, 1301 and 1302, each have a width of approximately 2m and a total length of approximately 4.5m, when the two cars 1301 and 1302 are competing with each other, the depth of field needs to be approximately three times their total length (approximately 13.5m). When focusing on the front of car 1301 at close range, the aperture value used to keep cars 1301 and 1302 within the background depth of field is F36. This is because the aperture and focus position can be adjusted via processing in the S607 (according to...). Figure 14The process of processing the flowchart is used to obtain the depth difference between cars 1301 and 1302, for example, thus moving the focus position 6m from the front of car 1301 to the far side. By doing so, at an aperture value of F18, the foreground depth of field and the background depth of field become 6.3m and 8.1m respectively (14.4m in total), and cars 1301 and 1302 are able to remain within the depth of field range.

[0109] In the shooting operation according to the second embodiment described above, even when multiple moving objects are detected as subjects in the depth direction, the positions of the multiple moving objects are predicted, and the multiple moving objects are captured while keeping them within the depth of field and with the aperture opened as wide as possible. In this case, hierarchical feature regions are detected for each of the multiple subjects, and the focus area is determined from the detected hierarchical feature regions. As a result, a high-quality image that is already focused on the part desired by the photographer can be obtained. Furthermore, by opening the aperture as wide as possible, the shutter speed can be increased, thereby obtaining an image with appropriate defocus effect, in which the blur of the subject has been reduced (the subject is clearly captured).

[0110] The present disclosure has been described in detail above based on its preferred embodiments, but the present disclosure is not limited to these specific embodiments, and various forms without departing from the spirit of the present disclosure are also included in the present disclosure. Furthermore, each of the above embodiments represents only one embodiment of the present disclosure, and the various embodiments can be appropriately combined.

[0111] For example, in the above embodiments, this disclosure has been described as an implementation of a digital still camera; however, this disclosure is not limited thereto and can be applied to electronic devices equipped with camera functions that acquire optical images of a subject as image data by means of a camera device (image sensor) and a camera optical system. Examples of such electronic devices equipped with camera functions (such as electronic devices equipped with cameras) include portable communication terminals (such as smartphones), tablet computers, etc., equipped with cameras.

[0112] According to this disclosure, when photographing multiple subjects at different positions in the depth direction, it is possible to obtain a photographed image focused on the part desired by the photographer.

[0113] Other embodiments

[0114] The embodiments of the present invention can also be implemented by the following method: providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0115] While this disclosure has been described with reference to embodiments, it should be understood that this disclosure is not limited to the disclosed embodiments. The scope of the appended claims should be accorded the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0116] This application claims the benefit of Japanese Patent Application No. 2024-150807, filed on September 2, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. An image pickup apparatus comprising: at least one processor and / or circuitry configured to function as: a subject detection unit that detects a predetermined subject from an image obtained by an image pickup unit including an image pickup optical system; a region detection unit that hierarchically detects a characteristic region of each predetermined subject of the predetermined subject; a selection unit that selects two subjects different in distance from the image pickup unit in a depth direction from among the predetermined subjects; a determination unit that determines a focus region from regions hierarchically detected for each of the two subjects; a control unit that sets an aperture value and a focus position that keep a portion corresponding to the focus region selected by the determination unit of the two subjects within a range of a depth of field; and a prediction unit that predicts positions of the two subjects in the depth direction after a predetermined period of time elapses, based on a subject image plane position obtained by a past focus detection for the two subjects and a defocus amount obtained by a most recent focus detection for the two subjects.

2. The image pickup apparatus according to claim 1, wherein the selection unit selects a close-range subject in the two subjects from among a subject closest to the image pickup unit, a subject at a center of the image, and a subject having a largest area in the image, which are detected by the subject detection unit.

3. The image pickup apparatus according to claim 2, wherein the image pickup optical system has a lens and an aperture, and the control unit sets the aperture to an open value and focuses on the close-range subject, and then determines whether the two subjects are within the range of the depth of field in a state where the aperture has been narrowed to a specified aperture value.

4. The image pickup apparatus according to claim 3, wherein in a case where the control unit determines that the two subjects are not within the range of the depth of field when the aperture has been set to the specified aperture value, the control unit sets the aperture to the open value and focuses on the close-range subject in the two subjects.

5. The image pickup apparatus according to claim 3, wherein in a case where the control unit determines that the two subjects are within the range of the depth of field when the aperture has been set to the specified aperture value, the control unit shifts the focus position to a far-range side while driving the aperture to an open side, and determines whether the two subjects are within the range of the depth of field.

6. The image pickup apparatus according to claim 5, wherein in a case where the control unit determines that the two subjects are not within the range of the depth of field, the control unit sets an aperture value and a focus distance at a time immediately before the determination in the shooting conditions.

7. The image pickup apparatus according to claim 6, wherein the control unit sets a shutter speed and an ISO sensitivity at which appropriate exposure can be obtained during shooting using the set aperture value.

8. The image pickup apparatus according to claim 1, wherein ​ In a case where accuracy of the predicted positions of the two subjects in the depth direction obtained by the prediction unit is not within a range of a predetermined value, the control unit focuses on a close-range subject among the two subjects.

9. The imaging apparatus according to any one of claims 1 to 8, wherein the at least one processor and / or circuitry is configured to further function as a storage unit that stores dictionary data in which a type of a specific subject, a higher-level corresponding to an entirety of the specific subject, and a portion corresponding to a part of the specific subject as a lower-level have been defined, the subject detection unit detects a subject belonging to the type defined in the dictionary data, and the region detection unit detects, from the image, a region including the entirety of the specific subject that has been defined as the higher-level and a region including the portion that has been defined as the lower-level as the characteristic region.

10. An imaging apparatus comprising: at least one processor and / or circuitry configured to function as: a subject detection unit that detects a predetermined subject from an image obtained by an imaging unit including an imaging optical system; a region detection unit that hierarchically detects a characteristic region of each predetermined subject of the predetermined subject; a selection unit that selects two subjects different in distance from the imaging unit in a depth direction from among the predetermined subject; a determination unit that determines a focus region from regions hierarchically detected for each of the two subjects; a control unit that shifts a focus position at the time of focusing on a focus region of a close-range subject among the two subjects to a far-range subject side, and obtains a minimum aperture value within a range of a depth of field of the two subjects.

11. A control method for an imaging apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an imaging unit; a step of hierarchically detecting a characteristic region of each predetermined subject of the predetermined subject; a step of selecting two subjects different in distance from the imaging unit in a depth direction from among the predetermined subject; a step of determining a focus region from regions hierarchically detected for each of the two subjects; a step of setting an aperture value and a focus position that keep a portion corresponding to the focus region determined for each of the two subjects within a range of a depth of field; and a step of predicting positions of the two subjects in the depth direction after a predetermined period of time elapses, based on a subject image plane position obtained through past focus detection for the two subjects and a defocus amount obtained through most recent focus detection for the two subjects.

12. A control method for an imaging apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an imaging unit; a step of hierarchically detecting a characteristic region of each predetermined subject of the predetermined subject; a step of selecting two subjects different in distance from the imaging unit in a depth direction from among the predetermined subject; a step of determining a focus area from regions detected hierarchically for each of the two subjects; and a step of shifting a focus position at the time of focusing on the focus area of the near subject of the two subjects to the far subject side, and obtaining a minimum aperture value of the two subjects within a range of a depth of field.

13. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method for an image pickup apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an image pickup unit; a step of detecting a characteristic region of each predetermined subject of the predetermined subject hierarchically; a step of selecting two subjects different in distance from the image pickup unit in a depth direction from among the predetermined subjects; a step of determining a focus area from regions detected hierarchically for each of the two subjects; a step of setting an aperture value and a focus position that keep a portion corresponding to the focus area determined for each of the two subjects within a range of a depth of field; and a step of predicting positions of the two subjects in the depth direction after a predetermined period of time elapses, based on a subject image plane position obtained through past focus detection for the two subjects and a defocus amount obtained through most recent focus detection for the two subjects.

14. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method for an image pickup apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an image pickup unit; a step of detecting a characteristic region of each predetermined subject of the predetermined subject hierarchically; a step of selecting two subjects different in distance from the image pickup unit in a depth direction from among the predetermined subjects; a step of determining a focus area from regions detected hierarchically for each of the two subjects; and a step of shifting a focus position at the time of focusing on the focus area of the near subject of the two subjects to the far subject side, and obtaining a minimum aperture value of the two subjects within a range of a depth of field.

15. A computer program product comprising a program for causing a computer to execute a control method for an image pickup apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an image pickup unit; a step of detecting a characteristic region of each predetermined subject of the predetermined subject hierarchically; a step of selecting two subjects different in distance from the image pickup unit in a depth direction from among the predetermined subjects; a step of determining a focus area from regions detected hierarchically for each of the two subjects; a step of setting an aperture value and a focus position that keep a portion corresponding to the focus area determined for each of the two subjects within a range of a depth of field; and a step of predicting positions of the two subjects in the depth direction after a predetermined period of time elapses, based on a subject image plane position obtained through past focus detection for the two subjects and a defocus amount obtained through most recent focus detection for the two subjects. a step of predicting positions of the two subjects in the depth direction after a predetermined period of time elapses, based on subject image plane positions obtained through past focus detection for the two subjects and amounts of defocus obtained through most recent focus detection for the two subjects.

16. A computer program product comprising a program for causing a computer to execute a control method for an image pickup apparatus, the control method comprising: a step of detecting a predetermined subject from an image obtained by an image pickup unit; a step of hierarchically detecting a characteristic region of each of the predetermined subjects of the predetermined subject; a step of selecting two subjects different in distance from the image pickup unit in a depth direction from among the predetermined subjects; a step of determining a focus region from regions hierarchically detected for each of the two subjects; and a step of shifting a focus position at the focus region focused on a near distance subject of the two subjects to a far distance subject side and obtaining a minimum aperture value of the two subjects within a range of a depth of field.

Images