Imaging device, control method for imaging device, and storage medium

Abstract

The present invention provides a photographing apparatus, a control method thereof, and a storage medium. The photographing apparatus includes: a photographing unit configured to photograph an image of a subject; a detection unit configured to detect the subject from the image photographed by the photographing unit; a setting unit configured to set a plurality of focus detection frames within a screen based on a detection result of the detection unit; a focus detection unit configured to detect a focus state and its reliability within each of the plurality of focus detection frames; and a selection unit configured to select a main focus detection frame for performing focus adjustment based on detection results of the detection unit and the focus detection unit, and the selection unit causes a method for selecting the main focus detection frame to vary according to a subject part detected by the detection unit.

Description

Technical Field

[0001] The present invention relates to focus adjustment technology in imaging devices. Background Art

[0002] In recent years, various AF methods (autofocus methods) using image sensors, such as the imaging plane phase difference AF method and the contrast AF method, have been put into practical use. In addition, for such AF methods, technology for achieving focusing by identifying the region of a main subject has been established.

[0003] As this method, Japanese Unexamined Patent Application Publication No. 2010-191073 performs control to detect adjacent blocks falling within a predetermined depth from among a plurality of AF frames, and selects a main AF frame from the blocks.

[0004] In addition, Japanese Unexamined Patent Application Publication No. 2015-041901 improves the recognition accuracy of the main subject region by using color information in addition to detecting blocks falling within a predetermined depth.

[0005] Furthermore, Japanese Unexamined Patent Application Publication No. 2019-121860 discloses a method that detects organs such as pupils included in a face, determines the reliability of detection results related to the detected organs, and sets a focus detection region in a region with high detection reliability. In this way, focusing can be performed in a state where the main subject has been recognized with higher accuracy.

[0006] However, since the technologies described in Japanese Unexamined Patent Application Publication No. 2010-191073, Japanese Unexamined Patent Application Publication No. 2015-041901, and Japanese Unexamined Patent Application Publication No. 2019-121860 detect the region of the main subject with high accuracy and set the focus detection region based on the detection results, focus detection is not always performed in a region suitable for focus detection. If the region determined to be the main subject is not a region suitable for focus detection, accurate focus adjustment may not be possible. Summary of the Invention

[0007] The present invention has been made in view of the above problems, and accurately performs focus adjustment in a region with a higher priority while avoiding a region where it is difficult to perform focus detection.

[0008] According to a first aspect of the present invention, there is provided an imaging device including: an imaging unit configured to capture an image of a subject; a detection unit configured to detect the subject from the image captured by the imaging unit; a setting unit configured to set a plurality of focus detection frames in the image based on a detection result of the detection unit; a focus detection unit configured to detect a focus state and its reliability within each of the plurality of focus detection frames; and a selection unit configured to select a main focus detection frame for performing focus adjustment based on detection results of the detection unit and the focus detection unit, wherein the selection unit is configured such that a method for selecting the main focus detection frame varies according to a part of the subject detected by the detection unit.

[0009] According to a second aspect of the present invention, there is provided a control method for an imaging device, the imaging device including an imaging unit for capturing an image of a subject, the control method including: detecting the subject from the image captured by the imaging unit; setting a plurality of focus detection frames in the image based on a detection result in the detection; performing focus detection to detect a focus state and its reliability within each of the plurality of focus detection frames; and selecting a main focus detection frame for performing focus adjustment based on detection results in the detection and the focus detection, wherein the selection is such that a method for selecting the main focus detection frame varies according to a part of the subject detected in the detection.

[0010] According to a third aspect of the present invention, there is provided a computer-readable storage medium storing a program for causing a computer to execute each step of the above control method.

[0011] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram showing the structure of a digital camera as an embodiment of the imaging device of the present invention.

[0013] Figure 2 is a flowchart showing the operation of a digital camera in an embodiment.

[0014] Figure 3 is a flowchart for describing an operation of setting an AF frame.

[0015] Figure 4A and Figure 4B is a diagram showing a concept of a detection area related to a human face.

[0016] Figure 5A and Figure 5BIt is a diagram showing the concept of a detection area related to the pupil, face, and body of a human.

[0017] Figure 6A and Figure 6B It is a diagram showing the concept of a detection area related to the pupil, face, and body of an animal.

[0018] Figure 7A and Figure 7B It is a flowchart showing the AF operation in a digital camera.

[0019] Figure 8 It is a flowchart showing the focus detection process.

[0020] Figure 9 It is a flowchart showing the operation for selecting the AF main frame.

[0021] Figure 10 It is a flowchart showing the operation for selecting the AF main frame in a way that prioritizes the detection center.

[0022] Figure 11 It is a flowchart showing the operation for selecting the AF main frame in a way that prioritizes the nearest subject in the central area.

[0023] Figure 12 It is a flowchart showing the operation for selecting the AF main frame in a way that prioritizes the reliability of the detection area.

[0024] Figure 13 It is a flowchart showing the operation for selecting the AF main frame in a way that prioritizes the reliability of the central area.

[0025] Figure 14 It is a flowchart showing the operation for selecting the AF main frame in a way that prioritizes prediction in the central area. Detailed Description of the Preferred Embodiment

[0026] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. In the embodiments, multiple features are described, but the invention does not require all such features, and multiple such features can be appropriately combined. Further, in the drawings, the same or similar structures are given the same reference numerals, and redundant descriptions thereof are omitted.

[0027] Figure 1 It is a block diagram showing the structure of a digital camera 300 as an embodiment of the imaging device of the present invention.

[0028] In Figure 1In this case, the digital camera 300 is configured such that the interchangeable lens 100 is detachably (replaceably) attached to the camera body 200 via a mounting unit (not shown) including an electrical contact unit 106.

[0029] The interchangeable lens 100 includes a photographing lens 101 having a zoom mechanism as a photographing optical system, an aperture and shutter 102 for controlling the amount of light, and a focusing lens 103 for focusing on an image sensor, which will be described later. A motor 104 drives the focusing lens 103, and a lens controller 105 controls the entire interchangeable lens 100.

[0030] The camera body 200 includes an image sensor 201. The image sensor 201 includes a large number of pixels having photodiodes that convert reflected light from a subject into an electrical signal. The A / D conversion unit 202 includes a CDS circuit that removes output noise of the image sensor 201 and a non-linear amplification circuit that operates before A / D conversion, and converts an analog signal from the image sensor 201 into a digital signal. The camera body 200 also includes an image processing unit 203, an AF processing unit 204, a format conversion unit 205, and a high-speed built-in memory (e.g., a random access memory; hereinafter referred to as a DRAM) 206.

[0031] The image recording unit 207 is composed of a recording medium such as a memory card and an interface for the medium. A timing generator 208 generates a timing signal for controlling the operation timing of the digital camera 300, and a system control unit 209 controls the operation of the entire digital camera such as a shooting sequence. A lens communication unit 210 enables communication between the camera body 200 and the interchangeable lens 100. A subject detection unit 211 detects a subject from a captured image signal.

[0032] An image display memory (hereinafter referred to as a VRAM) 212 stores an image for display. An image display unit 213 displays an image, and also performs display for assisting operation and display of the state of the camera. The image display unit 213 also displays a shooting screen and a focus detection area during shooting. An operation unit 214 includes operation members for operating the camera from the outside. A shooting mode switch 215 is an operation member for selecting a shooting mode such as a macro mode and a sports mode. A main switch 216 is a switch for turning on the power of the digital camera 300. When the release button is half-pressed, a release switch (SW1) 217 is turned on and causes shooting preparation operations such as AF and AE to start. When the release button is fully pressed, a release switch (SW2) 218 is turned on and causes the shooting operation to start.

[0033] The DRAM 206 serves as a buffer for temporary image storage, a working memory for image compression and decompression, and the like. The operation unit 214 includes various items. These various items include, for example, menu switches for configuring various types of settings such as the shooting function and image reproduction function settings of a photographing device, and operation mode switching switches for switching between a shooting mode and a reproduction mode, and the like.

[0034] The image sensor 201 is composed of a CCD or a CMOS sensor. The light beam of the photographing optical system of the interchangeable lens 100 forms an image on the light-receiving surface of the image sensor 201 and is converted into signal charges corresponding to the incident light amount by photodiodes. The signal charges accumulated in each photodiode are converted into voltage signals, and based on an instruction from the system control unit 209 and driving pulses from the timing generator 208, the voltage signals are sequentially read out from the image sensor 201.

[0035] Each pixel in the image sensor 201 used in the camera body 200 includes two (a pair of) photodiodes A and B and one microlens provided commonly for this pair of photodiodes A and B. Each pixel forms an optical image pair on this pair of photodiodes A and B by splitting incident light via the microlens, and outputs a pixel signal pair (A signal and B signal) to be used for an AF signal, which will be described later. In addition, a photographing signal (A + B signal) can be obtained by adding the outputs of this pair of photodiodes A and B.

[0036] The plurality of A signals and the plurality of B signals output from a plurality of pixels are synthesized together, which will generate an image signal pair as an AF signal (in other words, a focus detection signal) to be used in AF based on the imaging plane phase difference detection method (hereinafter referred to as imaging plane phase difference AF). The AF signal processing unit 204 calculates a phase difference (hereinafter referred to as an image displacement amount), which is a displacement amount between this image signal pair, by performing a correlation calculation on this image signal pair, and also calculates a defocus amount (and defocus direction and reliability (focus state)) of the photographing optical system based on this image displacement amount. In addition, it is assumed that the AF signal processing unit 204 calculates defocus amounts in a plurality of regions (focus detection regions) that can be specified on the screen.

[0037] The following will use Figure 2 to describe the operation of the digital camera 300 of this embodiment. Figure 2 is a flowchart showing the operation of the digital camera 300.

[0038] First, in step S201, the system control unit 209 checks the state of the release switch (SW1) 217. When the release switch is turned on, the system control unit 209 proceeds to step S202, and when the release switch is turned off, the system control unit 209 stands by.

[0039] In step S202, the system control unit 209 sets an AF frame (focus detection frame) to be described later for the AF signal processing unit 204, and proceeds to step S203.

[0040] The system control unit 209 performs an AF operation to be described later in step S203, and proceeds to step S204.

[0041] In step S204, the system control unit 209 checks the state of the release switch (SW1) 217. When the release switch is turned on, the process proceeds to step S205, otherwise, the process returns to step S201.

[0042] In step S205, the system control unit 209 checks the state of the release switch (SW2) 218. When the release switch is turned on, the process proceeds to step S206, otherwise, the process returns to step S204.

[0043] The system control unit 209 performs a shooting operation in step S206, and then returns to step S201.

[0044] Figure 3 is a flowchart for describing the process of setting a focus detection frame in the present embodiment.

[0045] First, in step S301, the system control unit 209 obtains subject detection information from the subject detection unit 211. In the present embodiment, it is assumed that a human or an animal such as a dog or a wild bird and a main area within the subject are detected as the subject. The main area indicates the pupil, face, and body of a human or an animal. As a method for detecting these, for example, a learning method or an image processing method based on deep learning, which is a known technique, is used. In the present embodiment, since a known method can be used as the subject detection method, a detailed description of the subject detection method is omitted.

[0046] In step S302, the system control unit 209 determines whether multiple main areas can be detected based on the detection result of the subject detection unit 211. When multiple main areas can be detected, the process proceeds to step S303, otherwise, the process proceeds to step S304.

[0047] Now use Figure 4A 、 4B 、5A and 5B to describe the concept of detection in the case where one main area is detected and in the case where multiple main areas are detected.Figure 4A shows the state where only face a is detected, while Figure 5A shows the state where pupil A, face B, and body C are detected. It is assumed that the type of the subject such as a human and an animal, and the center coordinates, horizontal size, and vertical size of each detected main area can be obtained from the subject detection unit 211.

[0048] In step S303, the system control unit 209 inputs the minimum value among the sizes of the detected main areas, that is, Figure 5A the smaller one of the horizontal size and the vertical size values of the area of pupil A in is regarded as MinA, and MinA is regarded as an AF frame size.

[0049] In step S305, the system control unit 209 obtains from the horizontal coordinates and horizontal sizes of the detected main areas Figure 5B the horizontal size H including all the main areas shown in, and determines the number of AF frames in the horizontal direction by dividing this H by the AF frame size MinA.

[0050] In step S307, the system control unit 209 obtains from the vertical coordinates and vertical sizes of the detected main areas Figure 5B the vertical size V including all the main areas shown in, and determines the number of AF frames in the vertical direction by dividing this V by the AF frame size MinA. Then, the setting of the AF frames ends.

[0051] In the case of an animal, the control flow is the same as in the case of a human, and the concept of the detection area and the setting of the AF frames are respectively as shown in Figure 6A and Figure 6B shown. Although this embodiment uses the main area with the minimum size to form a square AF frame, the AF frame size can vary between the horizontal direction and the vertical direction, and the number of AF frames that can be calculated by the system control unit 209 can be set.

[0052] In step S304, the system control unit 209 sets an AF frame with a predetermined size X for the detected face. As the size X, the pupil size estimated according to the face can be set, or a frame size that can ensure S / N and ensure sufficient focusing performance can be set considering the low-luminance environment. It is assumed that the estimated pupil size is set as the size X in this embodiment.

[0053] In step S306, the system control unit 209 sets the number Y of AF frames for the area including face a based on the size of the AF frame.

[0054] Figure 7A and Figure 7B are used to describe Figure 2Flowchart of the AF operation in step S203.

[0055] First, the system control unit 209 detects the defocus amount and reliability (reliability) by performing focus detection processing in step S401, and proceeds to step S402. The focus detection processing will be described later.

[0056] In step S402, the system control unit 209 selects the AF main frame using the reliability obtained in step S401, and proceeds to step S403. The selection of the AF main frame will be described later.

[0057] In step S403, the system control unit 209 checks whether the reliability of the defocus amount detected in step S401 is higher than a preset reliability threshold 2. When the reliability is higher than the reliability threshold 2, the process proceeds to step S404; otherwise, the process proceeds to step S413. Here, the reliability threshold 2 is set such that a reliability lower than the reliability threshold 2 cannot guarantee the accuracy of the defocus amount but can guarantee the direction of the focus position of the subject. In step S404, the system control unit 209 checks whether the defocus amount detected in step S401 is equal to or less than a preset Def amount threshold 2. When the defocus amount is equal to or less than the Def amount threshold 2, the process proceeds to step S405; otherwise, the process proceeds to step S412. Here, the Def amount threshold 2 is set (for example, setting a value greater than (five times) the depth of focus as the Def amount threshold 2) such that if the lens drive corresponding to a defocus amount equal to or less than the Def amount threshold 2 is performed within a predetermined number of times (for example, three times) or less, the defocus amount can control the focusing lens within the depth of focus.

[0058] In step S405, the system control unit 209 checks whether the focusing lens 103 is in a stopped state. When the focusing lens 103 is in a stopped state, the process proceeds to step S406; otherwise, the process proceeds to step S410.

[0059] In step S406, the system control unit 209 checks whether the reliability of the defocus amount detected in step S401 is higher than a preset reliability threshold 1. When the reliability is higher than the reliability threshold 1, the process proceeds to step S407; otherwise, the process proceeds to step S410. Here, the reliability threshold 1 is set such that a reliability higher than the reliability threshold 1 keeps the change in the accuracy of the defocus amount within a predetermined range (for example, within the depth of focus).

[0060] In step S407, the system control unit 209 checks whether the defocus amount detected in step S401 is equal to or less than a preset Def amount threshold 1. When the defocus amount is equal to or less than the Def amount threshold 1, the process proceeds to step S408; otherwise, the process proceeds to step S409. Here, the Def amount threshold 1 is set such that a detected defocus amount equal to or less than the Def amount threshold 1 enables control of the focusing lens within the depth of focus.

[0061] In step S408, the system control unit 209 determines that the current state is the in-focus state and ends this process.

[0062] In step S409, the system control unit 209 drives the focusing lens 103 with the defocus amount detected in step S401, and then returns to step S401.

[0063] By performing the operation sequence from step S405 to step S409, when the reliability detected in step S401 is higher than the reliability threshold 1, the defocus amount can be detected again in the state where the lens has stopped.

[0064] In step S410, the system control unit 209 drives the focusing lens 103 by a predetermined percentage of the defocus amount detected in step S401, and proceeds to step S411.

[0065] In step S411, the system control unit 209 issues an instruction to stop the focusing lens 103 and returns to step S401.

[0066] In step S412, the system control unit 209 drives the focusing lens 103 by a predetermined percentage of the defocus amount detected in step S401 and returns to step S401. Here, the predetermined percentage is set such that the lens driving amount is less than the defocus amount (e.g., 80%). In addition, the lens speed is set to be slower than, for example, the speed at which the lens driving is correctly performed within one frame period. This can prevent exceeding the focal position of the subject in the case where the detected defocus amount is inaccurate, and can also perform the next lens driving (overlap control) without stopping the lens while driving the lens.

[0067] In step S413, the system control unit 209 checks whether the defocus condition is satisfied. When the defocus condition has been satisfied, the process proceeds to step S414; otherwise, the process proceeds to step S415. Here, the defocus condition is a condition for determining that there is no subject to be focused, and is, for example, a case where the lens driving has been completed within the movable range of the focusing lens 103. That is, it is a condition where the focusing lens 103 has returned to the initial position after detecting both the far-side and near-side lens ends.

[0068] In step S414, the system control unit 209 determines that the current state is an out-of-focus state, and ends this process.

[0069] In step S415, the system control unit 209 checks whether the focusing lens 103 has reached the lens end on the far side or the near side. When it reaches the lens end, the process proceeds to step S416, otherwise the process proceeds to step S417.

[0070] In step S416, the system control unit 209 reverses the driving direction of the focusing lens 103 and returns to step S401. In step S417, the focusing lens 103 is driven in a predetermined direction, and the process returns to step S401. The focusing lens speed is set to, for example, the highest speed within the lens speed range that does not exceed the focal position when the defocus amount can be detected.

[0071] It will be used Figure 8 to describe the focus detection process in step S401.

[0072] First, in step S501, the system control unit 209 sets a focus detection area with an arbitrary range within the image sensor 201, and proceeds to step S502.

[0073] In step S502, the system control unit 209 obtains an image signal pair (A image and B image) for focus detection from the image sensor 201 for the focus detection area set in step S501, and proceeds to step S503.

[0074] In step S503, the system control unit 209 performs a process of adding and averaging the image pair obtained in step S502 in the vertical direction, and then proceeds to step S504. This process can reduce the influence of noise in the image signal.

[0075] In step S504, the system control unit 209 performs a filtering process for extracting a signal component in a predetermined frequency band from the result of adding and averaging in the vertical direction in step S503, and then proceeds to step S505.

[0076] In step S505, the system control unit 209 calculates a correlation quantity based on the signal generated by the filtering process in step S504, and proceeds to step S506.

[0077] In step S506, the system control unit 209 calculates a correlation change quantity based on the correlation quantity calculated in step S505, and proceeds to step S507.

[0078] In step S507, the system control unit 209 calculates an image displacement quantity based on the correlation change quantity calculated in step S506, and proceeds to step S508.

[0079] In step S508, the system control unit 209 calculates a reliability indicating the degree to which the image displacement amount calculated in step S507 can be trusted, and proceeds to step S509.

[0080] In step S509, the system control unit 209 converts the image displacement amount into a defocus amount, and ends the focus detection process.

[0081] Figure 9 is a flowchart showing Figure 7A the operation for selecting the main frame in step S402. When selecting the main focus detection frame, the main focus detection frame is sequentially searched from the subject with the highest priority according to the part of the subject.

[0082] First, in step S601, the system control unit 209 sets the main frame at the initial position as a pre - preparation for main frame selection.

[0083] In step S602, the system control unit 209 determines whether the subject detection unit 211 has detected the pupil of the subject. If the pupil has been detected, the process proceeds to step S604; if the pupil has not been detected, the process proceeds to step S603.

[0084] The system control unit 209 sets the pupil area as the main frame selection area in step S604, and performs the main frame selection with detection center priority, which will be described later, in the next step S605.

[0085] In step S607, the system control unit 209 determines whether the reliability of the selected main frame for the selection result of the main frame in the pupil area is equal to or higher than the main frame reliability threshold. When it is determined that the reliability is equal to or higher than the threshold, the process proceeds to step S614; otherwise, the process proceeds to step S603. Step S607 is used to determine whether the detected defocus amount of the main frame selected in the detection area of the pupil is equal to or less than a predetermined change. For example, the aforementioned reliability threshold 1 or the like can be set as the main frame reliability threshold. When it is determined in step S607 that it is difficult to select a main frame in the pupil area, the process proceeds to step S603 to select a main frame in the face area.

[0086] In step S603, the system control unit 209 determines whether the subject detection unit 211 has detected the face of the subject. When the face is detected, the process proceeds to step S608; when the face is not detected, the process proceeds to step S611.

[0087] In step S608, the system control unit 209 sets the face area as the main frame selection area, and in the next step S609, performs the main frame selection with detection center priority, which will be described later.

[0088] In step S610, the system control unit 209 determines whether the reliability of the selected frame is equal to or higher than the main frame reliability threshold based on the selection result of the main frame in the face area. When it is determined that the reliability is equal to or higher than the threshold, the process proceeds to step S614; otherwise, the process proceeds to step S611. Step S610 is used to determine whether the detection defocus amount of the main frame selected in the face detection area is equal to or less than a predetermined change. For example, similar to the case of the pupil area, a reliability threshold 1 or the like can be set as the main frame reliability threshold. When it is determined in step S610 that it is difficult to select a main frame in the face area, the process proceeds to step S611 to select a main frame in the body area.

[0089] In step S611, the system control unit 209 determines whether the subject detection unit 211 has detected the body of the subject. When the body has been detected, the process proceeds to step S612; when the body has not been detected, the process proceeds to step S614.

[0090] In step S612, the system control unit 209 sets the body area as the main frame selection area, and in the next step S613, performs the main frame selection with priority given to the nearest subject in the central area, which will be described later.

[0091] In step S614, the system control unit 209 determines whether the main frame is in the initial position to check whether a main frame can finally be set on one of the pupil, face, and body. When the main frame corresponds to the initial value, the process proceeds to step S615; otherwise, the main frame selection process ends.

[0092] In step S615, the system control unit 209 selects the main frame at multiple points. Note that in step S615, although a method of selecting the main frame in a predetermined area within the screen without using detection information may be adopted, for example, since this is not the main part of this embodiment, its detailed description is omitted.

[0093] Next, use Figure 10 and Figure 11 to describe the main frame selection process suitable for each detection part. In this embodiment, when the detection part is the pupil or the face, the detection part occupies a relatively small area inside the subject, so the main frame selection process that emphasizes the position of the detection center is used. Figure 10 is a flowchart of the operation for selecting the main frame in a manner that gives priority to the detection center.

[0094] From step S701 to step S704, the same process is performed for each focus detection frame within the focus detection area.

[0095] In step S702, the system control unit 209 determines whether the focus detection frame is closer to the center of the detection area than the focus detection frame currently set as the main frame. When the focus detection frame is closer, the process proceeds to step S703 to update the main frame; when the focus detection frame is farther, the process proceeds to step S704. The above process is executed for each focus detection frame, and the main frame selection process is completed.

[0096] Next, the main frame selection process in the case where the detection part is the body is described. When the detection part is the body, a relatively large area is detected inside the subject. Then, when selecting the main frame in the body area where the subject shape may be complex, the main frame selection process is performed while giving priority to the frame (closest subject first) where the subject exists in the central area within the detection area where the probability of the subject's existence is high. Figure 11 It is a flowchart showing the operation of selecting the main frame in a way that gives priority to the closest subject in the central area.

[0097] In step S801, the system control unit 209 configures the initial setting of the number of frames for setting the main frame search area in the next step S802.

[0098] The system control unit 209 sets the main frame search area in step S802 and proceeds to step S803. From step S803 to step S807, the same process is executed for each focus detection frame within the focus detection area.

[0099] In step S804, the system control unit 209 determines whether the focus detection frame is within the main frame search area; when the focus detection frame is within the main frame search area, the process proceeds to step S805. In step S805, the system control unit 209 determines whether the defocus amount of the focus detection frame corresponds to a distance shorter than the detection defocus amount of the currently set main frame. When the defocus amount of the focus detection frame corresponds to the shorter distance, the main frame is updated in the next step S806; when the defocus amount of the focus detection frame does not correspond to the shorter distance, the process proceeds to step S807, and thus the determination related to the next focus detection frame is entered. The above process is executed for each focus detection frame, and the process proceeds to step S808.

[0100] In step S808, the system control unit 209 determines whether the reliability of the detection defocus amount of the selected main frame is equal to or higher than a predetermined threshold. When it is determined that the reliability is equal to or higher than the predetermined threshold, the main frame selection process is completed. When it is determined in step S808 that the reliability is lower than the predetermined threshold, the process proceeds to steps S809 and S810, the main frame search area is enlarged, and the aforementioned steps S802 to S808 are executed. When the main frame search has been completed in the entire focus detection area in step S809, the main frame selection process is completed.

[0101] The application of this embodiment enables, when multiple subject parts are detected, to perform correct focus adjustment in areas with higher priority while avoiding areas where it is difficult to perform focus detection. For example, when the subject is an animal (such as a dog, a cat, or a bird) that moves around contrary to the photographer's intention, focus adjustment can be performed in parts with higher priority while avoiding parts where it is difficult to perform focus detection due to the low reliability of detecting the defocus amount.

[0102] Although one embodiment of the present invention has been described in detail so far, the present invention is not limited to the foregoing embodiment, and various embodiments that do not deviate from the gist of the present invention are also included in the present invention.

[0103] In this embodiment, when the pupil or face has been detected, the main frame selection process that emphasizes the position of the detection center is used. However, when the body has been detected, the main frame selection process that emphasizes the frame in which a subject exists in the central area with a high probability of subject presence within the detection area is used. For subjects such as the pupil and face that tend to have a relatively small size inside the subject, the main frame selection process that emphasizes the position of the detection center is performed on the assumption that there is almost no displacement in the detection position of the subject. However, for example, when displacement in the detection position needs to be considered, a process for selecting a focus detection frame with a more reliable detected defocus amount as the main frame can be performed. Figure 12 It is a flowchart showing an operation for selecting a main frame in such a way that a focus detection frame with a highly reliable detected defocus amount within the detection area is prioritized.

[0104] From step S901 to step S905, the same processing is performed for each focus detection frame within the focus detection area. In step S902, the system control unit 209 determines whether the focus detection frame is within the area of the detected part of the subject. When the focus detection frame is within this area, in the next step S903, it is determined whether this focus detection frame has a higher reliability than the focus detection frame currently set as the main frame; when this focus detection frame has a higher reliability, the process proceeds to step S904 to update the main frame. When it is determined that this focus detection frame has a lower reliability than the focus detection frame currently set as the main frame, the process proceeds to step S905, and this determination is made for each focus detection frame, and the main frame selection process is completed.

[0105] In addition, in this embodiment, when the body has been detected and the main frame has low reliability in the pupil or face, considering the complexity of the subject shape, for the body area, the main frame selection process that emphasizes the frame in which a subject exists in the central area with a high probability of subject presence is used. However, for example, when the complexity of the subject shape does not need to be considered, a process for selecting a focus detection frame with a more reliable detected defocus amount as the main frame can be performed. Figure 13It is a flowchart showing an operation of selecting a main frame in a manner that prioritizes the reliability of the central region.

[0106] In step S1001, the system control unit 209 is configured to perform an initial setting of the number of frames for setting the main frame search region in the next step S1002.

[0107] In step S1002, the system control unit 209 sets the main frame search region and proceeds to step S1003. From step S1003 to step S1007, the same processing is performed for each focus detection frame within the focus detection region.

[0108] In step S1004, the system control unit 209 determines whether the focus detection frame is within the main frame search region; when the focus detection frame is within the main frame search region, the process proceeds to step S1005. In step S1005, it is determined whether the focus detection frame has a higher reliability detection defocus amount compared to the currently set main frame. When the focus detection frame has a higher reliability detection defocus amount, in the next step S1006, the main frame is updated; when the focus detection frame has a lower reliability detection defocus amount, the process proceeds to S1007, and thus the determination related to the next focus detection frame is entered. The above processing is performed for each focus detection frame, and the process proceeds to step S1008.

[0109] In step S1008, the system control unit 209 determines whether the reliability of the detection defocus amount of the selected main frame is equal to or higher than a predetermined threshold. When it is determined in step S1008 that the reliability is equal to or higher than the predetermined threshold, the main frame selection process is completed. When it is determined in step S1008 that the reliability is low, the process proceeds to steps S1009 and S1010, the main frame search region is enlarged, and the aforementioned steps S1002 to S1008 are executed. When the main frame search is completed in the entire focus detection region in step S1009, the main frame selection process is completed.

[0110] In addition, in a scene where there is a high risk of no subject regardless of the detection part of the subject (such as when the subject is moving violently, etc.), the main frame selection process can be set based on a determination related to whether the focus adjustment mode is a mode for photographing a still object (stationary subject) or a mode for photographing a moving object (moving subject). In the case of the mode for photographing a moving object, the focus position is tracked by predicting the subject position in the object frame from the historical information of the subject positions in multiple past frames. The process of using the focus detection frame indicating a position close to the aforementioned predicted subject position as the main frame for priority selection will be described in this modified example. Figure 14 It is a flowchart showing an operation of selecting a main frame in a manner that prioritizes prediction in the central region.

[0111] In step S1101, the system control unit 209 is configured to perform an initial setting of the number of frames for setting the main frame search area in the next step S1102. In step S1102, the system control unit 209 sets the main frame search area and proceeds to step S1103.

[0112] From step S1103 to step S1107, the same processing is performed for each focus detection frame within the focus detection area. In step S1104, the system control unit 209 determines whether the focus detection frame is within the main frame search area; when the focus detection frame is within the main frame search area, the processing proceeds to step S1105. In step S1105, it is determined whether the focus detection frame is closer to the subject position predicted from the detected defocus amount and the current lens position than the currently set main frame. When the focus detection frame is closer, the main frame is updated in the next step S1106; when the focus detection frame is farther away, the processing proceeds to S1107, and thus a determination related to the next focus detection frame is made. The above processing is performed for each focus detection frame, and the processing proceeds to step S1108.

[0113] In step S1108, the system control unit 209 determines whether the reliability of the detected defocus amount of the selected main frame is equal to or higher than a predetermined threshold. When it is determined in step S1108 that the reliability is equal to or higher than the predetermined threshold, the main frame selection process is completed. When it is determined in step S1108 that the reliability is low, the processing proceeds to steps S1109 and S1110, the main frame search area is enlarged, and the aforementioned steps S1102 to S1108 are executed. When the main frame search has been completed throughout the focus detection area in step S1109, the main frame selection process is completed.

[0114] The application of this modification example enables correct focus adjustment to be performed in an area with a higher priority for a subject with intense movement while avoiding areas where it is difficult to perform focus detection.

[0115] Other embodiments

[0116] Embodiments of the present invention can also be implemented by the following method, that is, software (program) that executes the functions of the above embodiments is provided to a system or device through a network or various storage media, and a method in which a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0117] Although the present invention has been described with reference to the embodiments, it is to be understood that the present invention is not limited to the disclosed embodiments, but is defined by the scope of the following claims.

Claims

1. An imaging device, comprising: An imaging unit configured to capture an image of a subject; A detection unit configured to detect the subject from the image captured by the imaging unit; A setting unit configured to set a plurality of focus detection frames in the image based on a detection result of the detection unit; A focus detection unit configured to detect a focus state and its reliability within each of the plurality of focus detection frames; And A selection unit configured to select a main focus detection frame for focus adjustment based on detection results of the detection unit and the focus detection unit, wherein the selection unit selects a part to be focused on in the priority order of the pupil, face, and body of the subject detected by the detection unit, wherein the selection unit varies a method for selecting the main focus detection frame between a case where the pupil or face is selected as the part to be focused on and a case where the body is selected as the part to be focused on, and wherein the selection unit also varies the method for selecting the main focus detection frame between a case where the subject is a stationary subject and a case where the subject is a moving subject.

2. The imaging device according to claim 1, wherein, The selection unit is configured to vary a method for selecting the main focus detection frame according to a size of a part detected by the detection unit.

3. The imaging device according to claim 1, wherein, The selection unit is configured to select the main focus detection frame based on a detection result of the detection unit, a setting of the setting unit, and a detection result of the focus detection unit.

4. The imaging device according to claim 1, wherein, The setting unit is configured to set the plurality of focus detection frames for a region including the plurality of detection regions when the detection unit has detected a plurality of parts of the subject corresponding to a plurality of detection regions.

5. The imaging device according to claim 4, wherein, The setting unit is configured to set the plurality of focus detection frames based on a size of a smallest region among detection regions corresponding to a part of the subject detected by the detection unit.

6. The imaging device according to claim 1, wherein, The selection unit is configured to select the main focus detection frame in a manner that gives priority to the center of the part when the pupil is detected as a part of the subject.

7. The imaging device according to claim 1, wherein, The selection unit is configured to select the main focus detection frame in a manner that gives priority to the center of the part when the face is detected as a part of the subject.

8. The imaging device according to claim 1, wherein, The selection unit is configured to select the main focus detection frame in a manner that gives priority to the nearest subject when the body is detected as a part of the subject.

9. A control method for an imaging device, the imaging device including an imaging unit for capturing an image of a subject, the control method including: Detect the subject from the image captured by the imaging unit; Set a plurality of focus detection frames in the image based on a detection result in the detection; Perform focus detection to detect a focus state and its reliability within each of the plurality of focus detection frames; And Select a main focus detection frame for focus adjustment based on detection results in the detection and the focus detection, wherein the selection selects a part to be focused on in the priority order of the pupil, face, and body of the subject detected by the detection, wherein the selection varies a method for selecting the main focus detection frame between a case where the pupil or face is selected as the part to be focused on and a case where the body is selected as the part to be focused on, and Among them, the selection also causes the method for selecting the main focus detection frame to vary between the case where the subject is a stationary subject and the case where the subject is a moving subject.

10. A computer-readable storage medium storing a program for causing a computer to execute the steps of the control method according to claim 9.

11. A computer program product comprising a program for causing a computer to perform the steps of the control method according to claim 9.

Images