Focus detection device, focus detection method, imaging apparatus, and program

Abstract

To provide a technique capable of continuously focusing on a specific area even when a correct focus detection result cannot be obtained from the specific area of a subject to be subjected to autofocus control.SOLUTION: A focus detection apparatus includes an object detection unit configured to hierarchically detect one or a plurality of specific areas from an object in a captured image, a setting unit configured to set focus detection areas obtained by dividing the specific area into a plurality of areas, a focus detection unit configured to detect focus detection information for each of the focus detection areas, and a determination unit configured to determine a target area for performing an autofocus operation. The target area included in an upper area is determined based on a distribution of defocus amounts of a lower area including the upper area among the specific areas.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to focus detection technology. [Background technology]

[0002] Patent Document 1 describes a method for focusing on a specific area of ​​a detected subject with priority, while Patent Document 2 describes a method for focusing on a subject by avoiding an obstruction when the obstruction overlaps with the detected subject on the close side. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6071574 [Patent Document 2] Japanese Patent Publication No. 2022-137760 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Documents 1 and 2, when an obstruction such as clothing worn by the subject overlaps the specific area of ​​a detected subject, the depth difference between the subject and the obstruction is not large enough, and the obstruction is included in the subject detection area, making it impossible to avoid the obstruction. In this case, the focus is focused on the nearby obstruction, making it impossible to focus on the subject. Furthermore, when the specific area of ​​the detected subject is located at the edge of the subject's outline, an accurate focus detection result may not be obtained when attempting to focus based on the focus detection result of the specific area.

[0005] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to realize a technology that can continue to focus on a specific area of ​​a subject that is the target of autofocus control even if a correct focus detection result cannot be obtained from that specific area. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the focus detection device of the present invention has a subject detection means that hierarchically detects one or more specific areas from a subject in a captured image, a setting means that sets a focus detection area divided into multiple areas for the specific area, a focus detection means that detects focus detection information for each focus detection area, and a determination means that determines a target area for autofocus operation, wherein the determination means calculates the distribution of defocus amounts for the specific area and determines the target area included in the upper area based on the distribution of defocus amounts for lower areas including the upper area of ​​the specific area. [Effects of the Invention]

[0007] According to the present invention, even if a correct focus detection result cannot be obtained from a specific area of ​​a subject that is the target of autofocus control, it is possible to continue to focus on the specific area. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] 4 is a flowchart illustrating a control process during shooting of the imaging apparatus of the present embodiment. [Figure 3] 6 is a flowchart illustrating an example of an AF frame setting process in a control process during shooting according to the present embodiment. [Figure 4] 4A and 4B are diagrams illustrating an example of a single specific area detected from a subject and an AF frame set in the specific area in this embodiment. [Figure 5] 3A and 3B are diagrams illustrating examples of a plurality of specific areas detected from a subject and AF frames set in each specific area in the present embodiment. [Figure 6] 5 is a flowchart illustrating an example of an AF operation in a control process during shooting according to the present embodiment. [Figure 7] 6 is a flowchart illustrating a focus detection process in the AF operation of the present embodiment. [Figure 8] 6 is a flowchart illustrating a main frame selection process in the AF operation of the present embodiment. [Figure 9] 10 is a flowchart illustrating a detected subject main frame selection process in the main frame selection process of the present embodiment. [Figure 10] 10 is a flowchart illustrating a main frame selection region determination process in the detected subject main frame selection process of this embodiment. [Figure 11] 10 is a flowchart illustrating pupil-priority main frame selection processing in the main frame selection processing of the present embodiment. [Figure 12] 6A to 6C are diagrams for explaining a histogram creation method in the pupil-priority main frame selection process of the present embodiment. [Figure 13] 6A to 6C are diagrams illustrating an example of an AF frame around a pupil region in the pupil-priority main frame selection process of the present embodiment. [Figure 14] 6A and 6B are diagrams illustrating histograms created in the pupil-priority main frame selection process of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] Below, we will explain an example in which the focus detection device and imaging device of the present invention are applied to a digital camera with interchangeable lenses, but this is not the only example, and the invention can also be applied to, for example, digital cameras with integrated lenses, digital video cameras, smartphones with camera functions, tablet computers, web cameras such as surveillance cameras, medical cameras, etc.

[0011] The imaging apparatus of this embodiment includes the focus detection device of the present invention, and performs autofocus (AF) control using an imaging surface phase difference detection method based on a pair of imaging signals from the imaging section.

[0012] <Device configuration> First, the hardware configuration of the imaging device of this embodiment will be described with reference to FIG.

[0013] FIG. 1 is a block diagram illustrating the hardware configuration of an imaging apparatus according to this embodiment.

[0014] The imaging device of this embodiment includes a lens device (interchangeable lens) 100 and a camera body 200. The lens device 100 is mechanically and electrically connected to a lens mount 106 of the camera body 200. When the lens device 100 is attached to the camera body 200 via the lens mount 106, a lens controller 105 that controls the overall operation of the lens device 100 and a system control unit 209 that controls the overall operation of the camera become able to communicate with each other. The lens mount 106 includes a transmission path (bus) that allows for the exchange of synchronization signals, control signals, various types of data, and the like with the camera body 200.

[0015] Lens device 100 constitutes a photographing optical system that forms an optical image of a subject, which is light reflected from the subject, on imaging unit 201 of camera body 200. Lens device 100 includes photographing lens 101 including a zoom mechanism, aperture / shutter 102 that adjusts the amount of light of the subject image reflected by imaging unit 201, focus lens 103 that adjusts the focus state of the subject image reflected by imaging unit 201, drive unit 104 such as a motor that drives photographing lens 101, aperture / shutter 102, and focus lens 103, and lens controller 105.

[0016] The lens device 100 communicates with a system control unit 209 of the camera body 200 via a lens mount 106, and a lens controller 105 controls a drive unit 104 to operate an aperture / shutter 102 to adjust the brightness of the subject image, and controls the focus state of the subject image by displacing a focus lens 103.

[0017] The camera body 200 captures an image of a subject that has passed through the imaging optical system of the lens device 100 and generates an imaging signal.

[0018] The imaging unit 201 converts the subject image formed on the light receiving surface into an electrical signal using an image sensor such as a CCD or CMOS, and outputs the electrical signal to the A / D conversion unit 202. The A / D conversion unit 202 converts the analog signal input from the imaging unit 201 into a digital signal. The A / D conversion unit 202 also includes a CDS circuit that removes noise from the analog signal, and a nonlinear amplification circuit that nonlinearly amplifies the analog signal before converting it into a digital signal.

[0019] The image processing unit 203 outputs image data by performing predetermined pixel interpolation, resizing processing such as image reduction, and color conversion processing on the digital signal output from the A / D conversion unit 202. The image processing unit 203 also performs predetermined calculation processing using the image data, and the system control unit 209 performs AF processing and AE (auto exposure) processing based on the calculation results.

[0020] Each pixel of the imaging unit 201 of this embodiment includes a plurality (a pair) of photoelectric conversion elements (photodiodes) A ​​and B, and one microlens provided for the pair of photoelectric conversion elements A and B. Each pixel splits incident light using the microlens to form a pair of optical images on the pair of photoelectric conversion elements A and B, and outputs a pair of pixel signals (signal A and signal B) from the pair of photoelectric conversion elements A and B to be used as a focus detection signal, which will be described later. In addition, an imaging signal (signal A+signal B) is obtained by adding the outputs of the pair of photoelectric conversion elements A and B.

[0021] By combining a plurality of A signals and a plurality of B signals output from a plurality of pixels, a pair of image signals is obtained as a focus detection signal used for AF using an image plane phase difference detection method (hereinafter referred to as image plane phase difference AF). The AF signal processing unit 204 performs a correlation operation on the pair of image signals to calculate a phase difference (hereinafter referred to as image shift amount) that is the amount of shift between the pair of image signals, and further calculates the defocus amount (and defocus direction and reliability) of the imaging optical system from the image shift amount. Furthermore, when a plurality of specific regions are detected from the object detected by the object detection unit 211, the AF signal processing unit 204 calculates the defocus amount within each specific region and calculates a defocus distribution based on the calculated defocus amount.

[0022] The format conversion unit 205 converts the format of the image data generated by the image processing unit 203 in order to store the image data in the DRAM 206. The DRAM 206 is an example of a high-speed built-in memory, and is used as a high-speed buffer that temporarily stores image data, or as a working memory for image data compression / decompression processing.

[0023] The image recording unit 207 has a recording medium such as a memory card for recording captured images (still images and moving images) and an interface therefor.

[0024] The timing generating unit 208 supplies clock signals and control signals to the imaging unit 201 and the A / D conversion unit 202. The timing generating unit 208 also controls the reset timing of the charges accumulated in the imaging unit 201, thereby controlling the operations of accumulating and discharging charges in the imaging unit 201.

[0025] The system control unit 209 has a processor (CPU), memory (RAM and ROM), input / output circuits, timer circuits, etc., and controls the operation of the entire device by the CPU expanding and executing programs stored in the ROM into the working area of ​​the RAM.

[0026] The lens communication unit 210 communicates with the lens controller 105 of the lens device 100 attached to the camera body 200 via the lens mount 106 .

[0027] The subject detection unit 211 performs known subject detection processing on the imaging signal output from the A / D conversion unit 202, and detects a subject present in an imaging screen corresponding to the image data generated by the image processing unit 203. The subject detection unit 211 can repeatedly detect one or more specific regions of the subject in the image, stepwise from a higher-level region to a lower-level region. For example, if the subject is a person, the specific regions may be the whole body as a first hierarchical region, the face as a second hierarchical region contained in the first hierarchical region, and the pupils as a third hierarchical region contained in the second hierarchical region. Note that the subject is not limited to a person, and may be an animal, a vehicle, a train, etc., and the specific region may be determined based on the characteristics of the subject.

[0028] The VRAM 212 is a video memory in which data to be displayed on the display unit 213 is drawn. The data generated in the VRAM 212 is transferred to the display unit 213 at a predetermined frame rate, whereby an image is displayed on the display unit 213.

[0029] The display unit 213 includes a liquid crystal panel or an organic panel, and displays images, operation assistance, and the status of the camera. When taking a photograph, it also displays an imaging screen and an AF frame indicating the focus detection area.

[0030] The photographer operates the imaging device using the operation unit 214. The operation unit 214 includes, for example, a menu switch for performing various settings such as exposure compensation and aperture value settings, and settings for image playback, a zoom lever for instructing the zoom operation of the shooting lens, and an operation mode changeover switch between shooting mode and playback mode.

[0031] The shooting mode switch (SW) 215 includes a shooting mode changeover switch for selecting a shooting mode such as macro mode or sports mode.

[0032] The main switch (SW) 216 is a switch for turning on power to the system. The first switch (SW) 217 ​​is a switch for outputting a first switch signal SW1 to the system control unit 209 and performing shooting preparation operations such as AE processing and AF processing. The second switch (SW) 218 ​​is a switch for outputting a second switch signal SW2 to the system control unit 209 and issuing a shooting instruction while the first switch signal SW1 by the first switch 217 is on (shooting preparation state).

[0033] <Control processing during shooting> Next, the control process during image capture in this embodiment will be described with reference to FIG.

[0034] FIG. 2 is a flowchart illustrating an example of AF control processing in the still image shooting mode of this embodiment.

[0035] The control processing of this embodiment is realized by the system control unit 209 loading a program stored in ROM into RAM and executing it to control the components of the lens apparatus 100 and camera body 200.

[0036] Although this embodiment describes AF control processing in still image shooting mode, it is also effective for movie servo AF in movie shooting mode, which keeps the camera focused on a specific subject without user operation.

[0037] In step S201, the system control unit 209 determines whether or not a shooting preparation instruction has been received via the first switch 217. The system control unit 209 determines whether or not the first switch signal SW1 is on, and if it is not on, continues the determination, but if it is on, the process proceeds to step S202.

[0038] In step S202, the system control unit 209 performs an AF frame setting process, which will be described later, and then the process proceeds to step S203.

[0039] In step S203, the system control unit 209 performs an AF operation described later and advances the process to step S204.

[0040] In step S204, the system control unit 209 determines whether the first switch signal SW1 is on. If it is not on, the process returns to step S201. If it is on, the process advances to step S205.

[0041] In step S205, the system control unit 209 determines whether a shooting instruction has been received via the second switch 218. The system control unit 209 determines whether the second switch signal SW2 is on. If it is not on, the process returns to step S201. If it is on, the process advances to step S206.

[0042] In step S206, the system control unit 209 performs shooting processing and returns the process to step S201.

[0043] <AF Frame Setting Process> FIG. 3 is a flowchart illustrating the AF frame setting process in step S202 of FIG. 2.

[0044] In step S301, the system control unit 209 acquires detected subject information from the subject detection unit 211. In the subject detection process of the present embodiment, the detection target is a person, and one or more specific regions in the subject of the detection target are hierarchically detected. In the present embodiment, the whole body is the first hierarchical region, the face is the second hierarchical region, and the pupil is the third hierarchical region.

[0045] As a method for detecting a subject and a specific region, machine learning such as deep learning or image recognition processing can be used.

[0046] For example, the types of machine learning are as follows. (1) Support Vector Machine (2) Convolutional Neural Network (3) Recurrent Neural Network Furthermore, the image recognition process may involve detecting a face by extracting facial features such as the eyes, nose, and mouth using well-known pattern recognition technology. Note that the method for detecting a specific region is not limited to these examples, and other methods may also be used.

[0047] In step S302, the system control unit 209 determines whether or not multiple specific regions have been detected based on the detected subject information acquired from the subject detection unit 211. If multiple specific regions have been detected, the system control unit 209 proceeds to step S303; otherwise, the system control unit 209 proceeds to step S304.

[0048] Here, with reference to Figures 4 and 5, we will explain a state in which detection of a specific area from a detected subject is performed once and a single specific area is detected, and a state in which detection of a specific area from a detected subject is performed multiple times and multiple specific areas are detected.

[0049] Fig. 4(a) illustrates a state in which only a face 401 is detected as a specific region. Fig. 5(a) illustrates a state in which an eye 501, a face 502, and a whole body 503 are detected as specific regions. The subject detection unit 211 acquires the type of subject, such as a person or an animal, and the center coordinates, horizontal size, and vertical size of the specific region detected from the subject.

[0050] In step S303, the system control unit 209 sets the size of the AF frame to MinA, the size of the smallest area among the specific areas. In the example of Fig. 5, the smaller of the horizontal size and vertical size of the pupil 501 is set to MinA, and the set MinA is used as the size of one AF frame 504.

[0051] In step S305, the system control unit 209 calculates the horizontal size H in FIG. 5B that encompasses all specific areas from the horizontal coordinates and horizontal size of each specific area, and determines the number of horizontal AF frames by dividing H by the AF frame size MinA.

[0052] In step S307, the system control unit 209 calculates the vertical size V in Figure 5(b) that encompasses all specific areas from the vertical coordinates and vertical size of each specific area, determines the number of vertical AF frames by dividing V by the AF frame size MinA, and terminates the AF frame setting process.

[0053] In this embodiment, a square AF frame size is set using the minimum size of the specific area, but the AF frame size may be different between the horizontal and vertical sizes, or the number of AF frames may be set up to the number that can be calculated by the system control unit 209.

[0054] In step S304, the system control unit 209 sets an AF frame of a predetermined size X for the detected specific area. In this embodiment, for example, as shown in Fig. 4, when a face 401 is detected as the specific area, an AF frame 402 of size X is set based on the pupil size estimated from the face 401. Note that an AF frame size may be set that ensures an S / N ratio and provides sufficient focusing performance in consideration of low-illumination environments.

[0055] In step S306, the system control unit 209 sets the number of AF frames Y to be such that the detected specific area is included in the AF frame size set in step S304 and that the number of AF frames Y can be adapted to accommodate even if the specific area moves, and then ends the AF frame setting process. In this embodiment, for example, the number of AF frames Y is set to be such that the area of ​​the face 401 in FIG. 4A is included and that the number of AF frames Y can be adapted to accommodate even if the face 401 moves.

[0056] Note that the AF frame setting process of this embodiment does not necessarily need to be executed every time image data is input, and it may be performed once every time image data is input a plurality of times. In this case, when the AF frame setting process is not executed, the set AF frame is stored in the DRAM 206, and the AF frame stored at the timing closest in time may be read out and used.

[0057] <AF operation> FIG. 6 is a flowchart illustrating the AF operation in step S203 of FIG. 2.

[0058] In step S601, the system control unit 209 performs a focus detection process, calculates a defocus amount by the AF signal processing unit 204, and advances the process to step S602. Details of the focus detection process will be described later with reference to FIG. 7.

[0059] In step S602, the system control unit 209 performs a main frame selection process based on the detected subject information acquired in step S301 of FIG. 3, and advances the process to step S603. Details of the main frame selection process will be described later with reference to FIG. 8.

[0060] In step S603, the system control unit 209 calculates the driving amount of the focus lens based on the defocus amount of the main frame selected in step S602, and advances the process to step S604. The main frame is the target area for performing the AF operation.

[0061] In step S604, the system control unit 209 transmits the focus lens driving amount calculated in step S603 to the lens device 100 via the lens communication unit 210. The lens controller 105 of the lens device 100 drives the focus lens 103 based on the focus lens driving amount received from the system control unit 209.

[0062] <Focus detection process> FIG. 7 is a flowchart illustrating the focus detection process in step S601 of FIG. 6.

[0063] In step S701, the system control unit 209 sets a focus detection area of ​​a predetermined range within an imaging screen corresponding to image data, and the process proceeds to step S702.

[0064] In step S702, the system control unit 209 causes the AF signal processing unit 204 to acquire focus detection information (signals A and B) from the focus detection area set in step S701, and proceeds to step S703.

[0065] In step S703, the system control unit 209 performs row averaging processing in the vertical direction on the focus detection signals acquired in step S502, and then proceeds to step S704. This processing can reduce the influence of noise on the focus detection signals.

[0066] In step S704, the system control unit 209 performs filtering to extract signal components in a predetermined frequency band from the signal obtained by vertical row averaging in step S703, and then the process proceeds to step S705.

[0067] In step S705, the system control unit 209 calculates the amount of correlation from the signal that was filtered in step S704, and the process proceeds to step S706.

[0068] In step S706, the system control unit 209 calculates the amount of change in correlation from the amount of correlation calculated in step S705, and the process proceeds to step S707.

[0069] In step S707, the system control unit 209 calculates the amount of image shift from the amount of correlation change calculated in step S706, and the process proceeds to step S708.

[0070] In step S708, the system control unit 209 calculates the reliability of the image shift amount calculated in step S707, and the process proceeds to step S709.

[0071] In step S709, the system control unit 209 converts the amount of image shift calculated in step S707 into a defocus amount, and then ends the process.

[0072] <Main frame selection process> FIG. 8 is a flowchart illustrating the main frame selection process in step S602 of FIG.

[0073] In step S801, the system control unit 209 determines whether or not a subject has been detected by the subject detection unit 211, and if a subject has been detected, proceeds to step S803; if a subject has not been detected, proceeds to step S802.

[0074] In step S802, the system control unit 209 performs multi-point main frame selection processing without using detected subject information, and then ends the processing, because no subject has been detected by the subject detection unit 211. The multi-point main frame selection processing may be, for example, a method of selecting a main frame in a predetermined area within the imaging screen, but detailed description thereof will be omitted.

[0075] In step S803, the system control unit 209 determines whether or not the following conditions are met: a face and eyes are detected as specific regions of the subject detected by the subject detection unit 211, and the size of the face is equal to or larger than a predetermined size. If this condition is met, the system control unit 209 proceeds to step S805; if the condition is not met, the system control unit 209 proceeds to step S804.

[0076] In step S804, the system control unit 209 performs a detected subject main frame selection process, and then ends the process. Details of the detected subject main frame selection process will be described later with reference to FIG.

[0077] In step S805, the system control unit 209 performs pupil-priority main frame selection processing, and then ends the processing. Details of the pupil-priority main frame selection processing will be described later with reference to FIG.

[0078] <Detected subject main frame selection process> FIG. 9 is a flowchart illustrating the detection subject main frame selection process in step S804 of FIG.

[0079] In step S901, the system control unit 209 performs a main frame selection area determination process to determine a target area for main frame selection. Here, the main frame selection area determination process will be described with reference to FIG.

[0080] In step S1001, the system control unit 209 determines whether or not an eye has been detected as a specific region of the subject detected by the subject detection unit 211. If an eye has been detected, the system control unit 209 proceeds to step S1002, and if an eye has not been detected, the system control unit 209 proceeds to step S1003.

[0081] In step S1002, the system control unit 209 adds the pupil region as a main frame selection region, and proceeds to step S1003.

[0082] In step S1003, the system control unit 209 determines whether or not a face has been detected as a specific region of the subject detected by the subject detection unit 211. If a face has been detected, the system control unit 209 proceeds to step S1004, and if a face has not been detected, the system control unit 209 proceeds to step S1005.

[0083] In step S1004, the system control unit 209 adds the face region as the main frame selection region, and proceeds to step S1005.

[0084] In step S1005, the system control unit 209 determines whether or not the entire body has been detected as a specific region of the subject detected by the subject detection unit 211. If the entire body has been detected, the system control unit 209 proceeds to step S1006; if the entire body has not been detected, the system control unit 209 ends the process and proceeds to step S902 in FIG.

[0085] In step S1006, the system control unit 209 adds the whole body region as the main frame selection region, ends the process, and proceeds to step S902 in FIG.

[0086] 9 , in step S902, the system control unit 209 determines whether the number of AF frames that include the main frame selection region determined in step S901 is equal to or greater than a threshold. If the main frame selection region includes AF frames that are equal to or greater than the threshold, the system control unit 209 proceeds to process step S903, and if the main frame selection region does not include AF frames that are equal to or greater than the threshold, the system control unit 209 proceeds to process step S909. In this embodiment, the AF frame that includes the main frame selection region is an AF frame whose center is included in the main frame selection region, but this is not limiting. For example, the AF frame may be an AF frame that includes at least a portion of the main frame selection region, or an AF frame whose area overlaps the main frame selection region by a predetermined percentage or more.

[0087] In step S903, the system control unit 209 classifies the defocus amounts calculated for each AF frame including the main frame selection area into predetermined depths, creates a histogram of the number of AF frames at the AF frame positions illustrated in FIG. 14, and proceeds to step S904.

[0088] In step S904, the system control unit 209 determines whether the peak value (number of AF frames) of the histogram created in step S903 is equal to or greater than a predetermined value (predetermined number). If the peak value is equal to or greater than the predetermined value, the system control unit 209 proceeds to step S905, and if the peak value is less than the predetermined value, the system control unit 209 proceeds to step S909. In this embodiment, the maximum number of AF frames in the histogram is normalized by the total number of AF frames, and the value converted into a ratio is used as the peak value (bin).

[0089] In step S905, the system control unit 209 starts a loop process for all AF frames including the main frame selection area in order to select a main frame from the main frame selection area determined in step S901.

[0090] In step S906, the system control unit 209 determines whether the AF frame to be processed is included in the bin that indicates the peak value of the histogram. If the AF frame is included in the bin, the system control unit 209 proceeds to step S907. If the AF frame is not included in the bin, the system control unit 209 repeats the loop process for the other AF frames.

[0091] In step S907, the system control unit 209 determines whether the AF frame to be processed satisfies the condition that it is located closer to the center of the main frame selection area than the currently selected main frame. If this condition is met, the system control unit 209 proceeds to step S708; if the condition is not met, the system control unit 209 repeats the loop processing for the other AF frames. The center of the main frame selection area may be, for example, the center of the maximum width of the main frame selection area in the horizontal and vertical directions, or the center of gravity of the main frame selection area.

[0092] In step S908, the system control unit 209 updates the AF frame closest to the center of the main frame selection area to the main frame, and repeats the loop process for the other AF frames. By repeating this loop process, it is possible to select the AF frame closest to the center of the main frame selection area from among the AF frames included in the bin as the main frame. When the loop process is complete, the process ends and proceeds to step S603 in FIG. 6.

[0093] In step S909, because the system control unit 209 cannot perform the processes of steps S905 to S908 on the main frame selection area, it selects a main frame using center-priority main frame selection processing, terminates the processing, and proceeds to step S603 in Fig. 6. The center-priority main frame selection processing can be, for example, a method of setting the AF frame at the center of the subject detection area as the main frame, but a detailed description thereof will be omitted.

[0094] <Pupil priority main frame selection process> Fig. 11 is a flowchart illustrating the pupil-priority main frame selection process in step S805 in Fig. 8. Fig. 12, Fig. 13, and Fig. 14 are diagrams for explaining the pupil-priority main frame selection process.

[0095] In step S1101, the system control unit 209 classifies the defocus amounts calculated for each AF frame (A1 in FIG. 12) including the face region into predetermined depths to create a histogram.

[0096] In step S1102, the system control unit 209 starts loop processing of peripheral regions adjacent to the pupil region in order to select a main frame from the regions adjacent to the pupil region. The peripheral region adjacent to the pupil region is, for example, an AF frame (A2 in FIG. 12) whose center is included in the pupil region and an AF frame (A3 in FIG. 12) adjacent to the AF frame, but is not limited to this example and may be an AF frame included within a specific distance from the center of the pupil region.

[0097] In step S1103, the system control unit 209 determines whether the bin into which the AF frame to be processed (target bin) is classified is different from the bin currently selected as the main frame (selected bin). If the bin currently selected as the main frame (selected bin) is different, the system control unit 209 proceeds to step S1104; if the bin is the same, the system control unit 209 proceeds to step S1106. Note that an error value that does not match any bin in the histogram is registered as the initial value of the selected bin, and the system control unit 209 always proceeds to step S1104 the first time the loop processing is executed.

[0098] In step S1104, the system control unit 209 calculates the number of peripheral AF frames by adding the numbers of AF frames in adjacent bins for each of the bin into which the AF frame to be processed is classified and the selected bin, and determines which bin has the larger number of peripheral AF frames. If the bin into which the AF frame to be processed is classified has a larger number of peripheral AF frames, the process proceeds to step S1105. If the selected bin has a larger number of peripheral AF frames, the system control unit 209 does not update either the main AF frame or the selected bin, and repeats the loop process for the other AF frames.

[0099] In step S1105, the system control unit 209 sets the AF frame to be processed as the main frame, sets the bin into which the AF frame to be processed is classified as the selected bin, and repeats the loop process for the other AF frames.

[0100] The processes in steps S1104 and S1105 will now be described with reference to FIGS.

[0101] For example, suppose the current main frame is W1 in Figure 13, the currently selected bin is B4 in Figure 14, the AF frame to be processed is W2 in Figure 13, and the bin into which W2 is classified is B1 in Figure 14. In this case, the number of peripheral AF frames for the selected bin is the sum of the numbers of AF frames for bins B3, B4, and B5. The number of peripheral AF frames for the bin into which the AF frame to be processed is classified is the sum of the numbers of AF frames for bins B0, B1, and B2. In the example in Figure 14, the number of peripheral AF frames for the selected bin is greater than the number of peripheral AF frames for the bin into which the AF frame to be processed is classified, so neither the main frame nor the selected bin is updated.

[0102] In step S1104, the numbers of AF frames in adjacent bins were added together to determine the number of peripheral AF frames, but this does not have to be done depending on the characteristics of the subject, and the number of bins to be added may be changed. For example, if the width of the bin is set to be extremely narrow compared to the scale of the depth distribution of the actual face region, it is better to add more, and conversely, if the width is set to be appropriate or wide compared to the scale of the depth distribution of the face region, it is better not to add.

[0103] In step S1106, the system control unit 209 compares the AF frame to be processed with the current main frame to determine which is closer to the center of the pupil region. If the AF frame to be processed is closer to the center of the pupil region, the system control unit 209 proceeds to step S1107. If the current main frame is closer to the center of the pupil region, the system control unit 209 does not update the main frame and repeats the loop processing for the other AF frames.

[0104] In step S1107, the system control unit 209 sets the AF frame to be processed as the main frame, and repeats the loop process for the other AF frames.

[0105] The processes in steps S1106 and S1107 will now be described with reference to FIGS.

[0106] For example, suppose the current main frame is W1 in Fig. 13, the currently selected bin is B4 in Fig. 14, the AF frame to be processed is W4 in Fig. 13, and the bin into which W4 is classified is B4 in Fig. 14. In this case, when comparing W1 and W4 in Fig. 13 to determine which is closer to the center of the pupil region, W4 is closer, so the main frame is updated to W4, the AF frame to be processed.

[0107] If the loop processing in step S1105 is completed, the processing ends and the process proceeds to step S603 in FIG.

[0108] As described above, according to this embodiment, it is possible to select an AF frame that is closer to the center of the pupil area and that does not deviate significantly from the depth distribution of the face, so that even if a correct focus detection result cannot be obtained from a specific area of ​​the subject that is the target of autofocus control, it is possible to continue focusing on the specific area.

[0109] In this embodiment, an algorithm has been described that selects an AF frame around an pupil detection area that does not deviate too far from a detected face, but it can also be applied to selecting an AF frame around a face detection area that does not deviate too far from the depth distribution of the entire body, for example. Also, even if the detected subject types are different, if multiple specific areas of the subject can be detected hierarchically, an algorithm can be used that selects an AF frame around a higher-level area that does not deviate too far from a lower-level area.

[0110] [Other embodiments] This embodiment can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0111] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

[0112] The disclosure of this specification includes the following focus detection device, focus detection method, imaging device, and program. [Configuration 1] a subject detection means for hierarchically detecting one or more specific regions from a subject in a captured image; a setting means for setting a focus detection area divided into a plurality of areas for the specific area; a focus detection unit for detecting focus detection information for each of the focus detection areas; a determining means for determining a target area for performing an autofocus operation; the determining means determines a distribution of defocus amounts in the specific region; A focus detection device characterized in that the target area included in the upper region of the specific region is determined based on a distribution of defocus amounts of lower regions including the upper region of the specific region. [Configuration 2] The focus detection device described in configuration 1 is characterized in that, when the subject detection means detects the lower region having a predetermined size or larger and the upper region, the determination means determines the target region from the peripheral region of the upper region based on the distribution of defocus amounts of the lower region. [Configuration 3] The focus detection device according to configuration 2, wherein the determining means determines the target area based on the distribution of defocus amounts in the surrounding area of ​​the higher-order area and the distribution of defocus amounts in the lower-order area. [Configuration 4] The focus detection device according to configuration 3, wherein the determining means sets the target area based on a result of comparing peak values ​​of the distribution of defocus amounts in the surrounding areas of the higher-ranked area. [Configuration 5] 4. The focus detection device according to configuration 3, wherein the determining means determines the focus detection area of ​​the higher-level area based on the center of the higher-level area and the positions of the peripheral areas. [Configuration 6] If the subject detection means does not detect the lower region having a predetermined size or larger and the upper region, a distribution of defocus amounts is generated for the region among the lower regions in which the number of focus detection regions is equal to or larger than a threshold value; The focus detection device according to configuration 1, wherein the target area is determined from an area where the peak value of the generated distribution of defocus amounts is equal to or greater than a predetermined value. [Configuration 7] 7. The focus detection device according to configuration 6, wherein the determining means determines the target area based on the positions of the current target area and the lower-level area. [Configuration 8] A focus detection device described in any one of configurations 1 to 7, characterized in that the specific area includes at least one of a first area, a second area including the first area, and a third area including the second area. [Configuration 9] A focus detection device described in any one of configurations 1 to 8, characterized in that the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified by predetermined depth. [Configuration 10] 10. The focus detection device according to any one of configurations 1 to 9, wherein the subject is a person or an animal, and the specific region is the eyes, face, and entire body of the person or animal. [Configuration 11] 11. The focus detection device according to configuration 10, wherein the higher-level region is the pupil, and the lower-level region is the face or the whole body. [Configuration 12] An imaging means; a focus detection device according to any one of configurations 1 to 11; and a focus control unit that executes the autofocus operation on the target area. [Configuration 13] A focus detection method for a focus detection device, comprising: hierarchically detecting one or more specific regions from a subject in a captured image; setting a focus detection area divided into a plurality of areas for the specific area; detecting focus detection information for each of the focus detection areas; determining a region of interest for autofocus operation; In the determining step, a distribution of defocus amounts in the specific area is obtained, A focus detection method comprising determining the target area included in the upper region based on a distribution of defocus amounts of lower regions including the upper region among the specific regions. [Configuration 14] A program for causing a computer to function as the focus detection device according to any one of configurations 1 to 12. [Explanation of symbols]

[0113] 100... lens device, 200... camera body, 201... imaging section, 204... AF signal processing section, 209... system control section, 211... subject detection section

Claims

1. a subject detection means for hierarchically detecting one or more specific regions from a subject in a captured image; a setting means for setting a focus detection area divided into a plurality of areas for the specific area; a focus detection unit for detecting focus detection information for each of the focus detection areas; a determining means for determining a target area for performing an autofocus operation; the determining means determines a distribution of defocus amounts in the specific region; A focus detection device characterized in that the target area included in the upper region of the specific region is determined based on a distribution of defocus amounts of lower regions including the upper region of the specific region.

2. The focus detection device according to claim 1, characterized in that, when the subject detection means detects the lower region having a predetermined size or larger and the upper region, the determination means determines the target region from the peripheral region of the upper region based on the distribution of defocus amounts of the lower region.

3. 3. The focus detection device according to claim 2, wherein the determining means determines the target area based on the distribution of defocus amounts in areas surrounding the higher-order area and the distribution of defocus amounts in the lower-order area.

4. 4. The focus detection device according to claim 3, wherein the determining means sets the target area based on a result of comparing peak values ​​of the distribution of defocus amounts in the surrounding areas of the higher-ranked area.

5. 4. The focus detection device according to claim 3, wherein said determining means determines the focus detection area of ​​said upper region based on the center of said upper region and the positions of the peripheral regions.

6. If the subject detection means does not detect the lower region having a predetermined size or larger and the upper region, a distribution of defocus amounts is generated for the region among the lower regions in which the number of focus detection regions is equal to or larger than a threshold value; 2. The focus detection device according to claim 1, wherein the target area is determined from an area where the peak value of the generated distribution of defocus amounts is equal to or greater than a predetermined value.

7. 7. The focus detection device according to claim 6, wherein said determining means determines said target area based on the positions of a current target area and said lower-level area.

8. 2. The focus detection device according to claim 1, wherein the specific region includes at least one of a first region, a second region including the first region, and a third region including the second region.

9. 2. The focus detection device according to claim 1, wherein the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified for each predetermined depth.

10. 2. The focus detection device according to claim 1, wherein the subject is a person or an animal, and the specific region is the eyes, face, and entire body of the person or animal.

11. 11. The focus detection device according to claim 10, wherein the higher-order region is the pupil, and the lower-order region is the face or the whole body.

12. An imaging means; A focus detection device according to any one of claims 1 to 11; and a focus control unit that executes the autofocus operation on the target area.

13. A focus detection method for a focus detection device, comprising: hierarchically detecting one or more specific regions from a subject in a captured image; setting a focus detection area divided into a plurality of areas for the specific area; detecting focus detection information for each of the focus detection areas; determining a region of interest for autofocus operation; In the determining step, a distribution of defocus amounts in the specific area is obtained, A focus detection method comprising determining the target area included in the upper region based on a distribution of defocus amounts of lower regions including the upper region among the specific regions.

14. A program for causing a computer to function as the focus detection device according to any one of claims 1 to 11.

Images