Focus detection device, focus detection method, imaging apparatus, and program

Abstract

To provide a technique capable of continuously focusing on the same subject even when the direction of the subject of autofocus control is changed.SOLUTION: A focus detection apparatus includes an object detection unit configured to hierarchically detect one or a plurality of specific areas from an object in a captured image, a setting unit configured to set focus detection areas obtained by dividing the specific area into a plurality of areas, a focus detection unit configured to detect focus detection information for each of the focus detection areas, and a determination unit configured to determine a target area for performing an autofocus operation. When a target area included in an upper area is determined based on the distribution of the defocus amount of a lower area in the specific area, if the distribution of the defocus amount of the upper area is equal to or larger than a predetermined range, a range for obtaining the distribution of the defocus amount of the lower area is changed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to focus detection technology. [Background technology]

[0002] Patent Document 1 describes a method for setting a focus detection area that includes a face area, which is an area that is desired to be in focus, when detecting a person who is the target of autofocus control. Patent Document 2 describes a method for detecting the person's eyes, which are an area that is desired to be in focus, from within a focus detection area that includes a face area, and setting a focus detection area centered on the eyes. Recently, the performance of detecting people's eyes has improved, and it is now possible to detect the eyes even when the person is looking sideways. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-227080 [Patent Document 2] Japanese Patent Application Laid-Open No. 2001-215403 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned patent documents 1 and 2, when a focus detection area is set centered on the pupil of a person's face turned to the side, there is a possibility that the focus detection area will include something other than the person's face. In such cases, the focus detection result will be affected by the inclusion of high-contrast objects other than the person's face. Furthermore, when the focus detection area centered on the pupil includes the face area, the effect on the focus position is minor even if the subject moves or the photographer's hand shakes. However, when a focus detection area centered on the pupil of a person's face turned to the side, the effect is greater because the face area and areas other than the face included in the focus detection area centered on the pupil fluctuate.

[0005] The present invention has been made in view of the above-mentioned problems, and its purpose is to realize a technology that can continue to focus on the same subject that is the target of autofocus control even if the orientation of the subject changes. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the focus detection device of the present invention has a subject detection means that hierarchically detects one or more specific areas from a subject in a captured image, a setting means that sets a focus detection area divided into multiple areas for the specific area, a focus detection means that detects focus detection information for each focus detection area, and a determination means that determines a target area for autofocus operation, wherein the determination means determines the distribution of defocus amounts in the specific area, and when determining a target area to be included in a higher-level area based on the distribution of defocus amounts of lower-level areas in the specific area, if the distribution of defocus amounts of the higher-level area is equal to or greater than a predetermined range, changes the range for determining the distribution of defocus amounts of the lower-level area. [Effects of the Invention]

[0007] According to the present invention, even if the orientation of a subject that is the target of autofocus control changes, it is possible to continue to focus on the same subject. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] 4 is a flowchart illustrating a control process during shooting of the imaging apparatus of the present embodiment. [Figure 3] 6 is a flowchart illustrating an example of an AF frame setting process in a control process during shooting according to the present embodiment. [Figure 4] 4A and 4B are diagrams illustrating an example of a single specific area detected from a subject and an AF frame set in the specific area in this embodiment. [Figure 5] 3A and 3B are diagrams illustrating examples of a plurality of specific areas detected from a subject and AF frames set in each specific area in the present embodiment. [Figure 6] 5 is a flowchart illustrating an example of an AF operation in a control process during shooting according to the present embodiment. [Figure 7] 6 is a flowchart illustrating a focus detection process in the AF operation of the present embodiment. [Figure 8] 6 is a flowchart illustrating a main frame selection process in the AF operation of the present embodiment. [Figure 9] 10 is a flowchart illustrating a detected subject main frame selection process in the main frame selection process of the present embodiment. [Figure 10] 10 is a flowchart illustrating a main frame selection region determination process in the detected subject main frame selection process of this embodiment. [Figure 11] 10 is a flowchart illustrating pupil-priority main frame selection processing in the main frame selection processing of the present embodiment. [Figure 12] 6A to 6C are diagrams for explaining a histogram creation method in the pupil-priority main frame selection process of the present embodiment. [Figure 13] 6A to 6C are diagrams illustrating an example of an AF frame around a pupil region in the pupil-priority main frame selection process of the present embodiment. [Figure 14] 10A and 10B are diagrams illustrating an example of an AF frame around a pupil region in a profile view in the pupil-priority main frame selection process of the present embodiment. [Figure 15] 6A and 6B are diagrams illustrating histograms created in the pupil-priority main frame selection process of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] Below, we will explain an example in which the focus detection device and imaging device of the present invention are applied to a digital camera with interchangeable lenses, but this is not the only example, and the invention can also be applied to, for example, digital cameras with integrated lenses, digital video cameras, smartphones with camera functions, tablet computers, web cameras such as surveillance cameras, medical cameras, etc.

[0011] The imaging apparatus of this embodiment includes the focus detection device of the present invention, and performs autofocus (AF) control using an imaging surface phase difference detection method based on a pair of imaging signals from the imaging section.

[0012] <Device configuration> First, the hardware configuration of the imaging device of this embodiment will be described with reference to FIG.

[0013] FIG. 1 is a block diagram illustrating the hardware configuration of an imaging apparatus according to this embodiment.

[0014] The imaging device of this embodiment includes a lens device (interchangeable lens) 100 and a camera body 200. The lens device 100 is mechanically and electrically connected to a lens mount 106 of the camera body 200. When the lens device 100 is attached to the camera body 200 via the lens mount 106, a lens controller 105 that controls the overall operation of the lens device 100 and a system control unit 209 that controls the overall operation of the camera become able to communicate with each other. The lens mount 106 includes a transmission path (bus) that allows for the exchange of synchronization signals, control signals, various types of data, and the like with the camera body 200.

[0015] Lens device 100 constitutes a photographing optical system that forms an optical image of a subject, which is light reflected from the subject, on imaging unit 201 of camera body 200. Lens device 100 includes photographing lens 101 including a zoom mechanism, aperture / shutter 102 that adjusts the amount of light of the subject image reflected by imaging unit 201, focus lens 103 that adjusts the focus state of the subject image reflected by imaging unit 201, drive unit 104 such as a motor that drives photographing lens 101, aperture / shutter 102, and focus lens 103, and lens controller 105.

[0016] The lens device 100 communicates with a system control unit 209 of the camera body 200 via a lens mount 106, and a lens controller 105 controls a drive unit 104 to operate an aperture / shutter 102 to adjust the brightness of the subject image, and controls the focus state of the subject image by displacing a focus lens 103.

[0017] The camera body 200 captures an image of a subject that has passed through the imaging optical system of the lens device 100 and generates an imaging signal.

[0018] The imaging unit 201 converts the subject image formed on the light receiving surface into an electrical signal using an image sensor such as a CCD or CMOS, and outputs the electrical signal to the A / D conversion unit 202. The A / D conversion unit 202 converts the analog signal input from the imaging unit 201 into a digital signal. The A / D conversion unit 202 also includes a CDS circuit that removes noise from the analog signal, and a nonlinear amplification circuit that nonlinearly amplifies the analog signal before converting it into a digital signal.

[0019] The image processing unit 203 outputs image data by performing predetermined pixel interpolation, resizing processing such as image reduction, and color conversion processing on the digital signal output from the A / D conversion unit 202. The image processing unit 203 also performs predetermined calculation processing using the image data, and the system control unit 209 performs AF processing and AE (auto exposure) processing based on the calculation results.

[0020] Each pixel of the imaging unit 201 of this embodiment includes a plurality (a pair) of photoelectric conversion elements (photodiodes) A ​​and B, and one microlens provided for the pair of photoelectric conversion elements A and B. Each pixel splits incident light using the microlens to form a pair of optical images on the pair of photoelectric conversion elements A and B, and outputs a pair of pixel signals (signal A and signal B) from the pair of photoelectric conversion elements A and B to be used as a focus detection signal, which will be described later. In addition, an imaging signal (signal A+signal B) is obtained by adding the outputs of the pair of photoelectric conversion elements A and B.

[0021] By combining a plurality of A signals and a plurality of B signals output from a plurality of pixels, a pair of image signals is obtained as a focus detection signal used for AF using an image plane phase difference detection method (hereinafter referred to as image plane phase difference AF). The AF signal processing unit 204 performs a correlation operation on the pair of image signals to calculate a phase difference (hereinafter referred to as image shift amount) that is the amount of shift between the pair of image signals, and further calculates the defocus amount (and defocus direction and reliability) of the imaging optical system from the image shift amount. Furthermore, when a plurality of specific regions are detected from the object detected by the object detection unit 211, the AF signal processing unit 204 calculates the defocus amount within each specific region and calculates a defocus distribution based on the calculated defocus amount.

[0022] The format conversion unit 205 converts the format of the image data generated by the image processing unit 203 in order to store the image data in the DRAM 206. The DRAM 206 is an example of a high-speed built-in memory, and is used as a high-speed buffer that temporarily stores image data, or as a working memory for image data compression / decompression processing.

[0023] The image recording unit 207 has a recording medium such as a memory card for recording captured images (still images and moving images) and an interface therefor.

[0024] The timing generating unit 208 supplies clock signals and control signals to the imaging unit 201 and the A / D conversion unit 202. The timing generating unit 208 also controls the reset timing of the charges accumulated in the imaging unit 201, thereby controlling the operations of accumulating and discharging charges in the imaging unit 201.

[0025] The system control unit 209 has a processor (CPU), memory (RAM and ROM), input / output circuits, timer circuits, etc., and controls the operation of the entire device by the CPU expanding and executing programs stored in the ROM into the working area of ​​the RAM.

[0026] The lens communication unit 210 communicates with the lens controller 105 of the lens device 100 attached to the camera body 200 via the lens mount 106 .

[0027] The subject detection unit 211 performs known subject detection processing on the imaging signal output from the A / D conversion unit 202, and detects a subject present in an imaging screen corresponding to the image data generated by the image processing unit 203. The subject detection unit 211 can repeatedly detect one or more specific regions of the subject in the image, stepwise from a higher-level region to a lower-level region. For example, if the subject is a person, the specific regions may be the whole body as a first hierarchical region, the face as a second hierarchical region contained in the first hierarchical region, and the pupils as a third hierarchical region contained in the second hierarchical region. Note that the subject is not limited to a person, and may be an animal, a vehicle, a train, etc., and the specific region may be determined based on the characteristics of the subject.

[0028] The VRAM 212 is a video memory in which data to be displayed on the display unit 213 is drawn. The data generated in the VRAM 212 is transferred to the display unit 213 at a predetermined frame rate, whereby an image is displayed on the display unit 213.

[0029] The display unit 213 includes a liquid crystal panel or an organic panel, and displays images, operation assistance, and the status of the camera. When taking a photograph, it also displays an imaging screen and an AF frame indicating the focus detection area.

[0030] The photographer operates the imaging device using the operation unit 214. The operation unit 214 includes, for example, a menu switch for performing various settings such as exposure compensation and aperture value settings, and settings for image playback, a zoom lever for instructing the zoom operation of the shooting lens, and an operation mode changeover switch between shooting mode and playback mode.

[0031] The shooting mode switch (SW) 215 includes a shooting mode changeover switch for selecting a shooting mode such as macro mode or sports mode.

[0032] The main switch (SW) 216 is a switch for turning on power to the system. The first switch (SW) 217 ​​is a switch for outputting a first switch signal SW1 to the system control unit 209 and performing shooting preparation operations such as AE processing and AF processing. The second switch (SW) 218 ​​is a switch for outputting a second switch signal SW2 to the system control unit 209 and issuing a shooting instruction while the first switch signal SW1 by the first switch 217 is on (shooting preparation state).

[0033] <Control processing during shooting> Next, the control process during image capture in this embodiment will be described with reference to FIG.

[0034] FIG. 2 is a flowchart illustrating an example of AF control processing in the still image shooting mode of this embodiment.

[0035] The control processing of this embodiment is realized by the system control unit 209 loading a program stored in ROM into RAM and executing it to control the components of the lens apparatus 100 and camera body 200.

[0036] Although this embodiment describes AF control processing in still image shooting mode, it is also effective for movie servo AF in movie shooting mode, which keeps the camera focused on a specific subject without user operation.

[0037] In step S201, the system control unit 209 determines whether it has received a shooting preparation instruction via the first switch 217. The system control unit 209 determines whether the first switch signal SW1 is on. If it is not on, the determination continues. If it is on, the process proceeds to step S202.

[0038] In step S202, the system control unit 209 performs an AF frame setting process described later and proceeds to step S203.

[0039] In step S203, the system control unit 209 performs an AF operation described later and proceeds to step S204.

[0040] In step S204, the system control unit 209 determines whether the first switch signal SW1 is on. If it is not on, the process returns to step S201. If it is on, the process proceeds to step S205.

[0041] In step S205, the system control unit 209 determines whether it has received a shooting instruction via the second switch 218. The system control unit 209 determines whether the second switch signal SW2 is on. If it is not on, the process returns to step S201. If it is on, the process proceeds to step S206.

[0042] In step S206, the system control unit 209 performs a shooting process and returns to step S201.

[0043] <AF frame setting process> FIG. 3 is a flowchart illustrating the AF frame setting process in step S202 of FIG. 2.

[0044] In step S301, the system control unit 209 acquires detected subject information from the subject detection unit 211. In the subject detection process of this embodiment, the detection target is a person, and one or more specific regions of the subject to be detected are detected hierarchically. In this embodiment, the first hierarchical region is the whole body, the second hierarchical region is the face, and the third hierarchical region is the pupil.

[0045] Methods for detecting subjects and specific areas include machine learning such as deep learning and image recognition processing.

[0046] For example, the types of machine learning include: (1) Support Vector Machine (2) Convolutional Neural Network (3) Recurrent Neural Network Furthermore, the image recognition process may involve detecting a face by extracting facial features such as the eyes, nose, and mouth using well-known pattern recognition technology. Note that the method for detecting a specific region is not limited to these examples, and other methods may also be used.

[0047] In step S302, the system control unit 209 determines whether or not multiple specific regions have been detected based on the detected subject information acquired from the subject detection unit 211. If multiple specific regions have been detected, the system control unit 209 proceeds to step S303; otherwise, the system control unit 209 proceeds to step S304.

[0048] Here, with reference to Figures 4 and 5, we will explain a state in which detection of a specific area from a detected subject is performed once and a single specific area is detected, and a state in which detection of a specific area from a detected subject is performed multiple times and multiple specific areas are detected.

[0049] Fig. 4(a) illustrates a state in which only a face 401 is detected as a specific region. Fig. 5(a) illustrates a state in which an eye 501, a face 502, and a whole body 503 are detected as specific regions. The subject detection unit 211 acquires the type of subject, such as a person or an animal, and the center coordinates, horizontal size, and vertical size of the specific region detected from the subject.

[0050] In step S303, the system control unit 209 sets the size of the AF frame to MinA, the size of the smallest area among the specific areas. In the example of Fig. 5, the smaller of the horizontal size and vertical size of the pupil 501 is set to MinA, and the set MinA is used as the size of one AF frame 504.

[0051] In step S305, the system control unit 209 calculates the horizontal size H in FIG. 5B that encompasses all specific areas from the horizontal coordinates and horizontal size of each specific area, and determines the number of horizontal AF frames by dividing H by the AF frame size MinA.

[0052] In step S307, the system control unit 209 calculates the vertical size V in Figure 5(b) that encompasses all specific areas from the vertical coordinates and vertical size of each specific area, determines the number of vertical AF frames by dividing V by the AF frame size MinA, and terminates the AF frame setting process.

[0053] In this embodiment, a square AF frame size is set using the minimum size of the specific area, but the AF frame size may be different between the horizontal and vertical sizes, or the number of AF frames may be set up to the number that can be calculated by the system control unit 209.

[0054] In step S304, the system control unit 209 sets an AF frame of a predetermined size X for the detected specific area. In this embodiment, for example, as shown in Fig. 4, when a face 401 is detected as the specific area, an AF frame 402 of size X is set based on the pupil size estimated from the face 401. Note that an AF frame size may be set that ensures an S / N ratio and provides sufficient focusing performance in consideration of low-illumination environments.

[0055] In step S306, the system control unit 209 sets the number Y of AF frames that includes the detected specific area and can cope with the case where the specific area moves, with the AF frame size set in step S304, and ends the AF frame setting process. In the present embodiment, for example, the number Y of AF frames is set so as to include the area of the face 401 in Fig. 4(a) and can cope with the case where the face 401 moves.

[0056] Note that the AF frame setting process of the present embodiment does not have to be executed every time image data is input, and may be performed once every time image data is input a plurality of times. In this case, when the AF frame setting process is not executed, the set AF frame is stored in the DRAM 206, and the AF frame stored at the timing closest in time may be read out and used.

[0057] <AF operation> Fig. 6 is a flowchart illustrating the AF operation of step S203 in Fig. 2.

[0058] In step S601, the system control unit 209 performs a focus detection process, calculates a defocus amount by the AF signal processing unit 204, and advances the process to step S602. Details of the focus detection process will be described later with reference to Fig. 7.

[0059] In step S602, the system control unit 209 performs a main frame selection process based on the detected subject information acquired in step S301 of Fig. 3, and advances the process to step S603. Details of the main frame selection process will be described later with reference to Fig. 8.

[0060] In step S603, the system control unit 209 calculates the driving amount of the focus lens based on the defocus amount of the main frame selected in step S602, and advances the process to step S604. The main frame is the target area for performing the AF operation.

[0061] In step S604, the system control unit 209 transmits the focus lens driving amount calculated in step S603 to the lens device 100 via the lens communication unit 210. The lens controller 105 of the lens device 100 drives the focus lens 103 based on the focus lens driving amount received from the system control unit 209.

[0062] <Focus detection processing> FIG. 7 is a flowchart illustrating the focus detection process in step S601 of FIG.

[0063] In step S701, the system control unit 209 sets a focus detection area of ​​a predetermined range within an imaging screen corresponding to image data, and the process proceeds to step S702.

[0064] In step S702, the system control unit 209 causes the AF signal processing unit 204 to acquire focus detection information (signals A and B) from the focus detection area set in step S701, and proceeds to step S703.

[0065] In step S703, the system control unit 209 performs row averaging processing in the vertical direction on the focus detection signals acquired in step S502, and then proceeds to step S704. This processing can reduce the influence of noise on the focus detection signals.

[0066] In step S704, the system control unit 209 performs filtering to extract signal components in a predetermined frequency band from the signal obtained by vertical row averaging in step S703, and then the process proceeds to step S705.

[0067] In step S705, the system control unit 209 calculates the amount of correlation from the signal that was filtered in step S704, and the process proceeds to step S706.

[0068] In step S706, the system control unit 209 calculates the amount of change in correlation from the amount of correlation calculated in step S705, and the process proceeds to step S707.

[0069] In step S707, the system control unit 209 calculates the amount of image shift from the amount of correlation change calculated in step S706, and the process proceeds to step S708.

[0070] In step S708, the system control unit 209 calculates the reliability of the image shift amount calculated in step S707, and the process proceeds to step S709.

[0071] In step S709, the system control unit 209 converts the amount of image shift calculated in step S707 into a defocus amount, and then ends the process.

[0072] <Main frame selection process> FIG. 8 is a flowchart illustrating the main frame selection process in step S602 of FIG.

[0073] In step S801, the system control unit 209 determines whether or not a subject has been detected by the subject detection unit 211, and if a subject has been detected, proceeds to step S803; if a subject has not been detected, proceeds to step S802.

[0074] In step S802, the system control unit 209 performs multi-point main frame selection processing without using detected subject information, and then ends the processing, because no subject has been detected by the subject detection unit 211. The multi-point main frame selection processing may be, for example, a method of selecting a main frame in a predetermined area within the imaging screen, but detailed description thereof will be omitted.

[0075] In step S803, the system control unit 209 determines whether a face and eyes have been detected as specific regions of the subject detected by the subject detection unit 211. If a face and eyes have been detected, the system control unit 209 proceeds to step S805, and if a face and eyes have not been detected, the system control unit 209 proceeds to step S804.

[0076] In step S804, the system control unit 209 performs a detected subject main frame selection process, and then ends the process. Details of the detected subject main frame selection process will be described later with reference to FIG.

[0077] In step S805, the system control unit 209 performs pupil-priority main frame selection processing, and proceeds to step S806. Details of the pupil-priority main frame selection processing will be described later with reference to FIG.

[0078] In step S806, the system control unit 209 determines whether the distribution of defocus amounts in the pupil region calculated in the pupil-priority main frame selection process in step S805 is equal to or greater than a predetermined range. If the defocus distribution in the pupil region is less than the predetermined range, the system control unit 209 determines that the variation in defocus amounts is small and proceeds to step S603 in Fig. 6, but if the defocus distribution in the pupil region is equal to or greater than the predetermined range, the system control unit 209 proceeds to step S607.

[0079] In step S607, the system control unit 209 determines that the defocus distribution in the pupil region is below a predetermined range or more, that there is a large variation in the defocus amount, and that there is a possibility of perspective conflict occurring with the background, etc., and performs detected subject main frame selection processing to limit the range of the main frame selection region in the main frame selection region determination processing described later in FIG. 10, and then ends the processing.

[0080] <Detected subject main frame selection process> FIG. 9 is a flowchart illustrating the detection subject main frame selection process in steps S804 and S807 of FIG.

[0081] In step S901, the system control unit 209 performs a main frame selection area determination process to determine a target area for main frame selection. Here, the main frame selection area determination process will be described with reference to FIG.

[0082] In step S1001, the system control unit 209 determines whether or not an eye has been detected as a specific region of the subject detected by the subject detection unit 211. If an eye has been detected, the system control unit 209 proceeds to step S1002, and if an eye has not been detected, the system control unit 209 proceeds to step S1004.

[0083] In step S1002, it is determined whether the defocus distribution in the peripheral area adjacent to the pupil area is equal to or greater than a predetermined range. Here, the reason for determining the variation in the defocus distribution in the peripheral area adjacent to the pupil area will be explained. Consider the case where a person as a subject turns their face sideways, as shown in FIG. 14. In this case, the position of the pupil moves to the edge of the face area, causing the background to enter the pupil area. If a perspective conflict with the background occurs, the focus detection result may result in a back-focus state. This increases the variation in the defocus amount in the peripheral area adjacent to the pupil area, and because the histogram of the peripheral area of ​​the pupil area in the pupil-priority main frame selection process described later in FIG. 11 indicates a tendency toward back focus, the selected main frame will also be back-focused.

[0084] Furthermore, if you gradually turn your face away from the front, the number of frames competing for perspective will gradually increase, causing the focus detection result to gradually shift to a back focus state on the infinity side.As the subject gradually moves further away, a false detection will occur, and AF control will be performed with a back focus.

[0085] To avoid the above-described situation, if the defocus distribution around the pupil region is equal to or greater than a predetermined range in step S1002, the system control unit 209 determines that the variation in defocus amount around the pupil region is large due to the face being turned sideways, and limits the range of the main frame selection region. In this embodiment, if the variation in defocus amount around the pupil region is large, the system control unit 209 skips step S1003 and proceeds to step S1004, and does not add the pupil region to the main frame selection region. Also, if the defocus distribution around the pupil region is less than the predetermined range in step S1002, the system control unit 209 determines that the face has not been turned sideways and the variation in defocus amount around the pupil region is small, and proceeds to step S1003.

[0086] In the detected subject main frame selection process performed in step S804 of FIG. 8, since it is determined in step S803 that the pupil region has not been detected, the processes of steps S1002 and S1003 for limiting the range of the main frame selection region are not performed.

[0087] As described above, according to this embodiment, when there is a large variation in the defocus amount around the pupil region, the main frame selection region is changed so that the area around the pupil region is excluded from the main frame selection region for creating a histogram in step S903 of Fig. 9, which will be described later. Therefore, the area around the pupil region is selected as the main frame, and it is possible to avoid a situation in which perspective conflict with the background occurs.

[0088] In addition, in the video shooting mode, the main frame selection area may be changed under the following conditions. 1. Servo AF in video recording mode 2. Eye priority AF 3. The pupil area is more than a specified amount behind the focus, or the difference between the peak value of the histogram and the defocus amount of the pupil area is more than a specified value. Furthermore, if the above conditions 1 and 2 are met, the depth range in still image shooting mode is restricted (narrowed) on the infinity side and relaxed (widened) on the close-up side, and the main frame selection area is changed from the pupil area to the area where the histogram within the face area is maximized.By widening the depth range on the close-up side, it is detected that the face area is in back focus compared to the pupil area.

[0089] In condition 1, even if a perspective conflict with the background occurs in a still image, the image that was in focus up to that point is not wasted, but in the case of a video, if a perspective conflict with the background occurs, the entire video may become unusable, and in a video in which a face gradually turns away, it is desirable to have the back of the head in focus.

[0090] Under conditions 1 and 2 (in the case of pupil-priority AF with servo AF in video shooting mode), if the pupil area tends to be back-focused, the depth range in still image shooting mode is restricted (narrowed) on the infinity side and relaxed (widened) on the close-up side, and the main frame selection area is changed from the pupil area to the area within the face area where the histogram is maximum, thereby reducing the effects of perspective conflict and stabilizing the focus position by focusing on the face area and reducing the occurrence of perspective conflict with the background.

[0091] Furthermore, the main frame selection area may be changed between the moving image shooting mode and the still image shooting mode as follows. Eye-priority AF in video mode and eye-priority AF in still image mode During live view in video shooting mode and live view in still shooting mode In addition, the main frame selection area may be changed in the video shooting mode as follows. During video recording and standby In addition, the main frame selection area may be changed in the still image capture mode as follows. - Both eye and single eye recognition Face angle In step S1004, the system control unit 209 determines whether or not a face has been detected as a specific region of the subject detected by the subject detection unit 211. If a face has been detected, the system control unit 209 proceeds to step S1005, and if a face has not been detected, the system control unit 209 proceeds to step S1006.

[0092] In step S1005, the system control unit 209 adds the face region as the main frame selection region, and proceeds to step S1006.

[0093] In step S1006, the system control unit 209 determines whether or not the entire body has been detected as a specific region of the subject detected by the subject detection unit 211. If the entire body has been detected, the system control unit 209 proceeds to step S1007; if the entire body has not been detected, the system control unit 209 ends the process and proceeds to step S902 in FIG.

[0094] In step S1007, the system control unit 209 adds the whole body region as the main frame selection region, ends the process, and proceeds to step S902 in FIG.

[0095] 9 , in step S902, the system control unit 209 determines whether the number of AF frames that include the main frame selection region determined in step S901 is equal to or greater than a threshold. If the main frame selection region includes AF frames that are equal to or greater than the threshold, the system control unit 209 proceeds to process step S903, and if the main frame selection region does not include AF frames that are equal to or greater than the threshold, the system control unit 209 proceeds to process step S909. In this embodiment, the AF frame that includes the main frame selection region is an AF frame whose center is included in the main frame selection region, but this is not limiting. For example, the AF frame may be an AF frame that includes at least a portion of the main frame selection region, or an AF frame whose area overlaps the main frame selection region by a predetermined percentage or more.

[0096] In step S903, the system control unit 209 classifies the defocus amounts calculated for each AF frame including the main frame selection area into predetermined depths, creates a histogram of the number of AF frames at the AF frame positions illustrated in FIG. 15, and proceeds to step S904.

[0097] In step S904, the system control unit 209 determines whether the peak value (number of AF frames) of the histogram created in step S903 is equal to or greater than a predetermined value (predetermined number). If the peak value is equal to or greater than the predetermined value, the system control unit 209 proceeds to step S905, and if the peak value is less than the predetermined value, the system control unit 209 proceeds to step S909. In this embodiment, the maximum number of AF frames in the histogram is normalized by the total number of AF frames, and the value converted into a ratio is used as the peak value (bin).

[0098] In step S905, the system control unit 209 starts a loop process for all AF frames including the main frame selection area in order to select a main frame from the main frame selection area determined in step S901.

[0099] In step S906, the system control unit 209 determines whether the AF frame to be processed is included in the bin that indicates the peak value of the histogram. If the AF frame is included in the bin, the system control unit 209 proceeds to step S907. If the AF frame is not included in the bin, the system control unit 209 repeats the loop process for the other AF frames.

[0100] In step S907, the system control unit 209 determines whether the AF frame to be processed satisfies the condition that it is located closer to the center of the main frame selection area than the currently selected main frame. If this condition is met, the system control unit 209 proceeds to step S708; if the condition is not met, the system control unit 209 repeats the loop processing for the other AF frames. The center of the main frame selection area may be, for example, the center of the maximum width of the main frame selection area in the horizontal and vertical directions, or the center of gravity of the main frame selection area.

[0101] In step S908, the system control unit 209 updates the AF frame closest to the center of the main frame selection area to the main frame, and repeats the loop process for the other AF frames. By repeating this loop process, it is possible to select the AF frame closest to the center of the main frame selection area from among the AF frames included in the bin as the main frame. When the loop process is complete, the process ends and proceeds to step S603 in FIG. 6.

[0102] In step S909, because the system control unit 209 cannot perform the processes of steps S905 to S908 on the main frame selection area, it selects a main frame using center-priority main frame selection processing, terminates the processing, and proceeds to step S603 in Fig. 6. The center-priority main frame selection processing can be, for example, a method of setting the AF frame at the center of the subject detection area as the main frame, but a detailed description thereof will be omitted.

[0103] <Pupil priority main frame selection process> Fig. 11 is a flowchart illustrating the pupil-priority main frame selection process in step S805 in Fig. 8. Fig. 12, Fig. 13, and Fig. 15 are diagrams for explaining the pupil-priority main frame selection process.

[0104] In step S1101, the system control unit 209 classifies the defocus amounts calculated for each AF frame (A1 in FIG. 12) including the face region into predetermined depths to create a histogram.

[0105] In step S1102, the system control unit 209 starts loop processing of peripheral regions adjacent to the pupil region in order to select a main frame from the regions adjacent to the pupil region. The peripheral region adjacent to the pupil region is, for example, an AF frame (A2 in FIG. 12) whose center is included in the pupil region and an AF frame (A3 in FIG. 12) adjacent to the AF frame, but is not limited to this example and may be an AF frame included within a specific distance from the center of the pupil region.

[0106] In step S1103, the system control unit 209 determines whether the bin into which the AF frame to be processed (target bin) is classified is different from the bin currently selected as the main frame (selected bin). If the bin currently selected as the main frame (selected bin) is different, the system control unit 209 proceeds to step S1104; if the bin is the same, the system control unit 209 proceeds to step S1106. Note that an error value that does not match any bin in the histogram is registered as the initial value of the selected bin, and the system control unit 209 always proceeds to step S1104 the first time the loop processing is executed.

[0107] In step S1104, the system control unit 209 calculates the number of peripheral AF frames by adding the numbers of AF frames in adjacent bins for each of the bin into which the AF frame to be processed is classified and the selected bin, and determines which bin has the larger number of peripheral AF frames. If the bin into which the AF frame to be processed is classified has a larger number of peripheral AF frames, the process proceeds to step S1105. If the selected bin has a larger number of peripheral AF frames, the system control unit 209 does not update either the main AF frame or the selected bin, and repeats the loop process for the other AF frames.

[0108] In step S1105, the system control unit 209 sets the AF frame to be processed as the main frame, sets the bin into which the AF frame to be processed is classified as the selected bin, and repeats the loop process for the other AF frames.

[0109] Here, the processing of steps S1104 and S1105 will be described with reference to FIGS.

[0110] For example, suppose the current main frame is W1 in Figure 13, the currently selected bin is B4 in Figure 15, the AF frame to be processed is W2 in Figure 13, and the bin into which W2 is classified is B1 in Figure 15. In this case, the number of peripheral AF frames for the selected bin is the sum of the numbers of AF frames for bins B3, B4, and B5. The number of peripheral AF frames for the bin into which the AF frame to be processed is classified is the sum of the numbers of AF frames for bins B0, B1, and B2. In the example of Figure 15, the number of peripheral AF frames for the selected bin is greater than the number of peripheral AF frames for the bin into which the AF frame to be processed is classified, so neither the main frame nor the selected bin is updated.

[0111] In step S1104, the numbers of AF frames in adjacent bins were added together to determine the number of peripheral AF frames, but this does not have to be done depending on the characteristics of the subject, and the number of bins to be added may be changed. For example, if the width of the bin is set to be extremely narrow compared to the scale of the depth distribution of the actual face region, it is better to add more, and conversely, if the width is set to be appropriate or wide compared to the scale of the depth distribution of the face region, it is better not to add.

[0112] In step S1106, the system control unit 209 compares the AF frame to be processed with the current main frame to determine which is closer to the center of the pupil region. If the AF frame to be processed is closer to the center of the pupil region, the system control unit 209 proceeds to step S1107. If the current main frame is closer to the center of the pupil region, the system control unit 209 does not update the main frame and repeats the loop processing for the other AF frames.

[0113] In step S1107, the system control unit 209 sets the AF frame to be processed as the main frame, and repeats the loop process for the other AF frames.

[0114] The processes in steps S1106 and S1107 will now be described with reference to FIGS.

[0115] For example, suppose the current main frame is W1 in Fig. 13, the currently selected bin is B4 in Fig. 15, the AF frame to be processed is W4 in Fig. 13, and the bin into which W4 is classified is B4 in Fig. 15. In this case, when comparing which of W1 and W4 is closer to the center of the pupil region in Fig. 13, W4 is closer, so the main frame is updated to W4, the AF frame to be processed.

[0116] If the loop processing in step S1105 is completed, the processing ends and the process proceeds to step S603 in FIG.

[0117] As described above, according to this embodiment, it is possible to select an AF frame that is closer to the center of the pupil area and that does not deviate significantly from the depth distribution of the face, so that it is possible to continue to focus on the same subject even if the orientation of the subject that is the target of autofocus control changes.

[0118] In this embodiment, an algorithm has been described that selects an AF frame around an pupil detection area that does not deviate too far from a detected face, but it can also be applied to selecting an AF frame around a face detection area that does not deviate too far from the depth distribution of the entire body, for example. Also, even if the detected subject types are different, if multiple specific areas of the subject can be detected hierarchically, an algorithm can be used that selects an AF frame around a higher-level area that does not deviate too far from a lower-level area.

[0119] Furthermore, according to this embodiment, when the face is turned sideways and there is a large variation in the defocus distribution around the pupil region as shown in Fig. 14, there is a possibility that a subject other than the subject to be detected is present around the pupil region, or that a perspective conflict with the background is occurring. For this reason, it is possible to select an appropriate main frame for the subject by not adding the pupil region to the main frame selection region, and it is also possible to prevent erroneous detection in which the subject gradually moves away.

[0120] [Other embodiments] This embodiment can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0121] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

[0122] The disclosure of this specification includes the following focus detection device, focus detection method, imaging device, and program. [Configuration 1] a subject detection means for hierarchically detecting one or more specific regions from a subject in a captured image; a setting means for setting a focus detection area divided into a plurality of areas for the specific area; a focus detection unit for detecting focus detection information for each of the focus detection areas; a determining means for determining a target area for performing an autofocus operation; the determining means determines a distribution of defocus amounts in the specific region; A focus detection device characterized in that, when determining a target area to be included in a higher-level area based on the distribution of defocus amounts of lower-level areas in the specific area, if the distribution of defocus amounts of the higher-level area is equal to or greater than a predetermined range, the range for determining the distribution of defocus amounts of the lower-level area is changed. [Configuration 2] The focus detection device according to configuration 1, characterized in that the determination means limits the range for determining the distribution of defocus amounts in the lower region so as not to include the range for determining the distribution of defocus amounts in the upper region. [Configuration 3] The focus detection device described in configuration 2 is characterized in that, when the target area is set and the distribution of defocus amounts in the higher-level area exceeds a predetermined range, the range for obtaining the distribution of defocus amounts in the lower-level area is limited. [Configuration 4] The focus detection device according to any one of configurations 1 to 3, characterized in that when the lower region and the upper region are detected by the subject detection means, the determination means determines the target region from the peripheral region of the upper region based on the distribution of defocus amounts in the lower region. [Configuration 5] The focus detection device according to configuration 4, wherein the determining means determines the target area based on the distribution of defocus amounts in the surrounding area of ​​the higher-order area and the distribution of defocus amounts in the lower-order area. [Configuration 6] The focus detection device according to configuration 5, wherein the determining means sets the target area based on a result of comparing peak values ​​of the distribution of defocus amounts in the surrounding areas of the higher-ranked area. [Configuration 7] The focus detection device according to configuration 5, wherein the determining means determines the focus detection area of ​​the higher-level area based on the center of the higher-level area and the positions of the peripheral areas. [Configuration 8] If the lower region and the upper region are not detected by the subject detection means, a distribution of defocus amounts is generated for a region in the lower region where the number of focus detection regions is equal to or greater than a threshold value; 4. The focus detection device according to any one of configurations 1 to 3, wherein the target area is determined from an area where the peak value of the generated distribution of defocus amounts is equal to or greater than a predetermined value. [Configuration 9] 9. The focus detection device according to configuration 8, wherein the determining means determines the target area based on the positions of the current target area and the lower-level area. [Configuration 10] A focus detection device described in any one of configurations 1 to 9, characterized in that the specific area includes at least one of a first area, a second area including the first area, and a third area including the second area. [Configuration 11] A focus detection device described in any one of configurations 1 to 10, characterized in that the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified by predetermined depth. [Configuration 12] 12. The focus detection device according to any one of configurations 1 to 11, wherein the subject is a person or an animal, and the specific region is the eyes, face, and entire body of the person or animal. [Configuration 13] 13. The focus detection device according to configuration 12, wherein the higher-order region is the pupil, and the lower-order region is the face or the whole body. [Configuration 14] An imaging means; a focus detection device according to any one of configurations 1 to 13; and a focus control unit that executes the autofocus operation on the target area. [Configuration 15] 15. The imaging device according to configuration 14, wherein the determining means changes a range for determining a distribution of defocus amounts of the lower regions when determining the target region in a moving image shooting mode. [Configuration 16] the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified for each predetermined depth, The imaging device according to configuration 14 or 15, wherein the determining means narrows the depth range on the infinity side and widens the depth range on the close-up side in the still image shooting mode, and determines the target area from the lower area where the histogram is maximum. [Configuration 17] A focus detection method for a focus detection device, comprising: hierarchically detecting one or more specific regions from a subject in a captured image; setting a focus detection area divided into a plurality of areas for the specific area; detecting focus detection information for each of the focus detection areas; determining a region of interest for autofocus operation; In the determining step, a distribution of defocus amounts in the specific area is obtained, A focus detection method characterized by, when determining a target area to be included in a higher-level area based on the distribution of defocus amounts of lower-level areas in the specific area, if the distribution of defocus amounts of the higher-level area is equal to or greater than a predetermined range, changing the range for determining the distribution of defocus amounts of the lower-level area. [Configuration 18] A program for causing a computer to function as the focus detection device according to any one of claims 1 to 13. [Explanation of symbols]

[0123] 100... lens device, 200... camera body, 201... imaging section, 204... AF signal processing section, 209... system control section, 211... subject detection section

Claims

1. a subject detection means for hierarchically detecting one or more specific regions from a subject in a captured image; a setting means for setting a focus detection area divided into a plurality of areas for the specific area; a focus detection unit for detecting focus detection information for each of the focus detection areas; a determining means for determining a target area for performing an autofocus operation; the determining means determines a distribution of defocus amounts in the specific region; A focus detection device characterized in that, when determining a target area to be included in a higher-level area based on the distribution of defocus amounts of lower-level areas in the specific area, if the distribution of defocus amounts of the higher-level area is equal to or greater than a predetermined range, the range for determining the distribution of defocus amounts of the lower-level area is changed.

2. 2. The focus detection device according to claim 1, wherein the determining unit limits the range for determining the distribution of defocus amounts in the lower region so as not to include the range for determining the distribution of defocus amounts in the upper region.

3. 3. The focus detection device according to claim 2, wherein when the target area is set and the distribution of defocus amounts in the upper area exceeds a predetermined range, the range for calculating the distribution of defocus amounts in the lower area is limited.

4. The focus detection device according to claim 1, characterized in that, when the lower region and the upper region are detected by the subject detection means, the determination means determines the target region from the peripheral region of the upper region based on the distribution of defocus amounts in the lower region.

5. 5. The focus detection device according to claim 4, wherein the determining means determines the target area based on the distribution of defocus amounts in areas surrounding the higher-order area and the distribution of defocus amounts in the lower-order area.

6. 6. The focus detection device according to claim 5, wherein the determining means sets the target area based on a result of comparing peak values ​​of the distribution of defocus amounts in the surrounding areas of the higher-ranked area.

7. 6. The focus detection device according to claim 5, wherein said determining means determines the focus detection area of ​​said upper region based on the center of said upper region and the positions of the peripheral regions.

8. If the lower region and the upper region are not detected by the subject detection means, a distribution of defocus amounts is generated for a region in the lower region where the number of focus detection regions is equal to or greater than a threshold value; 2. The focus detection device according to claim 1, wherein the target area is determined from an area where the peak value of the generated distribution of defocus amounts is equal to or greater than a predetermined value.

9. 9. The focus detection device according to claim 8, wherein the determining means determines the target area based on the positions of the current target area and the lower-level area.

10. 2. The focus detection device according to claim 1, wherein the specific region includes at least one of a first region, a second region including the first region, and a third region including the second region.

11. 2. The focus detection device according to claim 1, wherein the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified for each predetermined depth.

12. 2. The focus detection device according to claim 1, wherein the subject is a person or an animal, and the specific region is the eyes, face, and entire body of the person or animal.

13. 13. The focus detection device according to claim 12, wherein the higher-order region is the pupil, and the lower-order region is the face or the whole body.

14. An imaging means; a focus detection device according to any one of claims 1 to 13; and a focus control unit that executes the autofocus operation on the target area.

15. 15. The imaging apparatus according to claim 14, wherein the determining unit changes a range for determining a distribution of defocus amounts of the lower-level regions when determining the target region in a moving image shooting mode.

16. the distribution of the defocus amount is a histogram of the number of focus detection areas in which the defocus amount is classified for each predetermined depth, The imaging device according to claim 14, wherein the determining means narrows the depth range on the infinity side and widens the depth range on the close-up side in the still image shooting mode, and determines the target area from the lower area where the histogram is maximum.

17. A focus detection method for a focus detection device, comprising: hierarchically detecting one or more specific regions from a subject in a captured image; setting a focus detection area divided into a plurality of areas for the specific area; detecting focus detection information for each of the focus detection areas; determining a region of interest for autofocus operation; In the determining step, a distribution of defocus amounts in the specific area is obtained, A focus detection method characterized by, when determining a target area to be included in a higher-level area based on the distribution of defocus amounts of lower-level areas in the specific area, if the distribution of defocus amounts of the higher-level area is equal to or greater than a predetermined range, changing the range for determining the distribution of defocus amounts of the lower-level area.

18. A program for causing a computer to function as the focus detection device according to any one of claims 1 to 13.

Images