Imaging device, control method thereof, program, and storage medium

Abstract

To accurately perform focusing in an area with higher priority while avoiding an area where focus detection is difficult.SOLUTION: An imaging apparatus comprises: an imaging unit that picks up an image of a subject; a detection unit that detects the subject from the image picked up by the imaging unit; a setting unit that sets a plurality of focus detection frames in a screen based on a result of detection made by the detection unit; a focus detection unit that detects a focus state and its reliability in each of the plurality of focus detection frames; and a selection unit that selects a main focus detection frame for performing focusing based on results of detection made by the detection unit and the focus detection unit. The selection unit makes different a method for selecting the main focus detection frame depending on a region of the subject detected by the detection unit.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a focus adjustment technique in an imaging device. [Background technology]

[0002] In recent years, various AF methods (autofocus methods) have been put to practical use, such as the image-surface phase-difference AF method and the contrast AF method using an image sensor. Furthermore, in these AF methods, technology has been established to identify and focus on the area of ​​the main subject.

[0003] As a method for achieving this, in Patent Document 1, adjacent clusters within a predetermined depth are detected from a plurality of AF frames, and a control is performed to select a main AF frame from among them.

[0004] Furthermore, in Patent Document 2, in addition to detecting masses within a predetermined depth, the accuracy of identifying the main subject region is improved by using color information.

[0005] Patent Document 3 discloses a method for detecting the pupils and other parts of the face, determining the reliability of the detection results for the detected parts, and setting the focus detection area to an area with a high detection reliability. This makes it possible to focus with the main subject position identified with higher accuracy. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2010-191073 A [Patent Document 2] JP 2015-041901 A [Patent Document 3] JP 2019-121860 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the techniques described in Patent Documents 1 to 3, the area of ​​the main subject is accurately detected and the focus detection area is set based on the detection result, so focus detection is not necessarily performed in an area suitable for focus detection. If the area determined to be the main subject is not suitable for focus detection, accurate focus adjustment may not be possible.

[0008] The present invention has been made in view of the above-mentioned problems, and has an object to perform accurate focus adjustment in areas with higher priority while avoiding areas where focus detection is difficult. [Means for solving the problem]

[0009] An imaging device according to the present invention comprises imaging means for imaging a subject, detection means for detecting the subject from an image captured by the imaging means, setting means for setting a plurality of focus detection frames within a screen based on a detection result of the detection means, focus detection means for detecting a focus state and its reliability in each of the plurality of focus detection frames, and selection means for selecting a main focus detection frame for focus adjustment based on detection results of the detection means and the focus detection means, wherein the selection means selects a part of the subject detected by the detection means to be focused on in the order of the pupil, the face, and the body, The selection means selects a region to be focused on depending on whether the eye or face is selected or the body is selected. The method for selecting the main focus detection frame is made different, and the selection means further makes the method for selecting the main focus detection frame different depending on whether the subject is a still subject or a moving subject. Effect of the Invention

[0010] According to the present invention, it is possible to perform accurate focus adjustment in areas with higher priority while avoiding areas where focus detection is difficult. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing the configuration of a digital camera as an embodiment of an imaging apparatus of the present invention. [Diagram 2]4 is a flowchart illustrating the operation of the digital camera according to an embodiment. [Diagram 3] 6 is a flowchart for explaining an AF frame setting operation. [Figure 4] FIG. 1 is a diagram showing the concept of a detection region for a human face. [Diagram 5] 1 is a diagram showing the concept of detection areas for a person's eyes, face, and body. [Figure 6] 1 is a diagram showing the concept of detection areas for the pupils, face, and body of an animal. [Figure 7] 5 is a flowchart illustrating an AF operation in the digital camera. [Figure 8] 6 is a flowchart illustrating a focus detection process. [Figure 9] 6 is a flowchart for explaining an AF main frame selection operation. [Figure 10] 6 is a flowchart for explaining a selection operation of the AF main frame with priority given to the detection center. [Figure 11] 11 is a flowchart for explaining the selection operation of the AF main frame with priority given to a central area close to the subject. [Figure 12] 11 is a flowchart for explaining a selection operation of an AF main frame with priority given to detection area reliability. [Figure 13] 11 is a flowchart for explaining a selection operation of an AF main frame with priority given to central area reliability. [Figure 14] 11 is a flowchart for explaining a selection operation of an AF main frame with priority given to central area prediction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0013] FIG. 1 is a block diagram showing the configuration of a digital camera 300 which is an embodiment of an imaging apparatus of the present invention.

[0014] In FIG. 1, a digital camera 300 is configured such that an interchangeable lens 100 is detachably (replaceably) attached to a camera body 200 via a mount portion (not shown) having an electrical contact unit 106.

[0015] The interchangeable lens 100 includes, as an imaging optical system, a photographing lens 101 including a zoom mechanism, an aperture and shutter 102 for controlling the amount of light, and a focus lens 103 for focusing on an image sensor described below. A motor 104 drives the focus lens 103, and a lens controller 105 controls the entire interchangeable lens 100.

[0016] The camera body 200 includes an image sensor 201. The image sensor 201 includes a large number of pixels each having a photodiode that converts reflected light from a subject into an electrical signal. The A / D conversion unit 202 includes a CDS circuit that removes output noise from the image sensor 201 and a nonlinear amplification circuit that performs A / D conversion before the A / D conversion, and converts an analog signal from the image sensor 201 into a digital signal. The camera body 200 further includes an image processing unit 203, an AF signal processing unit 204, a format conversion unit 205, and a high-speed built-in memory (for example, a random access memory, hereinafter referred to as DRAM) 206.

[0017] The image recording unit 207 is composed of a recording medium such as a memory card and its interface. The timing generator 208 generates timing signals that control the timing of the operation of the digital camera 300, and the system control unit 209 controls the overall operation of the digital camera, such as the shooting sequence. The lens communication unit 210 communicates between the camera body 200 and the interchangeable lens 100. The subject detection unit 211 detects the subject from the captured image signal.

[0018] An image display memory (hereinafter referred to as VRAM) 212 stores images for display. In addition to displaying images, an image display unit 213 displays operation assistance and the status of the camera. Furthermore, during shooting, it displays a shooting screen and a focus detection area. An operation unit 214 has an operation member for operating the camera from the outside. A shooting mode switch 215 is an operation member for selecting a shooting mode such as macro mode or sports mode. A main switch 216 is a switch for turning on the power to the digital camera 300. A release switch (SW1) 217 ​​is turned on when the release button is pressed halfway, and starts shooting preparation operations such as AF and AE. A release switch (SW2) 218 ​​is turned on when the release button is pressed all the way, and starts a shooting operation.

[0019] The DRAM 206 is used as a high-speed buffer as a temporary image storage means, a working memory for image compression / expansion, etc. The operation unit 214 includes, for example, the following: a menu switch for performing various settings such as the settings of the shooting function and image playback function of the imaging device, and an operation mode change switch between shooting mode and playback mode.

[0020] The image sensor 201 is composed of a CCD or CMOS sensor. A light beam passing through the photographing optical system of the interchangeable lens 100 is imaged on the light receiving surface of the image sensor 201 and converted by a photodiode into a signal charge according to the amount of incident light. The signal charge accumulated in each photodiode is converted into a voltage signal and sequentially read out from the image sensor 201 based on a drive pulse from a timing generator 208 in response to a command from a system control unit 209.

[0021] Each pixel of the image sensor 201 used in the camera body 200 includes two (a pair) of photodiodes A and B and one microlens provided in common to the pair of photodiodes A and B. Each pixel splits incident light with the microlens to form a pair of optical images on the pair of photodiodes A and B, and outputs a pair of pixel signals (signals A and B) used as an AF signal, which will be described later, from the pair of photodiodes A and B. In addition, an image signal (signal A+B) can be obtained by adding the outputs of the pair of photodiodes A and B.

[0022] A pair of image signals is obtained as an AF signal (in other words, a focus detection signal) used for AF by an image plane phase difference detection method (hereinafter referred to as image plane phase difference AF) by combining a plurality of A signals and a plurality of B signals output from a plurality of pixels. The AF signal processor 204 performs a correlation operation on the pair of image signals to calculate a phase difference (hereinafter referred to as an image shift amount) which is the amount of shift between the pair of image signals, and further calculates a defocus amount (and a defocus direction and reliability: a focus state) of the photographing optical system from the image shift amount. The AF signal processor 204 also calculates the defocus amount in a plurality of areas (focus detection areas) that can be specified on the screen.

[0023] The operation of the digital camera 300 of this embodiment will be described below with reference to Fig. 2. Fig. 2 is a flowchart showing the operation of the digital camera 300.

[0024] First, in step S201, the system control unit 209 checks the state of the release switch (SW1) 217, and if it is ON, the process proceeds to step S202, and if it is OFF, the process waits as is.

[0025] In step S202, the system control unit 209 sets an AF frame (focus detection frame), which will be described later, in the AF signal processing unit 204, and the process proceeds to step S203.

[0026] In step S203, the system control unit 209 performs an AF operation, which will be described later, and the process proceeds to step S204.

[0027] In step S204, the system control unit 209 checks the state of the release switch (SW1) 217, and if it is ON, proceeds to step S205, and if not, returns to step S201.

[0028] In step S205, the system control unit 209 checks the state of the release switch (SW2) 218, and if it is ON, proceeds to step S206, and if not, returns to step S204.

[0029] In step S206, the system control unit 209 performs a shooting operation, and then the process returns to step S201.

[0030] FIG. 3 is a flowchart illustrating the process of setting a focus detection frame in this embodiment.

[0031] First, in step S301, the system control unit 209 acquires subject detection information from the subject detection unit 211. In this embodiment, subjects include people, animals such as dogs and wild birds, and main areas within the subjects are detected. The main areas are the eyes, face and body of people and animals. As a detection method for these, a learning method using deep learning, which is a known technology, or an image processing method is used. In this embodiment, a known method can be used as a method for detecting the subject, and therefore a detailed description thereof will be omitted.

[0032] In step S302, system control unit 209 determines whether or not multiple main regions have been detected from the detection result of subject detection unit 211. If multiple main regions have been detected, the process proceeds to step S303; otherwise, the process proceeds to step S304.

[0033] Here, the detection concept when a single main area is detected and when multiple main areas are detected will be explained using Fig. 4 and Fig. 5. Fig. 4(a) shows a state where only face a is detected, and Fig. 5(a) shows a state where pupil A, face B, and body C are detected. It is assumed that it is possible to obtain from the subject detection unit 211 the type of subject, such as a person or animal, and the central coordinates, horizontal size, and vertical size of each detected main area.

[0034] In step S303, the system control unit 209 inputs the minimum value of the size of the detected main area, that is, the smaller of the horizontal and vertical sizes of the area of ​​pupil A in Figure 5 (a), into MinA and sets MinA as one AF frame size.

[0035] In step S305, the system control unit 209 calculates the horizontal size H shown in Figure 5(b) that encompasses all of the main areas from the horizontal coordinates and horizontal size of each detected main area, and determines the number of horizontal AF frames by dividing this H by the AF frame size MinA.

[0036] 5B that encompasses all of the main areas from the vertical coordinates and vertical sizes of the respective detected main areas, and determines the number of vertical AF frames by dividing V by the AF frame size MinA. Then, the AF frame setting is completed.

[0037] The control flow for animals is the same as for people, and the concepts of detection area and AF frame setting are shown in Figures 6(a) and 6(b), respectively. In this embodiment, a square AF frame is used using the minimum size of the main area, but the AF frame size may be made different in the horizontal and vertical directions, or the number of AF frames that can be calculated by the system control unit 209 may be set.

[0038] In step S304, the system control unit 209 sets an AF frame of a predetermined size X for the detected face. The size X may be set to the pupil size estimated from the face, or a frame size may be set that ensures a good S / N ratio and sufficient focusing performance in a low-illumination environment. In this embodiment, the estimated pupil size is set to size X.

[0039] In step S306, the system control unit 209 sets the number of AF frames Y such that the area of ​​the face a can be contained within the AF frame size.

[0040] FIG. 7 is a flowchart for explaining the AF operation in step S203 in FIG.

[0041] First, in step S401, the system control unit 209 performs focus detection processing to detect the defocus amount and reliability (degree of reliability), and proceeds to step S402. The focus detection processing will be described later.

[0042] In step S402, the system control unit 209 selects the AF main frame using the reliability obtained in step S401, and proceeds to step S403. The selection of the AF main frame will be described later.

[0043] In step S403, the system control unit 209 checks whether the reliability of the defocus amount detected in step S401 is higher than a preset reliability threshold 2. If the reliability is higher than the reliability threshold 2, the process proceeds to step S404, and if not, the process proceeds to step S413. Here, the reliability threshold 2 is set to a threshold value such that if the reliability is less than the reliability threshold 2, the accuracy of the defocus amount cannot be guaranteed but the focus position direction of the subject can be guaranteed. In step S404, the system control unit 209 checks whether the defocus amount detected in step S401 is equal to or less than a preset Def amount threshold 2. If it is equal to or less than the Def amount threshold 2, the process proceeds to step S405, and if not, the process proceeds to step S412. Here, the Def amount threshold 2 is set to a value (e.g., 5 times the focal depth) such that if the defocus amount is equal to or less than the Def amount threshold 2, the focus lens can be controlled within the focal depth by performing the lens drive by the defocus amount within a predetermined number of times (e.g., 3 times).

[0044] In step S405, the system control unit 209 checks whether the focus lens 103 is in a stopped state, and if it is in a stopped state, the process proceeds to step S406, and if not, the process proceeds to step S410.

[0045] In step S406, the system control unit 209 checks whether the reliability of the defocus amount detected in step S401 is higher than a preset reliability threshold 1. If the reliability is higher than the reliability threshold 1, the process proceeds to step S407, and if not, the process proceeds to step S410. Here, the reliability threshold 1 is set so that if the reliability is higher than the reliability threshold 1, the accuracy variation of the defocus amount falls within a predetermined range (for example, within the focal depth).

[0046] In step S407, the system control unit 209 checks whether the defocus amount detected in step S401 is equal to or less than a preset Def amount threshold 1. If it is equal to or less than Def amount threshold 1, the process proceeds to step S408, and if not, the process proceeds to step S409. Here, the Def amount threshold 1 is set to a value that indicates that the focus lens is controlled to be within the focal depth if the detected defocus amount is equal to or less than Def amount threshold 1.

[0047] In step S408, the system control unit 209 determines that the focus state is achieved, and ends this flow.

[0048] In step S409, the system control unit 209 drives the focus lens 103 by the defocus amount detected in step S401, and then returns to step S401.

[0049] By performing a series of operations from step S405 to step S409, if the reliability detected in step S401 is higher than the reliability threshold 1, the defocus amount can be detected again with the lens stopped.

[0050] In step S410, the system control unit 209 drives the focus lens 103 by a predetermined percentage of the defocus amount detected in step S401, and the process proceeds to step S411.

[0051] In step S411, the system control unit 209 instructs the focus lens 103 to stop, and the process returns to step S401.

[0052] In step S412, the system control unit 209 drives the focus lens 103 by a predetermined percentage of the defocus amount detected in step S401, and returns to step S401. Here, the predetermined percentage is set so that the lens drive amount is smaller than the defocus amount (e.g., 80%). Also, the lens speed is set so that it is slower than the speed at which the lens can be driven exactly in one frame time, for example. This makes it possible to prevent the focus position of the subject from being exceeded when the detected defocus amount is incorrect, and further makes it possible to drive the lens without stopping it while driving it to perform the next lens drive (overlap control).

[0053] In step S413, the system control unit 209 checks whether or not the out-of-focus condition is satisfied. If the out-of-focus condition is satisfied, the process proceeds to step S414. If not, the process proceeds to step S415. The out-of-focus condition here is a condition for determining that there is no subject to be focused on, such as when lens drive is completed over the entire movable range of the focus lens 103. In other words, the condition is when the focus lens 103 detects both the far-side and near-side lens ends and returns to the initial position.

[0054] In step S414, the system control unit 209 determines that the camera is out of focus, and ends this flow.

[0055] In step S415, the system control unit 209 checks whether the focus lens 103 has reached the lens end on the far side or the near side. If it has reached the lens end, the system control unit 209 proceeds to step S416, and if not, the system control unit 209 proceeds to step S417.

[0056] In step S416, system control unit 209 reverses the driving direction of focus lens 103 and returns to step S401. In step S417, focus lens 103 is driven in a predetermined direction and the process returns to step S401. The focus lens speed is set to the fastest speed within the range of lens speeds that will not cause the focus position to be passed when the defocus amount can be detected, for example.

[0057] The focus detection process in step S401 will be described with reference to FIG.

[0058] First, in step S501, the system control unit 209 sets a focus detection area of ​​an arbitrary range within the image sensor 201, and the process proceeds to step S502.

[0059] In step S502, the system control unit 209 acquires a pair of image signals (image A and image B) for focus detection from the image sensor 201 for the focus detection area set in step S501, and proceeds to step S503.

[0060] In step S503, the system control unit 209 performs processing for vertically averaging the pair of signals acquired in step S502, and then proceeds to step S504. This processing makes it possible to reduce the influence of noise on the image signal.

[0061] In step S504, the system control unit 209 performs filtering to extract signal components in a predetermined frequency band from the signal averaged in the vertical direction in step S503, and then proceeds to step S505.

[0062] In step S505, system control unit 209 calculates the amount of correlation from the signal that was filtered in step S504, and proceeds to step S506.

[0063] In step S506, system control unit 209 calculates the amount of correlation change from the amount of correlation calculated in step S505, and proceeds to step S507.

[0064] In step S507, the system control unit 209 calculates the amount of image shift from the amount of correlation change calculated in step S506, and the process proceeds to step S508.

[0065] In step S508, the system control unit 209 calculates the reliability indicating how reliable the image shift amount calculated in step S507 is, and proceeds to step S509.

[0066] In step S509, the system control unit 209 converts the amount of image shift into a defocus amount, and ends the focus detection process.

[0067] Fig. 9 is a flowchart for explaining the operation of selecting the main frame in step S402 in Fig. 7. In selecting the main focus detection frame, a search is made for the main focus detection frame starting from the subject with the highest priority according to the part of the subject.

[0068] First, in step S601, the system control unit 209 sets the main frame to an initial position as a preparation for main frame selection.

[0069] In step S602, system control unit 209 determines whether or not the pupil of the subject has been detected by subject detection unit 211. If the pupil has been detected, the process proceeds to step S604, and if not, the process proceeds to step S603.

[0070] In step S604, the system control unit 209 sets the selected region of the main frame to the pupil region, and in the following step S605, performs main frame selection with priority given to the detection center, which will be described later.

[0071] In step S607, the system control unit 209 determines whether the reliability of the selected main frame is equal to or greater than a main frame reliability threshold for the result of selecting the main frame in the pupil region. If it is determined that the reliability is equal to or greater than the threshold, the process proceeds to step S614; otherwise, the process proceeds to step S603. Step S607 is for determining whether the detected defocus amount of the main frame selected in the pupil detection region is within a predetermined variation. For example, the main frame reliability threshold may be set to the reliability threshold 1 described above. If it is determined in step S607 that it is difficult to select the main frame in the pupil region, the process proceeds to step S603, and the main frame is selected in the face region.

[0072] In step S603, system control unit 209 determines whether or not a subject's face has been detected by subject detection unit 211. If a face has been detected, the process proceeds to step S608, and if not, the process proceeds to step S611.

[0073] In step S608, the system control unit 209 sets the selected region of the main frame to the face region, and in the following step S609, performs main frame selection with priority given to the detection center, which will be described later.

[0074] In step S610, the system control unit 209 determines whether the reliability of the selected frame is equal to or greater than a main frame reliability threshold for the result of the main frame selection in the face region. If it is determined that the reliability is equal to or greater than the threshold, the process proceeds to step S614; otherwise, the process proceeds to step S611. Step S610 is for determining whether the detected defocus amount of the main frame selected in the face detection region is within a predetermined variation. For example, the main frame reliability threshold may be set to a reliability threshold of 1, similar to that in the case of the pupil region. If it is determined in step S610 that it is difficult to select the main frame in the face region, the process proceeds to step S611, and the main frame is selected in the body region.

[0075] In step S611, system control unit 209 determines whether or not the subject's body has been detected by subject detection unit 211. If the body has been detected, the process proceeds to step S612, and if not, the process proceeds to step S614.

[0076] In step S612, the system control unit 209 sets the selected region of the main frame to the body region, and in the following step S613, performs main frame selection with priority given to the central region closest to the main frame, which will be described later.

[0077] In step S614, the system control unit 209 determines whether the main frame is at the initial position in order to confirm whether the main frame has finally been set to the pupil, face, or body. If the main frame is at the initial value, the process proceeds to step S615; if not, the main frame selection process ends.

[0078] In step S615, the system control unit 209 selects a multi-point main frame. Note that in step S615, a method of selecting a main frame in a predetermined area on the screen without using detection information can be considered, but since this is not a main part of the present embodiment, detailed description will be omitted.

[0079] Next, the main frame selection process suitable for each detected part will be explained using Fig. 10 and Fig. 11. In this embodiment, when the detected part is an eye or a face, the area in the subject is relatively small, so the main frame selection process that prioritizes the detection center position is used. Fig. 10 is a flowchart of the main frame selection operation that prioritizes the detection center.

[0080] In steps S701 to S704, the same process is carried out for all focus detection frames within the focus detection area.

[0081] In step S702, the system control unit 209 determines whether the focus detection frame is closer to the center of the detection area than the focus detection frame currently set as the main frame. If it is closer, the process proceeds to step S703 to update the main frame, and if it is farther, the process proceeds to step S704. The above process is performed for all focus detection frames, and the main frame selection process is completed.

[0082] Next, the main frame selection process when the detected part is the body will be described. When the detected part is the body, a relatively large area is detected within the subject. When selecting the main frame in the body area where the subject shape may be complex, the main frame selection process places emphasis on a frame (closest area priority) in the detected area where the subject is not missing in the central area where the probability of the subject being present is high. Fig. 11 is a flowchart showing the main frame selection operation with close center area priority.

[0083] In step S801, the system control unit 209 initially sets the number of frames for setting a search region for the main frame in the following step S802.

[0084] In step S802, the system control unit 209 sets a main frame search area, and proceeds to step S803. In steps S803 to S807, the same process is carried out for all focus detection frames within the focus detection area.

[0085] In step S804, the system control unit 209 determines whether or not the focus detection frame is within the main frame search area, and if it is within the area, the process proceeds to step S805. In step S805, the system control unit 209 determines whether or not the focus detection frame with the detected defocus amount is closer than the currently set main frame. If it is closer, the system control unit 209 performs main frame update in the following step S806, and if it is not closer, the process proceeds to step S807 to determine the next focus detection frame. The above process is performed for all focus detection frames, and the process proceeds to step S808.

[0086] In step S808, the system control unit 209 determines whether the reliability of the detected defocus amount of the selected main frame is equal to or greater than a predetermined threshold. If it is determined that the reliability is equal to or greater than the predetermined threshold, the main frame selection process is completed. If it is determined in step S808 that the reliability is lower than the predetermined threshold, the process proceeds to steps S809 and S810, where the search area of ​​the main frame is expanded, and steps S802 to S808 described above are performed. If the main frame search of all focus detection areas is completed in step S809, the main frame selection process is completed.

[0087] By applying this embodiment, when multiple parts of a subject are detected, it is possible to perform accurate focus adjustment in a higher priority area while avoiding areas where focus detection is difficult. For example, when the subject is an animal that moves around against the photographer's will, such as a dog, cat, or bird, it is possible to perform focus adjustment in a higher priority area while avoiding areas where the reliability of the detected defocus amount is low and focus detection is difficult.

[0088] Although one embodiment of the present invention has been described in detail above, the present invention is not limited to the above embodiment, and various forms that do not depart from the gist of the present invention are also included in the present invention.

[0089] In this embodiment, when an eye or face is detected, a main frame selection process that prioritizes the detection center position is used, and when a body is detected, a main frame selection process that prioritizes a frame in the central area where the subject is likely to exist in the detected area and where the subject is not missing is used. For subjects such as eyes and faces that tend to be relatively small in size within the subject, a main frame selection process that prioritizes the detection center position is performed to almost eliminate the deviation of the subject's detection position. However, when it is necessary to consider the deviation of the detection position, a process may be performed to select a focus detection frame with a higher reliability of the detected defocus amount as the main frame. FIG. 12 is a flowchart showing a main frame selection operation that prioritizes a focus detection frame with a higher reliability of the detected defocus amount within the detection area.

[0090] In steps S901 to S905, the same process is performed for all focus detection frames within the focus detection area. In step S902, the system control unit 209 determines whether or not the focus detection frame is within the area of ​​the detection part for the subject. If it is within the area, in the following step S903, it is determined whether or not the focus detection frame has a higher reliability than the focus detection frame currently set as the main frame, and if so, the process proceeds to step S904 and the main frame is updated. If it is determined in step S903 that the focus detection frame has a lower reliability than the focus detection frame currently set as the main frame, the process proceeds to step S905, where this determination is performed for all focus detection frames and the main frame selection process is completed.

[0091] In addition, in this embodiment, when a body is detected and the reliability of the main frame is low for the eyes and face, the main frame selection process is used, which takes into consideration the complexity of the subject shape for the body area and prioritizes a frame in which the subject is not missing in the central area where the probability of the subject being present is high. However, in cases where the complexity of the subject shape does not need to be considered, a process may be performed to select a focus detection frame with a higher reliability of the detected defocus amount as the main frame. Figure 13 is a flowchart showing the main frame selection operation with priority given to the central area reliability.

[0092] In step S1001, the system control unit 209 initially sets the number of frames for setting a search region for the main frame in the following step S1002.

[0093] In step S1002, the system control unit 209 sets a main frame search area, and proceeds to step S1003. In steps S1003 to S1007, the same process is carried out for all focus detection frames within the focus detection area.

[0094] In step S1004, the system control unit 209 determines whether or not the focus detection frame is within the main frame search area, and if it is within the area, proceeds to step S1005. In step S1005, it determines whether or not the focus detection frame has a higher reliability of the detected defocus amount than the currently set main frame. If it is higher, the main frame is updated in the following step S1006, and if it is lower, proceeds to step S1007 to proceed to the determination of the next focus detection frame. The above process is performed for all focus detection frames, and proceeds to step S1008.

[0095] In step S1008, the system control unit 209 determines whether the reliability of the detected defocus amount of the selected main frame is equal to or greater than a predetermined threshold. If it is determined in step S1008 that the reliability is equal to or greater than the predetermined threshold, the main frame selection process is completed. If it is determined in step S1008 that the reliability is low, the process proceeds to steps S1009 and S1010, where the search area of ​​the main frame is expanded, and the above-mentioned steps S1002 to S1008 are performed. If the main frame search of all focus detection areas is completed in step S1009, the main frame selection process is completed.

[0096] In addition, in a scene where there is a great concern of missing the subject, regardless of the detected part of the subject, such as when the subject moves rapidly, the main frame selection process may be set based on whether the focus adjustment mode is for shooting a still object (still subject) or a moving object (moving subject). In the case of the moving object shooting mode, the subject position in the target frame is predicted from the history information of the subject position in the past multiple frames, and the focus position is tracked. In this modified example, a process of preferentially selecting a focus detection frame that indicates a position close to the predicted subject position as described above as the main frame will be described. FIG. 14 is a flowchart showing the main frame selection operation with center area prediction priority.

[0097] In step S1101, the system control unit 209 initially sets the number of frames for setting a main frame search area in the following step S1102. In step S1102, the system control unit 209 sets the main frame search area, and the process proceeds to step S1103.

[0098] In steps S1103 to S1107, the same process is performed for all focus detection frames within the focus detection area. In step S1104, the system control unit 209 determines whether or not the focus detection frame is within the main frame search area, and if it is within the area, proceeds to step S1105. In step S1105, it determines whether the focus detection frame is closer to the subject position predicted from the detected defocus amount and the current lens position than the currently set main frame. If it is closer, the main frame is updated in the following step S1106, and if it is lower, proceeds to step S1107 to determine the next focus detection frame. The above process is performed for all focus detection frames, and proceeds to step S1108.

[0099] In step S1108, the system control unit 209 determines whether the reliability of the detected defocus amount of the selected main frame is equal to or greater than a predetermined threshold. If it is determined in step S1108 that the reliability is equal to or greater than the predetermined threshold, the main frame selection process is completed. If it is determined in step S1108 that the reliability is low, the process proceeds to steps S1109 and S1110, where the search area of ​​the main frame is expanded, and the above-mentioned steps S1102 to S1108 are performed. If the main frame search of all focus detection areas is completed in step S1109, the main frame selection process is completed.

[0100] By applying this modified example, it becomes possible to perform accurate focus adjustment in areas with higher priority while avoiding areas where focus detection is difficult, even for subjects with rapid movement.

[0101] (Other embodiments) The present invention can also be realized by a process in which a program for realizing one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for realizing one or more functions.

[0102] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0103] 100: interchangeable lens, 101: photographing lens, 103: focus lens, 105: lens controller, 200: camera body, 201: imaging element, 209: system control unit, 211: subject detection unit, 300: digital camera

Claims

1. An imaging means for imaging a subject; a detection means for detecting a subject from an image captured by the imaging means; a setting means for setting a plurality of focus detection frames within a screen based on a detection result of the detection means; a focus detection unit that detects a focus state and its reliability in each of the plurality of focus detection frames; a selection means for selecting a main focus detection frame for performing focus adjustment based on the detection results of the detection means and the focus detection means, the selection means selects a part to be focused on in the order of eyes, face and body of the subject detected by the detection means; the selection means selects the main focus detection frame in a different manner depending on whether an eye or a face is selected as a part to be focused on or whether a body is selected as a part to be focused on; The imaging apparatus according to claim 1, wherein the selection means selects the main focus detection frame in a different manner depending on whether the subject is a still subject or a moving subject.

2. 2. The imaging device according to claim 1, wherein the selection means selects the main focus detection frame in a different manner depending on whether the focus adjustment mode is a mode for focusing on a still subject or a mode for focusing on a moving subject.

3. 3. The image pickup apparatus according to claim 1, wherein the selection means changes a method of selecting the main focus detection frame in accordance with a difference in size between different portions detected by the detection means.

4. 4. The imaging device according to claim 1, wherein the selection means selects the main focus detection frame based on the detection result of the detection means, the setting of the setting means, and the detection result of the focus detection means.

5. 2. The imaging device according to claim 1, wherein, when a plurality of different parts of a subject are detected by the detection means, the setting means sets the plurality of focus detection frames to an area that includes detection areas of the plurality of different parts.

6. 6. The image pickup apparatus according to claim 5, wherein the setting means sets the plurality of focus detection frames based on the size of a smallest area among the portions of the subject detected by the detection means.

7. 2. The image pickup apparatus according to claim 1, wherein, when an eye is detected as a part of the subject, the selection means selects the main focus detection frame by giving priority to the center of the part.

8. 2. The image pickup apparatus according to claim 1, wherein, when a face is detected as a part of the subject, the selection means selects the main focus detection frame by giving priority to the center of that part.

9. 2. The image pickup apparatus according to claim 1, wherein the selection means, when a body is detected as a part of the subject, selects the main focus detection frame by giving priority to a nearby subject.

10. A method for controlling an imaging device including an imaging unit that captures an image of a subject, comprising: a detection step of detecting a subject from an image captured by the imaging means; a setting step of setting a plurality of focus detection frames within a screen based on a detection result of the detection step; a focus detection step of detecting a focus state and its reliability in each of the plurality of focus detection frames; a selection step of selecting a main focus detection frame for performing focus adjustment based on the detection results of the detection step and the focus detection step, In the selection step, a part to be focused on is selected from the object detected in the detection step, in the order of eyes, face, and body; In the selection step, a method for selecting the main focus detection frame is made different depending on whether an eye or a face is selected as a part to be focused on or whether a body is selected as a part to be focused on; The method for controlling an image pickup apparatus, wherein the selection step further comprises changing a method for selecting the main focus detection frame depending on whether the subject is a still subject or a moving subject.

11. A program for causing a computer to execute each step of the control method according to claim 10.

12. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 10.

13. An imaging means for imaging a subject; a detection means for detecting a subject from an image captured by the imaging means; a setting means for setting a main focus detection frame within the part of the subject detected by the detection means; a focus detection unit for detecting a focus state of the main focus detection frame; a focus control unit that controls a focus lens based on a focus state of the main focus detection frame, the setting means selects a part to be focused on in the order of eyes, face and body of the subject detected by the detection means; The imaging device is characterized in that the setting means sets the main focus detection frame within the subject's body in a different manner depending on whether the eye or face is selected as the part to be focused on or whether the body is selected.

14. 14. The image pickup apparatus according to claim 13, wherein, when an eye is detected as a part of the subject, the setting means sets the main focus detection frame by giving priority to the center of the part.

15. 14. The image pickup apparatus according to claim 13, wherein, when a face is detected as a part of the subject, the setting means sets the main focus detection frame by giving priority to the center of that part.

16. 14. The image pickup apparatus according to claim 13, wherein the setting unit sets the main focus detection frame by giving priority to a closest subject when a body part is detected as a subject part.

17. 17. The image pickup apparatus according to claim 13, wherein the setting unit uses different methods for setting the main focus detection frame depending on whether the subject is a still subject or a moving subject.

18. 18. The imaging device according to claim 13, wherein the setting means uses a different method for setting the main focus detection frame depending on whether the focus adjustment mode is a mode for focusing on a still subject or a mode for focusing on a moving subject.

19. 19. The imaging apparatus according to claim 13, wherein the setting unit changes a method for setting the main focus detection frame according to a difference in size between different portions detected by the detection unit.

20. 20. The imaging apparatus according to claim 13, wherein the setting means sets the main focus detection frame based on a detection result of the detection means, a setting of the setting means, and a detection result of the focus detection means.

21. the setting means further sets a plurality of focus detection frames including a main focus detection frame corresponding to a part of the subject detected by the detection means; 14. The imaging device according to claim 13, wherein, when a plurality of different parts of a subject are detected by the detection means, the setting means sets the plurality of focus detection frames to an area including detection areas of the plurality of different parts.

22. 22. The image pickup apparatus according to claim 21, wherein the setting means sets the plurality of focus detection frames based on the size of a smallest area among the parts of the subject detected by the detection means.

23. A method for controlling an imaging device including an imaging unit that captures an image of a subject, comprising: a detection step of detecting a subject from an image captured by the imaging means; a setting step of setting a main focus detection frame within the part of the subject detected in the detection step; a focus detection step of detecting a focus state of the main focus detection frame; a focus control step of controlling a focus lens based on a focus state of the main focus detection frame, In the setting step, a part to be focused on is selected from the object detected in the detection step, in the order of eyes, face, and body; A control method for an imaging device, characterized in that in the setting step, a method for setting the main focus detection frame within the part of the subject is differentiated depending on whether an eye or face is selected as the part to be focused on or whether a body is selected.

24. A program for causing a computer to execute each step of the control method according to claim 23.

25. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 23.

Images