Control device, control method thereof, and program

Abstract

To realize a suitable focus detection operation even when objects at different distances from a main subject are mixed. [Solution] A control method is provided which comprises a subject detection step of detecting a subject area from an obtained image, a setting step of setting a focus detection area divided into a plurality of areas corresponding to the detected subject area, a focus detection step of detecting focus detection information for each of the plurality of areas, and a focus adjustment step of grouping the focus detection information for each of the plurality of focus detection areas, counting the number of pieces, and determining the group in which the number reaches a peak value, and then determining the area to be focused on based on information on the number of the plurality of focus areas and adjusting the focus.

Description

Technical Field

[0001] The present invention relates to a control device that utilizes distance information of a subject.

Background Art

[0002] In recent years, cameras equipped with an autofocus function for automatically adjusting the focal position of a photographic lens have become widespread. As means for performing the autofocus, various AF methods such as an imaging surface phase difference AF method and a contrast AF method using an image sensor have been put into practical use. Further, in various AF methods, there is a technique for specifying the area of a main subject and focusing on it. In Patent Document 1, control is performed to detect a lump within a predetermined depth adjacent to a plurality of AF frames and select a main frame from among them. Further, in Patent Document 2, in addition to means for detecting a lump within a predetermined depth, by taking color information into account, the accuracy of specifying the main subject area is improved.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Documents 1 and 2, in a situation where objects having different distances are mixed on the image to be captured, due to the influence of incident light from the objects having different distances, a desired focus adjustment result may not be obtained.

[0005] Therefore, an object of the present invention is to realize a suitable focus detection operation even when objects having different distances are mixed with respect to a main subject.

Means for Solving the Problems

[0006] As technical features of the present invention, a subject detection step of detecting a subject area of a specific subject from the obtained image, a setting step of setting a focus detection area divided into a plurality of areas corresponding to the detected subject area, a focus detection step of detecting a defocus amount for each of the plurality of areas, and a focus adjustment step of grouping the defocus amounts for each of the plurality of areas, counting the number, and determining an area to be focused based on the result of determining a group having the peak value of the number, and then performing focus adjustment. In the focus adjustment step, when there is a difference in defocus amount of a predetermined amount or more in an area within the subject area being , the defocus amounts for each of the plurality of areas are grouped, the number is counted, and among the areas belonging to the group having the peak value of the number when not being the closest, the group , an area closer to the detection center of the specific subject by the subject detection step is determined as the area to be focused.

Advantages of the Invention

[0007] According to the present invention, a suitable focus detection operation can be realized even when objects having different distances from the main subject are mixed.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4a

Figure 4b

Figure 5a

Figure 5b

Figure 6a

Figure 6b

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described in detail based on the accompanying drawings.

[0010] ● Configuration of Imaging Device FIG. 1 is a block diagram showing the configuration of an interchangeable-lens camera (hereinafter simply referred to as a camera) according to an embodiment of the present invention. The camera according to the embodiment of the present invention is an example of an imaging device equipped with a control device to which the present invention is applied, and performs focus adjustment by an imaging plane phase difference detection method using an output signal from an imaging element that images a subject image. The camera is composed of a lens device (interchangeable lens) 100 and a camera body 200. When the lens device 100 is attached to the camera body 200 via a mount portion having an electrical contact unit 106, a lens controller 105 that comprehensively controls the operation of the lens device 100 and a system control unit 209 that comprehensively controls the operation of the entire camera can communicate with each other.

[0011] First, the configuration of the lens device 100 will be described. The lens device 100 includes a photographing lens 101 including a zoom mechanism, a diaphragm and shutter 102 for controlling the light amount, a focus lens 103 for focusing on an imaging element described later, a motor 104 for driving the focus lens, and a lens controller 105.

[0012] Next, the configuration of the camera body 200 will be described. The camera body 200 is configured to be able to acquire an imaging signal from the light beam that has passed through the imaging optical system of the lens device 100. The camera body 200 includes an imaging element 201 that photoelectrically converts the reflected light from the subject into an electrical signal, an A / D conversion unit 202 that includes a CDS circuit for removing the output noise of the imaging element 201 and a non-linear amplification circuit performed before A / D conversion, an image processing unit 203, and an AF signal processing unit 204. Further, the camera body 200 includes a format conversion unit 205, a high-speed built-in memory (for example, a random access memory, hereinafter referred to as DRAM) 206, and an image recording unit 207 that includes a recording medium such as a memory card and its interface. Further, the camera body 200 includes a timing generator 208, a system control unit 209 that controls the system such as a shooting sequence, a lens communication unit 210 that communicates between the camera body and the interchangeable lens, a subject detection unit 211, and an image display memory 212. Note that the image display memory 212 is referred to as VRAM.

[0013] In addition to image display, the camera body 200 includes an image display unit 213 that displays, during shooting, a shooting screen and an indicator indicating an AF frame, in addition to displays for operation assistance and displays of the camera state. Further, the camera body 200 includes an operation unit 214 for operating the camera from the outside, a shooting mode switch 215 for selecting a shooting mode such as a macro mode and a sports mode, and a main switch 216 for turning on the power to the system.

[0014] In addition, the camera body 200 includes a switch (hereinafter referred to as SW1) 217 for performing a shooting standby operation such as AF and AE, and a shooting switch (hereinafter referred to as SW2) 218 for performing shooting after the operation of SW1. The DRAM of the built-in memory 206 is used as a high-speed buffer as a temporary image storage means, or as a working memory in image compression and decompression. The operation unit 214 includes, for example, the following. A menu switch for performing various settings such as the shooting function of the imaging device and settings during image playback, and an operation mode switching switch between the shooting mode and the playback mode.

[0015] The imaging device 201 is composed of a CCD or a CMOS sensor. Each pixel of the imaging device 201 used in this embodiment is composed of two (a pair of) photodiodes A and B and one microlens provided for this pair of photodiodes A and B. Each pixel divides the incident light with the microlens to form a pair of optical images on the pair of photodiodes A and B, and outputs a pair of pixel signals (A signal and B signal) used for the AF signal described later from the pair of photodiodes A and B. In other words, the imaging device 201 has a plurality of pixels capable of photoelectrically converting the light fluxes that have passed through different pupil regions of the imaging optical system and outputting a pair of focus detection signals. Also, by adding the outputs of the pair of photodiodes A and B, image data, which is an imaging signal (A + B signal), can be obtained.

[0016] By synthesizing the plurality of A signals and the plurality of B signals output from the plurality of pixels respectively, a pair of image signals as AF signals (in other words, focus detection signals) used for AF by the imaging plane phase difference detection method (hereinafter referred to as imaging plane phase difference AF) can be obtained. The AF signal processing unit 204 performs a correlation operation on the pair of image signals, calculates the phase difference (hereinafter referred to as image shift amount), which is the shift amount of the pair of image signals, and further calculates the defocus amount (and defocus direction and reliability) of the imaging optical system from the image shift amount. Also, the AF signal processing unit 204 is assumed to perform a plurality of operations in a predetermined region where the defocus amount can be specified.

[0017] Note that the imaging device 201 has a pupil division function, and is not limited to the above configuration as long as it is possible to perform AF (imaging plane phase difference AF) by the phase difference detection method using a pair of focus detection signals generated from the output of the imaging device 201. For example, a configuration having imaging pixels that output image signals corresponding to the captured images and a pair of focus detection pixels that receive a pair of pupil-divided light fluxes may be used.

[0018] ● Operation of the imaging device The operation of the imaging device according to the embodiment will be described below with reference to FIG. 2. FIG. 2 shows the flow of imaging control processing when performing still image imaging from the state of displaying a live view image. The system control unit 209 as a computer executes this processing according to a control program as a computer program. First, in S201, the state of SW1 (217) is checked. If it is ON, the process proceeds to S202. In S202, the system control unit 209 performs AF frame setting described later on the AF signal processing unit 204 and proceeds to S203. In S203, the AF operation described later is performed and the process proceeds to S204. In S204, the state of SW1 (217) is checked. If it is ON, the process proceeds to S205, and if not, the process returns to S201. In S205, the state of SW2 (218) is checked. If it is ON, the process proceeds to S206, and if not, the process returns to S204. In S206, after performing the shooting operation, the process returns to S201.

[0019] ●AF Frame Setting FIG. 3 is a flowchart for explaining the AF frame setting in S202 of FIG. 2. First, in S301, subject detection information is acquired from the subject detection unit 211. The subject in this embodiment is assumed to detect a person, an animal such as a dog or a wild bird, and further a main area within the subject. Here, the main area is the pupil, face, and body in a person or an animal. These detection methods use known learning methods based on machine learning and recognition processing by image processing means.

[0020] For example, the types of machine learning are as follows. (1) Support Vector Machine (2) Convolutional Neural Network (3) Recurrent Neural Network

[0021] As another example of recognition processing, there is known a method of extracting a skin color area from the gradation color of each pixel represented by image data and detecting a face based on the degree of matching with a face contour plate prepared in advance. Further, there is also a well-known method of performing face detection by extracting feature points of a face such as eyes, nose, mouth, etc. using pattern recognition technology. Furthermore, the detection method of the main area applicable to the present invention is not limited to these methods, and other methods may be used.

[0022] In S302, it is determined whether a plurality of main areas can be detected based on the detection result of the subject detection unit 211. If a plurality can be determined, the process proceeds to S303; otherwise, the process proceeds to S304.

[0023] Here, the detection images in the case where the detected main area is single and the case where it is plural will be described with reference to FIGS. 4 and 5. FIG. 4a shows a state where only face a is detected, and FIG. 5a shows a state where pupil A, face B, and body C are detected. It is assumed that the subject detection unit 211 can acquire the type of the subject such as a person or an animal, and the center coordinates, horizontal size, and vertical size in each detected main area.

[0024] In S303, the minimum detected main area, that is, the smaller value of the horizontal and vertical sizes of pupil A in FIG. 5a, is input to MinA, and MinA is set as one AF frame size. In S305, the horizontal size H in FIG. 5b that includes all the main areas is obtained from the horizontal coordinates and horizontal sizes of each detected main area, and the horizontal AF frame number is determined by dividing the H by the AF frame size MinA. In S307, the vertical size V in FIG. 5b that includes all the main areas is obtained from the vertical coordinates and vertical sizes of each detected main area, and the vertical AF frame number is determined by dividing the V by the AF frame size MinA, and the AF frame setting is completed. Even in the case of an animal, the control flow is the same as that in the case of a person, and the detection area and the image of the AF frame setting are shown in FIGS. 6a and 6b, respectively. In the present embodiment, the AF frame size of a square area using the minimum size is used, but the AF frame sizes may be different between horizontal and vertical, or the number of AF frames that can be calculated by the system control unit 209 may be set.

[0025] In S304, an AF frame of a predetermined size X is set for the detected face. The X may be set to the pupil size estimated from the face, or a frame size that can ensure an S / N and sufficient focusing performance considering low illuminance environments may be set. In this embodiment, X is set to the estimated pupil size. In S306, an AF frame number Y that includes the area of face a with the AF frame size and can handle cases where the face moves is set.

[0026] ●AF operation FIG. 7 is a flowchart for explaining the AF operation of S203. First, in S401, a focus detection process is performed to detect the defocus amount (including the defocus direction) and the reliability, and then proceed to S402. The focus detection process will be described later. In S402, the system control unit 209 calculates the detected defocus amount described later using the reliability obtained in S401, and then proceeds to S403. In S403, it is examined whether the reliability of the result detected in S402 is higher than a preset reliability threshold 2. If so, proceed to S404; otherwise, proceed to S413. Here, the reliability threshold 2 is set to a threshold such that if the reliability is less than the reliability threshold 2, the accuracy of the defocus amount cannot be guaranteed, but the in-focus position direction of the subject can be guaranteed. In S404, it is examined whether the defocus amount detected in S402 is smaller than a preset Def amount threshold 2. If so, proceed to S405; otherwise, proceed to S412. Here, the Def amount threshold 2 is set to a value (for example, five times the depth of focus) such that if the defocus amount is equal to or less than the Def amount threshold 2, the focus lens can be controlled within the depth of focus by driving the lens by the defocus amount within a predetermined number of times (for example, 3 times) thereafter. In S405, it is examined whether the focus lens 103 is in a stopped state. If so, proceed to S406; otherwise, proceed to S410. In S406, it is examined whether the reliability of the result detected in S402 is higher than a preset reliability threshold 1. If so, proceed to S407; otherwise, proceed to S410. Here, the reliability threshold 1 is set such that if the reliability is equal to or higher than the reliability threshold 1, the variation in the accuracy of the defocus amount is within a predetermined range (for example, within the depth of focus). In S407, it is examined whether the defocus amount detected in S401 is smaller than a preset Def amount threshold 1. If so, proceed to S408; otherwise, proceed to S409. Here, the Def amount threshold 1 is set to a value such that if the detected defocus amount is equal to or less than the Def amount threshold 1, it means that the focus lens is controlled within the depth of focus. In S408, it is determined that the in-focus state is achieved and this flowchart ends. In S409, after driving the focus lens 103 by the defocus amount detected in S401, proceed to S401.By performing a series of S405 to S409, when the reliability detected in S401 is higher than the reliability threshold 1, focus determination can be made with the lens stopped.

[0027] In S410, the focus lens 103 is driven by a predetermined ratio with respect to the defocus amount detected in S401 and proceeds to S411. In S411, an instruction to stop the focus lens 103 is given and it proceeds to S401. In S412, since the defocus amount is not less than the Def amount threshold 2, after driving the focus lens 103 by a predetermined ratio with respect to the defocus amount detected in S401, it proceeds to S401. Here, the predetermined ratio is set so that the lens driving amount decreases with respect to the defocus amount (for example, 80%). Also, the set lens speed is set to be slower than the speed at which the lens can be driven exactly in the time of one frame, for example. Thereby, when the detected defocus amount is incorrect, it is possible to prevent going beyond the subject focus position, and further, it is possible to perform the next lens driving while driving without stopping the lens (overlap control).

[0028] In S413, it is checked whether the out-of-focus condition is satisfied. If so, it proceeds to S414, and if not, it proceeds to S415. Here, the out-of-focus condition is a predetermined condition for determining that there is no subject to be focused. For example, when the lens driving is completed over the entire movable range of the focus lens 103, that is, when the focus lens 103 detects both the far-side and near-side lens ends and returns to the initial position, such a condition is set. In S414, it is determined that it is in an out-of-focus state and this flow ends. In S415, it is checked whether the focus lens 103 has reached the far-side or near-side lens end. If so, it proceeds to S416, and if not, it proceeds to S417. In S416, the driving direction of the focus lens 103 is reversed and it proceeds to S401. In S417, the focus lens 103 is driven in a predetermined direction and it proceeds to S401. The focus lens speed is set to the fastest speed within the range of the lens speed such that it does not pass through the focus position when the defocus amount can be detected, for example.

[0029] ● Focus detection processing The focus detection process of S401 will be described with reference to FIG. 8. First, in S501, a focus detection area within an arbitrary range in the image data output from the imaging device 201 is set, and the process proceeds to S502. The focus detection area set here is an area corresponding to each of the AF frames set in S202. In S502, a pair of image signals (A image, B image) for focus detection from the imaging device 201 corresponding to the focus detection area set in S501 is acquired, and the process proceeds to S503. In S503, after performing row addition averaging processing on the pair of signals acquired in S502 in the vertical direction, the process proceeds to S504. This process can reduce the influence of noise in the image signal. In S504, after performing filter processing to extract signal components in a predetermined frequency band from the signal vertically row-averaged in S503, the process proceeds to S505. Here, signal components in a plurality of frequency bands (high frequency band, middle frequency band, low frequency band) are extracted, and the subsequent processes from S505 onwards are performed for each of them. In S505, a correlation amount is calculated from the signal filter-processed in S504, and the process proceeds to S506. In S506, a correlation change amount is calculated from the correlation amount calculated in S505, and the process proceeds to S507. In S507, an image shift amount is calculated from the correlation change amount calculated in S506, and the process proceeds to S508. In S508, a reliability representing how reliable the image shift amount calculated in S507 is is calculated, and the process proceeds to S509. In S509, the image shift amount is converted into a defocus amount (including the defocus direction), and the focus detection process ends. As a result, the defocus amount (including the defocus direction) and the reliability are obtained in S401.

[0030] ● Detection Defocus Amount Calculation The calculation of the detection defocus amount in S402 will be described with reference to FIG. 9. First, in S601, it is determined whether or not it is a condition for performing the first obstacle avoidance process. The condition for performing the first obstacle avoidance process is, for example, when the type of the subject detected by the subject detection unit 211 is an animal. This is based on the fact that animals are often photographed together with obstacles having a relatively low spatial frequency such as over the orifice (gauge). If it is a condition for performing the first obstacle avoidance process, the process proceeds to S602; otherwise, it proceeds to S603. By proceeding to S603, the first obstacle avoidance process is omitted. In S602, the first obstacle avoidance process described later is performed using the detection result of the subject detection unit 211 and the defocus amount calculated in the process of S509, and the process proceeds to S606. In S606, it is confirmed whether or not it is determined that there is a first obstacle in S602. If it is determined that there is a first obstacle, the calculation of the detection defocus amount is terminated. If it is not determined that there is a first obstacle or the first obstacle cannot be determined, the process proceeds to S603. If it is determined in S601 that it is a condition for performing the first obstacle avoidance process, the process of S602 is executed, and it is determined in S606 that there is a first obstacle, the second obstacle avoidance process is omitted.

[0031] In S603, it is determined whether it is a condition for performing the second obstacle avoidance process. The condition for performing the second obstacle avoidance process is, for example, when the type of the subject detected by the subject detection unit 211 is a person. This is based on the fact that a person is often photographed together with obstacles having a relatively high spatial frequency such as through a net. (For example, photographing through a net in sports watching such as a baseball field net or a volleyball net.) If it is a condition for performing the second obstacle avoidance process, the process proceeds to S604. Otherwise, it is determined that the type of the subject is a third type of subject that is neither an animal nor a person, and the process proceeds to S605 in a form omitting the first and second obstacle detection processes. In S604, the second obstacle avoidance process described later is performed using the detection result of the subject detection unit 211 and the defocus amount calculated in the process of S509, and the process proceeds to S607. In S607, it is confirmed whether it is determined that there is a second obstacle in S604. If it is determined that there is a second obstacle, the detection defocus amount calculation is terminated. If it is not determined that there is a second obstacle or the second obstacle determination is impossible, the process proceeds to S605.

[0032] In S605, the normal detection defocus amount calculation described later is performed using the detection result of the subject detection unit 211 and the defocus amount calculated in the process of S509, and the detection defocus amount calculation is terminated.

[0033] ● First obstacle avoidance process FIG. 10 is a flowchart for explaining the first obstacle avoidance process of S602. In the present embodiment, a histogram which is an image analysis means is used. Since it is a general technique, details of the histogram are omitted.

[0034] First, in S701, when focus detection is performed on signals in a plurality of frequency bands in the process of S509, it is set to use the defocus amount detected by the signal in the highest band among them, and the process proceeds to S702. In S702, the subject detection unit 211 determines whether the face of the subject is detected. If the face is detected, the process proceeds to S703; if not, the process proceeds to S704. In S703, the subject detection unit 211 determines whether the body of the subject is detected. If the body is detected, the process proceeds to S705; if not, the process proceeds to S706. In S705, for the AF frame including all the main regions, the defocus amount set in the process of S701 is counted for each predetermined depth to create a histogram. In this embodiment, the defocus amount itself is histogrammed, but considering a moving subject, a predicted value (subject distance) corresponding to the subject position may be obtained based on the defocus amount calculated for each AF frame, and this predicted value may be histogrammed. Also, in S706, for each AF frame set within a region that is a predetermined multiple of the face frame, the defocus amount calculated is counted for each predetermined depth to create a histogram.

[0035] In S708, it is determined whether the peak value of the histogram (the number of AF frames of the histogram peak) created in S705 or S706 is equal to or greater than a predetermined value. In this embodiment, the peak value of the histogram is normalized by the total number of AF frames and converted into a ratio for use. If the peak value is equal to or greater than a predetermined ratio, the process proceeds to S711; if it is less than the predetermined ratio, the process proceeds to S709. In S709, it is determined that the first obstacle cannot be determined, and the first obstacle avoidance process ends. In S711, it is determined whether the bin that takes the peak value of the histogram obtained in S708 is the closest to the end of the histogram. If so, the process proceeds to S710; if not, the process proceeds to S712. Here, a histogram is a data analysis means for visualizing the distribution of numerical values by dividing discrete numerical values into groups at a certain width and displaying them in a bar graph form. A bin is a group (one bar in a bar graph) divided at a certain width. It is determined whether the closest bar in the created histogram is the peak. If the peak is the closest, it means there is no obstacle because the subject is the closest. Otherwise, it is determined that there is an obstacle because something can be seen in front.

[0036] In S712, if it is determined that there is a first obstacle existing on the front side with respect to the main subject, the process proceeds to S713. In S710, if it is determined that there is no first obstacle existing on the front side with respect to the main subject, the first obstacle avoidance process ends. In S713, in order to select a main frame from the set AF frames, a loop process is performed for all the set AF frames, which is a series of processes from S714 to S717 performed while paying attention to a certain AF frame. Also, the initial value of the main frame is set in advance with information (such as the total number of frames + 1) that can determine that the main frame has not been selected, and the figure is omitted. In S714, it is determined whether the AF frame being focused on is an AF frame counted as a histogram peak. If so, the process proceeds to S715; otherwise, the loop process of S713 is repeated. In S715, it is determined whether the pupils have been detected. If the pupils have been detected, the process proceeds to S717; otherwise, the process proceeds to S716. In S716, if the coordinates of the AF frame being focused on are closer to the face detection center than the currently selected main frame, the AF frame being focused on is set as the main frame in S718. In S717, if the coordinates of the AF frame being focused on are closer to the pupil detection center than the current main frame, the AF frame being focused on is set as the main frame in S718.

[0037] In S704, the subject detection unit 211 determines whether a body is detected. If detected, the process proceeds to S719; if not detected, the process proceeds to S730. In S719, a histogram is created in the full-body detection area to obtain the histogram peak. In S720, it is determined whether the peak value of the histogram created in S719 is equal to or greater than a predetermined ratio. If it is equal to or greater than the predetermined ratio, the process proceeds to S727; if it is less than the predetermined ratio, the process proceeds to S730. In S727, it is determined whether the bin taking the peak value of the histogram obtained in S720 is the closest. If so, the process proceeds to S729; if not, the process proceeds to S728. In S728, it is determined that there is a first obstacle existing on the front side with respect to the main subject, and the process proceeds to S721. In S721, a loop process is performed for all the set AF frames, which is a series of processes from S722 to S724 that focus on a certain AF frame to select the main frame from the AF frames set in the same way as S713. In S722, it is determined whether the AF frame being focused on is the AF frame counted as the histogram peak. If so, the process proceeds to S723; if not, the loop process of S721 is repeated. In S723, if the coordinates of the AF frame being focused on are closer to the center of the full-body detection than the currently selected main frame, the AF frame being focused on is set as the main frame in S724.

[0038] In S729, it is determined that there is no first obstacle existing on the front side with respect to the main subject, and the first obstacle avoidance process ends. In S730, it is determined that the first obstacle cannot be determined, and the first obstacle avoidance process ends.

[0039] In S725, it is determined whether the main frame has been selected according to the above-described flow, based on whether it is the initial value. If the main frame is the initial value, the process proceeds to S726; if not, the first obstacle avoidance process ends. In S726, normal main frame selection described later is performed, and the first obstacle avoidance process ends.

[0040] If there is no difference in the defocus amount of more than a predetermined amount in the area within the subject area, the process proceeds to S726, and main frame selection may be performed by means such as selecting the main frame in a predetermined area within the screen. Also, in main frame selection, it may be varied according to whether the subject is a still object or a moving object. More specifically, when the subject is a still object, the area to be focused on is determined based on the information on the number of focused areas as described above, and when the subject is a moving object, the area to be focused on is determined based on past focus detection information. Also, in main frame selection, it may be varied according to whether it is a mode for photographing a still object or a mode for photographing a moving object. More specifically, in the case of the mode for photographing a still object, the area to be focused on is determined based on the information on the number of focused areas as described above, and in the case of the mode for photographing a moving object, the area to be focused on is determined based on past focus detection information.

[0041] ●Second obstacle avoidance process The focus detection results used in the second obstacle avoidance process will be described in advance.

[0042] In the second obstacle avoidance process, first, in the focus detection process of S401 in FIG. 7 and the filter process of S504 in FIG. 8, the signal bands of the first band, the second band, and the third band are extracted. Then, processing is performed using the first focus detection result, the second focus detection result, and the third focus detection result obtained by performing focus detection processing on each signal band.

[0043] The first band is the band used for normal focus detection and is set to a high frequency range. Also, the second band is the band used for the second obstacle determination in S2003 described later and for a subject with a large defocus amount, and is set to a low frequency range. Also, the third band is the band used when it is determined that there is an obstacle in the second obstacle determination of S2003. The third band is set as low as possible within the range of the signal band where focus detection accuracy is ensured so as not to be affected by obstacles, and is set to a band lower than the first band and higher than the second band.

[0044] Hereinafter, the flow of the second obstacle avoidance process in the present embodiment will be described.

[0045] FIG. 12 shows a flowchart of the second obstacle avoidance process. The operation in FIG. 12 is executed by the system control unit 209.

[0046] In step S2001 of FIG. 12, the system control unit 209 (focus detection accuracy determination means) determines whether the focus correction value of the third focus detection result is equal to or less than a predetermined value. If it is equal to or less than the predetermined value, the process proceeds to S2002. If it is greater than the predetermined value, the process proceeds to S2004, the detection defocus amount is not selected in this process, and the second obstacle avoidance process is terminated as the second obstacle determination being impossible.

[0047] The focus correction value corrects the best image plane position difference for each band of the signal caused by the spherical aberration of the lens. When there is a difference between the band of the imaging signal and the band of the focus detection signal, it corrects the focus position detected by the focus detection signal. Generally, the design value of the difference in the best image plane position between the imaging band and the focus detection band is stored as the correction value for correction. However, when there are individual variations in the lens, the spherical aberration varies, and the variation amount becomes a correction error. Since the correction error increases as the correction value increases, it is not desirable to perform focus detection under the condition that the above correction value increases.

[0048] Therefore, in this step, the obstacle determination is performed only under the condition that the focus correction value of the third focus detection result selected when it is determined that there is an obstacle is small. When the focus correction value is large, the obstacle determination is made impossible, and the normal focus detection operation is performed.

[0049] Next, in step S2002, the system control unit 209 (focus detection accuracy determination means) determines whether the shading (SHD) difference between the focus detection signals (A image, B image) obtained in S502 of the focus detection process in S401 is equal to or less than a predetermined value. If it is equal to or less than the predetermined value, the process proceeds to S2003. If it is greater than the predetermined value, the process proceeds to S2004, and as the second obstacle determination is not possible, the detected defocus amount is not selected in this process, and the second obstacle avoidance process ends. The SHD difference is the difference in the levels and inclinations of the A image and the B image.

[0050] When the SHD difference is large, a difference occurs in the shapes of the A image and the B image, and a focus detection error due to the shape difference occurs. In particular, when performing filter processing in the low-frequency range processing, since many components of the level and inclination remain, it is easily affected by the SHD difference. The third band selected when it is determined that there is an obstacle is set in the low frequency range so as not to be affected by the obstacle, so it is easily affected by the SHD difference.

[0051] Therefore, in this step, the obstacle determination is performed only when the SHD difference is small and the accuracy of the third focus detection result does not decrease. When the SHD difference is large, the obstacle determination is not possible, and the normal focus detection operation is performed.

[0052] Next, in step S2003, the system control unit 209 (obstacle determination means) performs a second obstacle determination. Details of the second obstacle determination will be described later.

[0053] Next, in step S2005, according to the result determined in S2003, if it is determined that there is an obstacle, the process proceeds to S2006. If it is determined that there is no obstacle, the detected defocus amount is not selected in this process, and the second obstacle avoidance process ends. In step S2006, the third focus detection result is selected as the detected defocus amount, and the process proceeds to S2007. In S2007, normal main frame selection described later is performed, and the process ends.

[0054] ● Second obstacle determination process Next, the flow of the second obstacle determination process in the present embodiment will be described.

[0055] Fig. 13 shows a flowchart of the second obstacle determination process. The operation in Fig. 13 is executed by the system control unit 209 (obstacle determination means).

[0056] In step S2101 of Fig. 13, the system control unit 209 acquires the center position of the subject area acquired in S301 of Fig. 3 as the representative position. Fig. 14 is a schematic diagram showing a state where there is an obstacle in front of the subject, representing a state where there is a net, which is an obstacle, in front of (the closest side to) the face of a person. As shown in Fig. 14, a focus detection area is set for the detected subject area, and the center position thereof is used as the representative position. In this embodiment, the center position of the subject area is acquired as the representative position, but it is not limited thereto. The second focus detection results within the subject area may be classified, and the position closest to the center among the areas belonging to the most frequent class may be used as the representative position.

[0057] Next, in step S2102, the system control unit 209 calculates the difference between the first focus detection result and the second focus detection result at the representative position acquired in step S2101. As shown in Fig. 14, when both the face and the net are present within one AF frame, since both signals are mixed, the focus detection results differ depending on the signal band used for focus detection, and it may detect the net position or the position between the face and the net instead of the face.

[0058] Fig. 15 shows the responses in the frequency bands of the face and the net. Fig. 15(a) shows the frequency responses of the first band of the face and the net, and Fig. 15(b) shows the frequency responses of the second band of the face and the net, respectively. As shown in Fig. 15(a), in the first band, which is the high frequency band, the response of the net is higher than that of the face. As shown in Fig. 15(b), in the second band, which is the low frequency band, the response of the face is higher than that of the net. Therefore, in the first band, since the response of the net is higher, the first focus detection result detects the focus position of the net, and in the second band, since the response of the face is higher, the second focus detection result detects the focus position of the face.

[0059] Thus, if there is a difference in the focus detection results depending on the frequency band, it can be said that the subject and the obstacle are mixed within the AF frame. In particular, when the focus detection result in the high frequency band is closer to the camera than the focus detection result in the low frequency band, it is highly likely that an obstacle such as a net, which is a high frequency subject, overlaps in front of the subject such as a face. Utilizing this relationship, the determination of the obstacle is performed in steps S2103 to S2105.

[0060] In step S2103, when the difference calculated in step S2102 indicates that the first focus detection result is closer to the camera than the second focus detection result, the system control unit 209 proceeds to step S2104, assuming a high possibility of an obstacle. When it indicates the infinite side, the process proceeds to step S2108, where it is determined that there is no obstacle, and the second obstacle determination ends. Here, when the first focus detection result, which is more likely to detect the focus position of a high frequency obstacle such as a net, is closer to the camera than the second focus detection result in the low frequency band, it is determined that there is a high possibility that an obstacle such as a net overlaps in front of the subject such as a face.

[0061] Next, in step S2104, when the difference calculated in step S2102 is greater than the threshold value 1, the system control unit 209 proceeds to step S2105, assuming a high possibility of an obstacle. When it is less than or equal to the threshold value 1, the process proceeds to step S2108, where it is determined that there is no obstacle, and the second obstacle determination ends. Here, in order to consider the variation in the focus detection results, the threshold value 1 is set as the range in which the focus detection results can vary. If the difference is greater than or equal to the threshold value 1, it is determined that it is not a variation in the detection results but a high possibility that an obstacle overlaps the subject.

[0062] Next, in step S1205, the system control unit 209 calculates the ratio X of the first focus detection result within the focus detection area being closer to the camera than the second focus detection result at the representative position.

[0063] Next, in step S1206, when the ratio X calculated in step S1205 is greater than the threshold value 2, the system control unit 209 proceeds to step S2107, determines that there is an obstacle, and ends the second obstacle determination. When the ratio X is less than or equal to the threshold value 2, the process proceeds to step S2108, determines that there is no obstacle, and ends the second obstacle determination. Here, it is determined based on whether the high-frequency first focus detection result, which detects the nearer side from the low-frequency second focus detection result, spreads over a sufficiently wide range. A high-frequency obstacle such as a net is likely to spread over a wider range than a subject such as a face. Therefore, if the high-frequency first focus detection result indicates the nearer side in a wide range within the focus detection area, it is determined that there is a high possibility that an obstacle such as a net spreads in front of a subject such as a face.

[0064] In this embodiment, the subject area is set as the face area detected by the subject detection means. However, it is not limited thereto, and it may be set as an area for other detected subjects. Further, the detection types may be divided in advance into a group with a high frequency and a group with a low frequency, and the second obstacle avoidance process may be performed only for the subject in the group with a low frequency of the subject.

[0065] Also, in this embodiment, the focus detection area is set with respect to the subject area. However, based on the subject area, it may be set in an area wider than the subject area, and for the area outside the subject area, only the focus detection result in the first band may be calculated.

[0066] ● Normal detection defocus amount calculation The normal detection defocus amount calculation in S605 will be described with reference to FIG. 16. First, in S1001, when focus detection is performed on signals in a plurality of frequency bands in the process of S509, it is set to use the defocus amount detected by the signal in the highest band among them, and the process proceeds to S1002. In S1002, normal main frame selection described later is performed, and the normal detection defocus amount calculation is ended.

[0067] ● Normal main frame selection The normal main frame selection of S726, S2007, and S1002 will be described with reference to FIG. 17. First, in S1101, the subject detection unit 211 determines whether the face of the subject has been detected. If the face is detected, the process proceeds to S1102; otherwise, it proceeds to S1107. In S1107, the subject detection unit 211 determines whether the body of the subject has been detected. If the body is detected, the process proceeds to S1108; otherwise, it proceeds to S1109.

[0068] In S1102, it is determined whether the pupils are detected and whether the AF frame at the pupil center is within a predetermined depth from the focus lens position when calculating the defocus amount. Generally, the subject detection information is more accurate when in focus. When the defocus amount is above the predetermined depth, there is a possibility of false detection of the subject detection information, so the above-mentioned predetermined conditions are used. When the pupils are detected and the pupil center frame is within the predetermined depth, the process proceeds to S1104 to set the AF frame at the pupil center as the main frame; otherwise, the process proceeds to S1103.

[0069] In S1103, a loop process is performed for all the set AF frames, in which the process of S1105 that focuses on a certain AF frame to select the main frame from the set AF frames is carried out. Also, the initial value of the main frame is set with information (such as the total number of frames + 1) that can determine that the main frame has not been selected, and the figure is omitted. In S1105, it is determined whether the AF frame being focused on is closer and has a defocus amount within the predetermined depth than the selected main frame. If the condition is met, the AF frame being focused on is set as the main frame (S1106).

[0070] In S1109, it is determined whether the main frame has been selected according to the above flow by whether it is the initial value. If the main frame is the initial value, the process proceeds to S1110; otherwise, the main frame selection process ends. In S1110, means such as selecting the main frame in a predetermined area within the screen without using the detection information can be considered, but since it is not the main technology of this case, the details are omitted.

[0071] In addition, if the difference in the defocus amount in the area within the subject area is not more than a predetermined amount, the process proceeds to S1110, and main frame selection may be performed by means such as selecting the main frame in a predetermined area within the screen. Also, in the main frame selection, it may be made to differ according to whether it is a mode for photographing a still subject or a mode for photographing a moving subject. More specifically, in the case of the mode for photographing a still subject, the main frame is determined based on the information of the detection area as described above, and in the case of the mode for photographing a moving subject, the main frame is determined based on the past focus detection information.

[0072] By applying this embodiment, it is possible to suitably identify a scene in which objects having different distances from the main subject are mixed. Also, even when objects having different distances from the main subject are mixed, a suitable focus detection operation can be realized. For example, it is possible to reduce a phenomenon such as focusing on a duck when a wild bird or the like is bred beyond the duck in an environment such as a zoo. An image of the histogram in the above environment is shown in FIG. 11. Also, for example, it is possible to reduce a phenomenon such as focusing on a net when photographing a sports scene or the like through a net.

[0073] As described above, the present invention has been described in detail based on its preferred embodiments. However, the present invention is not limited to these specific embodiments, and various forms within the scope not departing from the gist of the present invention are also included in the present invention. Some of the above-described embodiments may be appropriately combined. Further, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly or to a system or apparatus having a computer capable of executing the program using wired / wireless communication and the program is executed, it is also included in the present invention. Therefore, in order to realize the functional processing of the present invention by a computer, the program code itself supplied to and installed in the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In that case, as long as it has the functions of the program, the form of the program is not limited, such as object code, a program executed by an interpreter, and script data supplied to an OS. As a recording medium for supplying the program, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / photo-magnetic storage medium, or a non-volatile semiconductor memory may be used. Further, as a method for supplying the program, a method in which a computer program forming the present invention is stored in a server on a computer network and a connected client computer downloads and programs the computer program is also conceivable.

Explanation of Reference Numerals

[0074] 100 Lens device 103 Focus lens 105 Lens controller 200 Camera body 201 Image sensor 204 AF signal processing unit 209 System control unit 210 Lens communication unit 211 Subject detection unit

Claims

1. subject detection means for detecting a subject area of a specific subject from the obtained image; setting means for setting a focus detection area divided into a plurality of areas corresponding to the detected subject area; focus detection means for detecting a defocus amount for each of the plurality of areas; focus adjustment means for grouping the defocus amounts for each of the plurality of areas, counting the number, determining an area to be focused based on the result of determining a group having the peak value, and performing focus adjustment, The focus adjustment means is a case where there is a difference in defocus amount of a predetermined amount or more in an area within the subject area, and when grouping the defocus amounts for each of the plurality of areas, counting the number, and the group having the peak value is not the closest, among the areas belonging to the group, an area closer to the detection center of the specific subject by the subject detection means is determined as the area to be focused. A control device characterized by this.

2. The control device according to claim 1, wherein the subject detection means detects at least one of a person's pupil, a person's face, a person's whole body, an animal's pupil, an animal's face, and an animal's whole body as a subject area.

3. An imaging device comprising the control device according to claim 1 or 2, and imaging means capable of photoelectrically converting light beams passing through different pupil regions of an imaging optical system and outputting a focus detection signal.

4. a subject detection step of detecting a subject area of a specific subject from the obtained image; a setting step of setting a focus detection area divided into a plurality of areas corresponding to the detected subject area; a focus detection step of detecting a defocus amount for each of the plurality of areas; A focus adjustment step of grouping the defocus amounts for each of the plurality of areas, counting the number thereof, determining an area to be focused based on the result of determining a group having the peak value in the number, and performing focus adjustment; In the focus adjustment step, when there is a difference in defocus amount of a predetermined amount or more in an area within the subject area, and the group having the peak value in the number obtained by grouping and counting the defocus amounts for each of the plurality of areas is not the closest, among the areas belonging to the group, an area closer to the detection center of a specific subject by the subject detection step is determined as the area to be focused. A control method characterized by this.

5. A program for causing a computer included in a control device to function as each means of the control device according to claim 1 or 2.

Images