Imaging device and control method thereof

Abstract

To reduce perspective conflict and overresponse when moving from a large part to a small part of an object during AF motion tracking.SOLUTION: An image capturing device provided herein is configured to perform focus detection processing for detecting the focusing state on an object or a portion of the object, and determination processing for determining whether to switch a focusing state detection target from the object to a portion of the object or not according to given conditions.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to an imaging device having a focus adjustment mechanism. [Background technology]

[0002] Digital cameras and other imaging devices have traditionally been equipped with an automatic focus adjustment (AF) function for the convenience of users. In recent years, models equipped with a function to detect faces through image analysis have become common. Some models also feature motion prediction control, which predicts the AF focus position from past history in order to allow AF tracking of moving objects or people.

[0003] With such a camera, for example, when photographing a person approaching from a distance, the subject is initially small and face detection does not work, so AF is performed on the torso area. As the person gradually approaches, AF is performed while also using the motion prediction mentioned earlier. As the person then gets even closer, the area of ​​the person in the image increases, and face detection is activated. Since a photo with the face in focus is generally desired, the AF target area is shifted from the torso to the face. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-178480 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when the AF target is shifted from the torso to the face, because the face is a smaller subject compared to the torso, if the phase difference detection method is used for AF, perspective conflict occurs between the face and the background, and the focus detection result may be behind the face. In this case, even though the focus was set on the torso, as soon as the focus shifts to the face, it becomes back focus and the face cannot be focused on.

[0006] Furthermore, when motion prediction is performed, even if a person is actually approaching at a constant speed, the focus detection result suddenly changes and the next motion prediction result becomes even further back in focus. This results in over-response and can lead to even more severe out-of-focus images.

[0007] As a countermeasure, Patent Document 1 gives a method of judging whether or not a subject has been captured based on reliability. However, it is difficult to detect perspective conflicts with high accuracy. Users also want to be able to focus on faces using AF. [Means for solving the problem]

[0008] A technical feature of the present invention is an imaging device having a first detection means for detecting an object from an image, a second detection means for detecting a part of the object from the image, a focus detection means for detecting a focus state of the object and the part of the object, and a history means for storing a history of focus detection results of the object and the part of the object detected by the focus detection means, when the focus detection means detects a focus state of the object detected by the first detection means and the variance of the focus detection results of the object detected by the first detection means stored in the history means is equal to or greater than a certain value, the focus detection means adjusts the focus on the object detected by the first detection means; When the variance of the focus detection results of the subject detected by the first detection means stored in the history means is less than a certain level, focusing is performed on a part of the subject detected by the second detection means. Effect of the Invention

[0009] According to the present invention, stable focus tracking can be achieved. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an interchangeable lens camera system. [Diagram 2] FIG. 1 is a diagram showing a pixel arrangement in image plane phase-difference AF. [Diagram 3]FIG. 11 is a flowchart of a focus adjustment operation. [Figure 4] FIG. 4 is a flowchart of a focus detection process. [Diagram 5] 5A and 5B are explanatory diagrams of focus detection areas in focus detection processing. [Figure 6] 5A and 5B are explanatory diagrams of image signals in focus detection processing. [Figure 7] 6A and 6B are diagrams illustrating the relationship between a shift amount and a correlation amount in focus detection processing. [Figure 8] 6A and 6B are diagrams illustrating the relationship between the shift amount and the correlation change amount in the focus detection process. [Figure 9] 13 is a flowchart of a prediction process in a focus adjustment operation. [Figure 10] 13 is a flowchart of a selection process in a focus adjustment operation. [Figure 11] 11A and 11B are diagrams illustrating an example of a selection process in a focus adjustment operation. [Figure 12] 11A and 11B are diagrams illustrating an example of a selection process in a focus adjustment operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0012] (First embodiment) (judging whether the robot has moved from the body to the face based on the size of the face, and resetting the history) <Configuration of imaging device> In this embodiment, an example in which the present invention is applied to an image pickup apparatus of an interchangeable lens type in which the lens unit and the image pickup apparatus are detachable will be described.

[0013] FIG. 1 is a block diagram showing the configuration of the main part of the imaging device.

[0014] 1, an imaging device 20 that is detachable from an interchangeable lens unit 10 is mainly composed of a camera control unit 212 that controls the operation of the entire camera system including the lens unit 10. The imaging device is controlled based on a control program stored in an internal ROM and RAM (not shown) of the camera control unit 212 and various data required for control.

[0015] Furthermore, the lens unit 10 has a lens control unit 106 that controls the overall operation of the lens, and the lens control unit 106 and the camera control unit 212 are capable of communicating with each other via a terminal provided on the lens mount.

[0016] First, the configuration of lens unit 10 will be described. Fixed lens 101, aperture 102, and focus lens 103 constitute an imaging optical system. Aperture 102 is driven by aperture drive unit 104 and controls the amount of light incident on image sensor 201, which will be described later. Focus lens 103 is driven by focus lens drive unit 105 and the focal distance of the imaging optical system changes depending on the position of focus lens 103. Aperture drive unit 104 and focus lens drive unit 105 are controlled by lens control unit 106 and determine the opening amount of aperture 102 and the position of focus lens 103.

[0017] The lens operation unit 107 is a group of input devices that allow the user to make settings related to the operation of the lens unit 10, such as switching between AF / MF modes, adjusting the position of the focus lens by MF, setting the operating range of the focus lens, setting the image stabilization mode, etc. When the lens operation unit 107 is operated, the lens control unit 106 performs control according to the operation.

[0018] The lens control unit 106 controls the aperture driving unit 104 and the focus lens driving unit 105 in response to control commands and control information received from a camera control unit 212 (described later), and also transmits lens control information to the camera control unit 212 .

[0019] Next, a description will be given of the configuration of the imaging device 20. The imaging device 20 is configured so as to be able to obtain an imaging signal from a light beam that has passed through the imaging optical system of the lens unit .

[0020] The image sensor 201 is composed of a CCD or CMOS sensor. A light beam incident from the photographing optical system of the lens unit 10 forms an image on the light receiving surface of the image sensor 201 and is converted into a signal charge according to the amount of incident light by a photodiode provided in each pixel arranged in the image sensor 201. The signal charge accumulated in each photodiode is sequentially read out from the image sensor 201 as a voltage signal according to the signal charge by a drive pulse output by a timing generator 214 according to a command from a camera control unit 212.

[0021] Each pixel of the image sensor 201 used in this embodiment is composed of two (a pair) photodiodes A and B and one microlens provided for the pair of photodiodes A and B. Each pixel splits incident light with the microlens to form a pair of optical images on the pair of photodiodes A and B, and outputs a pair of pixel signals (A signal and B signal) used as an AF signal described later from the pair of photodiodes A and B. In addition, an image signal (A+B signal) can be obtained by adding the outputs of the pair of photodiodes A and B.

[0022] A signals output from a plurality of pixels are synthesized with each other, and B signals are synthesized with each other. This provides a pair of image signals as AF signals (in other words, focus detection signals) used for AF using an image plane phase difference detection method (hereinafter referred to as image plane phase difference AF). An AF signal processor 204, which will be described later, performs a correlation calculation on the pair of image signals to calculate a phase difference (hereinafter referred to as image shift amount) which is the amount of shift between the pair of image signals, and further calculates a defocus amount (and defocus direction) of the photographing optical system from the image shift amount. The defocus amount and defocus direction are focus detection results and indicate the focus state.

[0023] FIG. 2(a) shows a pixel configuration that does not support image plane phase difference AF, and FIG. 2(b) shows a pixel configuration that supports image plane phase difference AF. In both figures, a Bayer array is used, with R indicating a red color filter, B indicating a blue color filter, and Gr and Gb indicating green color filters. In the pixel configuration shown in FIG. 2(b) that supports image plane phase difference AF, two photodiodes A and B that are divided into two in the horizontal direction of the figure are provided in a pixel that corresponds to one pixel (shown surrounded by a solid line) in the pixel configuration that does not support image plane phase difference AF shown in FIG. 2(a). Note that the pixel division method shown in FIG. 2(b) is merely an example, and the pixel may be divided into two in the vertical direction of the figure, or into two each in the horizontal and vertical directions (a total of four divisions). In addition, the same image sensor may include multiple types of pixels that are divided by different division methods.

[0024] The CDS / AGC / AD converter 202 performs correlated double sampling, gain adjustment, and AD conversion to remove reset noise on the AF signal and the imaging signal read out from the image sensor 201. The converter 202 outputs the imaging signal and AF signal that have been subjected to these processes to an image input controller 203 and an AF signal processor 204, respectively.

[0025] The image input controller 203 stores the imaging signal output from the converter 202 in the SDRAM 209 as an image signal via the bus 21. The image signal stored in the SDRAM 209 is read out by the display control unit 205 via the bus 21 and displayed on the display unit 206. In a recording mode in which the image signal is recorded, the image signal stored in the SDRAM 209 is recorded by the recording medium control unit 207 in a recording medium 208 such as a semiconductor memory.

[0026] The ROM 210 stores control programs and processing programs executed by the camera control unit 212, as well as various data required for executing these programs. The flash ROM 211 stores various setting information related to the operation of the camera 20 set by the user.

[0027] The subject detection unit 2121 in the camera control unit 212 detects a specific subject based on the imaging signal input from the image input controller 203, and determines the position of the specific subject in the imaging signal. In addition, the imaging signal is continuously input from the image input controller 203, and when the detected specific subject moves, the position of the destination is determined and the position of the specific subject is tracked. Examples of the specific subject include a human subject, an animal, a vehicle such as a car, and a subject that exists at a position specified in the imaging screen by a user using the camera operation unit 213. Furthermore, the organ detection unit 2122 detects a human face, a face of a person riding a vehicle, etc., from the subjects detected by the subject detection unit 2121, and determines a specific position in the imaging signal. As will be described later, information on the position and size of the detected specific subject is mainly used to set the area for performing AF.

[0028] The AF signal processing unit 204, which serves as a focus detection device, performs correlation calculations on a pair of image signals, which are AF signals output from the converter 202, and calculates the image shift amount and reliability of the pair of image signals. The reliability is calculated using the degree of coincidence between two images and the steepness of the correlation change amount, which will be described later. The AF signal processing unit 204 also sets the position and size of a focus detection area, which is an area within the imaging screen where focus detection and AF are performed. The AF signal processing unit 204 outputs information on the image shift amount (detection amount) and reliability calculated in the focus detection area to the camera control unit 212. Details of the processing performed by the AF signal processing unit 204 will be described later.

[0029] The AF control unit 2123 in the camera control unit 212 instructs the lens control unit 106 to move the focal position based on the converted defocus amount. Furthermore, the AF control unit 2123 predicts a future image plane position using a prediction unit 2124, calculates a lens driving amount required for the focus lens 103 to come to the predicted image plane position, and instructs the lens control unit 106.

[0030] The storage unit 2125 stores in the memory circuit 215 the subject image plane position calculated from the image capturing time and the focus amount.

[0031] The selection unit 2126 compares the image plane velocity calculated by the prediction unit 2124, which is based on the defocus amount of the human region detected by the subject detection unit 2121, with the image plane velocity calculated by the prediction unit 2124, which is based on the defocus amount of the face region detected by the organ detection unit 2122. Then, it selects whether the focus lens is to be driven toward the image plane predicted position of the subject detection unit 2121 or the organ detection unit 2122.

[0032] The camera control unit 212 controls each unit in the imaging device 20 while exchanging information with them. In addition, the camera control unit 212 executes various processes corresponding to user operations such as power ON / OFF, changing various settings, imaging processing, AF processing, and playback processing of recorded images, in response to an input from a camera operation unit 213 based on a user operation. Furthermore, the camera control unit 212 transmits control commands for the lens unit 10 (lens control unit 106) and information on the imaging device 20 to the lens control unit 106, and acquires information on the lens unit 10 from the lens control unit 106. The camera control unit 212 is configured by a microcomputer, and controls the entire camera system including the interchangeable lens 10 by executing a computer program stored in the ROM 210.

[0033] The camera control unit 212 calculates a defocus amount using the image shift amount in the focus detection area calculated by the AF signal processing unit 204, and controls the driving of the focus lens 103 via the lens control unit 106 based on the defocus amount.

[0034] <Focus adjustment procedure> The following describes the processing performed by the imaging device 20. The camera control unit 212 performs the following processing in accordance with an imaging processing program, which is a computer program.

[0035] 3 is a flowchart showing the photographing process of the imaging device 20, in particular the procedure of the focus adjustment operation performed by the AF control unit 2123. S denotes a step.

[0036] First, the camera control unit 212 determines whether or not to execute a focus adjustment operation in response to the camera settings and an input from the camera operation unit 213 (S301).

[0037] When it is determined that the focus adjustment operation is to be performed, the focus detection process is performed in step S302. The focus detection process will be described in detail later with reference to FIG.

[0038] In S303, a pre-shooting prediction process is performed. In the pre-shooting prediction process, if the shooting start switch is in an on state as described below, the prediction unit 2124 predicts the image plane position of the subject from the time of phase difference detection in the focus detection process in S302 to the time of shooting by the image sensor 201. If the shooting start switch is in an off state, the prediction unit 2124 predicts the image plane position of the subject until the next phase difference detection. Details of the prediction method will be described later with reference to FIG. 9.

[0039] In S304, using the image plane velocity calculated in S303, it is selected whether to move the focus lens 103 to focus on the image plane position of the human region detected by the subject detection unit 2121 or the image plane position of the face region detected by the organ detection unit 2122. Details of the selection method will be described later with reference to FIG.

[0040] In S305, the lens drive amount required to move the focus lens 103 so as to focus on the subject image plane position predicted in S303 and selected in S304 is calculated and notified to the lens control unit 106.

[0041] Next, in S306, the state of the shooting start switch is determined, and if the switch is on, the process proceeds to shooting in S307, and if the switch is off, the process proceeds to S310.

[0042] In S307, the image captured by the image sensor 201 is stored in the memory circuit 215. In S308, the image plane position of the subject at the time of the next phase difference detection is predicted by the prediction unit 2124, and in S309, the lens drive amount required to move the focus lens 103 so as to focus on the image plane position predicted in S308 is calculated and transmitted to the lens control unit 106.

[0043] In S310, it is determined whether the shooting preparation switch is off. If the switch is off, the process ends, and if the switch is on, the process returns to S302 and the above process is repeated.

[0044] <Focus detection processing> An example of the operation of the focus detection process performed in S302 will be described with reference to the flowchart in FIG.

[0045] First, the AF signal processing unit 204 acquires a pair of image signals as AF signals from a plurality of pixels included in the focus detection area of ​​the image sensor 201 (S401). FIG. 5(a) shows an example of a focus detection area 502 on a pixel array 501 of the image sensor 201. The shift areas 503 on both sides of the focus detection area 502 are areas required for correlation calculation. Therefore, an area 504 combining the focus detection area 502 and the shift area 503 is a pixel area required for correlation calculation. p, q, s, and t in the figure each represent coordinates in the horizontal direction (x-axis direction), p and q respectively represent the x coordinates of the start point and end point of the pixel area 504, and s and t respectively represent the x coordinates of the start point and end point of the focus detection area 502. FIG. 6 also shows an example of a pair of image signals for AF acquired from a plurality of pixels included in the focus detection area 502 shown in FIG. 5(a). A solid line 601 represents one image signal A, and a dashed line 602 represents the other image signal B. FIG. 6(a) shows image signals A and B before shifting, and FIGS. 6(b) and (c) show the image signals A and B after being shifted in the positive and negative directions, respectively, from the state of FIG. 6(a).

[0046] Next, the AF signal processor 204 calculates the correlation amount of the pair of image signals while relatively shifting the pair of acquired image signals by one pixel (one bit) at a time (S402). In each of a plurality of pixel lines (hereinafter referred to as scanning lines) provided in the focus detection area, both image signals A601 and B602 are shifted by one bit in the direction of the arrow as shown in FIG. 6B and FIG. 6C. In this way, the correlation amount of the pair of image signals A601 and B602 is calculated, and one correlation amount is calculated by averaging each correlation amount. Here, the pair of image signals is relatively shifted by one pixel at a time to calculate the correlation amount, but it is also possible to shift the image signals by more pixel units, for example, by shifting them by two pixels at a time. In addition, one correlation amount is calculated by averaging the correlation amounts of each scanning line, but it is also possible to perform averaging on the pair of image signals of each scanning line, and then calculate the correlation amount for the pair of image signals obtained by averaging. When the shift amount is i, the minimum shift amount is ps, the maximum shift amount is qt, x is the start coordinate of the focus detection area 502, and y is the end coordinate of the focus detection area 502, the correlation amount COR can be calculated by the following equation (1).

[0047]

number

[0048] 7A shows an example of the relationship between the shift amount and the correlation amount COR. The horizontal axis is the shift amount, and the vertical axis is the correlation amount COR. Among extreme values ​​702 and 703 in a correlation amount 701 that changes with the shift amount, the degree of match between a pair of image signals A and B is highest at a shift amount corresponding to a smaller correlation amount.

[0049] Next, the AF signal processing unit 204 calculates a correlation change amount from the correlation amount calculated in S402 (S403). If the difference between the correlation amounts for every other shift in the waveform of the correlation amount 701 shown in Fig. 7(a) is calculated as the correlation change amount, and the shift amount is i, the minimum shift amount is ps, and the maximum shift amount is qt, the correlation change amount ΔCOR can be calculated by the following formula (2).

[0050]

number

[0051] Next, the AF signal processor 204 calculates the image shift amount using the correlation change amount calculated in S403 (S404). FIG. 8(a) shows an example of the relationship between the shift amount and the correlation change amount ΔCOR, where the horizontal axis represents the shift amount and the vertical axis represents the correlation change amount ΔCOR. The correlation change amount 801, which changes with the shift amount, goes from positive to negative at 802 and 803. A state in which the correlation change amount is 0 is called a zero cross, and the matching degree of a pair of image signals A and B is highest. Therefore, the shift amount that gives the zero cross is the image shift amount. FIG. 8(b) shows an enlarged view of the portion indicated by 802 in FIG. 8(a). 804 is a part of the correlation change amount 801. The shift amount (k-1+α) that gives the zero cross is divided into an integer part β (=k-1) and a decimal part α. The decimal part α can be calculated by the following formula (3) from the similarity relationship between the triangles ABC and ADE in the figure.

[0052]

number

[0053] Moreover, the integer part β can be calculated from FIG. 8(b) using the following formula (4). (Formula 4) β=k-1 (4)

[0054] That is, the image shift amount PRD can be calculated from the sum of α and β. When there are multiple zero crossings of the correlation change amount ΔCOR as shown in FIG. 8(a), the one in the vicinity of which the change in the correlation change amount ΔCOR has a greater steepness is determined to be the first zero crossing. This steepness is an index showing the ease of AF, and the larger the value, the easier it is to perform accurate AF. The steepness maxder can be calculated using the following formula (5).

[0055]

number

[0056] In this embodiment, when there are multiple zero crosses in the correlation change amount, the first zero cross is determined based on the steepness of the zero cross, and the shift amount that gives this first zero cross is taken as the image shift amount.

[0057] Next, the AF signal processing unit 204 calculates the defocus amount of the focus detection area using the image shift amount of the focus detection area calculated in S404 (S405). Then, in S406, the obtained focus detection information is stored in the memory circuit 215. Here, the defocus amount of each focus detection area, the shooting time of image signal A and image signal B, and the image plane speed are stored.

[0058] <Pre-shooting prediction> The pre-imaging prediction for predicting a future image plane position from the past multiple image plane positions and changes in the image capturing times, which is performed by the prediction unit 2124 in S303, will be described with reference to FIG.

[0059] First, in S901, the defocus amount is calculated from the phase difference detected by the AF signal processing unit 204 in the area detected by the object detection unit 2121. Then, in the next S902, the image plane position corresponding to the calculated defocus amount and its shooting time are calculated. Generally, a certain amount of charge accumulation time is required until an image signal is obtained from the image sensor 201. Therefore, the image plane position of the object is calculated by adding this defocus amount to the relative extension amount of the focus lens 103, with the midpoint between the accumulation start time and end time being set as the shooting time. Then, in the next S903, data of the pair of the image plane position and the shooting time is stored in the memory circuit 215. The data structure of the memory for storing is a queue, and data is stored in order up to a predetermined number, but the newest data is overwritten on the oldest data for data thereafter. Then, the process proceeds to S904, where it is determined whether the number of data stored in the memory circuit 215 is capable of performing statistical calculation. If the result of this determination is that the number of data required for statistical calculation is sufficient, the process proceeds to S905, where a prediction formula for statistical calculation is determined. In determining the prediction formula by statistical calculation in S905, the coefficients α, β, and γ are statistically determined by multiple regression analysis in a prediction function f(t) as shown in formula (6). In addition, the value of n in formula (6) is determined so that the prediction error is minimized when prediction is performed on samples of multiple representative moving object prediction shooting scenes. (Formula 6) f(t)=α+βt+γtn (6)

[0060] After the prediction formula is determined in S905, the process proceeds to S906, where the image plane position at a predetermined future time is predicted, and the lens driving amount required for the focus lens 103 to focus on the image plane position is calculated. On the other hand, if it is determined in S904 that the number of data is insufficient, the process proceeds to S907, where the lens driving amount is calculated using the defocus amount calculated without using statistical calculation. The above process is also performed on the area detected by the organ detection unit 2122.

[0061] <Outline of the necessity for selection of the area to be focused> The necessity for and overview of the selection process performed in S304 will be described with reference to FIG.

[0062] 11A shows a person as a subject, reference numeral 1101. When focusing on this person, reference numerals 1111 and 1112 correspond to the focus detection area 502 in FIG.

[0063] Basically, in portraits, the focus should be on the face. The distance between the face and the body may differ, and even if the distance is the same, the focus detection result may change due to differences in spatial frequency and contrast. Therefore, 1111 tries to recognize the face and focus on the coordinates of the face, but it does not always focus exactly on the face due to the accuracy of face detection and the movement of the subject. If the subject is out of focus on the face as in 1111, the correlation amount is pulled by the background image, and the defocus becomes backward, resulting in a state of perspective conflict. In this case, if the focus is on the person's torso as in 1112, which is larger than the face, perspective conflict is unlikely to occur despite the difference in distance between the face and torso, and more accurate focusing is possible.

[0064] FIG. 11(b), (c), and (d) each show how a person is brought into focus. The subject approaches the camera in the order of (b), (c), and (d). FIG. 11(e) shows the defocus amount of the face and torso at that time. Reference numeral 1121 in FIG. 11(b) represents the focus detection area of ​​the face, and its defocus amount is def1b. Reference numeral 1122 in FIG. 11(b) represents the focus detection area of ​​the torso, and its defocus amount is def2b. Reference numeral 1131 in FIG. 11(c) represents the focus detection area of ​​the face, and its defocus amount is def1c. Reference numeral 1132 in FIG. 11(c) represents the focus detection area of ​​the torso, and its defocus amount is def2c. Reference numeral 1141 in FIG. 11(d) represents the focus detection area of ​​the face, and its defocus amount is def1d. Reference numeral 1142 in FIG. 11(d) represents the focus detection area of ​​the torso, and its defocus amount is def2d. If all the images were focused on the face, as in the previous example, def1b, def1c, and def1d would be used in that order. However, as mentioned above, the focus detection area for def1b extends beyond the face, so there is a possibility of perspective conflict.

[0065] If you want to focus on the torso, you would use def2b, def2c, and def2d in that order. In that case, perspective conflict is unlikely to occur as mentioned above, but at a distance of about Fig. 11(d), the difference in distance between the face and the torso becomes non-negligible. Therefore, you could consider a control that looks at the size of the subject's face as a specified condition, and switches from the torso to the face along the way, going from def2b, def2c, to def1d.

[0066] Fig. 11(f) is a graph plotting each defocus in Fig. 11(d). The vertical axis represents the defocus amount, and the horizontal axis represents time. The subject gradually approaches, as indicated by the dotted line 1151. Also, the defocus of each face here represents an example of gradual perspective conflict.

[0067] Here, as mentioned above, when the size of the subject's face is checked and control is performed to move from the body to the face in the middle, from def2b, def2c to def1d, and def1d also has perspective conflict due to the movement of the subject. In this case, as explained in Fig. 9, predictive control of defocus is performed, and it is measured as if the approaching movement was suddenly braked, and the curve 1152 is predicted. If there was a constant amount of perspective conflict all the time and predictive control was not performed, the error would have been about 1162, but the predictive control predicts the position 1161, resulting in an even greater deviation from the focus than the focus detection result without prediction. Therefore, if the focus moves from the body to the face at point (d), the frequency of the phenomenon called over-response can be suppressed by erasing the data of the past image plane positions and shooting times of (b) and (c) by eliminating the past history.

[0068] <Selection of the area to be focused> A specific flow of the selection process performed in S304 will be described with reference to FIG.

[0069] In S1001, the size of the face detected by the organ detection unit 2122 is obtained. In S1002, it is determined whether the size of the face is equal to or larger than a certain level. If it is equal to or smaller than the certain level, the subject to perform AF is moved to S1003 while remaining in the subject detection area, and if it is equal to or larger than the certain level, it is moved to the face detection area (S1004). In S1003, it is selected to drive the focus lens to the image plane position of the subject detection area, and the process ends. In S1004, it is selected to drive the focus lens to the image plane position of the face detection area. In S1005, the area selected last time was the subject detection area, and this time the face detection area was selected in S1004, and as a result, if the area has been moved from the entire body / torso to the face, the process goes to S1006, and if not, the process ends. In S1006, the data of the past image plane position and shooting time pair stored in the memory circuit 215 in S903 is erased, and the process ends.

[0070] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the present invention.

[0071] As described above, according to the first embodiment of the present invention, when transitioning from an entire person / torso detected in a subject detection area to a face detection area, the size of the face is checked to make it difficult to transition to a small face, thereby reducing the frequency of perspective conflicts between a small face and the background.

[0072] In addition, after switching from the entire person / torso detected in the subject detection area to the face detection area, the frequency of over-responses after the switch can be reduced by deleting the history of data on past image plane positions and shooting times.

[0073] (Second embodiment) (Dispersion of history / judgment of transfer from torso to face based on sudden acceleration / deceleration) Except for the sequences and processing flows described below, the configuration and control are the same as those in the first embodiment.

[0074] In the above explanation, a specific flow of the selection process performed in S304 was explained using Fig. 10. At this time, in S1002, it was determined whether the face size was a certain size or more. However, if this is taken as a past history, for example as a predetermined condition, and the variance of def2b, def2c, def2d, etc. in Fig. 11(e) is large, it may be determined that the person is currently performing irregular movements and is therefore not suitable for prediction or jumping onto the head.

[0075] As a specified condition, if it is determined that the most recent state is experiencing rapid acceleration or deceleration, such as the latter half of def2b, def2c, def2d, etc. in Figure 11(e), it may also be determined that this is not appropriate for prediction or transition to the head.

[0076] Also, the above dispersion and sudden acceleration / deceleration are judged from the defocus history of the torso, def2b, def2c, def2d, etc., but it is also possible to judge by adding the defocus after the transfer to the head at the end of the history. Specifically, it will be def2b, def2c, def1d.

[0077] As described above, according to the second embodiment of the present invention, when transitioning from an entire person / torso detected in a subject detection area to a face detection area, past history is checked to determine whether to transition to a face, thereby reducing the frequency of perspective conflicts between a small face and the background.

[0078] (Third embodiment) (Predicting the face position in a narrow area by tracing the DEFMAP history after the transfer) Except for the sequences and processing flows described below, the configuration and control are the same as those of the first or second embodiment.

[0079] The necessity and outline of the selection process performed in S304 of the third embodiment will be described with reference to FIG.

[0080] FIG. 11(a) shows an example of two focus detection areas 1111 and 1112. In this embodiment, a plurality of focus detection areas are arranged in a grid pattern as shown in FIG. 12(a). Each of them represents a focus detection area 502. For the 31 focus detection areas in the figure, the defocus information and time of each focus detection area are simultaneously saved in S903. Furthermore, the focus detection area 502 in FIG. 5(a) may be narrowed to have narrow defocus information of a narrow area by the focus detection area 502 in FIG. 5(b). By narrowing the area horizontally, perspective conflict is less likely to occur, but the large defocus ability is reduced instead. FIG. 12(b) shows an example in which the defocus information of each of the above 31 focus detection areas is a standard focus detection area and a narrow focus detection area, and the defocus information of the past history is stored. The position of 1201 corresponds to the focus detection area 1, the position of 1202 corresponds to the focus detection area 16 at the center of the face, and the position of 1203 corresponds to the focus detection area 29 at the center of the body.

[0081] In this state, the selection process is performed in S304 in Fig. 10. If S1003 is selected when the subject is approaching the camera and the face is still small, a prediction is made using the defocus history of the standard focus detection area of ​​the torso. The history of the standard focus detection area of ​​focus detection area 29 indicated by dotted line 1212 is used. If S1004 is selected when the subject is approaching and the face has become larger, a prediction is made using the defocus history of the narrow focus detection area of ​​the face. The history of the narrow focus detection area of ​​focus detection area 16 indicated by dotted line 1211 is used. In this case, the history information is not cleared in S1006.

[0082] In this embodiment, the memory capacity of the past history is increased, but instead of clearing the past history as in S1006, the defocus history of the face location is used going back in time, and narrow-frame defocus information for small subjects such as faces is used. This makes it difficult for perspective conflicts to occur.

[0083] As described above, according to the third embodiment, after switching from the whole person / torso detected in the subject detection area to the face detection area, the defocus history of the face location is used going back in time. Also, defocus information of a narrow focus detection area for a small subject such as a face is used. This makes it possible to reduce the frequency of perspective conflicts.

[0084] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0085] In addition, in each of the above-mentioned embodiments, the present invention has been described as being applied to a digital camera, but the present invention is not limited to this example. That is, the present invention may be applied to any device equipped with an image sensor. That is, the present invention is applicable to any device capable of capturing images, such as a mobile phone terminal, a portable image viewer, a television equipped with a camera, a digital photo frame, a music player, a game console, or an electronic book reader.

[0086] In the above embodiment, the lens barrel is integrated with the imaging device, and the imaging device is attached to the lens barrel. However, the imaging device may be an interchangeable lens type imaging device. The lens barrel type imaging device is configured with an interchangeable lens unit (lens barrel) and a camera body (imaging device). The lens control unit, which controls the overall operation of the lens unit, and the camera control unit, which controls the overall operation of the camera system including the lens unit, are configured to be able to communicate with each other through terminals provided on the lens mount. [Explanation of symbols]

[0087] 10 Lens unit 103 Focus Lens 105 Focus lens drive unit 106 Lens control unit 20 Imaging device 201 Image sensor 204 AF signal processing section 205 Display control unit 206 Display section 207 Recording medium control section 208 Recording media 212 Camera control unit 2121 Object detection unit 2122 Organ Detection Unit 2123 AF control section 213 Camera operation unit

Claims

1. An imaging device having a first detection means for detecting an object from an image, a second detection means for detecting a part of the object from the image, a focus detection means for detecting a focus state of the object and the part of the object, and a history means for storing a history of focus detection results of the object and the part of the object detected by the focus detection means, when the focus detection means detects a focus state of the subject detected by the first detection means and the variance of the focus detection results of the subject detected by the first detection means stored in the history means is equal to or greater than a certain value, the focus detection means adjusts the focus on the subject detected by the first detection means; an imaging apparatus characterized in that, when the variance of the focus detection results of the subject detected by the first detection means and stored in the history means is not greater than a certain value, focusing is performed on a portion of the subject detected by the second detection means.

2. An imaging device having a first detection means for detecting an object from an image, a second detection means for detecting a part of the object from the image, a focus detection means for detecting a focus state of the object and the part of the object, and a history means for storing a history of focus detection results of the object and the part of the object detected by the focus detection means, when the focus detection means detects a focus state of the object detected by the first detection means, if the focus detection result of the object detected by the first detection means stored in the history means shows a sudden acceleration or deceleration, the focus detection means adjusts the focus on the object detected by the first detection means; an imaging device characterized in that, when the focus detection result of the subject detected by the first detection means and stored in the history means does not show sudden acceleration or deceleration, focusing is performed on a portion of the subject detected by the second detection means.

3. 3. The imaging apparatus according to claim 1, wherein the subject detected by the first detection means is an entire body or a torso, and the part of the subject detected by the second detection means is a face.

4. a first detection step of detecting a subject from an image, a second detection step of detecting a portion of the subject from an image, a focus detection step of detecting a focus state of the subject and the portion of the subject, and a history step of storing a history of focus detection results of the subject and the portion of the subject detected by the focus detection step, and when a variance of the focus detection results of the subject detected by the first detection step stored in the history step is equal to or greater than a certain value while the focus state of the subject detected by the first detection step is being detected by the focus detection step, focusing on the subject detected by the first detection step; and a focusing step of focusing on a portion of the subject detected by the second detection means when the variance of the focus detection results of the subject detected by the first detection means and stored in the history means is not equal to or greater than a certain level.

5. a first detection step of detecting a subject from an image, a second detection step of detecting a portion of the subject from an image, a focus detection step of detecting a focus state of the subject and the portion of the subject, and a history step of storing a history of focus detection results of the subject and the portion of the subject detected by the focus detection step, and when the focus detection result of the subject detected by the first detection means and stored in the history step indicates a rapid acceleration or deceleration while the focus state of the subject detected by the first detection step is being detected by the focus detection step, focusing is performed on the subject detected by the first detection step, and a focusing step of focusing on a portion of the subject detected by the second detection means when the focus detection result of the subject detected by the first detection means and stored in the history means does not indicate rapid acceleration or deceleration.

Images