Focus detection device, imaging device, and focus detection method
The focus detection device and method address the challenge of maintaining focus on moving subjects by dynamically selecting focus areas based on reliability, enhancing focus stability and reducing background issues.
Patent Information
- Application Number
- JP2023169013
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-09-29
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2039-01-31
AI Technical Summary
Existing focus detection methods struggle to maintain stable focus on moving subjects, often resulting in background omission or perspective conflicts due to difficulties in aligning the AF area with the subject, especially when the AF area is enlarged for moving objects, and prediction of future focus positions is unreliable.
A focus detection device and method that utilize a phase difference detection system to select between a first and second focus detection area based on the reliability of focus detection, using a first focus detection area when reliable and a second, wider area when reliability is low, and adjust focus accordingly.
Enables stable focus detection on moving subjects by suppressing background omission and perspective conflicts, ensuring accurate focus adjustment even with rapidly moving objects.
Smart Images

Figure 0007596476000006 
Figure 0007596476000007 
Figure 0007596476000008
Abstract
Description
[Technical field]
[0001] The present invention relates to an autofocus (AF) technology in an imaging device such as a digital camera. [Background technology]
[0002] In the AF of an imaging device, focus detection can be performed in an AF area (focus detection area) that the user sets arbitrarily as the area on the imaging screen where the user wants to focus, and the focus lens can be driven based on the result. However, it is difficult for the user to move (pan) the imaging device so that the AF area continues to focus on a moving subject. If the AF area moves away from the subject, the subject may be in focus on the background (background loss) or the focus may fluctuate due to perspective conflict.
[0003] Patent Document 1 discloses a method for setting a small (narrow) AF area when it is determined that the subject is not moving, and for setting a large (wide) AF area when it is determined that the subject is moving. Patent Document 2 discloses a method for predicting a future focus position for a moving subject. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Japanese Patent Application Publication No. 02-093419 [Patent Document 2] JP 2001-021794 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, if the AF area is enlarged when the subject is moving, as in the method disclosed in Patent Document 1, perspective conflicts may occur and it may be difficult to pinpoint focus on a small subject or part of a subject. Also, even if the future focus position is predicted as disclosed in Patent Document 2, it is not possible to specify the AF area to focus on if the prediction is incorrect.
[0006] The present invention provides a focus detection device and the like that is capable of performing stable focus detection for a moving subject by suppressing background loss and perspective conflict. [Means for solving the problem]
[0007] A focus detection device according to one aspect of the present invention performs focus detection using a phase difference detection method on a subject captured in a focus detection area in an image capture screen. The focus detection device includes a selection means for selecting a first focus detection area or a second focus detection area including the first focus detection area and its periphery as the focus detection area, a focus detection means for performing focus detection using a focus detection signal from the focus detection area selected by the selection means, and an information acquisition means for acquiring first information related to whether or not a subject moving within the image capture screen can be continuously captured in the first focus detection area. The selection means selects the first focus detection area when the first information indicates that the subject can be continuously captured in the first focus detection area, and when the first information indicates that the subject cannot be continuously captured in the first focus detection area, selects the first focus detection area when the reliability of the focus detection result in the first focus detection area is higher than a predetermined reliability, and selects the second focus detection area when the reliability is lower than the predetermined reliability. The focus detection means is characterized in that, when a first focus detection area is selected, it acquires a focus detection signal generated using a signal output from a first pixel area of an image sensor that captures an image of a subject, and, when a second focus detection area is selected, it acquires a focus detection signal generated using a signal output from a second pixel area of the image sensor that includes the first pixel area and its surroundings.
[0008] A focus adjustment device and an imaging device including the focus detection device described above also constitute another aspect of the present invention.
[0009] A focus detection method according to another aspect of the present invention is a method for performing focus detection by a phase difference detection method on a subject captured in a focus detection area in an image capture screen. The focus detection method includes the steps of selecting a first focus detection area and a second focus detection area including the first focus detection area and its periphery as focus detection areas, performing focus detection using a focus detection signal from the focus detection area selected in the focus detection area selection step, and acquiring first information related to whether or not a subject moving in the image capture screen can be continuously captured in the first focus detection area. In the focus detection area selection step, if the first information indicates that the subject can be continuously captured in the first focus detection area, the first focus detection area is selected, and if the first information indicates that the subject cannot be continuously captured in the first focus detection area, the first focus detection area is selected if the reliability of the focus detection result in the first focus detection area is higher than a predetermined reliability, and the second focus detection area is selected if the reliability is lower than the predetermined reliability. In the step of performing focus detection, if a first focus detection area is selected, a focus detection signal generated using a signal output from a first pixel area of an image sensor that captures an image of a subject is obtained, and if a second focus detection area is selected, a focus detection signal generated using a signal output from a second pixel area of the image sensor that includes the first pixel area and its surroundings is obtained.
[0010] Note that a computer program for causing a computer of an image capturing apparatus to execute processing according to the focus detection method constitutes another aspect of the present invention. Effect of the Invention
[0011] According to the present invention, stable focus detection can be performed on a moving subject. [Brief description of the drawings]
[0012] [Figure 1] 1 is a block diagram showing the configuration of a camera system including an interchangeable lens digital camera equipped with a focus adjustment device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a pixel arrangement of an image sensor for performing image-plane phase-difference AF in the camera. [Diagram 3] 4 is a flowchart showing an imaging process in the first embodiment. [Figure 4] 4 is a flowchart showing focus detection processing in the imaging processing. [Diagram 5] A diagram showing the AF area. [Figure 6] 5A to 5C are diagrams for explaining a pair of phase difference AF signals in focus detection processing. [Figure 7] 5A and 5B are diagrams illustrating the relationship between the shift amount and correlation amount of a pair of phase difference AF signals. [Figure 8] 6A and 6B are diagrams illustrating the relationship between the shift amount and the correlation change amount of a pair of phase difference AF signals. [Figure 9] 5 is a flowchart showing an AF area selection process in the imaging process. [Figure 10] 11 is a flowchart showing a prediction process in the imaging process. [Figure 11] 4A and 4B are diagrams for explaining first and second AF areas within an imaging screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 shows the configuration of a camera system including an interchangeable lens digital camera (hereinafter referred to as the camera body) 20 as an imaging device according to an embodiment of the present invention, and an interchangeable lens unit (hereinafter simply referred to as the lens unit) 10 that is detachably attached to the camera body 20.
[0014] The lens unit 10 has an imaging optical system that forms an image of a subject by focusing light from a subject (not shown), and a lens control unit 106 that is configured with a CPU or the like and controls all operations (processing) of the lens unit 10. The imaging optical system includes, in order from the subject side to the image side, a fixed lens 101, an aperture 102, and a focus lens 103.
[0015] The aperture 102 is driven by an aperture drive unit 104 to change its opening diameter, thereby controlling the amount of light incident on an image sensor 201 (described later). The focus lens 103 is driven in the optical axis direction by a focus lens drive unit 105 to perform focus adjustment. The aperture drive unit 104 and the focus lens drive unit 105 are controlled by a lens control unit 106.
[0016] In addition, in the lens unit 10, the lens operation unit 107 includes an operation member for the user to set the operation of the lens unit 10, such as switching between AF (autofocus) and MF (manual focus), setting the image stabilization operation ON / OFF, etc. The lens control unit 106 performs control according to the operation of the lens operation unit 107.
[0017] The lens control unit 106 is capable of communicating with a camera control unit 212 provided in the camera body 20. The lens control unit 106 controls the aperture driving unit 104 and the focus lens driving unit 105 in response to control commands and control information received from the camera control unit 212, and transmits optical information of the lens unit 10 and other lens information to the camera control unit 212.
[0018] In the camera body 20, the image sensor 201 is composed of a CCD sensor or a CMOS sensor, and photoelectrically converts (captures) an object image formed by a light beam passing through an imaging optical system. The image sensor 201 outputs a phase difference AF signal and an imaging signal, which will be described below, in response to a timing signal output by a timing generator 214 in accordance with a command from a camera control unit 212.
[0019] FIG. 2 shows the configuration of the image sensor 201 used in this embodiment. FIG. 2 shows the entire image sensor 201 and an enlarged view of a part of it. The image sensor 201 has a plurality of R, G, and B pixels arranged in a Bayer array. Each pixel is provided with a pair of photoelectric conversion units (sub-pixels) 201a and 201b divided into two in the horizontal direction, and a single microlens 201c shared by these. A light beam that has passed through different regions of the exit pupil of the imaging optical system is incident on the pair of sub-pixels 201a and 201b via the microlens 201c. This performs pupil division. The pair of sub-pixels 201a and 201b output A and B signals by photoelectrically converting the incident light. The image sensor 201 then combines A and B signals from a plurality of pixels included in an AF area, which will be described later, to output A and B image signals as a pair of phase difference AF signals. The pixels that output the A and B image signals are called focus detection pixels. Furthermore, the image sensor 201 adds the A image signal and the B image signal together and outputs an image signal (A+B signal).
[0020] The CDS / AGC / AD converter 202 performs correlated double sampling to remove reset noise, gain adjustment, and AD conversion on the phase difference AF signal and the imaging signal output from the image sensor 201. The converter 202 outputs the phase difference AF signal and the imaging signal that have been subjected to these processes to an AF signal processing unit 204 and an image input controller 203, respectively.
[0021] The AF signal processing unit 204 performs correlation calculations on a pair of phase difference AF signals (A image signal and B image signal) to calculate a phase difference (hereinafter referred to as image shift amount) which is the amount of shift between the A image signal and the B image signal, and also calculates its reliability (hereinafter referred to as focus detection reliability). In this embodiment, the focus detection reliability is calculated using the degree of coincidence between two images and the steepness of the amount of correlation change, which will be described later. In addition, the AF signal processing unit 204 sets the position and size of an AF area (focus detection area) which is an area in the imaging screen where focus adjustment is performed by AF. The AF signal processing unit 204 outputs information on the amount of image shift and the focus detection reliability to the camera control unit 212. Details of the processing performed by the AF signal processing unit 204 will be described later.
[0022] The AF control unit (control means) 2123 in the camera control unit 212 calculates the defocus amount of the imaging optical system using the information on the image shift amount and focus detection reliability from the AF signal processing unit 204. Then, it transmits a focus control command including information on the drive amount of the focus lens 103 converted from the defocus amount to the lens control unit 106. The lens control unit 106 controls the focus lens driving unit 105 to drive the focus lens 103 by the received drive amount. As a result, the image plane position moves so that a focused subject image is formed on the image sensor 201. The image plane position where the focused subject image is formed on the image sensor 201 is called the focused image plane position. The AF signal processing unit 204 and the camera control unit 212 constitute a focus detection device and a focus adjustment device.
[0023] Although FIG. 2 shows a case where each pixel has sub-pixels divided into two in the horizontal direction, it is also possible to have sub-pixels divided into two in the vertical direction, or to have sub-pixels divided into two each in the horizontal and vertical directions (a total of four divisions).
[0024] The image input controller 203 stores the imaging signal output from the converter 202 in the SDRAM 209 as an image signal via the bus 21. The image signal stored in the SDRAM 209 is read out by the display control unit 205 via the bus 21 and displayed on a display unit 206 provided on the rear surface of the camera body 20. In a recording mode in which the image signal is recorded, the image signal stored in the SDRAM 209 is recorded by the recording medium control unit 207 in a recording medium 208 such as a semiconductor memory.
[0025] The ROM 210 stores various data and computer programs for the camera control unit 212 to execute control and processing. The flash ROM 211 stores various setting information related to the operation of the camera 20 set by the user.
[0026] The camera operation unit 213 includes operation members such as a main switch for turning the power of the camera body 20 ON / OFF, an image capture preparation switch for starting AF / AE processing and the like, and an image capture start switch for starting image capture recording processing. The operation members also include a playback switch for performing playback processing of captured images generated and recorded by the image capture recording processing, and dials for performing various camera settings. The camera operation unit 213 outputs operation signals to the camera control unit 212 in response to user operations on these operation members.
[0027] The subject detection unit 2121 in the camera control unit 212 detects a specific subject from the imaging signal input from the image input controller 203, and determines the position of the specific subject in the imaging signal (image). The specific subject is a person's face present in the imaging screen, a subject present at a position specified by the user through the camera operation unit 213, etc. The subject detection unit 2121 also determines whether the specific subject is a moving or stationary object depending on whether the position of the specific subject has changed in multiple frames of the imaging signal continuously input from the image input controller 203. If the specific subject is a moving object, the subject detection unit 2121 acquires information on the position, size, and amount of movement of the specific subject, which is the difference between the position before movement and the position after movement. The information on the position, size, and amount of movement of the specific subject is mainly used to set the AF area.
[0028] The storage unit 2125 stores in the memory circuit 215 the in-focus image plane position calculated from the defocus amount and the acquisition times (focus detection times) of the A-image and B-image signals for calculating the defocus amount.
[0029] The camera control unit 212 controls each unit in the camera body 20 while exchanging information with them. The camera control unit 212 also performs power ON / OFF, AF / AE processing, imaging and recording processing, and playback processing of recorded images, and changes various camera settings, in response to operation signals from the camera operation unit 213. Furthermore, the camera control unit 212 transmits various control commands for the lens unit 10 (lens control unit 106) and information about the camera body 20 to the lens control unit 106, and obtains information about the lens unit 10 from the lens control unit 106. The camera control unit 212 is configured by a microcomputer, and controls the entire camera system by executing computer programs stored in the ROM 210.
[0030] Next, the process performed by the camera control unit 212 will be described. The camera control unit 212 performs the following process according to an imaging process program, which is a computer program stored in the ROM 210. The flowchart in Fig. 3 shows the imaging process including the AF operation that performs focus adjustment control. S means a step.
[0031] First, in S301, the camera control unit 212 determines whether or not to start an AF operation depending on whether the imaging preparation switch of the camera operation unit 213 is ON. If the AF operation is to be executed, the camera control unit 212 proceeds to S302 and performs a focus detection process. The details of the focus detection process will be described later.
[0032] Next, in S303, the camera control unit 212 as a selection unit performs an AF area selection process to select a first AF area (first focus detection area) or a second AF area (second focus detection area) as an AF area (hereinafter referred to as an in-use AF area) to be used as an area in the imaging screen to allow the user to capture a subject in an AF operation. Specifically, the camera control unit 212 selects the first AF area or the second AF area as the in-use AF area depending on whether it is possible to capture (include) a subject in the first AF area and whether the focus detection reliability, which is the reliability of the image shift amount detected in the first AF area, is high. Details of the AF area selection process will be described later.
[0033] Next, in S304, the camera control unit 212 performs a pre-imaging prediction process. Specifically, if the imaging start switch is ON, the camera control unit 212 causes the prediction unit 2124 to predict the focused image plane position from the detection of the image shift amount in the focus detection process of S302 for the AF area selected in S303 to the imaging recording process. Also, if the imaging start switch is OFF, the prediction unit 2124 predicts the focused image plane position until the next image shift amount is detected. Details of the method of predicting the focused image plane position by the prediction unit 2124 will be described later.
[0034] Next, in S305, the camera control unit 212 calculates the driving amount of the focus lens 103 required to move the image plane position to the in-focus image plane position predicted in S304, and transmits this to the lens control unit .
[0035] Next, in S306, the camera control unit 212 determines whether or not the imaging start switch is ON. If it is ON, the process proceeds to S307, and if not, the process proceeds to S310.
[0036] In S307, the camera control unit 212 performs imaging and recording processing, and stores the captured image obtained thereby in the memory circuit 215. Next, in S308, the camera control unit 212 causes the prediction unit 2124 to predict the focused image plane position at the time of the next image shift amount detection. Next, in S309, the camera control unit 212 calculates the driving amount of the focus lens 103 required to move the image plane position to the focused image plane position predicted in S308, and transmits this to the lens control unit 106. Then, the camera control unit 212 proceeds to S310.
[0037] In S310, the camera control unit 212 determines whether or not the imaging preparation switch is OFF. If it is OFF, this process ends, and if it is ON, the process returns to S302 and the above process is repeated.
[0038] 11(a) shows a first AF region 1101 and a second AF region 1102 set on the image sensor 201 (within an image capture screen 1100). When an AF operation is performed, an AF frame is displayed on the display unit 206 so as to overlap with a live view image observed by the user. The first AF region 1101 has a size equivalent to that of the AF frame. The second AF region 1002 includes the first AF region 1001 and its surroundings, and is a larger region than the first AF region 1101.
[0039] In FIG. 11B, three vertically striped regions are arranged in a first pixel region corresponding to the first AF region 1101 of the image sensor 201, and are first focus detection pixel rows in which a plurality of focus detection pixels from which a pair of phase difference AF signals are read out are arranged in the horizontal direction. In FIG. 11C, ten diagonally striped regions are arranged in a second pixel region corresponding to the second AF region 1102 of the image sensor 201, and are second focus detection pixel rows in which a plurality of focus detection pixels from which a pair of phase difference AF signals are read out are arranged in the horizontal direction. The second pixel region also includes the above-mentioned three first focus detection pixel rows. Some of the ten second focus detection pixel rows are arranged between the three first focus detection pixel rows in the first pixel region. That is, a second focus detection pixel row different from the first focus detection pixel row is provided in a portion of the second pixel region included in the first pixel region.
[0040] Furthermore, the second focus detection pixel row has the same horizontal size (length) as the first focus detection pixel row. It is desirable that each focus detection pixel row is of a size that makes it difficult for perspective conflict to occur when capturing an image of a small subject or a part of a subject. Note that the positions and sizes of the first and second pixel regions and each focus detection pixel row shown in Figures 11(a) to (c) are examples, and other positions and sizes may be used.
[0041] The flowchart in Fig. 4 shows the focus detection process performed in S302 of the imaging process shown in Fig. 3. In S401, the AF signal processing unit 204 acquires an A image signal and a B image signal, which are a pair of phase difference AF signals, from a focus detection pixel row in a pixel area in the image sensor 201 that corresponds to the AF area.
[0042] Calculation of the image shift amount performed by the AF signal processor 204 will be described with reference to FIG. 5 and FIGS. 6(a) to 6(c). FIG. 5 shows a correlation calculation area 504 including a focus detection pixel array 502 on the image sensor 201 and shift areas 503 on both sides of the focus detection pixel array 502. The focus detection pixel array 502 is an area in which the amount of correlation between the A image signal 601 and the B image signal 602 shown in FIG. 6(a) is calculated by correlation calculation. The shift area 503 is an area required to shift the A image signal 601 and the B image signal 602 in the plus and minus directions in order to perform the correlation calculation, as shown in FIGS. 9(b) and 9(c). In FIG. 5 and FIGS. 6(a) to 6(c), p, q, s, and t respectively represent horizontal coordinates, and p to q represent the correlation calculation area 504. Furthermore, s to t represent the arrangement area of the focus detection pixel array 502.
[0043] In the next step S402, the AF signal processing unit 204 calculates the amount of correlation between the A image signal 601 and the B image signal 602 while relatively shifting the A image signal 601 and the B image signal 602 by one pixel (one bit) in the positive or negative direction. Specifically, it calculates the sum of the absolute values of the differences between the A and B image signals 601, 602 after the shift. The AF signal processing unit 204 then calculates the amount of correlation COR using the following formula (1), where the shift amount is i, the minimum shift amount is ps, the maximum shift amount is qt, and the start coordinates and end coordinates of the focus detection pixel array 502 are x and y, respectively.
[0044]
number
[0045] The amount of shift for each of the A image signal 601 and the B image signal 602 may be a number of pixels (for example, two pixels).
[0046] 7A shows an example of how correlation amount (COR) 701 changes with the shift amount. The horizontal axis represents the shift amount, and the vertical axis represents the correlation amount. The correlation amount 701 has extreme values 702 and 703. A smaller correlation amount 701 indicates a higher degree of similarity between the A image signal 601 and the B image signal 602, that is, a higher degree of match.
[0047] Next, in S403, the AF signal processing unit 204 calculates a correlation change amount from the difference between the correlation amounts obtained at the shift amounts i-1 and i+1, for example, from the correlation amounts 701 calculated in S402. Specifically, the correlation change amount ΔCOR is calculated by the following formula (2).
[0048]
number
[0049] 7(b) shows an example of how the correlation change amount (ΔCOR) 705 changes for each shift amount. The horizontal axis represents the shift amount, and the vertical axis represents the correlation change amount. The correlation change amount 705 has zero crossing points 706 and 707 where its value goes from positive to zero and then negative. When the correlation change amount becomes zero, the degree of match between the A image signal and the B image signal is highest. The shift amount when the correlation change amount becomes zero is the amount of image shift.
[0050] Fig. 8(a) shows an enlarged view of the correlation change amount 705 near the zero cross point 706 shown in Fig. 7(b). In S404, the AF signal processing unit 204 calculates the image shift amount PRD by dividing it into an integer part β and a decimal part α. The AF signal processing unit 204 calculates the decimal part α by the following formula (3) based on the similarity relationship between triangles ABC and ADE shown in the figure.
[0051]
number
[0052] Furthermore, the AF signal processing unit 204 calculates the integer part β by the following formula (4), as shown in FIG. β=k-1 (4) Then, the AF signal processor 204 calculates the image shift amount PRD from the sum of α and β.
[0053] Furthermore, when there are multiple zero crossing points 706, 707 as shown in Fig. 7(b), the AF signal processor 204 determines the zero crossing point with the largest steepness maxder of the change in the correlation change amount at each zero crossing point as the first zero crossing point. The steepness maxder indicates that the larger the value, the easier it is to perform AF. The AF signal processor 204 calculates the steepness maxder using the following formula (5).
[0054]
number
[0055] Then, the AF signal processor 204 determines the shift amount that gives the first zero cross point as the image shift amount PRD.
[0056] Next, in S405, the AF signal processing unit 204 calculates the defocus amount using the image shift amount calculated in S404. Furthermore, the AF signal processing unit 204 also calculates the focus detection reliability, which is the reliability of the image shift amount, in other words, the defocus amount. Specifically, the focus detection reliability is calculated as follows. Note that the calculation method of the focus detection reliability described below is merely an example, and the focus detection reliability may be calculated depending on the contrast of the subject and whether the defocus amounts (focus detection results) calculated multiple times in the past are continuous or not. In other words, when the contrast of the subject is high or the defocus amount changes continuously, the focus detection reliability is increased, and when the contrast of the subject is low or the defocus amount changes discontinuously, the focus detection reliability is decreased.
[0057] The AF signal processing unit 204 defines the degree of coincidence between the A image signal and the B image signal by the steepness maxder of the change in the correlation change amount described above and the degree of coincidence between the A image signal and the B image signal, that is, the degree of coincidence fnclvl of the two-image coincidence. The higher the value of the degree of coincidence between the two images, the higher the accuracy of the amount of image shift, that is, the amount of defocus. FIG. 8(b) shows an enlarged view of the correlation amount 701 near the extreme value 702 shown in FIG. 7(a). The AF signal processing unit 204 calculates the degree of coincidence between the two images according to the value of steepness maxder using the following formula (6).
[0058]
number
[0059] Then, in S406, the AF signal processing unit 204 stores the defocus amount obtained in each focus detection pixel row, the focus detection reliability, and the focus detection times at which the A-image and B-image signals were obtained, as focus detection information in the memory circuit 215. In this manner, the focus detection process ends.
[0060] The flowchart in Fig. 9 shows the AF area selection process performed in S303 of the imaging process shown in Fig. 3. In S901, the camera control unit 212 (subject detection unit 2121) as an information acquisition unit acquires the amount of movement of a specific subject (moving object) detected within a predetermined number of frames, that is, a predetermined time, from the imaging signal from the image input controller 203. This amount of movement of the specific subject is first information related to whether or not it is possible to keep capturing a subject moving within the imaging screen in the first focus detection area.
[0061] Next, in S902, the camera control unit 212 as a determination unit determines whether or not it is possible (easy) for the user to continue to capture the specific subject within the first AF area, using the amount of movement of the specific subject acquired in S901. In other words, it determines whether or not the first information indicates that the specific subject moving within the imaging screen can be continuously captured in the first AF area.
[0062] If the amount of movement of the specific subject is smaller than a predetermined value, the camera control unit 212 determines that the specific subject is a subject with little movement and can (is easy) be continuously captured in the first AF area, which is a narrow area, and proceeds to S905, where it selects (sets) the first AF area as the AF area in use.On the other hand, if the amount of movement of the specific subject is larger than the predetermined value, it determines that the specific subject is a subject with rapid movement and cannot (is difficult) be continuously captured in the first AF area, and proceeds to S903.
[0063] Whether or not the specific subject can be continuously captured in the first AF area can be determined not only by the amount of movement of the specific subject within a predetermined time, but also by the shake detection result based on the output of a gyro sensor (shake sensor) mounted on the camera body 20 or the lens unit 10. That is, the amount of shake of the imaging screen relative to the subject is acquired as first information, and the determination can be made based on the magnitude of the shake amount. Specifically, when the detected shake is smaller than a predetermined amount, it may be determined that the specific subject can be continuously captured in the first AF area, and when the shake is larger than the predetermined amount, it may be determined that the specific subject cannot be continuously captured in the first AF area.
[0064] Furthermore, whether or not the specific subject can be continuously captured in the first AF area may be determined using information on the focal length of the imaging optical system as the first information. Specifically, if the focal length is shorter than a predetermined distance, it may be determined that the specific subject can be continuously captured in the first AF area, and if the focal length is longer than the predetermined distance, it may be determined that the specific subject cannot be continuously captured in the first AF area.
[0065] In S903, the camera control unit 212 acquires the focus detection reliability for the defocus amount calculated from a pair of phase difference AF signals from the focus detection pixel array in the first AF region and stored in the memory circuit 215 in S302.
[0066] Next, in S904, the camera control unit 212 determines whether or not the focus detection reliability in the first AF area acquired in S903 is higher than a predetermined reliability. If the focus detection reliability is higher than the predetermined reliability, the camera control unit 212 proceeds to S906 and selects the first AF area as the AF area in use. On the other hand, if the focus detection reliability is lower than the predetermined reliability, the camera control unit 212 proceeds to S907 and selects the second AF area as the AF area in use. In this way, the AF area selection process ends.
[0067] The flowchart in Fig. 10 shows the pre-imaging prediction process performed in S304 of the imaging process shown in Fig. 3. In the pre-imaging prediction process, the in-focus image plane position at a future time is calculated (predicted) from changes in the in-focus image plane position obtained from the results of focus detection in the past several times and the focus detection times corresponding to those in-focus image plane positions. In this embodiment, the in-focus image plane position at a future time is predicted using statistical calculation. However, the in-focus image plane position at a future time may be predicted using other methods.
[0068] First, in S1001, the prediction unit 2124 acquires the defocus amount in the used AF area (the first or second AF area) selected in S303 from among the defocus amounts stored in the memory circuit 215 in S302.
[0069] Next, in S1002, the prediction unit 2124 calculates the in-focus image plane position and focus detection time corresponding to the acquired defocus amount. Generally, a certain amount of charge accumulation time is required before an output signal is obtained from the image sensor 201. For this reason, the prediction unit 2124 sets the time between the start time and end time of charge accumulation (for example, the central time) as the focus detection time. Then, the prediction unit 2124 calculates the in-focus image plane position by adding the acquired defocus amount to the current position of the focus lens 103.
[0070] Next, in S1003, the prediction unit 2124 stores the calculated in-focus image plane positions and the corresponding focus detection time data in the memory circuit 215. The memory circuit 215 sequentially stores up to a predetermined number of in-focus image plane positions and focus detection time data, and after the predetermined number of stored data is reached, the oldest stored data is overwritten with the newest data.
[0071] Next, in S1004, the prediction unit 2124 judges whether or not the number of data stored in the memory circuit 215 has reached the number required for performing statistical calculations. If the number of stored data is sufficient, the prediction unit 2124 proceeds to S1005, and if not, proceeds to S1007.
[0072] In S1005, the prediction unit 2124 determines a prediction formula for predicting the in-focus image plane position at a future time. In this embodiment, a prediction function f(t) shown in equation (7) is used as the prediction formula. The prediction unit 2124 statistically determines the coefficients α, β, and γ in equation (7) by multiple regression analysis. n in equation (7) is a value that minimizes the prediction error when prediction is performed on samples of multiple representative moving object prediction imaging scenes.
[0073] f(t) = α + β t + γ t n (7) Having thus determined the prediction formula, the prediction unit 2124 proceeds to S1006, where it calculates the focused image plane position at a predetermined future time using formula (7), and calculates the drive amount of the focus lens 103 required to move the actual image plane position to that focused image plane position. Then, it transmits the calculated drive amount to the lens control unit 106 as the drive amount of the focus lens 103 at the predetermined future time. As a result, the focus lens 103 is driven at the predetermined future time, and the actual image plane position moves to the calculated focused image plane position.
[0074] On the other hand, in S1007, the prediction unit 2124 calculates the drive amount of the focus lens 103 from the defocus amount calculated without using statistical calculation (i.e., without prediction), and transmits the calculated drive amount to the lens control unit 106. As a result, the focus lens 103 is driven in a direction in which the subject is brought into focus.
[0075] As described above, in this embodiment, the size of the active AF area is selected depending on whether or not the user can (is easy to) keep a specific moving subject in the active AF area. This makes it possible to perform stable AF operation by suppressing background loss and perspective conflict for small moving subjects or parts of subjects. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0076] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]
[0077] 10 Interchangeable lens unit 103 Focus Lens 20 Camera body 201 Image sensor 204 AF signal processing section 212 Camera control unit
Claims
1. A focus detection device that performs focus detection using a phase difference detection method on a subject captured in a focus detection area of an imaging screen, a selection means for selecting, as the focus detection area, a first focus detection area and a second focus detection area including the first focus detection area and its surroundings; a focus detection unit that performs the focus detection using a focus detection signal from the focus detection area selected by the selection unit; an information acquisition unit that acquires first information relating to whether or not the subject moving within the imaging screen can be continuously captured in the first focus detection area, The selection means is selecting the first focus detection area when the first information indicates that it is possible to continue capturing the object in the first focus detection area; when the first information indicates that it is not possible to continue capturing the subject in the first focus detection area, if a reliability of the focus detection result in the first focus detection area is higher than a predetermined reliability, the first focus detection area is selected, and if the reliability is lower than the predetermined reliability, the second focus detection area is selected; The focus detection device is characterized in that, when the first focus detection area is selected, the focus detection means acquires the focus detection signal generated using a signal output from a first pixel area of an image sensor that captures an image of a subject, and, when the second focus detection area is selected, acquires the focus detection signal generated using a signal output from a second pixel area of the image sensor that includes the first pixel area and its surroundings.
2. the first information is information indicating whether a movement amount of the subject within a predetermined time is smaller or larger than a predetermined amount, 2. The focus detection device of claim 1, wherein if the amount of movement is smaller than the predetermined amount, it indicates that it is possible to continue to capture the subject in the first focus detection area, and if the amount of movement is greater than the predetermined amount, it indicates that it is not possible to continue to capture the subject in the first focus detection area.
3. the first information is information indicating whether an amount of shake of an imaging device including the imaging element, detected by a shake sensor, is smaller or larger than a predetermined amount; 2. The focus detection device according to claim 1, wherein when the amount of shake is smaller than the predetermined amount, it indicates that it is possible to continue to capture the subject in the first focus detection area, and when the amount of shake is greater than the predetermined amount, it indicates that it is not possible to continue to capture the subject in the first focus detection area.
4. the first information is information indicating whether a focal length of an imaging optical system is shorter or longer than a predetermined distance, 2. The focus detection device according to claim 1, wherein when the focal length is shorter than the predetermined distance, it indicates that it is possible to continue to capture the subject in the first focus detection area, and when the focal length is longer than the predetermined distance, it indicates that it is not possible to continue to capture the subject in the first focus detection area.
5. The focus detection device of claim 1, characterized in that the reliability is high when the result of the focus detection in the first focus detection area changes continuously, and is low when the result of the focus detection in the first focus detection area changes discontinuously.
6. A focus detection device according to any one of claims 1 to 5, a control unit that performs focus adjustment control using a result of the focus detection by the focus detection device.
7. a prediction means for calculating a focused image plane position at a first future time by using the results of the focus detection performed a plurality of times in the past; 7. The focus adjustment device according to claim 6, wherein the control means performs focus adjustment control so that an actual image plane position moves to the in-focus image plane position at the first future time.
8. an imaging element for capturing an image of a subject; An imaging device comprising the focus adjustment device according to claim 6.
9. A focus detection method for performing focus detection using a phase difference detection method on a subject captured in a focus detection area of an imaging screen, comprising: selecting, as the focus detection area, a first focus detection area and a second focus detection area including the first focus detection area and its surroundings; performing the focus detection using a focus detection signal from the focus detection area selected in the step of selecting the focus detection area; acquiring first information relating to whether or not the subject moving within the imaging screen can be continuously captured in the first focus detection area; In the step of selecting a focus detection area, selecting the first focus detection area when the first information indicates that it is possible to continue capturing the object in the first focus detection area; when the first information indicates that it is not possible to continue capturing the subject in the first focus detection area, if a reliability of the focus detection result in the first focus detection area is higher than a predetermined reliability, the first focus detection area is selected, and if the reliability is lower than the predetermined reliability, the second focus detection area is selected; A focus detection method characterized in that, in the step of performing focus detection, if the first focus detection area is selected, the focus detection signal is generated using a signal output from a first pixel area of an image sensor that captures an image of a subject, and if the second focus detection area is selected, the focus detection signal is generated using a signal output from a second pixel area of the image sensor that includes the first pixel area and its surroundings.
10. 10. A computer program causing a computer of an image pickup apparatus to execute a process according to the focus detection method of claim 9.
Citation Information
Patent Citations
Automatic focus detecting device
JP1990093419A
Auto-focusing adjusting device, and optical instrument
JP2001021794A
Camera
JP2010014883A
Image pickup apparatus, and control method for the same
JP2016224277A
Focus control device and imaging device
JP2018197845A
