Focusing device, focusing method, and program

Abstract

To provide an imaging apparatus that can achieve stable AF.SOLUTION: A focusing imaging apparatus has: acquisition means that acquires a first amount of defocus from the phase difference between a pair of images in a first direction, and a second amount of defocus from the phase difference between a pair of images in a second direction; and focusing means that performs control of focusing of an imaging optical system by using the amounts of defocus acquired by the acquisition means. When a predetermined condition is satisfied, the focusing means performs control of focusing by fixedly selecting and using one of the first amount of defocus and the second amount of defocus that are sequentially acquired.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a focus adjustment device and a focus adjustment method that perform phase difference focus detection using an image sensor. [Background technology]

[0002] As an auto focus detection (AF) method for an imaging device, an image plane phase difference focus detection method in which an image sensor is used as a focus detection sensor is known. A configuration has been proposed in which focus detection is performed in a first pupil division direction and in a second pupil division direction different from the first pupil division direction, and one of the results of multiple focus detections is adopted (see Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3584691 [Patent Document 2] JP 2015-22264 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the configurations of Patent Documents 1 and 2, if the adopted detection result is unstable, the AF operation may become unstable. For example, if the focus detection result in the first pupil division direction and the focus detection result in the second pupil division direction are significantly different and the focus detection results are adopted alternately, the lens will move alternately between the two focus detection results.

[0005] An object of the present invention is to provide a focus adjustment device and a focus adjustment method that can achieve stable AF. [Means for solving the problem]

[0006] A focus adjustment device as one aspect of the present invention has an acquisition means for acquiring a first defocus amount of the imaging optical system from a phase difference between a pair of images in a first direction based on a signal from a first focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system, and acquiring a second defocus amount of the imaging optical system from the phase difference between a pair of images in a second direction different from the first direction based on a signal from a second focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system different from the first focus detection pixel, and a focus adjustment means that controls focus adjustment of the imaging optical system using the defocus amount acquired by the acquisition means, wherein the focus adjustment means controls focus adjustment by fixedly selecting one of the first defocus amount and the second defocus amount that are sequentially acquired when a predetermined condition is satisfied. Effect of the Invention

[0007] According to the present invention, it is possible to provide a focus adjustment device and a focus adjustment method that are capable of achieving stable AF. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of an imaging device according to a first embodiment. [Diagram 2] 1 is a schematic diagram of a pixel array of an image sensor according to a first embodiment. [Diagram 3] 1A and 1B are a plan view and a cross-sectional view of a pixel according to a first embodiment. [Figure 4] FIG. 2 is an explanatory diagram of pixels and pupil division according to the first embodiment. [Diagram 5] 4 is a diagram showing a correspondence relationship between an image sensor and pupil division according to the first embodiment. FIG. [Figure 6] 5A to 5C are diagrams illustrating the relationship between the defocus amount and the image shift amount of the first and second focus detection signals according to the first embodiment. [Figure 7] 5 is a flowchart showing a focus detection method according to the first embodiment. [Figure 8] 13A and 13B are diagrams illustrating an example of a composition for focus detection according to a second embodiment. [Figure 9] 10 is a flowchart showing a focus detection method according to a second embodiment. [Figure 10] FIG. 11 is a schematic diagram of a pixel array of an image sensor according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to refer to the same components, and duplicated descriptions will be omitted. [First embodiment] 1 is a block diagram of a lens-interchangeable single-lens reflex digital camera 10, which is an example of an imaging device according to this embodiment. Digital camera 10 has a lens unit 100 (interchangeable lens) and a camera body 120. Lens unit 100 is detachably attached to camera body 120 via a mount M, which is indicated by a dotted line in FIG. 1.

[0010] In this embodiment, the digital camera is configured to have interchangeable lenses, but the lens unit (image pickup optical system) and the camera body may be integrated. The present invention is not limited to digital cameras, and can be applied to other image pickup devices such as video cameras and smartphones.

[0011] Lens unit 100 includes a first lens group 101, an aperture 102, a second lens group 103, a focus lens group (hereinafter simply referred to as a "focus lens") 104, and a drive / control system. In this manner, lens unit 100 includes focus lens 104 and is a photographing lens (image pickup optical system) that forms a subject image.

[0012] The first lens group 101 is disposed at the tip of the lens unit 100 and is held so as to be movable forward and backward along the optical axis OA. The diaphragm 102 adjusts the amount of light during shooting by adjusting its aperture diameter, and functions as a shutter for adjusting exposure time during still image shooting. The diaphragm 102 and the second lens group 103 are integrally movable along the optical axis OA, and realize a zoom function in conjunction with the forward and backward movement of the first lens group 101. The focus lens 104 is movable along the optical axis OA, and the subject distance (focus distance) at which the lens unit 100 focuses changes depending on the position of the focus lens 104. By controlling the position of the focus lens 104 in a direction parallel to the optical axis OA (optical axis direction), focus adjustment (focus control) for adjusting the focus distance of the lens unit 100 is possible.

[0013] The drive / control system includes a zoom actuator 111, an aperture actuator 112, a focus actuator 113, a zoom drive circuit 114, an aperture drive circuit 115, a focus drive circuit 116, a lens MPU (processor) 117, and a lens memory 118. The zoom drive circuit 114 uses the zoom actuator 111 to move the first lens group 101 and the third lens group 103 along the optical axis OA, thereby controlling the angle of view of the lens unit 100 (performing a zoom operation). The aperture drive circuit 115 uses the aperture actuator 112 to drive the aperture 102, thereby controlling the aperture diameter and opening / closing operation of the aperture 102. The focus drive circuit 116 uses the focus actuator 113 to move the focus lens 104 along the optical axis OA, thereby controlling the focal distance of the lens unit 100 (performing focus control). The focus drive circuit 116 also has a function of detecting the current position (lens position) of the focus lens 104 using the focus actuator 113.

[0014] The lens MPU 117 performs all calculations and control related to the lens unit 100, and controls the zoom driving circuit 114, the aperture driving circuit 115, and the focus driving circuit 116. The lens MPU 117 is also connected to the camera MPU 125 via the mount M to communicate commands and data. For example, the lens MPU 117 detects the position of the focus lens 104, and notifies the camera MPU 125 of lens position information in response to a request from the camera MPU 125. The lens position information includes information such as the position of the focus lens 104 in the optical axis direction, the position and diameter of the exit pupil in the optical axis direction when the optical element is not moving, and the position and diameter of the lens frame that limits the light beam of the exit pupil in the optical axis direction. The lens MPU 117 also controls the zoom driving circuit 114, the aperture driving circuit 115, and the focus driving circuit 116 in response to a request from the camera MPU 125. The lens memory 118 stores optical information required for automatic focus adjustment (AF control). The camera MPU 125 controls the operation of the lens unit 100 by executing programs stored in, for example, an internal non-volatile memory or the lens memory 118 .

[0015] The camera body 120 has an optical low-pass filter 121, an image sensor 122, and a drive / control system. The optical low-pass filter 121 and the image sensor 122 function as an image sensor (image capture means) that photoelectrically converts a subject image (optical image) formed via the lens unit 100 and outputs image data. In this embodiment, the image sensor 122 photoelectrically converts the subject image formed via the lens unit 100 and outputs an image capture signal and a focus detection signal as image data.

[0016] The optical low-pass filter 121 reduces false colors and moiré in a captured image. The image sensor 122 is composed of a CMOS image sensor and its peripheral circuits, and is arranged with m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or more). In this embodiment, the image sensor 122 also serves as a focus detection element, has a pupil division function, and has pupil division pixels capable of phase difference detection focus detection (phase difference AF) using image data (image signal). The image processing circuit 124 generates data for phase difference AF and data for display, recording, and subject detection based on the image data output from the image sensor 122.

[0017] The drive / control system functions as a focus adjustment device. The drive / control system includes an image sensor drive circuit 123, an image processing circuit 124, a camera MPU (processor, control device) 125, a display 126, a group of operation switches (operation SW) 127, and a memory (recording means) 128. The drive / control system also includes a phase difference AF section (selection means, focus detection means) 129, a flicker detection section 130, an AE section 131, and a white balance adjustment section 132. The image sensor drive circuit 123 controls the operation of the image sensor 122, and also A / D converts the image signal (image data) output from the image sensor 122 and transmits it to the camera MPU 125. The image processing circuit 124 performs general image processing performed in digital cameras, such as gamma conversion, color interpolation processing, and compression encoding processing, on the image signal output from the image sensor 122. The image processing circuit 124 also generates a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection. In this embodiment, a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection are generated, but for example, a signal for AE, a signal for white balance adjustment, and a signal for subject detection may be generated as a common signal. Also, the combination of common signals is not limited to this.

[0018] The camera MPU 125 performs all calculations and control related to the camera body 120. That is, the camera MPU 125 controls the image sensor drive circuit 123, the image processing circuit 124, the display 126, the operation switch group 127, the memory 128, the phase difference AF section 129, the flicker detection section 130, the AE section 131, and the white balance adjustment section 132. The camera MPU 125 is connected to the lens MPU 117 via a signal line of the mount M to communicate commands and data. The camera MPU 125 issues to the lens MPU 117 a request to acquire lens position information and optical information unique to the lens unit 100, and a request to drive the lens by a predetermined drive amount.

[0019] The camera MPU 125 incorporates a ROM 125a storing a program for controlling the operation of the camera body 120, a RAM (camera memory) 125b for storing variables, and an EEPROM 125c for storing various parameters. The camera MPU 125 also executes focus detection processing based on the program stored in the ROM 125a. In the focus detection processing, a known correlation calculation process is executed using a pair of image signals obtained by photoelectrically converting optical images formed by light beams passing through different pupil regions (pupil partial regions) of the imaging optical system.

[0020] The display 126 is comprised of an LCD etc. and displays information regarding the shooting mode of the digital camera 10, a preview image before shooting and a confirmation image after shooting, and an in-focus state display image at the time of focus detection, etc. The operation switch group 127 is comprised of a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, etc. The memory 128 is a removable flash memory and records shot images.

[0021] The phase difference AF unit 129 performs focus detection processing by a phase difference detection method based on an image signal (a signal for phase difference AF) of focus detection image data obtained from the image sensor 122 and the image processing circuit 124. Specifically, the image processing circuit 124 generates a pair of image data formed by light beams passing through a pair of pupil regions of the imaging optical system as focus detection data, and the phase difference AF unit 129 detects the amount of focus deviation based on the amount of deviation of the pair of image data. In this way, the phase difference AF unit 129 performs phase difference AF (image plane phase difference AF) based on the output of the image sensor 122 without using a dedicated AF sensor. The phase difference AF unit 129 has an acquisition unit 129a and a calculation unit 129b. Note that at least a part of the means of the phase difference AF unit 129 (a part of the acquisition unit 129a or the calculation unit 129b) may be provided in the camera MPU 125. The phase difference AF unit 129 also controls the position of the focus lens 104 using the focus detection result and performs focus detection operation.

[0022] The flicker detection unit 130 performs flicker detection based on a signal for flicker detection obtained from the image processing circuit 124, thereby performing adjustment control of an appropriate exposure amount that does not cause flicker. The AE unit 131 performs exposure adjustment processing to make the shooting conditions appropriate by performing photometry based on a signal for AE obtained from the image sensor 122 and the image processing circuit 124. Specifically, photometry is performed based on the signal for AE, and the exposure amount at the aperture value, shutter speed, and ISO sensitivity currently set is calculated. From the difference between the calculated exposure amount and a predetermined appropriate exposure amount, an appropriate aperture value, shutter speed, and ISO sensitivity to be set at the time of shooting are calculated, and set as the shooting conditions, thereby performing exposure adjustment processing. The AE unit 131 functions as an exposure adjustment means that calculates the exposure conditions at the time of shooting using the photometry result, and controls the aperture value, shutter speed, and ISO sensitivity of the aperture 102. The white balance adjustment unit 132 performs white balance adjustment processing based on a signal for white balance adjustment obtained from the image sensor 122 and the image processing circuit 124. Specifically, the white balance of a signal for white balance adjustment is calculated, and the color weights are adjusted based on the difference from a predetermined appropriate white balance, thereby performing the white balance adjustment process.

[0023] In this way, the digital camera 10 can perform a combination of phase-difference AF, photometry (exposure adjustment), and white balance adjustment with subject detection. This allows the digital camera 10 to select a position (image height range) for performing phase-difference AF, photometry, and white balance adjustment according to the result of subject detection. (Image sensor) 2 is a schematic diagram of an array of imaging pixels (and focus detection pixels) of a two-dimensional CMOS image sensor of the image sensor 122, and shows the pixel array in a range of 4 columns x 4 rows. In this embodiment, in a pixel group 200 of 2 columns x 2 rows, a pixel 200R having a spectral sensitivity of R (red), pixels 200Ga and 200Gb having a spectral sensitivity of G (green), and a pixel 200B having a spectral sensitivity of B (blue) are arranged at the upper left, upper right, lower left, and lower right, respectively. Each pixel is composed of a first focus detection pixel 201 and a second focus detection pixel 202, and in pixel 200Gb, the first focus detection pixel 201 and the second focus detection pixel 202 are arranged along the column direction.

[0024] Fig. 3(a) is a plan view of the pixel 200Ga as viewed from the light receiving surface side (+z-axis side) of the image sensor 122. Fig. 3(b) is a cross-sectional view of the aa cross section of Fig. 3(a) as viewed from the -y-axis side.

[0025] The pixel 200Ga is formed with photoelectric conversion units 301 and 302 that are N-H divided (2 divisions) in the x-axis direction and N-V divided (1 division) in the y-axis direction. The photoelectric conversion units 301 and 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively. The photoelectric conversion units 301 and 302 may be pin structure photodiodes with an intrinsic layer sandwiched between a p-type layer and an n-type layer, or may be pn junction photodiodes with the intrinsic layer omitted as necessary.

[0026] The light incident on the pixel 200Ga is collected by the microlens 305, dispersed by the color filter 306, and then received by the photoelectric conversion units 301 and 302. If necessary, the spectral transmittance of the color filter may be changed for each subpixel, or the color filter may be omitted. In the photoelectric conversion units 301 and 302, pairs of electrons and holes are generated according to the amount of received light, and after separation by a depletion layer, the negatively charged electrons are accumulated in an n-type layer (not shown), and the holes are discharged to the outside of the image sensor 122 through a p-type layer connected to a constant voltage source (not shown). The electrons accumulated in the n-type layers of the photoelectric conversion units 301 and 302 are transferred to a capacitance unit (FD) via a transfer gate and converted into a voltage signal.

[0027] Fig. 4 is an explanatory diagram of pixels and pupil division in the first embodiment 1, and shows a cross-sectional view of the aa cross section in Fig. 3(a) as viewed from the +y side and the pupil plane (pupil distance DS) of the image sensor 122. In Fig. 4, the x-axis and y-axis of the cross-sectional view are inverted with respect to Fig. 3 in order to correspond to the coordinate axes of the pupil plane of the image sensor 122.

[0028] The first pupil partial region 501 of the first focus detection pixel 201 is in a roughly conjugate relationship with the light receiving surface of the photoelectric conversion unit 301, whose center of gravity is decentered on the -x-axis side, via the microlens 305, and represents a pupil region that can receive light at the first focus detection pixel 201. The first pupil partial region 501 has its center of gravity decentered on the +x-axis side on the pupil plane. The second pupil partial region 502 of the second focus detection pixel 202 is in a roughly conjugate relationship with the light receiving surface of the photoelectric conversion unit 302, whose center of gravity is decentered on the +x-axis direction, via the microlens 305, and represents a pupil region that can receive light at the second focus detection pixel 202. The second pupil partial region 502 has its center of gravity decentered on the -x-axis side on the pupil plane. The pupil region 500 is a pupil region that can receive light from the entire pixel 200Ga when all of the photoelectric conversion units 301, 302 (the first focus detection pixel 201 and the second focus detection pixel 202) are combined.

[0029] 5 is a diagram showing the correspondence between the image sensor 122 and pupil division in the first embodiment. Light beams that have passed through different pupil partial regions, a first pupil partial region 501 and a second pupil partial region 502, are incident on each pixel of the image sensor 122 at different angles and are received by a first focus detection pixel 201 and a second focus detection pixel 202 that are divided into 2×1 regions. In pixel 200Ga, pupil region 500 is pupil-divided into two in the horizontal direction, while in pixel 200Gb, pupil region 500 is pupil-divided into two in the vertical direction.

[0030] The image sensor 122 has an array of imaging pixels, each having a first focus detection pixel 201 and a second focus detection pixel 202. The first focus detection pixel 201 and the second focus detection pixel 202 receive a light beam that passes through a first pupil partial region 501 of the imaging optical system and a second pupil partial region 502 of the photographing optical system different from the first pupil partial region 501, respectively. Furthermore, each imaging pixel receives a light beam that passes through a pupil region 500 which is a combination of the first pupil partial region 501 and the second pupil partial region 502.

[0031] In this embodiment, the imaging pixels are composed of the first focus detection pixel 201 and the second focus detection pixel 202, but the present invention is not limited to this. If necessary, the imaging pixels, the first focus detection pixel 201, and the second focus detection pixel 202 may be configured as separate pixels, and the first focus detection pixel 201 and the second focus detection pixel 202 may be partially arranged in a portion of the imaging pixel array.

[0032] In this embodiment, focus detection is performed by collecting light receiving signals from the first focus detection pixel 201 of each pixel to generate a first focus detection signal, and collecting light receiving signals from the second focus detection pixel 202 of each pixel to generate a second focus detection signal. Furthermore, an imaging signal (captured image) with a resolution of the effective pixel number N is generated by adding up the signals from the first focus detection pixel 201 and the second focus detection pixel 202 for each pixel. Note that the method of generating each signal is not limited to the method of this embodiment. For example, the second focus detection signal may be generated from the difference between the imaging signal and the first focus detection signal. (Relationship between defocus amount and image shift amount) FIG. 6 is a diagram showing the relationship between the defocus amount and the image shift amount of the first focus detection signal and the second focus detection signal. The image sensor 122 is disposed on the image sensor 800, and the pupil surface of the image sensor 122 is divided into a first pupil partial region 501 and a second pupil partial region 502, as in FIG. 4 and FIG. 5. The defocus amount d is defined as a magnitude |d| of the distance from the imaging position of the subject to the image sensor 800, and a front focus state in which the imaging position of the subject is on the subject side of the image sensor 800 is defined as a negative sign (d<0). Also, a back focus state in which the imaging position of the subject is on the opposite side of the subject from the image sensor 800 is defined as a positive sign (d>0). In a focused state in which the imaging position of the subject is on the image sensor (focus position) 800, the defocus amount d is 0. The object 801 shows an example of a focused state (d=0), and the object 802 shows an example of a front focus state (d<0). The front focus state (d<0) and the back focus state (d>0) are combined to form a defocus state (|d|>0).

[0033] In the front focus state (d<0), the light beam from the subject 802 that passes through the first pupil partial region 501 (second pupil partial region 502) is once collected and then spreads to a width Γ1 (Γ2) centered on the center of gravity position G1 (G2) of the light beam, forming a blurred image on the imaging surface 800. The blurred image is received by the first focus detection pixel 201 (second focus detection pixel 202) constituting each pixel, and a first focus detection signal (second focus detection signal) is generated. Thus, the first focus detection signal (second focus detection signal) is recorded at the center of gravity position G1 (G2) on the imaging surface 800 as a subject image in which the subject 802 is blurred to a width Γ1 (Γ2). The blur width Γ1 (Γ2) of the subject image increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. Similarly, the magnitude |p| of the image shift amount p (= the difference G1-G2 in the center of gravity positions of the light beams) of the subject image between the first focus detection signal and the second focus detection signal also increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. In the back-focus state (d>0), the direction of the image shift of the subject image between the first focus detection signal and the second focus detection signal is the opposite to that in the front-focus state, but the same is true.

[0034] As the magnitude of the defocus amount of the imaging signal obtained by adding the first focus detection signal and the second focus detection signal, or the first focus detection signal and the second focus detection signal, increases, the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases. Therefore, the phase difference AF unit 129 converts the image shift amount into a detected defocus amount using a conversion coefficient calculated based on the base line length, based on the relationship in which the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases as the magnitude of the defocus amount of the imaging signal increases. The defocus calculation of the first focus detection signal and the second focus detection signal with pupil division in the horizontal direction, such as pixel 200Ga, is referred to as a horizontal eye defocus calculation. Also, the defocus calculation of the first focus detection signal and the second focus detection signal with pupil division in the vertical direction, such as pixel 200Gb, is referred to as a vertical eye defocus calculation. (Issues with two or more focus detection means) As described above, this embodiment has a first focus detection means (first focus detection pixel) with a pupil divided in the horizontal direction, and a second focus detection means (second focus detection pixel) with a pupil divided in the vertical direction. In the case of image plane phase difference AF, the focus lens 104 is moved according to the finally calculated defocus amount, so that the finally calculated value is one. Generally, a method of adopting the optimal focus detection result each time is known, but if the adopted detection result is unstable, the AF operation may become unstable. For example, if the focus detection result in the first pupil division direction and the focus detection result in the second pupil division direction are significantly different, and the focus detection results are adopted alternately, the lens will move alternately between the two focus detection results. If the lens drive is unstable, the quality of the AF will be impaired.

[0035] In this embodiment, a method for achieving stable AF in an image pickup apparatus that has focus detection means with at least two pupil division directions and is capable of acquiring the results of each will be described. (Focus detection method) FIG. 7 is a flowchart showing the focus detection method of this embodiment.

[0036] In step S101, it is determined whether or not the focus detection fixation condition is satisfied. If it is determined that the focus detection fixation condition is satisfied, the process of step S102 is executed, and if it is determined that the focus detection fixation condition is not satisfied, the process of step S105 is executed.

[0037] In this embodiment, it is possible to obtain focus detection results consisting of two pupil division directions, but in order to achieve stable AF, when the focus detection fixed condition is entered, the focus detection means is fixed to one of the two pupil division directions. The focus detection fixed condition is, for example, when at least one of the two focus detection results falls within a predetermined depth and the detected defocus amount is equal to or less than a certain value. In addition, it may be a case where the contrast evaluation value is equal to or greater than a predetermined value, or a condition using edge determination (edge ​​information) in combination with subject recognition. In addition, the greater the difference between the two focus detection results, the more effective the present invention is, so it is preferable to include a case where the difference between the two focus detection results is equal to or greater than a certain value in the focus detection fixed condition.

[0038] In step S102, the focus detection result selection means (phase difference AF section 129) selects the focus detection means to be fixed according to a predetermined criterion, and fixes the selected focus detection means. In this embodiment, the focus detection result selection means selects the focus detection means having a high reliability of the focus detection result. The reliability is, for example, the variability of the focus detection result predicted in the process of calculating the focus detection result, the contrast in the correlation direction of the focus detection, and the baseline length. The smaller the predicted variability of the focus detection result, the higher the contrast, and the longer the baseline length, respectively, the higher the reliability.

[0039] In step S103, the lens is driven until the focus detection fixing condition is satisfied. If the subject is not found and an appropriate focus detection result is not obtained, search driving is performed in a certain direction. If an appropriate focus detection result is obtained from at least one of the two focus detection results, the lens is driven according to that focus detection result. If the focus detection fixing condition is not satisfied even after a certain time has passed, it may be determined that the focus is not in focus.

[0040] In step S104, the lens is driven in accordance with the result of focus detection by the fixed focus detection means.

[0041] In step S105, it is determined whether the focusing condition is satisfied. If it is determined that the focusing condition is satisfied, this flow is ended, and if it is determined that the focusing condition is not satisfied, the process of step S104 is executed. The focusing condition is, for example, when the focus detection result of a fixed focus detection means is within a certain depth that is used for focusing determination. For a more strict determination, it is preferable to include in the focusing condition a case where the reliability is equal to or greater than a certain value.

[0042] If the focusing conditions are not satisfied after a certain period of time has elapsed, it may be determined that the focus is not achieved. Also, when it is determined that the focus is not achieved, the focus detection means may be fixed to a different focus detection means from the selected focus detection means, and the AF operation may be performed again. [Second embodiment] The basic configuration of the imaging device of this embodiment is similar to that of the imaging device of embodiment 1. In this embodiment, only the configuration different from that of embodiment 1 will be described, and a description of the common configuration will be omitted.

[0043] In this embodiment, a case will be described in which the focus detection fixing condition is set to a condition that uses edge determination in combination with subject recognition.

[0044] FIG. 8 is a diagram showing an example of the composition of focus detection in this embodiment. FIG. 8(a) shows a case where a main subject 804 and a perspective competing object 805 exist in front of the main subject 804 in an AF area 803. In the image plane phase difference AF, if a perspective competing object 805 with a different focal plane is included, a focus detection error may occur. In FIG. 8(a), the focus detection result may be pulled by the perspective competing object 805 in front. In recent years, the advancement of AI has made it possible to recognize the main subject 804 and the perspective competing object 805. In this embodiment, the perspective competing object 805 is judged when recognizing the main subject, and a stable AF is realized by fixing it to one of the focus detection means in the two pupil division directions. In the case of FIG. 8(a), since the perspective competing object 805 is a vertical line, the focus detection result of the first focus detection means with the pupil divided in the horizontal direction is easily affected by the perspective competing object 805. On the other hand, the second focus detection means with the pupil divided in the vertical direction is not easily affected when the perspective competing object 805 is a vertical line.

[0045] In Fig. 8(b), there is a main subject 807 within AF area 806, and a perspective competing object 808 behind main subject 807. In Fig. 8(b), the focus detection result may be influenced by perspective competing object 808 behind. In the case of Fig. 8(b), because perspective competing object 808 is a horizontal line, the focus detection result of the first focus detection means with the pupil divided in the horizontal direction is less likely to be affected by perspective competing object 808. On the other hand, the second focus detection means with the pupil divided in the vertical direction is more likely to be affected if perspective competing object 808 is a horizontal line.

[0046] FIG. 9 is a flowchart showing the focus detection method of this embodiment.

[0047] In step S111, object recognition is performed. For example, it is recognized whether or not the AF area includes a perspective competing object other than the main object.

[0048] In step S112, it is determined whether or not the focus detection fixation condition is satisfied. If it is determined that the focus detection fixation condition is satisfied, the process of step S113 is executed, and if it is determined that the focus detection fixation condition is not satisfied, the process of step S116 is executed. In the subject recognition mode, the focus detection fixation condition is, for example, when a perspective competing object other than the main subject is recognized and there is a difference between the first focus detection result and the second focus detection result that is equal to or greater than a predetermined value.

[0049] In step S113, the focus detection result selection means selects, from the first pupil division direction and the second pupil division direction, the focus detection means with the pupil division direction that is least affected by perspective conflicting objects, from the subject recognition result obtained in step S111. For example, in the case of Fig. 8(a), the second focus detection means with the pupil divided in the vertical direction is selected, and in the case of Fig. 8(b), the first focus detection means with the pupil divided in the horizontal direction is selected.

[0050] In step S114, the lens is driven in accordance with the result of focus detection by the fixed focus detection means.

[0051] In step S115, it is determined whether the focusing conditions are satisfied. If it is determined that the focusing conditions are satisfied, this flow is ended, and if it is determined that the focusing conditions are not satisfied, the process of step S114 is executed. Note that if the focusing conditions are not satisfied even after a certain period of time has elapsed, it may be determined that the focusing is not achieved. Also, when it is determined that the focusing is not achieved, the focusing means may be fixed to a different focusing means from the selected focusing means, and the AF operation may be performed again. In step S116, the lens is driven.

[0052] In step S117, it is determined whether or not the focus detection fixation condition is satisfied. If it is determined that the focus detection fixation condition is satisfied, the process of step S113 is executed, and if it is determined that the focus detection fixation condition is not satisfied, the process of step S118 is executed. In this step, if an appropriate focus detection result is obtained from at least one of the two focus detection results, the lens is driven according to that focus detection result. If the subject is not found and an appropriate focus detection result is not obtained, search drive is performed in a certain direction.

[0053] In step S118, it is determined whether the focusing condition is satisfied. If it is determined that the focusing condition is satisfied, this flow is ended, and if it is determined that the focusing condition is not satisfied, the process of step S116 is executed. Note that if the focusing condition is not satisfied even after a certain period of time has passed, it may be determined that the focusing condition is not satisfied. (Regarding the configuration of focus detection pixels) In the first and second embodiments, in the focus detection pixel configuration of FIG. 2, two focus detection means are provided: a first focus detection means with pupil division in the horizontal direction, and a second focus detection means with pupil division in the vertical direction, with the arrangement direction being different by 90 degrees. In other words, the first focus detection means and the second focus detection means are arranged so as to have a positional relationship of point symmetry with the center of the pixel group as the center of symmetry. However, the present invention is not limited to the focus detection pixel configuration. It is applicable if there are two focus detection means with different pupil division areas, and it is also applicable to a focus detection pixel configuration in which images are divided into four, 211, 212, 213, and 214, under one ML as shown in FIG. 10. In this case, in addition to horizontal division and vertical division, there are a maximum of four focus detection stages, including diagonal pupil division by 212 / 213 and diagonal pupil division by 211 / 214. Even in such a case, when the focus detection fixed condition is entered, the present invention is effective by selecting and fixing one focus detection means. The division is not limited to four, but may be 2n (n is an integer of 2 or more) in the first direction and the second direction.

[0054] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) an acquisition means for acquiring a first defocus amount of the imaging optical system from a phase difference between a pair of images in a first direction based on a signal from a first focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system, and acquiring a second defocus amount of the imaging optical system from the phase difference between the pair of images in a second direction different from the first direction based on a signal from a second focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system that is different from the first focus detection pixel; a focus adjustment unit that controls focus adjustment of the imaging optical system using the defocus amount acquired by the acquisition unit, The focus adjustment device, characterized in that, when a predetermined condition is satisfied, the focus adjustment means selects and uses a fixed one of the first defocus amount and the second defocus amount which are sequentially acquired to control the focus adjustment. (Configuration 2) The focus adjustment device described in configuration 1, characterized in that the first focus detection pixel and the second focus detection pixel are arranged in a point-symmetric positional relationship with the center of the pixel group as the center of symmetry. (Configuration 3) The focus adjustment device described in configuration 1, characterized in that each of the first focus detection pixel and the second focus detection pixel has a photoelectric conversion unit divided into 2n (n is an integer equal to or greater than 2) in the first direction and the second direction, and the multiple photoelectric conversion units have a function of photoelectrically converting multiple subject images formed by divided light beams from the imaging optical system, and outputting focus detection signals used to detect a phase difference. (Configuration 4) 4. The focus adjustment device according to any one of configurations 1 to 3, wherein the predetermined condition is that a detected defocus amount is equal to or less than a predetermined threshold value. (Configuration 5) 4. The focus adjustment device according to any one of configurations 1 to 3, wherein the predetermined condition is that a difference between the first defocus amount and the second defocus amount is equal to or greater than a predetermined threshold value. (Configuration 6) 6. The focus adjustment device according to any one of configurations 1 to 5, wherein the focus adjustment means selects the one of the focus detection results using at least one of a variation in focus detection results, a base line length, and a contrast. (Configuration 7) 6. The focus adjustment device according to any one of configurations 1 to 5, wherein the focus adjustment means selects the one of the two by using edge information other than that of the main subject. (Configuration 8) an image sensor including a pixel group having the first focus detection pixel and the second focus detection pixel; 8. An imaging device comprising the focus adjustment device according to any one of configurations 1 to 7. (Method 1) an acquisition step of acquiring a first defocus amount of the imaging optical system from a phase difference between a pair of images in a first direction based on a signal from a first focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system, and acquiring a second defocus amount of the imaging optical system from the phase difference between the pair of images in a second direction different from the first direction based on a signal from the second focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system that is different from the first focus detection pixel; a focus adjustment step of controlling focus adjustment of the imaging optical system by using the defocus amount acquired in the acquisition step, A focus adjustment method characterized in that, in the focus adjustment process, when a predetermined condition is satisfied, one of the first defocus amount and the second defocus amount which are sequentially acquired is fixedly selected and used to control the focus adjustment. (Configuration 9) A program for causing a computer to execute the focus adjustment method according to Method 1. [Explanation of symbols]

[0055] 122 Image sensor 129 Phase difference AF section (selection means, focus detection means) 129a Means of acquisition

Claims

1. an acquisition means for acquiring a first defocus amount of the imaging optical system from a phase difference between a pair of images in a first direction based on a signal from a first focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system, and acquiring a second defocus amount of the imaging optical system from the phase difference between a pair of images in a second direction different from the first direction based on a signal from a second focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system that is different from the first focus detection pixel; a focus adjustment unit that controls focus adjustment of the imaging optical system using the defocus amount acquired by the acquisition unit, a focus adjustment means for controlling focus adjustment by fixedly selecting one of the first defocus amount and the second defocus amount which are sequentially acquired when a predetermined condition is satisfied, and using the selected one.

2. 2. The focus adjustment device according to claim 1, wherein the first focus detection pixel and the second focus detection pixel are arranged in a point-symmetrical positional relationship with a center of a pixel group as a center of symmetry.

3. 2. The focus adjustment device according to claim 1, wherein each of the first focus detection pixel and the second focus detection pixel has a photoelectric conversion unit that is divided into 2n (n is an integer equal to or greater than 2) in the first direction and the second direction, and the multiple photoelectric conversion units have a function of photoelectrically converting a plurality of subject images formed by divided light beams from the imaging optical system, and outputting focus detection signals used for detecting a phase difference.

4. 4. The focus adjustment device according to claim 1, wherein the predetermined condition is that a detected defocus amount is equal to or less than a predetermined threshold value.

5. 4. The focus adjustment device according to claim 1, wherein the predetermined condition is that a difference between the first defocus amount and the second defocus amount is equal to or greater than a predetermined threshold value.

6. 4. The focus adjustment device according to claim 1, wherein the focus adjustment means selects the one of the two by using at least one of a variation in focus detection results, a base line length, and a contrast.

7. 4. The focus adjustment device according to claim 1, wherein the focus adjustment means selects the one of the two by using edge information other than that of the main subject.

8. an image sensor including a pixel group having the first focus detection pixel and the second focus detection pixel; An imaging device comprising the focus adjustment device according to claim 1.

9. an acquisition step of acquiring a first defocus amount of the imaging optical system from a phase difference between a pair of images in a first direction based on a signal from a first focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system, and acquiring a second defocus amount of the imaging optical system from the phase difference between the pair of images in a second direction different from the first direction based on a signal from a second focus detection pixel that photoelectrically converts an image formed by a light beam that has passed through a pupil partial region of the imaging optical system different from the first focus detection pixel; a focus adjustment step of controlling focus adjustment of the imaging optical system by using the defocus amount acquired in the acquisition step, A focus adjustment method characterized in that, in the focus adjustment process, when a predetermined condition is satisfied, one of the first defocus amount and the second defocus amount which are sequentially acquired is fixedly selected and used to control the focus adjustment.

10. A program for causing a computer to execute the focus adjustment method according to claim 9.

Images