Imaging apparatus, control method thereof, program and storage medium

Abstract

To provide an imaging apparatus capable of stable focus adjustment even under an environment difficult for focus detection.SOLUTION: An imaging apparatus includes: an imaging device with a plurality of aligned unit pixels having one microlens and a plurality of photoelectric conversion parts corresponding to the one microlens; a reading part configured to read out a signal of a photoelectric conversion part from the imaging device; a first calculation part configured to calculate a defocus amount in a plurality of focus detection frames in an imaging screen according to the signal; a generation part configured to generate a histogram showing a frequency distribution of respective defocus amounts of the plurality of focus detection frames; a storage part configured to store a histogram generated in a prescribed cycle; an addition part configured to add a plurality of histograms aligned in a time sequence among histograms stored at the storage part; and a second calculation part configured to calculate a defocus amount of a subject according to a result of the addition part.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a focus adjustment technique for an imaging device such as a digital camera or a video camera. [Background technology]

[0002] Patent Document 1 discloses a technology that eliminates the need for a dedicated AF sensor and achieves high-speed phase-difference AF by providing an imaging element with a function for detecting the phase difference of a subject image.

[0003] Patent document 2 discloses a technology in which the sensitive area of ​​the light receiving element (pixel) of a part of an image sensor is decentered with respect to the optical axis of an on-chip microlens to provide a pupil division function and perform phase difference focus detection.

[0004] Patent Document 3 discloses a technology for calculating a frequency distribution (histogram) of subject distances (defocus amounts), selecting multiple subject distances based on the peaks of appearance frequency, and acquiring multiple captured images focused at each subject distance.

[0005] Patent Document 4 discloses a technology that, when performing continuous shooting using servo AF, identifies peaks containing the same subject from changes in the peak position of the histogram between captured images (between frames), and tracks the subject between frames with high accuracy. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-292686 [Patent Document 2] Japanese Patent Application Laid-Open No. 2014-232181 [Patent Document 3] Japanese Patent Application Laid-Open No. 2014-232181 [Patent Document 4] Japanese Patent Publication No. 2022-171437 Summary of the Invention [Problem to be solved by the invention]

[0007] In the conventional imaging devices described above, mainly when capturing moving images, image signals are displayed and focus detection is performed using focus detection signals. In such cases, it is difficult to achieve both short-period image updates for a comfortable image display and highly accurate focus detection in low-luminance environments.

[0008] The imaging devices described in Patent Documents 3 and 4 create a defocus amount histogram from multiple focus detection frames in order to identify the main subject on which focus is to be tracked. The techniques described in these patent documents calculate the defocus amount and generate the histogram from a focus detection signal obtained from a single frame. Therefore, in environments where focus detection is difficult, such as the low-brightness environments described above, the number of focus detection frames that can provide reliable focus detection results decreases, making it difficult to obtain a sufficient number of samples for creating the histogram.

[0009] The present invention has been made in view of the above-mentioned problems, and its object is to provide an imaging apparatus that can perform stable focus adjustment even in an environment where focus detection is difficult. [Means for solving the problem]

[0010] An imaging device according to the present invention is characterized by comprising: an imaging element in which a plurality of unit pixels are arranged, each unit pixel having one microlens and a plurality of photoelectric conversion units corresponding to the one microlens; a readout means for reading out a signal from the photoelectric conversion unit from the imaging element; a first calculation means for calculating a defocus amount in a plurality of focus detection frames within an imaging screen based on the signal; a generation means for generating a histogram indicating a frequency distribution of the defocus amount for each of the plurality of focus detection frames; a storage means for storing the histograms generated at a predetermined cycle; an addition means for adding together a plurality of histograms arranged in chronological order from among the histograms stored in the storage means; and a second calculation means for calculating a defocus amount of a subject based on the result of the addition means. [Effects of the Invention]

[0011] According to the present invention, stable focus adjustment can be performed even in an environment where focus detection is difficult. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the configuration of an imaging apparatus according to an embodiment of the present invention; [Figure 2] FIG. 2 is a schematic diagram of an arrangement of imaging pixels (and focus detection pixels) of an image sensor. [Figure 3] 3A and 3B are a plan view and a cross-sectional view of a pixel of an imaging element. [Figure 4] FIG. 2 is a schematic explanatory diagram showing the correspondence between pixels and pupil divisions. [Figure 5] FIG. 2 is a schematic diagram showing the correspondence between an image sensor and pupil division. [Figure 6] FIG. 10 is a diagram showing the relationship between the defocus amount and the image shift amount. [Figure 7] 10 is a flowchart of a focus adjustment operation. [Figure 8] FIG. 4 is a diagram showing the arrangement of focus detection frames in a portrait scene. [Figure 9] 10A and 10B are diagrams showing examples of histograms calculated based on the defocus amounts of the focus detection frames. [Figure 10]FIG. 10 is a diagram conceptually illustrating frame addition processing of a histogram. [Figure 11] 10 is a flowchart showing the operation of a histogram frame addition process. [Figure 12] 10 is a flowchart of a process for calculating a defocus amount. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0014] (Configuration of imaging device 10) Fig. 1 is a block diagram showing the configuration of an image capture device 10 according to one embodiment of the present invention. In Fig. 1, the image capture device 10 is a single-lens reflex digital camera with interchangeable lenses.

[0015] The imaging device 10 takes the form of a camera system having a lens unit 100 (interchangeable lens) and a camera body 120. The lens unit 100 is detachably attached to the camera body 120 via a mount M shown by a dotted line in FIG. 1. However, this embodiment is not limited to the configuration shown in FIG. 1 and can also be applied to an imaging device (digital camera) in which the lens unit (imaging optical system) and the camera body are integrated. Furthermore, the present invention is not limited to digital cameras and can also be applied to other imaging devices such as video cameras.

[0016] Lens unit 100 has an optical system consisting of a first lens group 101, an aperture 102, a second lens group 103, a focus lens group (hereinafter simply referred to as "focus lens") 104, and a drive / control system. Thus, lens unit 100 is a photographing lens (image pickup optical system) that includes focus lens 104 and forms a subject image.

[0017] The first lens group 101 is disposed at the tip of the lens unit 100 and is held so as to be able to move back and forth in the optical axis direction OA. The diaphragm 102 adjusts the amount of light during shooting by adjusting its aperture diameter, and also functions as a shutter for adjusting the exposure time during still image shooting. The diaphragm 102 and the second lens group 103 are movable together in the optical axis direction OA, and a zoom function is achieved in conjunction with the forward and backward movement of the first lens group 101. The focus lens 104 is movable in the optical axis direction OA, and the subject distance (focusing distance) at which the lens unit 100 focuses changes depending on its position. Controlling the position of the focus lens 104 in the optical axis direction OA enables focus adjustment (focus control) to adjust the focusing distance of the lens unit 100.

[0018] The drive / control system includes a zoom actuator 111 , an aperture actuator 112 , a focus actuator 113 , a zoom drive circuit 114 , an aperture drive circuit 115 , a focus drive circuit 116 , a lens MPU 117 , and a lens memory 118 .

[0019] The zoom driving circuit 114 drives the first lens group 101 and the second lens group 103 in the optical axis direction OA using the zoom actuator 111, and controls the angle of view of the optical system of the lens unit 100 (performs a zoom operation). The iris driving circuit 115 drives the iris 102 using the iris actuator 112, and controls the opening diameter and opening / closing operation of the iris 102. The focus driving circuit 116 drives the focus lens 104 in the optical axis direction OA using the focus actuator 113, and controls the focal length of the optical system of the lens unit 100 (performs focus control). The focus driving circuit 116 also functions as a position detection unit that detects the current position (lens position) of the focus lens 104 using the focus actuator 113.

[0020] The lens MPU 117 (processor) performs all calculations and controls related to the lens unit 100, and controls the zoom drive circuit 114, the aperture drive circuit 115, and the focus drive circuit 116. The lens MPU 117 is also connected to the camera MPU 125 via the mount M to exchange commands and data.

[0021] For example, the lens MPU 117 detects the position of the focus lens 104 and notifies the camera MPU 125 of the lens position information in response to a request from the camera MPU 125. This lens position information includes information such as the position of the focus lens 104 in the optical axis direction OA, the position and diameter of the exit pupil in the optical axis direction OA when the optical system is not moving, and the position and diameter of the lens frame that limits the light beam from the exit pupil in the optical axis direction OA.

[0022] Furthermore, the lens MPU 117 controls the zoom driving circuit 114, the aperture driving circuit 115, and the focus driving circuit 116 in response to requests from the camera MPU 125. The lens memory 118 stores optical information necessary for automatic focus adjustment (AF control). The camera MPU 125 controls the operation of the lens unit 100 by executing programs stored in, for example, an internal nonvolatile memory or the lens memory 118.

[0023] The camera body 120 has an optical low-pass filter 121, an image sensor 122, and a drive / control system. The optical low-pass filter 121 and the image sensor 122 function as an imaging section that photoelectrically converts an object image (optical image) formed via the lens unit 100 and outputs image data. In this embodiment, the image sensor 122 photoelectrically converts an object image formed via the imaging optical system and outputs the image data, as well as an imaging signal and a focus detection signal. In this embodiment, the first lens group 101, the aperture 102, the second lens group 103, the focus lens 104, and the optical low-pass filter 121 constitute an imaging optical system.

[0024] The optical low-pass filter 121 reduces false colors and moiré in captured images. The image sensor 122 is composed of a CMOS image sensor and its peripheral circuits, and is arranged with m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or greater). The image sensor 122 of this embodiment also serves as a focus detection element, has a pupil-splitting function, and has pupil-splitting pixels that are capable of phase-difference detection focus detection (phase-difference AF) using image data (image signals). The image processing circuit 124 generates data for phase-difference AF and image data for display, recording, and subject detection based on the image data output from the image sensor 122.

[0025] The drive / control system includes an image sensor drive circuit 123, an image processing circuit 124, a camera MPU 125, a display 126, a group of operation switches (operation SW) 127, a memory 128, a phase difference AF unit 129 (image plane phase difference focus detection unit, control unit), an AE unit 130 (control unit), a white balance adjustment unit 131 (control unit), and a subject detection unit 132 (detection unit).

[0026] The image sensor drive circuit 123 controls the operation of the image sensor 122, and also A / D converts the image signal (image data) output from the image sensor 122 and transmits it to the camera MPU 125. The image processing circuit 124 performs general image processing performed in digital cameras, such as gamma conversion, color interpolation processing, and compression encoding processing, on the image signal output from the image sensor 122. The image processing circuit 124 also generates a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection.

[0027] In this embodiment, a signal for phase difference AF, a signal for AE, a signal for white balance adjustment, and a signal for subject detection are generated, but for example, the signal for AE, the signal for white balance adjustment, and the signal for subject detection may be generated as a common signal. Also, the combination of common signals is not limited to this.

[0028] Camera MPU 125 (processor, control device) performs all calculations and controls related to camera body 120. That is, camera MPU 125 controls image sensor drive circuit 123, image processing circuit 124, display 126, operation switch group 127, memory 128, phase difference AF unit 129, AE unit 130, white balance adjustment unit 131, and subject detection unit 132.

[0029] The camera MPU 125 is connected to the lens MPU 117 via a signal line of the mount M, and exchanges commands and data with the lens MPU 117. The camera MPU 125 issues requests to the lens MPU 117 to acquire the lens position and to drive the lens by a predetermined drive amount, and also issues requests to acquire optical information specific to the lens unit 100 from the lens MPU 117.

[0030] The camera MPU 125 incorporates a ROM 125a that stores a program for controlling the operation of the camera body 120, a RAM 125b (camera memory) that stores variables, and an EEPROM 125c that stores various parameters. The camera MPU 125 also executes focus detection processing based on the program stored in the ROM 125a. In the focus detection processing, a known correlation calculation process is executed using a pair of image signals obtained by photoelectrically converting optical images formed by light beams that have passed through different pupil regions (pupil partial regions) of the imaging optical system.

[0031] The display 126 is composed of an LCD or the like, and displays information about the shooting mode of the imaging device 10, a preview image before shooting and a confirmation image after shooting, an in-focus state display image during focus detection, etc. The operation switch group 127 is composed of a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, etc. The memory 128 (storage unit) is a removable flash memory that records shot images.

[0032] The phase-difference AF unit 129 performs focus detection processing using a phase-difference detection method based on image signals (signals for phase-difference AF) of focus detection image data obtained from the image sensor 122 and the image processing circuit 124. More specifically, the image processing circuit 124 generates a pair of image data formed by light beams passing through a pair of pupil regions of the imaging optical system as focus detection data, and the phase-difference AF unit 129 detects the amount of focus deviation based on the amount of deviation between the pair of image data. As described above, the phase-difference AF unit 129 of this embodiment performs phase-difference AF (image-surface phase-difference AF) based on the output of the image sensor 122 without using a dedicated AF sensor. In this embodiment, the phase-difference AF unit 129 includes an acquisition unit 129a and a calculation unit 129b. The operation of each of these units will be described later.

[0033] At least a part of phase difference AF unit 129 (a part of acquisition unit 129a or calculation unit 129b) may be provided in camera MPU 125. Details of the operation of phase difference AF unit 129 will be described later. Phase difference AF unit 129 functions as a focus control unit that controls the position of focus lens 104 using the focus detection result.

[0034] The AE unit 130 performs exposure adjustment processing to optimize the shooting conditions by performing photometry based on AE signals obtained from the image sensor 122 and the image processing circuit 124. Specifically, it performs photometry based on the AE signals and calculates the exposure amount at the currently set aperture value, shutter speed, and ISO sensitivity. From the difference between the calculated exposure amount and a predetermined appropriate exposure amount, it calculates the appropriate aperture value, shutter speed, and ISO sensitivity to be set during shooting and sets these as the shooting conditions, thereby performing exposure adjustment processing. The AE unit 130 functions as an exposure adjustment unit that calculates the exposure conditions during shooting using the photometry results and controls the aperture value, shutter speed, and ISO sensitivity of the aperture 102.

[0035] The white balance adjustment unit 131 performs white balance adjustment processing based on a signal for white balance adjustment obtained from the image sensor 122 and the image processing circuit 124. Specifically, the white balance adjustment processing is performed by calculating the white balance of the signal for white balance adjustment and adjusting the weight of colors based on the difference from a predetermined appropriate white balance.

[0036] The subject detection unit 132 performs subject detection processing based on the subject detection signal generated by the image processing circuit 124. The subject detection processing detects the type and state of the subject (detection attribute), and the position and size of the subject (detection area).

[0037] In this way, the imaging device 10 of this embodiment can perform a combination of phase difference AF, photometry (exposure adjustment), white balance adjustment, and subject detection, and can select the position (image height range) for performing phase difference AF, photometry, and white balance adjustment depending on the results of subject detection.

[0038] (Configuration of image sensor 122) FIG. 2 is a schematic diagram of the array of imaging pixels (and focus detection pixels) of the image sensor 122. FIG. 2 shows the pixel (imaging pixel) array of the two-dimensional CMOS sensor (image sensor 122) of this embodiment in an area of ​​4 columns x 4 rows, and the focus detection pixel array in an area of ​​8 columns x 4 rows. In this embodiment, the pixels 200R, 200G, and 200B comprising the 2 columns x 1 row focus detection pixel array are each referred to as a unit pixel. In this embodiment, the 2 columns x 2 rows pixel group 200 shown in FIG. 2 has a pixel 200R having R (red) spectral sensitivity arranged in the upper left, a pixel 200G having G (green) spectral sensitivity arranged in the upper right and lower left, and a pixel 200B having B (blue) spectral sensitivity arranged in the lower right. Furthermore, each pixel is composed of a first focus detection pixel 201 and a second focus detection pixel 202 arranged in a 2 columns x 1 row.

[0039] 2 are arranged on a surface, making it possible to acquire a captured image (focus detection signal). In this embodiment, the pixel period P is 4 μm, the number of pixels N is 5,575 columns horizontally by 3,725 rows vertically = approximately 20.75 million pixels, the column-direction period PAF of the focus detection pixels is 2 μm, and the number of focus detection pixels NAF is 11,150 columns horizontally by 3,725 rows vertically = approximately 41.5 million pixels.

[0040] Figure 3(a) is a plan view of one pixel 200G of the imaging element 122 shown in Figure 2, viewed from the light receiving surface side (+z side) of the imaging element 122, and Figure 3(b) is a cross-sectional view of the aa section of Figure 3(a) viewed from the -y side.

[0041] 3, in the pixel 200G of this embodiment, a microlens 305 for collecting incident light is formed on the light-receiving side of each pixel, and a photoelectric conversion unit 301 and a photoelectric conversion unit 302 are formed that are divided into N-H divisions (two divisions) in the x direction and N-V divisions (one division) in the y direction. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively.

[0042] The photoelectric conversion units 301 and 302 may be pin structure photodiodes with an intrinsic layer sandwiched between a p-type layer and an n-type layer, or may be pn junction photodiodes with the intrinsic layer omitted as necessary. In each pixel, a color filter 306 is formed between the microlens 305 and the photoelectric conversion units 301 and 302. Furthermore, as necessary, the spectral transmittance of the color filter 306 may be changed for each sub-pixel (focus detection pixel), or the color filter 306 may be omitted.

[0043] Light incident on pixel 200G shown in FIG. 3 is collected by microlens 305, dispersed by color filter 306, and then received by photoelectric conversion unit 301 and photoelectric conversion unit 302. In photoelectric conversion unit 301 and photoelectric conversion unit 302, electron-hole pairs are generated according to the amount of received light, and after being separated by a depletion layer, the negatively charged electrons are accumulated in an n-type layer (not shown). Meanwhile, the holes are discharged to the outside of image sensor 122 through a p-type layer connected to a constant voltage source (not shown). The electrons accumulated in the n-type layers (not shown) of photoelectric conversion unit 301 and photoelectric conversion unit 302 are transferred to a capacitance unit (FD) via a transfer gate and converted into a voltage signal.

[0044] Fig. 4 is a schematic diagram illustrating the correspondence between the pixel structure of this embodiment shown in Fig. 3 and pupil division. Fig. 4 shows a cross-sectional view of the aa cross section of the pixel structure of this embodiment shown in Fig. 3(a) as viewed from the +y side, and the pupil plane (pupil distance Ds) of the image sensor 122. In Fig. 4, the x-axis and y-axis of the cross-sectional view are reversed with respect to Fig. 3 in order to correspond to the coordinate axes of the pupil plane of the image sensor 122.

[0045] 4, the first partial pupil region 501 of the first focus detection pixel 201 is in a generally conjugate relationship with the light receiving surface of the photoelectric conversion unit 301, whose center of gravity is decentered in the -x direction, due to the microlens, and represents the pupil region that can receive light by the first focus detection pixel 201. The center of gravity of the first partial pupil region 501 of the first focus detection pixel 201 is decentered on the +X side on the pupil plane.

[0046] 4, the second partial pupil region 502 of the second focus detection pixel 202 is in a generally conjugate relationship with the light receiving surface of the photoelectric conversion unit 302, whose center of gravity is decentered in the +x direction, due to the microlens, and represents the pupil region that can receive light at the second focus detection pixel 202. The second partial pupil region 502 of the second focus detection pixel 202 has its center of gravity decentered on the -X side on the pupil plane. Also, in FIG. 4, pupil region 500 is the pupil region that can receive light by the entire pixel 200G when the photoelectric conversion unit 301 and the photoelectric conversion unit 302 (first focus detection pixel 201 and second focus detection pixel 202) are all combined.

[0047] Image plane phase-difference AF uses microlenses on the image sensor to divide the pupil, which is affected by diffraction. In Figure 4, the pupil distance to the pupil plane of the image sensor is several tens of mm, while the diameter of the microlenses is several microns. As a result, the aperture value of the microlenses is several tens of thousands, causing diffraction blurring on the order of several tens of mm. As a result, the image on the light-receiving surface of the photoelectric conversion unit does not show a clear pupil region or partial pupil region, but rather shows the light-receiving sensitivity characteristics (incident angle distribution of light-receiving rate).

[0048] 5 is a schematic diagram showing the correspondence between the image sensor 122 and pupil division. The image sensor 122 is disposed on an imaging plane 600. Light beams that pass through different pupil partial regions, the first pupil partial region 501 and the second pupil partial region 502, are incident on each pixel of the image sensor 122 at different angles and are received by the first focus detection pixel 201 and the second focus detection pixel 202, which are divided into 2×1 regions. In this embodiment, the pupil region is divided into two in the horizontal direction. If necessary, the pupil may also be divided vertically.

[0049] The image sensor 122 of this embodiment has an array of imaging pixels, each having a first focus detection pixel 201 and a second focus detection pixel 202. The first focus detection pixel 201 receives a light beam that passes through a first pupil partial region 501 of the imaging optical system. The second focus detection pixel 202 receives a light beam that passes through a second pupil partial region 502 of the imaging optical system that is different from the first pupil partial region 501. The imaging pixel receives a light beam that passes through a pupil region that is the combined first pupil partial region 501 and second pupil partial region 502 of the imaging optical system.

[0050] In the image sensor 122 of this embodiment, each imaging pixel is made up of a first focus detection pixel 201 and a second focus detection pixel 202. If necessary, the imaging pixel, the first focus detection pixel 201, and the second focus detection pixel 202 may be configured as separate pixels, and the first focus detection pixel 201 and the second focus detection pixel 202 may be partially arranged in a portion of the imaging pixel array.

[0051] In this embodiment, focus detection is performed by collecting light reception signals from the first focus detection pixels 201 of each pixel of the image sensor 122 to generate a first focus signal, and collecting light reception signals from the second focus detection pixels 202 of each pixel to generate a second focus signal. Furthermore, an image pickup signal (captured image) with a resolution of N effective pixels is generated by adding the signals from the first focus detection pixels 201 and second focus detection pixels 202 for each pixel of the image sensor 122. The method of generating each signal is not limited to the methods described above, and for example, the second focus detection signal may be generated from the difference between the image pickup signal and the first focus signal.

[0052] (Relationship between defocus amount and image shift amount) The relationship between the defocus amount and the image shift amount based on the first focus detection signal and the second focus detection signal acquired by the image sensor 122 of this embodiment will be described below.

[0053] 6 is a schematic diagram illustrating the relationship between the defocus amount based on the first focus detection signal and the second focus detection signal and the image shift amount between the first focus detection signal and the second focus detection signal. The image sensor 122 is disposed on an imaging plane 600. As in FIGS. 4 and 5, the pupil plane of the image sensor 122 is divided into a first pupil partial region 501 and a second pupil partial region 502.

[0054] The defocus amount d is defined as the distance from the subject's imaging position to the imaging plane, with magnitude |d|, and a front-focus state in which the subject's imaging position is closer to the subject than the imaging plane is defined as a negative sign (d<0). A back-focus state in which the subject's imaging position is on the opposite side of the subject than the imaging plane is defined as a positive sign (d>0). In a focused state in which the subject's imaging position is on the imaging plane (focus position), d=0. In FIG. 6, subject 601 shows an example of a focused state (d=0), and subject 602 shows an example of a front-focus state (d<0). The front-focus state (d<0) and the back-focus state (d>0) are combined to form a defocused state (|d|>0).

[0055] In a front-focus state (d<0), a light beam from the subject 602 that passes through the first pupil partial region 501 (second pupil partial region 502) is first focused and then spreads to a width Γ1 (Γ2) around the center of gravity G1 (G2) of the light beam, forming a blurred image on the imaging surface 600. The blurred image is received by the first focus detection pixels 201 (second focus detection pixels 202) that constitute the pixels arranged on the image sensor 122, and a first focus detection signal (second focus detection signal) is generated. Therefore, the first focus detection signal (second focus detection signal) is recorded as a subject image in which the subject 602 is blurred to a width Γ1 (Γ2) at the center of gravity G1 (G2) on the imaging surface 600. The blur width Γ1 (Γ2) of the subject image increases roughly proportionally as the magnitude of the defocus amount d, |d|, increases. Similarly, the magnitude |p| of the image shift amount p (= the difference G1-G2 in the center of gravity positions of the light beams) of the subject image between the first focus detection signal and the second focus detection signal also increases roughly proportionally as the magnitude |d| of the defocus amount d increases. The same is true in the back-focus state (d>0), although the direction of the image shift of the subject image between the first focus detection signal and the second focus detection signal is opposite to that in the front-focus state.

[0056] As the magnitude of the defocus amount of the first focus detection signal and the second focus detection signal, or the image capture signal obtained by adding the first focus detection signal and the second focus detection signal, increases, the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal also increases. Therefore, in this embodiment, the phase difference AF section 129 utilizes the relationship in which the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases as the magnitude of the defocus amount of the image capture signal increases, and converts the image shift amount into a detected defocus amount using a conversion coefficient calculated based on the base length.

[0057] (AF operation flow) 7 is a flowchart of the AF operation. This AF operation is performed for each frame by the camera MPU 125 executing a control program stored in the ROM 125a. For this AF operation, the camera MPU 125 cooperates with the lens MPU 117 as necessary.

[0058] 8 is a diagram showing the arrangement of multiple focus detection frames within the imaging screen in a portrait scene. It shows a subject detection frame 800 displayed within the imaging screen of display 126, and the arrangement of multiple focus detection frames 801 used to calculate the defocus amount at that time. Subject detection frame 800 is displayed on display 126 based on the subject position and size detected by subject detection unit 132. The arrangement of focus detection frames 801 is also set based on the subject position and size.

[0059] 7, camera MPU 125 sets a part of the angle of view as a defocus amount calculation area based on the size of subject detection frame 800 or the detected subject area. Note that the defocus amount calculation area does not have to be set based on the above-described subject detection area. For example, the entire angle of view may be set as the defocus amount calculation area.

[0060] In S702, the camera MPU 125 calculates contrast information using the focus detection signal for each of the multiple focus detection frames in the defocus amount calculation area set in S701. Here, the amplitude (maximum value - minimum value) of the focus detection signal is calculated as the contrast information. Note that the contrast information is not limited to the signal amplitude described above, and may be set using other parameters that can evaluate contrast, such as the maximum value, minimum value, or absolute sum of the differences between each pixel signal of the focus detection signal.

[0061] In the next step S703, the defocus amount and reliability information are calculated for each of the multiple focus detection frames. Here, reliability is a numerical value that is graded based on contrast information, saturation information, defect information, etc., of the focus detection signal, and the higher this numerical value, the higher the reliability of the defocus amount. These reliability determination calculations are well-known techniques and are not a main part of this embodiment, so detailed explanations will be omitted.

[0062] In S704, the camera MPU 125 calculates a histogram based on the defocus amount and reliability of each focus detection frame calculated in S703. The histogram is created using the frequency distribution of the defocus amount of focus detection frames whose reliability is equal to or greater than a predetermined threshold.

[0063] In S705, the camera MPU 125 stores in the memory 128 the histogram created up to S704.

[0064] FIG. 9 shows an example of a histogram calculated in S704 based on the defocus amount of each focus detection frame. The horizontal axis in FIG. 9 represents the defocus amount, with the left side representing infinity and the right side representing close focus. The vertical axis represents the frequency of defocus amounts belonging to each class. The camera MPU 125 sets the bin interval (width of each class) and range of the histogram using a predetermined bin interval and range. Note that the method for setting the bin interval and range of the histogram is not limited to this. For example, they may be set based on at least one of the subject distance (distance in the depth direction of the subject), the subject size, and the aperture value at the time of shooting. The aperture value at the time of shooting can be used to calculate the subject depth size on the image plane by value conversion. By setting the bin interval according to the subject depth size based on the aperture value, the resolution when representing the subject on the histogram can be appropriately set regardless of conditions.

[0065] In S706, the camera MPU 125 performs histogram frame addition processing, which is a feature of this embodiment. The histogram frame addition processing will be described later. This processing makes it possible to calculate a stable histogram even in an environment where the S / N ratio of the focus detection signal is poor and focus detection is difficult.

[0066] In S707, the camera MPU 125 calculates the defocus amount for driving the lens using the histogram created in S706. The process of calculating the defocus amount will also be described later.

[0067] In S708, the camera MPU 125 calculates the lens drive amount based on the defocus amount calculated up to S707, and drives the focus lens 104. With the above processing, the AF operation is completed.

[0068] Figure 10 is a diagram conceptually illustrating the addition of histograms between frames. In Figure 10, the horizontal axis represents time, and image signal readout is repeated at a predetermined cycle Ts. The frame corresponding to (n-5)Ts is the first frame after shooting begins, and frames to the right are more recent, with nTs being the current frame.

[0069] In Figure 10, the vertical axis, from top to bottom, shows the representative time for each frame in the first row, the histogram acquired for each frame in the second row, the period for frame addition in the third row, and the histogram after frame addition in the fourth row. In the frame addition shown in the third row, frame addition is not performed when there is no histogram to go back and add, as in the first frame, or when the histogram for one frame has a sufficient number of focus detection frame counts. In other cases, the number of frames to be added is determined so that the number of focus detection frame counts within a predetermined defocus range is equal to or greater than a predetermined number. As shown in the second row, the histogram acquired for each frame may have a small number of counts for each bin, making it difficult to identify the peak bin. Even in such an environment, adding histograms obtained from multiple consecutive frames makes it possible to calculate a histogram with a stable number of focus detection frame counts, as shown in the fourth row.

[0070] FIG. 11 is a flowchart illustrating the operation of the histogram frame addition process S706. This embodiment proposes a method for obtaining a highly reliable and stable histogram in environments where focus detection is difficult due to a poor S / N ratio of the focus detection signal, such as low-brightness environments. In environments where focus detection is difficult, even when focus detection is performed using multiple focus detection frames, the number of reliable focus detection frames is limited, making it difficult to identify the peak bin in a histogram that can be created from a single frame. Therefore, this embodiment adds histograms obtained within a period that can be considered to represent the same scene between frames to create a histogram that is less susceptible to unusual detected defocus amounts and allows for stable peak determination.

[0071] In S801, the camera MPU 125 acquires the latest histogram of the current frame from among the histograms stored in the memory 128 in chronological order.

[0072] In S802, the camera MPU 125 predicts the defocus amount of the current frame based on the defocus amount of the subject in the previous frame (or in multiple past frames), thereby obtaining a predicted defocus amount (predicted value).

[0073] In S803, the camera MPU 125 calculates a main class and a sample count range to monitor the count number of the focus detection frame for the histogram. The histogram summation between frames controls the number of frames to be added based on the count number of the focus detection frame within the sample count range in the summed histogram.

[0074] The main class and sample count range will be described using FIG. 9 mentioned above. In FIG. 9, the data section including the predicted defocus amount acquired in S802 is set as the main class (the shaded area in FIG. 9). In addition, a predetermined defocus range centered on the main class is set as the sample count range, as a range for monitoring the count number of the focus detection frame after addition. The camera MPU 125 sets the sample count range using a predetermined defocus range. Note that the method for setting the sample count range is not limited to this, and it may be set based on at least one of the subject distance (distance in the depth direction of the subject), the size of the subject, and the aperture value at the time of shooting.

[0075] In S804, the camera MPU 125 tally up the count numbers of the focus detection frames included in the sample count range.

[0076] In S805, the camera MPU 125 determines whether the count number of the focus detection frame included in the sample count range is equal to or greater than a predetermined threshold. If the camera MPU 125 determines that the count number of the focus detection frame is less than the predetermined threshold, the process proceeds to S806. If the camera MPU 125 determines that the count number is equal to or greater than the threshold, the process proceeds to S809.

[0077] In S806, the camera MPU 125 determines whether the number of added frames in the histogram is equal to or greater than a predetermined upper limit of the number of added frames. If the number of added frames is less than the upper limit, the camera MPU 125 proceeds to S807. If the number of added frames is equal to or greater than the upper limit, the camera MPU 125 proceeds to S809.

[0078] In S807, the camera MPU 125 acquires the histogram of the previous frame, and in S808 adds the addition result of the histogram of the current frame and the histogram of the previous frame for each bin to create a new histogram. By performing the flow of steps S804 to S808 above, going back frame by frame, it is possible to obtain a histogram that ensures a sufficient count number of focus detection frames.

[0079] Note that while histogram addition control is based on the count number of focus detection frames included within a predetermined range of the histogram, control of the number of frame additions is not limited to this. For example, control may be based on whether the maximum count number of the histogram exceeds a predetermined threshold. Furthermore, the threshold value for the count number of focus detection frames and the upper limit for the number of frame additions described above may be set as a single threshold value in advance, or the threshold value may be changed based on the shooting conditions and the output of the focus detection pixels.

[0080] Specifically, when shooting with a small aperture, shooting a low-brightness scene, or shooting a low-contrast subject, which may result in greater variation in focus detection results, the threshold for the count number of the focus detection frame is adjusted to be relatively small, and the upper limit for the number of frames added is adjusted to be large. This enables control according to the camera settings and the shooting scene. Note that scenes shot with a small aperture or a low-brightness scene can be determined from the exposure conditions during shooting, and whether or not the subject is low-contrast can be determined from the signal level of the focus detection pixel signal.

[0081] In S805, if the count number of the focus detection frame is equal to or greater than a predetermined threshold, or in S806, if the frame addition number is equal to or greater than the upper limit, the camera MPU 125 proceeds to S809. In S809, the camera MPU 125 stores the histogram frame addition result in the memory 128 and ends the histogram frame addition process.

[0082] FIG. 12 is a flowchart showing the operation of calculating the defocus amount from the histogram.

[0083] In S901 and S902, the camera MPU 125 acquires from the memory 128 the histogram and the predicted defocus amount calculated in S706.

[0084] In S903, the camera MPU 125 performs matching processing on the histogram of the previous frame. In S903, the camera MPU 125 performs matching processing on the shapes of the histograms calculated for the current frame and the frame before. A known correlation calculation process is used for the matching processing. The correlation value is calculated by shifting the histogram shape of the current frame and the histogram shape of the frame before, and it is determined that the smaller the shift amount that results in the correlation value, the higher the degree of matching.

[0085] In S904, the camera MPU 125 resets the dominant class using the histogram after frame addition. The class corresponding to the defocus amount closest to the predicted defocus amount obtained in S902 is set as the predicted class. The class shifted by the shift amount calculated from the dominant class of the previous frame based on the result of S903 is set as the matching class. In S904, if the predicted class and the matching class are within a predetermined range of each other, the matching class is reset as the dominant class of the current frame. On the other hand, if the predicted class and the matching class are farther apart from each other than the predetermined range, the predicted class is reset as the dominant class of the current frame.

[0086] In S905, the camera MPU 125 selects a focus detection frame within the subject area detected by the subject detection unit 132 from the focus detection frames belonging to the main class identified in S904, and calculates the average value of the detected defocus amounts of the selected focus detection frames. This completes the defocus calculation process.

[0087] Note that, here, the average defocus amount is calculated by selecting a focus detection frame within the subject area detected from the main class selected in S904, but this embodiment is not limited to this configuration. For example, a configuration may be adopted in which a representative point is selected from the main class identified in S904. Alternatively, a configuration may be adopted in which the average defocus amount or the closest defocus amount is used from all focus detection frames included in the main class identified in S904.

[0088] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to these specific embodiments, and various forms within the scope of the invention that do not deviate from the gist of the invention are also included in the present invention.

[0089] For example, in the above-described embodiment, step S808, which performs histogram addition, employs a method of simply adding the histogram addition result and the histogram addition result from one frame back. Here, if focus tracking is always possible in response to subject movement, the center position of the histogram corresponds to the subject position, and so histogram alignment is not necessary. However, if the focus position during focus detection shifts from frame to frame, adding the histograms as they are may result in an error in the peak position in the histogram addition result due to the addition of histograms with different peak positions.

[0090] Therefore, in the histogram addition in S808, the histogram from one frame back to be added is added in accordance with the position of the histogram addition result, making it possible to add accurate histograms even when histograms are calculated with different focus conditions for each frame.

[0091] Specifically, before performing the histogram addition in S808, the matching process between histograms performed in S903 is performed. This calculates the amount of deviation (amount of movement) between the histograms, and adds the histogram from one frame back to be added together with the position of the histogram addition result, enabling accurate histogram addition.

[0092] The disclosure of this specification includes the following imaging apparatus, its control method, program, and storage medium.

[0093] (Item 1) an imaging element in which a plurality of unit pixels are arranged, each unit pixel having one microlens and a plurality of photoelectric conversion units corresponding to the one microlens; a readout means for reading out a signal from the photoelectric conversion unit from the imaging element; a first calculation means for calculating defocus amounts in a plurality of focus detection frames within an imaging screen based on the signal; a generation means for generating a histogram indicating a frequency distribution of the defocus amount for each of the plurality of focus detection frames; a storage means for storing the histogram generated at a predetermined interval; an adding means for adding a plurality of histograms arranged in time series among the histograms stored in the storage means; a second calculation means for calculating a defocus amount of the object based on the result of the addition means; An imaging device comprising:

[0094] (Item 2) 2. The imaging device according to item 1, wherein the first calculation means further calculates the reliability of the defocus amount from information on the contrast of the signal.

[0095] (Item 3) 3. The imaging device according to item 2, wherein the generation means generates the histogram using the defocus amount of a focus detection frame whose reliability is higher than a predetermined value.

[0096] (Item 4) The imaging device according to any one of items 1 to 3, characterized in that the first calculation means further calculates a predicted value of the current defocus amount of the subject based on information on past defocus amounts of the subject.

[0097] (Item 5) 5. The imaging device according to item 4, wherein the adding means sets an interval in the histogram that includes the predicted value of the defocus amount as a main class, and counts the frequency distribution of the defocus amount for a predetermined range of defocus amounts centered on the main class.

[0098] (Item 6) Item 6. The imaging device according to item 5, wherein the adding means controls the number of histograms to be added so that the count number in a range of a predetermined defocus amount centered on the main class of the histogram exceeds a threshold value.

[0099] (Item 7) 7. The imaging device according to item 6, wherein the threshold value is changed according to the imaging conditions and the output of the signal.

[0100] (Item 8) 8. The imaging device according to any one of items 1 to 7, wherein the adding means calculates the amount of movement of the shapes of the multiple histograms stored in the storage means at the predetermined period, and adds up the histograms after moving them according to the amount of movement.

[0101] (Item 9) 9. The imaging device according to any one of items 1 to 8, wherein the adding means adds a histogram going back to the current histogram for each predetermined period.

[0102] (Item 10) 10. The imaging device according to any one of items 1 to 9, wherein the plurality of photoelectric conversion units receive a light beam that passes through a part of an exit pupil of an imaging optical system.

[0103] (Item 11) A method for controlling an imaging device including an imaging element in which a plurality of unit pixels are arranged, each unit pixel having one microlens and a plurality of photoelectric conversion units corresponding to the one microlens, comprising: a reading step of reading out a signal from the photoelectric conversion unit from the imaging element; a first calculation step of calculating defocus amounts for a plurality of focus detection frames within an imaging screen based on the signal; a generation step of generating a histogram indicating a frequency distribution of the defocus amount for each of the plurality of focus detection frames; a storage step of storing the histogram generated at a predetermined cycle; an adding step of adding together a plurality of histograms arranged in time series among the histograms stored in the storing step; a second calculation step of calculating a defocus amount of the object based on the result of the addition step; 10. A method for controlling an imaging device, comprising:

[0104] (Item 12) Item 12. A program for causing a computer to execute each step of the control method described in Item 11.

[0105] (Item 13) A computer-readable storage medium storing a program for causing a computer to execute each step of the control method described in item 11.

[0106] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions.

[0107] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0108] 10: imaging device, 100: lens unit, 101: first lens group, 102: aperture, 103: second lens group, 104: focus lens, 117: lens MPU, 118: lens memory, 120: camera body, 122: imaging element, 125: camera MPU, 128: memory, 129: phase difference AF unit, 132: subject detection unit

Claims

1. an imaging element in which a plurality of unit pixels are arranged, each unit pixel having one microlens and a plurality of photoelectric conversion units corresponding to the one microlens; a readout means for reading out a signal from the photoelectric conversion unit from the imaging element; a first calculation means for calculating defocus amounts in a plurality of focus detection frames within an imaging screen based on the signal; a generation means for generating a histogram indicating a frequency distribution of the defocus amount for each of the plurality of focus detection frames; a storage means for storing the histogram generated at a predetermined interval; an adding means for adding a plurality of histograms arranged in time series among the histograms stored in the storage means; a second calculation means for calculating a defocus amount of the object based on the result of the addition means; An imaging device comprising:

2. 2. The imaging apparatus according to claim 1, wherein the first calculation means further calculates the reliability of the defocus amount from information on the contrast of the signal.

3. 3. The image pickup apparatus according to claim 2, wherein the generating means generates the histogram using defocus amounts of focus detection frames having higher reliability than a predetermined level.

4. 2. The imaging apparatus according to claim 1, wherein the first calculation means further calculates a predicted value of the current defocus amount of the subject based on information about past defocus amounts of the subject.

5. 5. The imaging device according to claim 4, wherein the adding means sets an interval in the histogram that includes the predicted value of the defocus amount as a main class, and counts the frequency distribution of the defocus amount within a predetermined range of defocus amounts centered on the main class.

6. 6. The imaging device according to claim 5, wherein the adding means controls the number of histograms to be added so that the number of counts in a range of a predetermined defocus amount centered on the main class of the histogram exceeds a threshold value.

7. 7. The imaging device according to claim 6, wherein the threshold value is changed in accordance with imaging conditions and the output of the signal.

8. 2. The imaging device according to claim 1, wherein the adding means calculates the amount of movement of the shapes of the plurality of histograms stored in the storage means at the predetermined period, and adds the histograms after moving them according to the amount of movement.

9. 2. The imaging device according to claim 1, wherein the adding means adds a histogram going back to the current histogram for each predetermined period.

10. 2. The imaging device according to claim 1, wherein the plurality of photoelectric conversion units receive light beams that pass through a part of an exit pupil of an imaging optical system.

11. A method for controlling an imaging device including an imaging element in which a plurality of unit pixels are arranged, each unit pixel having one microlens and a plurality of photoelectric conversion units corresponding to the one microlens, comprising: a reading step of reading out a signal from the photoelectric conversion unit from the imaging element; a first calculation step of calculating defocus amounts for a plurality of focus detection frames within an imaging screen based on the signal; a generation step of generating a histogram indicating a frequency distribution of the defocus amount for each of the plurality of focus detection frames; a storage step of storing the histogram generated at a predetermined cycle; an adding step of adding together a plurality of histograms arranged in time series among the histograms stored in the storing step; a second calculation step of calculating a defocus amount of the object based on the result of the addition step; 10. A method for controlling an imaging device, comprising:

12. A program for causing a computer to execute each step of the control method according to claim 11.

13. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 11.

Images