Focus detection device, focus detection method, and imaging apparatus

Abstract

To provide a focus detection device and a focus detection method that achieve highly accurate focus detection based on signals for focus detection obtained from an image pickup device.SOLUTION: A focus detection device has: generation means that generates a first signal for focus detection and a second signal for focus detection based on signals obtained from an image pickup device; image deviation amount calculation means that calculates the image deviation amount between the first signal for focus detection and the second signal for focus detection; coefficient calculation means that calculates a conversion coefficient for converting the image deviation amount into a defocus amount of an imaging optical system; and detection means that applies the conversion coefficient to the image deviation amount to detect the defocus amount. The coefficient calculation means calculates the conversion coefficient based on at least one of a pupil eccentricity amount and an incident pupil distance according to an image height at a focus detection position.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to a focus detection device, a focus detection method, and an imaging device, and more particularly to a focus detection device, a focus detection method, and an imaging device that perform focus detection using an imaging element. [Background technology]

[0002] An image sensor that enables image-plane phase-difference AF is known (Patent Document 1). In Patent Document 1, some of the pixels arranged in a matrix are designated as dedicated pixels (focus detection pixels) configured to output signals for phase-difference AF, and focus detection of the imaging lens is performed based on the phase difference between signal pairs obtained from the focus detection pixels. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-156823 Summary of the Invention [Problem to be solved by the invention]

[0004] The accuracy of image-plane phase-difference AF is affected by the quality of the focus detection signal, which in turn is affected by manufacturing errors in the image sensor.

[0005] In one aspect, the present invention provides a focus detection device and a focus detection method that realizes highly accurate focus detection based on a focus detection signal obtained from an image sensor, taking into account manufacturing errors of the image sensor. [Means for solving the problem]

[0006] The above-mentioned object can be achieved by a focus detection device comprising: a generation means for generating a first focus detection signal and a second focus detection signal based on signals obtained from an image sensor; an image shift amount calculation means for calculating an image shift amount between the first focus detection signal and the second focus detection signal; a coefficient calculation means for calculating a conversion coefficient for converting the image shift amount into a defocus amount of the imaging optical system; and a detection means for detecting the defocus amount by applying the conversion coefficient to the image shift amount, wherein the coefficient calculation means calculates the conversion coefficient based on at least one of an amount of pupil eccentricity corresponding to the image height at the focus detection position and an entrance pupil distance. [Effects of the Invention]

[0007] According to the present invention, there are provided a focus detection device and a focus detection method that realize highly accurate focus detection based on a focus detection signal obtained from an image sensor, taking into account manufacturing errors of the image sensor. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the functional arrangement of an image capture apparatus as a focus detection apparatus according to a first embodiment. [Figure 2] FIG. 1 is a diagram schematically illustrating a pixel array in a first embodiment. [Figure 3] 1A and 1B are a schematic plan view and a schematic cross-sectional view of a pixel according to a first embodiment; [Figure 4] FIG. 10 is a diagram for explaining the correspondence between pixels of the image sensor and pupil intensity distribution in the first embodiment; [Figure 5] FIG. 10 is a diagram for explaining the correspondence between the image sensor and the pupil intensity distribution of the first embodiment. [Figure 6] FIG. 1 is a diagram for explaining pupil division in the imaging optical system and the image sensor 107 according to the first embodiment; [Figure 7] FIG. 10 is a diagram showing an example of a light intensity distribution inside a pixel in the first embodiment; [Figure 8] FIG. 1 is a diagram showing partial pupil regions according to a first embodiment; [Figure 9] FIG. 10 is a diagram showing the relationship between the defocus amount and the image shift amount in the first embodiment. [Figure 10]FIG. 10 is a diagram for explaining the correspondence between the image sensor and the pupil intensity distribution when there is a change in the amount of pupil decentration or the entrance pupil distance in the first embodiment; [Figure 11] FIG. 10 is a diagram for explaining the image height position at which the optical axis deviation is measured when correcting the image height-dependent pupil decentration amount and entrance pupil distance change in the first embodiment. [Figure 12] FIG. 1 is a diagram showing the relationship between the sensor entrance pupil, pupil decentration amount, entrance pupil distance, lens frame vignetting of the imaging optical system, and pupil partial area of ​​the image sensor of the first embodiment. [Figure 13] FIG. 1 is a diagram showing the relationship between the sensor entrance pupil, pupil decentration amount, entrance pupil distance, lens frame vignetting of the imaging optical system, and pupil partial area of ​​the image sensor of the first embodiment. [Figure 14] Flowchart showing a focus detection method according to the first embodiment [Figure 15] Relationship diagram of lens frame vignetting in the imaging optical system of the first embodiment [Figure 16] 1 is a flowchart showing a conversion coefficient calculation method according to a first embodiment; [Figure 17] FIG. 10 is a diagram showing a method for generating a virtual focus detection signal according to the first embodiment. [Figure 18] FIG. 10 is a diagram showing an example of change in the virtual pupil intensity distribution in the first embodiment; [Figure 19] 10A and 10B are diagrams showing examples of a reference pupil intensity distribution and an individual intensity distribution according to the second embodiment; [Figure 20] FIG. 10 is a diagram for explaining a method for calculating correction values ​​of conversion coefficients according to a second embodiment; [Figure 21] FIG. 10 is a diagram showing a pixel array according to a third embodiment; [Figure 22] FIG. 10 is a diagram showing a pixel structure of a third embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below based on exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Furthermore, although multiple features are described in the embodiments, not all of them are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] In the following embodiments, the present invention will be described with reference to an imaging device such as a digital camera. However, the present invention can also be implemented in any electronic device capable of processing signals from an imaging element. Such electronic devices include computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, and drive recorders. These are merely examples, and the present invention can also be implemented in other electronic devices.

[0011] ●(First embodiment) [Overall configuration] 1 is a block diagram showing an example of the functional configuration of an image capture device 100 having a focus detection device according to an embodiment. The image capture device 100 is a digital camera system equipped with a camera body and an interchangeable lens (imaging optical system or imaging optical system) that is detachable from the camera body. However, the focus detection device according to an embodiment can also be applied to an image capture device in which the camera body and imaging optical system are integrated.

[0012] The imaging optical system has a first lens group 101, an aperture 102, a second lens group 103, and a third lens group 104, and forms an optical image of a subject on the imaging surface of an image sensor 107. The first lens group 101 is arranged at the front (subject side) of the multiple lens groups that make up the imaging optical system, and can move forward and backward along an optical axis OA. The aperture 102 has an adjustable opening size, and functions as a mechanical shutter when capturing a still image.

[0013] The second lens group 103 moves forward and backward along the optical axis OA together with the aperture 102, and changes the focal length (angle of view) of the imaging optical system in conjunction with the forward and backward movement of the first lens group 101. The third lens group 105 is a focus lens that moves forward and backward along the optical axis OA to change the focal length of the imaging optical system.

[0014] The optical low-pass filter 106 is provided to reduce false colors and moire that occur in captured images. The image sensor 107 is, for example, a CMOS image sensor or a CCD image sensor having a pixel array (also called a pixel area) in which m pixels are arranged horizontally and n pixels are arranged vertically in a two-dimensional manner. Each pixel is provided with a color filter in a primary color Bayer array and an on-chip microlens. The image sensor 107 may also be a three-chip color image sensor.

[0015] The zoom actuator 111 moves the first lens group 101 and the second lens group 103 in the optical axis direction, for example, by rotating a cam barrel (not shown). The aperture actuator 112 drives the aperture 102. The focus actuator 114 moves the third lens group 105 in the optical axis direction. Note that a mechanical shutter separate from the aperture 102 and a shutter actuator that drives the mechanical shutter may also be provided.

[0016] The flash 115 is a light source that illuminates the subject. The flash 115 is equipped with a flashlight device using a xenon tube or a continuous-light-emitting LED (light-emitting diode). The AF (autofocus) auxiliary light source 116 projects a predetermined pattern image through a projection lens. This improves focus detection capability for low-brightness or low-contrast subjects.

[0017] The CPU 121 controls the overall operation of the imaging device 100. The CPU 121 includes a calculation unit, a ROM, a RAM, an A / D converter, a D / A converter, and a communication interface circuit. The CPU 121 loads a program stored in a ROM 135 into a RAM 136 and executes it to control each unit of the imaging device 100 and realize functions of the imaging device 100, such as autofocus detection (AF), imaging, image processing, and recording. Some of the functions realized by the CPU 121 executing the program may be implemented by a hardware circuit separate from the CPU 121. Some of the circuits may also be reconfigurable circuits such as FPGAs. For example, to reduce the time required for focus detection calculations (described later), some of the calculations may be executed by a dedicated hardware circuit.

[0018] The pixel signal acquisition unit 121a, signal generation unit 121b, focus detection unit 121c, and lens information acquisition unit 121d are functional blocks that represent functions realized by the CPU 121 executing a program. The communication interface circuit included in the CPU 121 supports one or more standards for wired and wireless communication. The imaging device 100 can communicate with external devices directly or via other devices through the communication interface circuit.

[0019] A flash control circuit 122 controls the lighting of the flash 115 in synchronization with the imaging operation. An assist light source drive circuit 123 controls the lighting of the AF assist light source 116 in synchronization with the focus detection process. An image sensor drive circuit 124 controls the imaging operation of the image sensor 107, and also A / D converts the signal obtained by the imaging operation and sends it to the CPU 121. An image processing circuit 125 can apply various image processing to image data, such as gamma conversion, color interpolation, encoding, decoding, evaluation value generation, and feature region detection.

[0020] The focus driving circuit 126 drives the focus actuator 114 based on the focus detection result of the focus detection unit 121c and moves the third lens group 105 along the optical axis OA, thereby adjusting the focal length of the imaging optical system.

[0021] The aperture drive circuit 128 drives the aperture actuator 112 to control the aperture diameter and opening / closing of the aperture 102. The zoom drive circuit 129 drives the zoom actuator 111 in response to, for example, a user instruction, and changes the focal length (angle of view) of the imaging optical system by moving the first lens group 101 and the second lens group 103 along the optical axis OA.

[0022] The lens communication circuit 130 communicates with the interchangeable lens attached to the camera body under the control of the CPU 121. The lens communication circuit 130 acquires lens information from the interchangeable lens, for example, and supplies the information to a lens information acquisition unit 121d of the CPU 121. The lens communication circuit 130 can also transmit information and commands from the camera body to the interchangeable lens.

[0023] The interchangeable lens and the camera body have mounts that are mechanically detachable. The mounts of the interchangeable lens and the camera body are provided with multiple contacts that are configured to come into contact when the interchangeable lens is attached. The interchangeable lens and the camera body are electrically connected through the contacts, allowing communication and power supply.

[0024] The interchangeable lens is provided with a board equipped with a lens CPU, lens memory, lens gyro, etc. (not shown). The lens CPU uses correction values ​​and the like stored in the lens memory to execute various programs. The lens memory also stores aberration information, which is the optical characteristics of the lens, and the exit pupil distance LPO, etc. The CPU 121 (lens information acquisition unit 121d) can acquire lens information such as the current focus state FS, zoom state ZS, and exit pupil distance LPO of the interchangeable lens via the lens communication circuit 130.

[0025] The display unit 131 has, for example, an LCD (liquid crystal display device). The display unit 131 displays information about the imaging mode of the imaging device 100, a preview image before imaging, a confirmation image after imaging, an in-focus state display image during focus detection, and the like. The operation unit 132 is configured with a power switch, a release switch, a zoom operation switch, an imaging mode selection switch, and the like. The release switch has two switches: SW1 that is turned on when pressed halfway, and SW2 that is turned on when pressed fully. The recording medium 133 is, for example, a semiconductor memory card that is detachable from the imaging device 100, and still image data and video data obtained by imaging are recorded on the recording medium 133.

[0026] Note that if the display unit 131 is a touch display, a touch panel or a combination of a touch panel and a GUI displayed on the display unit 131 may be used as the operation unit 132. For example, when a tap operation on the touch panel is detected during live view display, the image area corresponding to the tap position can be configured to perform focus detection as a focus detection area.

[0027] It is also possible to calculate contrast information of the captured image data using the image processing circuit 125, and have the CPU 121 perform contrast AF. In contrast AF, the focus lens group 105 is moved to change the focal length of the imaging optical system, and contrast information is calculated sequentially, and the focus lens position at which the contrast information reaches its peak is set as the focal position.

[0028] In this way, the imaging device 100 can perform both imaging surface phase difference AF and contrast AF, and can selectively use one or a combination of both depending on the situation.

[0029] [Image sensor] The pixel array and pixel structure of the image sensor 107 will be described with reference to Figures 2 and 3. The left-right direction in Figure 2 is the x-direction (horizontal direction), the up-down direction is the y-direction (vertical direction), and the direction perpendicular to the x-direction and y-direction (direction perpendicular to the paper surface) is the z-direction (optical axis direction). In the example shown in Figure 2, the pixel (unit pixel) array of the image sensor 107 is shown as an area of ​​4 columns x 4 rows, and the sub-pixel array is shown as an area of ​​8 columns x 4 rows.

[0030] In the pixel group 200 of 2 columns x 2 rows, for example, a pixel 200R having a spectral sensitivity of a first color R (red) is arranged at the upper left position, a pixel 200G having a spectral sensitivity of a second color G (green) is arranged at the upper right and lower left position, and a pixel 200B having a spectral sensitivity of a third color B (blue) is arranged at the lower right position. Furthermore, each pixel (unit pixel) is divided into two in the x direction (Nx divisions) and one in the y direction (Ny divisions), resulting in a division number of 2 (division number N LF =Nx×Ny) first subpixel 201 and second subpixel 202 (the first to Nth subpixels LF It is composed of multiple sub-pixels (sub-pixels).

[0031] In the example shown in FIG. 2, each pixel of the image sensor 107 is divided into two sub-pixels arranged in the horizontal direction, and the number of divisions N is calculated from the image signal (LF data) obtained by one image capture. LF It is possible to generate a number of viewpoint images equal to the number of sub-pixels in the image sensor 107, and a captured image obtained by combining all the viewpoint images. Note that the pixels may be divided in two directions, and there is no limit to the number of divisions in each direction. Therefore, it can be said that the viewpoint images are images generated from signals of some of the sub-pixels among the plurality of sub-pixels, and the captured image is an image generated from signals of all the sub-pixels. In this embodiment, as an example, the pixel period P in the horizontal and vertical directions of the image sensor 107 is set to 6 μm, and the number of horizontal pixels N H =6000, number of vertical pixels N V = 4000. Therefore, the total number of pixels N = N H ×N V = 24 million. In addition, the horizontal period of the sub-pixels P S If the total number of sub-pixels is N S =N H ×(P / P S )×N V =48 million.

[0032] FIG. 3(a) shows a plan view of one pixel 200G of the image sensor 107 shown in FIG. 2, as viewed from the light receiving surface side (+z side) of the image sensor 107. The z-axis is set perpendicular to the plane of FIG. 3(a), with the positive direction of the z-axis defined as the front side. The y-axis is set perpendicular to the z-axis, with the positive direction of the y-axis being the up-down direction, and the x-axis is set perpendicular to the z-axis and y-axis, with the positive direction of the x-axis being the right side. FIG. 3(b) shows a cross-sectional view taken along the aa section line in FIG. 3(a) as viewed from the -y side.

[0033] As shown in Figures 3(a) and 3(b), in pixel 200G, a microlens 305 is formed on the light receiving surface side (+z direction) of each pixel, and incident light is condensed by this microlens 305. Furthermore, a plurality of photoelectric conversion units, namely, first photoelectric conversion unit 301 and second photoelectric conversion unit 302, are formed, which are divided into two in the x (horizontal) direction and one in the y (vertical) direction, with a division number of 2. The first photoelectric conversion unit 301 and second photoelectric conversion unit 302 correspond to the first subpixel 201 and second subpixel 202 in Figure 2, respectively. More generally, the photoelectric conversion unit of each pixel is divided into Nx divisions in the x direction and Ny divisions in the y direction, and the division number of the photoelectric conversion unit is N LF = Nx × Ny, the first to Nth LF The photoelectric conversion units are 1st to Nth LF It corresponds to a sub-pixel.

[0034] The first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 are two independent pn junction photodiodes, each consisting of a p-type well layer 300 and two divided n-type layers 301 and 302. If necessary, an intrinsic layer may be sandwiched between them to form a pin structure photodiode. In each pixel, a color filter 306 is formed between the microlens 305 and the first photoelectric conversion unit 301 or second photoelectric conversion unit 302. If necessary, the spectral transmittance of the color filter 306 may be changed for each pixel or each photoelectric conversion unit, or the color filter may be omitted.

[0035] Light incident on pixel 200G is collected by microlens 305 and further dispersed by color filter 306, after which it is received by first photoelectric conversion unit 301 and second photoelectric conversion unit 302, respectively. In first photoelectric conversion unit 301 and second photoelectric conversion unit 302, pairs of electrons and holes (positive holes) are generated according to the amount of received light, and after they are separated by a depletion layer, the electrons are accumulated. Meanwhile, the holes are discharged to the outside of image sensor 107 through a p-type well layer connected to a constant voltage source (not shown). The electrons accumulated in first photoelectric conversion unit 301 and second photoelectric conversion unit 302 are transferred to a capacitance unit (FD) via a transfer gate and converted into a voltage signal.

[0036] In this embodiment, the microlenses 305 correspond to the optical system in the image sensor 107. The optical system in the image sensor 107 may be configured to use microlenses as in this embodiment, or may be configured to use materials with different refractive indices, such as waveguides. The image sensor 107 may be a back-illuminated image sensor having circuits and the like on the surface opposite to the surface having the microlenses 305, or may be a stacked image sensor having some circuits, such as the image sensor drive circuit 124 and the image processing circuit 125. A material other than silicon may be used for the semiconductor substrate, and for example, an organic material may be used as the photoelectric conversion material.

[0037] [Pupil division] Next, the pupil division function of the image sensor 107 of this embodiment will be described with reference to FIGS. 4 shows a cross-sectional view of the aa cross section of pixel 200G arranged in the image sensor 107 of this embodiment shown in FIG. 3(a) as viewed from the +y side, and also shows the pupil plane at a position a distance Z in the z-axis direction (direction of the optical axis OA) from the image sensor 107's image sensor plane 600. Note that in FIG. 4, the x-axis and y-axis of the cross-sectional view are reversed compared to FIG. 3 in order to correspond to the coordinate axes of the exit pupil plane. The image sensor 107's image sensor plane 600 is disposed at the image sensor plane of the image sensor optical system.

[0038] The pupil intensity distribution (first pupil intensity distribution) 501 has a substantially conjugate relationship, via the microlens 305, with the light receiving surface of the first photoelectric conversion unit 301, whose center of gravity is decentered in the -x direction. Therefore, the first pupil intensity distribution 501 corresponds to the pupil region that can receive light by the first subpixel 201. The center of gravity of the first pupil intensity distribution 501 is decentered to the +xp side on the pupil plane. Similarly, the pupil intensity distribution (second pupil intensity distribution) 502 has a substantially conjugate relationship, via the microlens 305, with the light receiving surface of the second photoelectric conversion unit 302, whose center of gravity is decentered in the +x direction.

[0039] Therefore, the second pupil intensity distribution 502 corresponds to the pupil region that can receive light by the second subpixel 202. The center of gravity of the second pupil intensity distribution 502 is decentered toward the -xp side on the pupil plane. Furthermore, the pupil intensity distribution 500 is the pupil region that can receive light by the entire pixel 200G when the first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 (the first subpixel 201 and the second subpixel 202) are all combined. In other words, the first pupil intensity distribution 501 is decentered toward the +xp side on the pupil plane with respect to the center of the pupil intensity distribution 500, and the second pupil intensity distribution 502 is decentered toward the -xp side on the pupil plane with respect to the center of the pupil intensity distribution 500.

[0040] Next, the sensor entrance pupil of the image sensor 107 will be described with reference to Fig. 5. Fig. 5 shows a schematic diagram of the arrangement of microlenses 305 according to the image height of the pixel. The Z axis passes through the center of the image sensor (image height = 0), and the image height increases as the distance from the center increases.

[0041] In the image sensor 107, the microlenses 305 are arranged so that the intersection between the optical axis of the microlenses 305 and the photoelectric conversion unit approaches the center of the image sensor (the origin of the image height) as the image height of the corresponding pixel increases. Note that the intersection between the center of the image sensor and the optical axis of the imaging optical system moves when the lens or the image sensor is driven by the optical image stabilization mechanism, but remains approximately the same.

[0042] In this way, the position of the microlens 305 is shifted toward the center of the image sensor 107 by an amount according to the image height of the corresponding pixel. As a result, on the pupil plane at a position distance Z from the image sensor 107, the first pupil intensity distributions 501 corresponding to the first photoelectric conversion units 301 of the first subpixels 201 of pixels at different image heights become roughly consistent. Similarly, the second pupil intensity distributions 502 corresponding to the second photoelectric conversion units 302 of the second subpixels 202 become roughly consistent.

[0043] As a result, the first pupil intensity distribution 501 and the second pupil intensity distribution 502 of all pixels of the image sensor 107 can be made to roughly match on the pupil plane located distance Z away from the image sensor 107. Hereinafter, the first pupil intensity distribution 501 and the second pupil intensity distribution 502 will be referred to as the "sensor entrance pupil" of the image sensor 107, and the distance Z will be referred to as the "entrance pupil distance" of the image sensor 107. By making the entrance pupil distance equal for all pixels, it is possible to suppress a decrease in the amount of light in pixels in peripheral regions with large image heights.

[0044] 6 shows the optical division (pupil division) of the exit pupil of the imaging optical system by the sensor entrance pupil (first pupil intensity distribution 501 and second pupil intensity distribution 502) of the image sensor 107. The exit pupil 400 of the imaging optical system has a size corresponding to the aperture of the diaphragm 102. Of the light beams from the subject, those that pass through a pupil partial region (first pupil partial region) 601, which is an overlapping region between the exit pupil 400 of the imaging optical system and the pupil intensity distribution 501, are received by the first sub-pixel 201 (first photoelectric conversion unit 301). Similarly, of the light beams from the subject, those that pass through a pupil partial region (second pupil partial region) 602, which is an overlapping region between the exit pupil 400 of the imaging optical system and the pupil intensity distribution 502, are received by the second sub-pixel 202 (second photoelectric conversion unit 302).

[0045] 4 shows a first pupil partial region 601 and a second pupil partial region 602 on the pupil plane at a position separated by entrance pupil distance Z from the image sensor 107. The exit pupil of the imaging optical system is divided into the first pupil partial region 601 and the second pupil partial region 602 by the sensor entrance pupil (first pupil intensity distribution 501 and second pupil intensity distribution 502) of the image sensor 107. In other words, the first pupil partial region 601 and the second pupil partial region 602 are the areas (products) common to the exit pupil of the imaging optical system and each of the first pupil intensity distribution 501 and the second pupil intensity distribution 502.

[0046] Fig. 7 is a diagram schematically showing the light intensity distribution when light is incident on the microlens 305 formed in each pixel. The light intensity distribution shown in Fig. 7 is calculated by the FDTD (Finite Difference Time Domain) method for the light intensity distribution inside the image sensor 107. As an example, it is assumed here that a right-handed circularly polarized plane wave with a wavelength λ=540 nm is incident from above the microlens 305 parallel to the optical axis of the microlens.

[0047] 7A shows the light intensity distribution in a cross section parallel to the optical axis of the microlens. The microlens optical system of each pixel is composed of a microlens 305, a planarization layer, a sealing layer, an insulating layer, etc. The microlens optical system may include multiple microlenses. The pixel period is 2a, the focal length of the microlens optical system is f, and the aperture angle of the microlens optical system is 2φ. Furthermore, the refractive index at the focal position of the microlens optical system is n. Furthermore, the coordinate on the optical axis is z. The coordinate z has the focal position as the origin (z=0), with the microlens side (top of the figure) being negative and the opposite side to the microlens being positive. Furthermore, H is the principal point.

[0048] The numerical aperture NA of the microlens optical system is defined by the following formula (1). NA=n·sinφ (1) The aperture value F of the microlens optical system is defined by the following equation (2). F=1 / (2n sinφ)=f / (2n a) (2)

[0049] Light incident on a pixel is focused at a focal point by a microlens optical system. However, due to the effect of diffraction caused by the wave nature of light, the diameter of the focused spot cannot be made smaller than the diffraction limit Δ and is therefore finite. Assuming that the intensity distribution of the focused spot is close to an Airy pattern, the diffraction limit Δ can be roughly calculated using the following equation (3), where λ is the wavelength of the incident light. Δ=1.22 (λ / (n·sinφ))=2.44·λ·F (3)

[0050] The size of the light receiving surface of the photoelectric conversion unit is approximately 1 to 3 μm on a side, and if the diameter of the light-condensing spot of the microlens is equal to the diffraction limit Δ, it is approximately 1 μm. Therefore, the sensor entrance pupil (first pupil intensity distribution 501 and second pupil intensity distribution 502), which is in a conjugate relationship with the light receiving surface of the photoelectric conversion unit via the microlens, does not have clear pupil division due to diffraction blur, and becomes a light receiving rate distribution (pupil intensity distribution) that depends on the angle of incidence of light.

[0051] Figure 7(B) shows a schematic diagram of the light intensity distribution in a cross section perpendicular to the optical axis of the microlens at the focal position of the microlens. At the focal position (z = 0), the diameter of the focused spot is the diffraction limit Δ and is at its minimum. Rear focal depth of the microlens optical system +z D and the front focal depth -z D is calculated by the following equation (4) with the diffraction limit Δ as the permissible circle of confusion. D <z<+z D is. ±z D =±n F Δ (4)

[0052] If the intensity distribution of the focused spot is close to a Gaussian distribution, the diameter w of the focused spot at the coordinate z can be roughly expressed by the following equation (5). w(z)=Δ{(1+(z / z R ) 2} 1 / 2 (5) where z Ris the Rayleigh length, and the coefficient αR=0.61π≒1.92, z R = αRz D is defined as:

[0053] In the calculation example shown in Figure 7, the wavelength λ is 540 nm, the pixel period 2a is 4.3 μm, the focal length f of the microlens optical system is 5.8 μm, the refractive index at the focal position n is 1.46, and the aperture value F of the microlens optical system is 0.924. In this case, the diffraction limit Δ is about 1.22 μm, and the focal depth ±z D is approximately ±1.65 μm.

[0054] 8 shows an example of the relationship between the light incidence angle and the light-receiving rate for the first pupil partial region 601 and the second pupil partial region 602 of this embodiment. The horizontal axis represents the light incidence angle θ (which can be converted into pupil coordinates), and the vertical axis represents the light-receiving rate. Graph line L1 shown as a solid line in Fig. 8 represents the light-receiving rate distribution along the X-axis of the first pupil partial region 601 in Fig. 4, and graph line L2 shown as a dotted line represents the light-receiving rate distribution along the X-axis of the second pupil partial region 602.

[0055] The angle of light incidence on the first pupil partial region 601 and the second pupil partial region 602 is limited by the exit pupil, which is defined by the lens frame and aperture frame of the imaging optical system. For this reason, as shown in Fig. 8, the light-receiving efficiency of the first pupil partial region L1 and the second pupil partial region L2 drops sharply when the angle of incidence reaches a certain value. On the other hand, for the angle of incidence in the range where the pupil is divided by the microlens, the light-receiving efficiency changes gradually because the boundaries of the pupil partial regions become blurred due to the effects of diffraction. Although an example in which the pupil region is divided into two horizontally is shown here, the pupil region may also be divided vertically.

[0056] Each pixel of the image sensor 107 has multiple subpixels that share a single microlens. Each subpixel receives light beams that pass through different partial regions of the pupil region of the imaging optical system. In this embodiment, two subpixels share a single microlens, but more subpixels may share a single microlens. By sharing a single microlens, signals from all the subpixels can be combined at each pixel to be used as an imaging signal, and the signals from each subpixel can be used as a focus detection signal. However, a pixel having only the first subpixel 201 and a pixel having only the second subpixel 202 may also be provided. In this case, the pixel having only one subpixel is a pixel dedicated to focus detection. Similarly, a pixel having a photoelectric conversion region that is not divided into subpixels may also be provided. This pixel receives light beams that pass through the combined area of ​​the pupil partial regions 601 and 602 and serves as a pixel dedicated to imaging.

[0057] In this embodiment, a first focus detection signal is generated based on the pixel signal of the first sub-pixel 201 of each pixel of the image sensor 107, and a second focus detection signal is generated based on the pixel signal of the second sub-pixel 202 of each pixel. The first focus detection signal and the second focus detection signal are used to calculate the defocus amount of the imaging optical system, and phase-difference AF is performed to adjust the focus of the imaging optical system in accordance with the defocus amount.

[0058] Furthermore, by adding and reading out the signals of the first sub-pixel 201 and the second sub-pixel 202 for each pixel of the image sensor 107, it is possible to generate an imaging signal (captured image) having a resolution of the number of effective pixels N. Furthermore, the method of generating each signal is not limited to that of this embodiment, and for example, the second focus detection signal may be generated by subtracting the first focus detection signal from the imaging signal.

[0059] Note that, although the pixel configuration of the image sensor 107 is configured to include the first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 relative to the microlens 305, the present invention is not limited to this. For example, pupil division may be performed by including a light-shielding layer having an opening in a portion between the microlens and the photoelectric conversion unit. By varying the positions of the openings in the light-shielding layer between adjacent pixels, signals corresponding to the first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 can be acquired.

[0060] [Relationship between defocus amount and image shift amount] The relationship between the amount of image shift and the amount of defocus for the first and second focus detection signals will now be described. Fig. 9 is a diagram showing the relationship between the amount of defocus and the amount of image shift. The image sensor 107 is disposed on an imaging surface 600, and the exit pupil 400 of the imaging optical system is divided into two pupil partial regions 601 and 602, similar to Figs. 4 and 6.

[0061] The magnitude |d| of the defocus amount d represents the distance from the imaging position of the subject image to the imaging plane 600. In a front-focus state, where the imaging position of the subject image is closer to the subject than the imaging plane 600, the direction is defined as a negative sign (d<0), and in the opposite back-focus state, the direction is defined as a positive sign (d>0). In a focused state, where the imaging position of the subject image is on the imaging plane (i.e., the in-focus position), d=0. For example, subject 701 is in-focus because it is imaged in a focused state (d=0). Furthermore, the imaging position of subject 702 is closer to the subject than the imaging plane 600 (d<0), so it is in front-focus. Hereinafter, the front-focus state (d<0) and the back-focus state (d>0) will be collectively referred to as the defocus state (|d|>0).

[0062] In a front-focus state (d<0), the light beam received from the subject 702 that passes through the first pupil partial region 601 (or the second pupil partial region 602) is collected and then spreads to a width Γ1 (or Γ2) centered on the center of gravity position G1 (or G2) of the light beam. In this case, the image of the subject 702 is blurred on the imaging surface 600. The blurred image is received by the first sub-pixels 201 (or the second sub-pixels 202) that constitute each pixel arranged in the image sensor 107, and a first focus detection signal (or a second focus detection signal) is generated.

[0063] Therefore, the first focus detection signal (or the second focus detection signal) is stored in memory as image data of a (blurred) subject image having a width Γ1 (or Γ2) at the center of gravity position G1 (or G2) on the imaging surface 600. The width Γ1 (or Γ2) of the subject image increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. Similarly, if the image shift amount between the first focus detection signal and the second focus detection signal is "p," the magnitude of the image shift amount |p| increases as the magnitude |d| of the defocus amount d increases.

[0064] 9, the image shift amount p can be defined as the difference in the center of gravity positions of the light beams, "G1-G2," and its magnitude |p| increases roughly in proportion to the increase in the magnitude of the defocus amount |d|. Note that in a back-focus state (d>0), the direction of the image shift between the first focus detection signal and the second focus detection signal is opposite to that in the front-focus state, but the magnitude of the defocus amount |p| is proportional to the magnitude of the defocus amount |d|.

[0065] In this way, as the defocus amount of the first focus detection signal and the second focus detection signal, or the defocus amount of the image signal obtained by adding the first and second focus detection signals, increases, the image shift amount of the first focus detection signal and the second focus detection signal increases.

[0066] Therefore, phase-difference AF can be achieved by detecting the amount of image shift between the first and second focus detection signals and converting the amount of image shift into a defocus amount. The amount of image shift between the first and second focus detection signals can be calculated by shifting the first and second focus detection signals relative to each other and finding the shift amount that provides good correlation (signal agreement).

[0067] So far, we have explained the principle of achieving phase-difference AF by pupil-dividing the exit pupil of the imaging optical system into a first pupil partial region 601 and a second pupil partial region 602 using a first pupil intensity distribution 501 and a second pupil intensity distribution 502 at the image sensor 107. In reality, the first pupil partial region 601 and the second pupil partial region 602 will have more complex shapes because they are affected by the amount of pupil decentering of the sensor entrance pupil of the image sensor 107, the entrance pupil distance, and lens frame vignetting of the imaging optical system.

[0068] The relationship between the pupil decentration amount of the image sensor 107, the entrance pupil distance, and the lens frame vignetting of the imaging optical system, and the first pupil partial area 601 and the second pupil partial area 602 will be described with reference to FIG.

[0069] [Pupil decentering of the sensor entrance pupil] The sensor entrance pupil (pupil intensity distributions 501, 502) of the image sensor 107 may have pupil decentering (eccentricity of the pupil intensity distribution relative to the optical axis) or an error in the entrance pupil distance due to manufacturing errors or the like.

[0070] 10 shows a state in which the distance from the microlens 305 to the light receiving surfaces of the first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 in all pixels of the image sensor 107 is zs+dzs (error+dzs) with respect to the design value zs due to manufacturing errors, etc. Furthermore, as the distance zs becomes zs+dzs, the entrance pupil distance of the image sensor 107 becomes Z+dZ (error+dZ) with respect to the design value Z.

[0071] 10 also shows a state in which the microlens 305 is decentered (by an amount of decentering dxs) in the pupil division direction (the division direction of the photoelectric conversion units) from the design value relative to the first photoelectric conversion unit 301 and the second photoelectric conversion unit 302 in all pixels of the image sensor 107. In addition, as the optical axis position of the microlens 305 is decentered by dxs from the design value, the entrance pupil (pupil intensity distributions 501, 502) of the image sensor 107 is decentered (by an amount of pupil decentering dX) in the pupil division direction (the division direction of the photoelectric conversion units).

[0072] Errors in the entrance pupil distance and pupil decentering can also occur due to assembly errors when attaching the image sensor 107 to the camera body. The pupil decentering amount dX and the entrance pupil distance error +dZ vary in magnitude depending on the image height in the x and y directions due to errors that occur in each manufacturing process. That is, the pupil decentering amount dX and the entrance pupil distance error +dZ depend on the image height. Therefore, as will be described later, when calculating a conversion coefficient for converting the image shift amount to a defocus amount, a highly accurate conversion coefficient can be calculated by taking into account the pupil decentering amount dX and the entrance pupil distance Z+dZ that correspond to the image height.

[0073] [Image height dependence of pupil decentration and entrance pupil distance] 11 shows an example of a method for efficiently storing information on pupil decentration amount dX and entrance pupil distance Z+dZ corresponding to a plurality of image heights. FIG. 11 shows the pixel area of ​​the image sensor 107, and stores information on pupil decentration amount dX and entrance pupil distance error +dZ for positions 1001 to 1005 indicated by black circles. This information can be measured after the camera body is manufactured and stored in ROM 135, for example. The design value Z of the entrance pupil distance is also stored in ROM 135.

[0074] When the pixel area is divided into four quadrants (top, bottom, left, and right) using a Cartesian coordinate system with the center as the origin, the pupil decentering amount dX and the entrance pupil distance error +dZ have values ​​that depend on the image height, and therefore each quadrant exhibits different optical characteristics. Here, the image height is a value with a sign that depends on the direction, with the center of the image sensor 107 (image area) as the origin (image height = 0).

[0075] Here, the pixel coordinates of position 100n (n = 1 to 5) are (xi, yi) (i = 0 to 4). Also, position 1001 is assumed to be equal to the center of the image area. The optical characteristics of the image sensor 107 basically change continuously as the image height changes. Therefore, by storing information on pupil decentration amount dX and entrance pupil distance error +dZ for one or more discrete positions for each quadrant, the pupil decentration amount dX and entrance pupil distance error +dZ for each pixel can be determined by interpolation. Hereinafter, information on pupil decentration amount dX and entrance pupil distance error +dZ will be collectively referred to as error information.

[0076] In the example of Figure 11, positions 1002 to 1005 are arranged so as to be point-symmetric with respect to the center of the pixel area (position 1001) and symmetric with respect to the coordinate axes. In this case, x1 = x4, y1 = y2, x2 = x3, y3 = y4. Two or more positions that satisfy the same conditions may be arranged in each quadrant.

[0077] An example of calculating error information by interpolation will be described below. The pupil decentration amount dX(x,y) and entrance pupil distance error +dZ(x,y) for pixel coordinates (x,y) included in the second quadrant (upper left) can be calculated as follows: First, the error information of the position 1002 and the position 1003 is linearly interpolated to calculate the error information of the position 1006 (x0, (y1+y2) / 2). Next, the error information of the position 1003 and the position 1004 is linearly interpolated to calculate the error information of the position 1007 ((x2+x3) / 2,0). The error information at the pixel coordinates included in the rectangular area having vertices at positions 1001, 1006, 1003, and 1007 can be calculated by subjecting the error information at positions 1001, 1006, 1003, and 1007 to two-dimensional linear interpolation (bilinear interpolation).

[0078] By calculating pixel coordinates included in other quadrants in the same way, error information can be calculated for pixel coordinates included in a rectangular area with positions 1002 to 1005 as vertices.

[0079] If simplicity of calculation is prioritized over accuracy of correction, error information at pixel coordinates within a rectangular area having vertices at positions 1002 to 1005 may be calculated by two-dimensional linear interpolation (bilinear interpolation) of the error information at positions 1002 to 1005. In this case, the error information at position 1001 is not used.

[0080] On the other hand, the error information at the pixel coordinates of each quadrant not included in the rectangular area having vertices at positions 1002 to 1005 can be calculated by extrapolating the error information at the pixel coordinates included in the rectangular area having vertices at positions 1002 to 1005. Alternatively, the error information at the closest pixel coordinates among the pixel coordinates included in the rectangular area having vertices at positions 1002 to 1005, or the error information at the closest pixel coordinates among positions 1002 to 1005, may be used.

[0081] Next, we will explain how to calculate a conversion coefficient that takes error information into account. The conversion coefficient is used to convert the image shift amount of the focus detection signal into a defocus amount. Here, we calculate a conversion coefficient K[dX(x,y), Z+dZ(x,y)] that takes into account the pupil decentering amount dX(x,y) at pixel coordinates (x,y) and the entrance pupil distance error +dZ(x,y). Using such a conversion coefficient enables highly accurate focus detection for focus detection areas located at various image heights.

[0082] The image sensor 107 is basically optically designed based on the image height at the center position. Therefore, if error information is stored for one or more pixel coordinates in each quadrant in addition to the center position (x0, y0), it is possible to calculate conversion coefficients with high accuracy. A specific method for calculating the conversion coefficients will be described later.

[0083] Note that the case where conversion coefficients are calculated after error information at pixel coordinates (x, y) has been described here. However, conversion coefficients that reflect error information may be calculated and stored for positions 1001 to 1005, and conversion coefficients at any pixel coordinates may be calculated in the same manner as for the error information. Also, instead of storing the entrance pupil distance error +dZ for positions 1001 to 1005 and the entrance pupil distance design value z, the entrance pupil distance Z+dZ that includes an error may be stored for positions 1001 to 1005.

[0084] 11 are merely examples. The image height of the pixel coordinates for storing the error information in each quadrant may be different from that in FIG. 11, or error information may be stored for a plurality of pixel coordinates with different image heights in each quadrant.

[0085] It should be noted that the model of pupil decenter amount and entrance pupil distance in the above description is merely an example. In the present invention, the pupil decenter amount and entrance pupil distance may be expressed by any model as long as it is possible to calculate the pupil decenter amount and / or entrance pupil distance, which change depending on the image height, for any pixel coordinates.

[0086] [Lens frame vignetting] Next, vignetting caused by the lens frame for pixels in the peripheral region of the pixel region (region with large image height) will be described with reference to Figures 12 and 13. The lens frame is a frame-shaped member that holds the lenses that make up the imaging optical system.

[0087] 12 shows, in the xz plane from the +y side, the relationship between the pupil decentering amount of the sensor entrance pupil of the image sensor 107, the entrance pupil distance, and the lens frame vignetting of the imaging optical system, and the pupil partial areas (first pupil partial area 601, second pupil partial area 602) for pixels in the peripheral area. The entrance pupil (pupil intensity distributions 501, 502) of the image sensor 107 corresponds to that in Fig. 10, and is assumed to be decentered by pupil decentering amount dX in the pupil division direction (division direction of the photoelectric conversion unit) on the pupil plane at entrance pupil distance Z+dZ.

[0088] The light beam from the subject 703 reaches the imaging plane 600 after the angle of incidence is limited by the first frame F1 on the subject side of the imaging optical system, the third frame F3 which is the aperture 102 of the imaging optical system, and the second frame F2 on the image side of the imaging optical system. Of the light beams incident on the imaging optical system from the subject 703, the amount of light beams that are blocked (vignetted) by the first to third frames F1 to F3 and cannot reach the imaging plane 600 increases as the pixels become closer to the peripheral region.

[0089] Of the light beams vignetted by the first to third frames F1 to F3 of the imaging optical system, the light beams that pass through a first pupil partial region 601, which is an overlapping region with the pupil intensity distribution 501, are received by the first subpixel 201 (first photoelectric conversion unit 301). Similarly, of the light beams vignetted by the first to third frames F1 to F3 of the imaging optical system, the light beams that pass through a second pupil partial region 602, which is an overlapping region with the pupil intensity distribution 502, are received by the second subpixel 202 (second photoelectric conversion unit 302).

[0090] In this embodiment, the first to third frames F1 to F3 are described as frame-shaped members having concentric circular openings centered on the optical axis, but at least a portion of the opening does not have to be formed as an arc. For example, the opening may be polygonal or D-shaped.

[0091] 13 shows a first pupil partial region 601 and a second pupil partial region 602 on the pupil plane at a position separated by the entrance pupil distance Z+dZ of the image sensor 107. The exit pupil, which is the overlap of the apertures of the first to third frames F1 to F3 of the imaging optical system, is divided into the first pupil partial region 601 and the second pupil partial region 602 by the sensor entrance pupil (first pupil intensity distribution 501, second pupil intensity distribution 502) of the image sensor 107. The shapes of the first pupil partial region 601 and the second pupil partial region 602 are affected by the aperture shapes of the first to third frames F1 to F3, the positions of the frames in the z direction, the pixel image height, and error information (pupil eccentricity amount dX, entrance pupil distance error+dZ(x, y)), etc.

[0092] [Calculation of defocus amount] 14, a description will be given of a defocus amount calculation process performed by the CPU 121 executing a focus detection program. The focus detection program may be stored in the ROM 135 or the recording medium 133.

[0093] In this embodiment, the CPU 121 of the imaging device 100 executes the focus detection process, but a personal computer (PC) or a dedicated device may also execute the focus detection process as a focus detection device. The focus detection process described below may also be executed using a hardware circuit such as an ASIC or FPGA.

[0094] In S101, the CPU 121 sets the position of the focus detection area. The focus detection position may be, for example, the coordinates of the pixel closest to the center or center of gravity of the focus detection area. Here, the focus detection position is expressed as the coordinates (x AF ,y AF The focus detection area is set by the following formula: The focus detection area is, for example, a rectangular area. The focus detection area can be determined by any known method.

[0095] The center or center of gravity of the pixel area substantially coincides with the optical axis of the imaging optical system. However, strictly speaking, there is a certain degree of variation in the positions of the image sensor 107 and the imaging optical system. Therefore, the origin position may be determined taking into account the range of variation. Furthermore, the coordinates of the focus detection position may be set taking into account the amount of shift in the optical axis position caused by the optical image stabilization mechanism.

[0096] Next, the coordinate (x AF ,y AF ) and, using equation (6), the image height r AF Calculate. r AF ={x AF 2 +y AF 2} 1 / 2 (6)

[0097] In S102, the CPU 121 (lens information acquisition unit 121d) acquires lens information according to the optical conditions. Specifically, the coordinates (x AF ,y AF ) and lens information according to optical conditions such as the focus state FS and zoom state ZS of the imaging optical system (interchangeable lens). The lens information is obtained by determining the distance Z from the imaging plane of the imaging optical system (the imaging plane of the image sensor 107) to a predetermined pupil distance Z f The center C1(x) of the first frame (object-side frame) F1 on the object side of the imaging optical system is projected onto the pupil plane at a distance of AF ,y AF ,FS,ZS) and radius R1(x AF ,y AF , FS, ZS). Furthermore, the frame information includes the center C2(x AF ,y AF ,FS,ZS) and radius R2(x AF ,y AF ,FS,ZS).

[0098] In addition, the lens information is the aperture value F(x AF ,y AF ,FS,ZS) and the exit pupil distance LPO(x AF ,y AF , FS, ZS). The lens information is stored in advance in a nonvolatile memory or the like of the imaging optical system, and the CPU 121 (lens information acquisition unit 121d) can acquire the lens information by communicating with the imaging optical system via the lens communication circuit 130. Alternatively, lens information corresponding to the attached interchangeable lens may be used from lens information stored in advance in the ROM 135 of the imaging device 100. Note that the lens information can be acquired at any timing, but typically may be acquired when the lens is replaced or when the imaging element 100 is started up.

[0099] Lens information may also be acquired from an external device, for example, via a network. Optical conditions may also include information other than the focus state FS and zoom state ZS. For example, the optical conditions may include the attachment status of an extender, the amount of tilt or shift, and the attachment status of various optical filters.

[0100] In step S102, the CPU 121 next calculates the image height r AF From the exit pupil distance LPO of the aperture frame of the imaging optical system, the specified pupil distance Z f The center C3(x) of the third frame F3 of the imaging optical system projected onto the pupil coordinates of AF ,y AF ,FS,ZS) is calculated using equation (7). C3(x AF , y AF , FS, ZS) = r AF {1-Z f / LPO(x AF , y AF , FS, ZS)} (7)

[0101] The CPU 121 also calculates the image height r AF and the aperture value F of the imaging optical system, the specified pupil distance Z f The radius R3(x AF ,y AF .FS,ZS) Calculated using equation (8). R3(x AF , y AF , FS, ZS) = Z f / (4F 2 -1) 1 / 2 (8)

[0102] Furthermore, the CPU 121 calculates a first aperture parameter a1 and a second aperture parameter a2 based on the lens information. The first aperture parameter a1 and the second aperture parameter a2 are parameters related to vignetting by the lens frame. f1 shows an example of an exit pupil where the apertures of the first to third frames F1 to F3 of the imaging optical system overlap on a pupil plane located at a distance of 100 mm. The straight line passing through the centers C1 to C3 of the apertures is the optical axis.

[0103] The first aperture parameter a1 (0≦a1≦1) is the distance A1 from the center C3 of the opening of the third frame F3 to the vertex P1 of the first frame F1, normalized by the radius R3 of the third frame. Here, when the opening of the first frame F1 is a circle, the vertex P1 is the point on the circumference that is the shortest distance from the center C3. Similarly, the second aperture parameter a2 (0≦a2≦1) is the distance A2 from the center C3 of the third frame to the vertex P2 of the second frame, normalized by the radius R3 of the third frame. The first aperture parameter a1 is calculated using equation (9A), and the second aperture parameter a2 is calculated using equation (9B). a1 = (R1-|C1-C3|) / R3 (9A) a2 = (R2-|C2-C3|) / R3 (9B)

[0104] In S103, the CPU 121 (pixel signal acquisition unit 121a) acquires focus detection signals from the image sensor 107. Specifically, the CPU 121 acquires, for each pixel 200 of the image sensor 107, a pixel signal (A image signal) obtained from the first sub-pixel 201 and a pixel signal (B image signal) obtained from the second sub-pixel 202. As described above, these signals may be read out from the sub-pixels, or one may be read out from the sub-pixel and the other may be acquired by subtracting the A image signal or the B image signal from the image signal (A+B image signal). Note that the focus detection signals may be acquired by capturing an image by the image sensor 107 and stored in a recording medium in advance.

[0105] Furthermore, based on the acquired pixel signals, the CPU 121 (signal generation unit 121b) generates a first focus detection signal corresponding to a different first pupil partial region 601 of the imaging optical system, and a second focus detection signal corresponding to a second pupil partial region 602. In other words, each focus detection signal is generated based on signals from pixels corresponding to the same pupil partial region.

[0106] A pixel is divided into Nx sub-pixels in the horizontal direction and Ny sub-pixels in the vertical direction, and the sub-pixel signal group LF obtained from one pixel (total number Nx × Ny = NLF) is defined as follows: Of the sub-pixel signal group LF, the signal obtained from the iS-th sub-pixel in the column direction (1≦iS≦Nx) and the jS-th sub-pixel in the row direction (1≦jS≦Ny) is defined as the k-th focus detection signal, where k=Nx(jS-1)+iS(1≦k≦NLF). The k-th focus detection signal Ik(j,i), which is the i-th sub-pixel in the column direction and the j-th sub-pixel in the row direction and corresponds to the k-th pupil region of the imaging optical system, is generated using the following equation (10):

number

[0107] In this embodiment, Nx=2, Ny=1, and NLF=2, each pixel has two sub-pixels divided in the x direction, and a first focus detection signal and a second focus detection signal are generated. From the pixels arranged as shown in FIG. 2, the signal of the first sub-pixel 201 is selected for each pixel. This generates a first focus detection signal I1(j,i), which is an RGB signal in a Bayer array having a resolution of N pixels (= number of horizontal pixels NH × number of vertical pixels NV) corresponding to a first pupil partial region 601 of the imaging optical system. Similarly, a second focus detection signal I2(j,i) corresponding to a second pupil partial region 602 of the imaging optical system is generated by selecting the signal of the second sub-pixel 202.

[0108] In S103, the CPU 121 (signal generation unit 121b) as a generation means generates a k-th focus detection luminance signal Yk(i,j)(k=1,2) from the k-th focus detection signal Ik(k=1,2), which is an RGB signal in a Bayer array. Specifically, the CPU 121 (signal generation unit 121b) matches the color centroids of each RGB color for each position (j,i) and generates the k-th focus detection luminance signal Yk(i,j)(k=1,2) using the following equation (11):

number

[0109] At this time, to improve focus detection accuracy, a light intensity correction process that corrects shading caused by pupil division can be applied to the kth focus detection luminance signal Yk. Like the conversion coefficient, shading caused by pupil division is also affected by the amount of pupil decentration, the entrance pupil distance, and the aperture frame of the imaging optical system. Therefore, the light intensity correction value used in the light intensity correction process may be calculated by an external device, with the result (intermediate value) stored in ROM 135. By applying part of the calculation process externally in advance, the load on the CPU 121 related to the calculation of the light intensity correction value can be reduced.

[0110] Here, it is assumed that the light intensity correction value SHD is calculated in advance for each of a plurality of image heights, taking into account the amount of pupil decentration, the entrance pupil distance, and the influence of the aperture frame of the imaging optical system, and stored in the ROM 135. The CPU 121 (signal generation unit 121b) obtains an appropriate light intensity correction value SHD from the ROM 135 according to the lens information and the focus detection position, and applies it to the k-th focus detection luminance signal Yk as shown in equation (12), thereby correcting the light intensity. Y k (j, i)=SHD k (j, i) Y k (j, i) (12)

[0111] Furthermore, the CPU 121 (signal generation unit 121b) applies one-dimensional band-pass filtering to each of the kth focus detection luminance signals Yk (k=1, 2) in the pupil division direction (column direction), thereby generating a first focus detection signal dYA and a second focus detection signal dYB that are limited to components of approximately wavenumber kAF.

[0112] The one-dimensional band-pass filter used here may be, for example, a first-order differential filter [1, 5, 8, 8, 8, 8, 5, 1, -1, -5, -8, -8, -8, -8, -5, -1]. The passband of the one-dimensional band-pass filter can be adjusted as needed. Generally, in phase-difference AF, focus detection is performed in a large defocus state, so the passband of the filter is configured to include a low-frequency band. However, when focus detection is performed from a large defocus state to a small defocus state, the passband of the filter may be adjusted to a higher frequency band depending on the defocus state.

[0113] In S104, the CPU 121 (focus detection unit 121c) as an image shift amount calculation unit calculates the image shift amount based on the first focus detection signal dYA and the second focus detection signal dYB. AF ,i AF ) centered on the j2 (-n2 ≦ j2 ≦ n2)-th wave number in the row direction and the i2 (-m2 ≦ i2 ≦ m2)-th wave number in the column direction, which is the pupil division direction, is used as the first focus detection signal dYA(j AF +j2,i AF +i2). Furthermore, the second focus detection signal is dYB(j AF +j2,i AF +i2). The shift amount is s (-ns≦s≦ns), and each position (j AF ,i AF ) correlation amount COR EVEN (j AF ,i AF , s) is calculated by equation (13A), and the correlation amount COR ODD (j AF ,i AF , s) is calculated by equation (13B). The smaller the correlation amount calculated here, the higher the correlation.

number

[0114] Correlation amount COR ODD (j AF ,i AF ,s) is the shift amount, and the correlation amount COREVEN (j AF ,i AF , s) is the correlation amount when shifted by half phase -1 with respect to the shift amount for obtaining the signal.

[0115] In S104, the correlation amount COR EVEN (j AF ,i AF ,s) and the correlation amount COR ODD (j AF ,i AF , s), the real value shift amount at which the correlation amount becomes the minimum value is calculated by sub-pixel calculation, and the average value is calculated. Then, the focus detection position (j AF ,i AF ) image shift amount q det Calculate.

[0116] [Calculating the conversion factor] In S105, the CPU 121 (focus detection unit 121c) as a coefficient calculation means calculates the image shift amount q det The defocus amount d det A conversion coefficient K for converting the image signal into a signal is calculated. A method for calculating the conversion coefficient K will be described in detail using the flowchart shown in FIG. 16. In the following, the processing of each step will be described as being executed by the CPU 121 (focus detection unit 121c). However, some steps may be executed by an external device, and the results (intermediate values) may be stored in the ROM 135. By applying some of the calculation steps externally in advance, the load on the CPU 121 related to the calculation of the conversion coefficient K can be reduced.

[0117] In step S201, the CPU 121 (focus detection unit 121c) calculates a virtual defocus amount d vir Set the virtual defocus amount d vir is a hypothetical defocus amount set to determine the conversion coefficient K. For example, the CPU 121 sets a predetermined defocus amount that allows the subject to be sufficiently blurred as the virtual defocus amount d vir The virtual defocus amount d vir is the front focus state (d vir <0, even in the back focus state (d vir>0).

[0118] In addition, multiple virtual defocus amounts d vir As will be described later, the conversion coefficient K is set to a virtual defocus amount d vir and the image shift amount q det In other words, the conversion coefficient K is calculated as the ratio of the virtual defocus amount d vir The vertical axis represents the image shift amount q det In the Cartesian coordinate system, the virtual defocus amount d vir and the image shift amount q det Therefore, the virtual defocus amount d vir By setting the above, the conversion coefficient K can be calculated with high accuracy.

[0119] In S202, the CPU 121 (focus detection unit 121c) The sensor entrance pupil of the image sensor 107 (first pupil intensity distribution 501, second pupil intensity distribution 502), For each individual image sensor 107, the coordinates of the focus detection position (x AF ,y AF ) for each pupil decentration dX(x AF ,y AF ), For each individual image sensor 107, the coordinates of the focus detection position (x AF ,y AF ) for each entrance pupil distance Z+dZ(x AF ,y AF )

[0120] Furthermore, the CPU 121 (focus detection unit 121c) First virtual focus detection signal VI1(j,i|d vir ,dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) Second virtual focus detection signal VI2(j,i|d vir ,dX(x AF ,yAF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) Generate.

[0121] The first virtual focus detection signal VI1 and the second virtual focus detection signal VI2 are Virtual defocus amount d vir , The sensor entrance pupil of the image sensor 107 (first pupil intensity distribution 501, second pupil intensity distribution 502), Focus detection position coordinates (x AF ,y AF ), For each individual image sensor 107, the coordinates of the focus detection position (x AF ,y AF ) for each pupil decentration dX(x AF ,y AF ), For each individual image sensor 107, the coordinates of the focus detection position (x AF ,y AF ) for each entrance pupil distance Z+dZ(x AF ,y AF ), and Frame information of the imaging optical system (aperture value F, aperture frame exit pupil distance LPO, first aperture parameter a1, second aperture parameter a2) It should be noted that the frame information of the imaging optical system is not limited to the example given here.

[0122] Of these parameters, The sensor entrance pupil of the image sensor 107 (first pupil intensity distribution 501, second pupil intensity distribution 502), The pupil decentration amount dX(x, y) for each individual image sensor 107 and for each coordinate of the focus detection position, and Entrance pupil distance Z+dZ(x,y) for each individual image sensor 107 and for each coordinate of the focus detection position may vary for each individual image sensor 107. These parameters that may vary for each individual image sensor 107 may be measured, for example, when the image sensor 107 is manufactured, and may be stored as adjusted values ​​or fixed values ​​in a nonvolatile memory included in the image sensor 107.

[0123] On the other hand, the coordinates of the focus detection position (x AF ,y AF ) is determined based on a user operation or by the CPU 121 each time an image is captured, regardless of the user operation. AF ,y AF ) is obtained during the calculation of the conversion coefficient K.

[0124] Furthermore, the frame information of the imaging optical system (aperture value F, aperture frame exit pupil distance LPO) may also change for each image capture. Furthermore, the first aperture parameter a1 and the second aperture parameter a2 depend on the position of the frame of the imaging optical system as shown in Equations (9A) and (9B), and therefore may change for each image capture. The CPU 121 (lens information acquisition unit 121d) can acquire these parameters related to the state of the imaging optical system by communicating with the imaging optical system (interchangeable lens) via the lens communication circuit 130. By acquiring parameters from the imaging optical system as needed, it is possible to calculate a more accurate conversion coefficient K using parameters according to imaging conditions such as aperture value and focal length. Note that the imaging conditions may include other items.

[0125] To calculate the conversion coefficient K, use the parameter (dX(x AF ,y AF ),Z+dZ(x AF ,y AF ), x AF ,y AF In addition to the parameters (F, LPO), the centers C1 and C2 and radii R1 and R2 of the first and second frames of the imaging optical system are required. In this embodiment, the total number of parameters required to calculate the conversion coefficient K can be reduced by using the first aperture parameter a1 and the second aperture parameter a2, which indicate the proportion of lens frame vignetting.

[0126] FIG. 17 shows the first virtual focus detection signal VI I 10A and 10B are diagrams illustrating a method for generating a second virtual focus detection signal VI2. First, the CPU 121 (signal generating unit 121b) A first pupil intensity distribution 501 and a second pupil partial region 502 of the image sensor 107, Focus detection position coordinates (x AF ,y AF ), Pupil decentering amount dX(x AF ,y AF ), Entrance pupil distance Z+dZ(x AF ,y AF ), -F-stop value F of the imaging optical system, The exit pupil distance LPO of the aperture frame of the imaging optical system, A first aperture parameter a1 (0≦a1≦1) and a second aperture parameter a2 (0≦a2≦1) are obtained.

[0127] Of these parameters, the CPU 121 (signal generating unit 121b) can acquire values ​​that do not depend on the imaging conditions from values ​​stored in advance in the ROM 135, or acquire them at the start of the calculation process of the conversion coefficient K and store them in the RAM 136 for use. Furthermore, the CPU 121 (signal generating unit 121b) can acquire values ​​that are based on the imaging conditions at the start of the calculation process of the conversion coefficient K and store them in the RAM 136 for use.

[0128] Next, the CPU 121 (signal generating unit 121b) calculates a predetermined pupil distance Z from the exit pupil distance LPO of the aperture frame of the imaging optical system. f The CPU 121 (signal generating unit 121b) calculates the center C3 of the third frame (aperture frame) F3 of the imaging optical system projected onto the pupil coordinates of the image forming optical system by using the formula (7). f The radius R3 of the third frame (aperture frame) F3 of the imaging optical system projected onto the pupil coordinates is calculated using equation (8).

[0129] Next, based on a model (FIG. 15) in which the first to third frames F1 to F3 intersect at the same point, the CPU 121 (signal generation unit 121b) calculates a radius R1' of the first frame used to generate a virtual focus detection signal from the first aperture parameter a1 using equation (14A). Similarly, the CPU 121 (signal generation unit 121b) calculates a radius R2' of the second frame used to generate a virtual focus detection signal from the second aperture parameter a2 using equation (14B).

[0130] Generally, the first to third frames F1 to F3 do not intersect at the same point, but by modeling them so that they intersect at the same point, it is possible to calculate the shapes of the first frame F1 and the second frame F2 with high accuracy using fewer parameters, thereby improving the accuracy of calculation of the conversion coefficient K. R1'={(a1×R3) 2 +R3 2} / {2(a1×R3)} (14A) R2'={(a2×R3) 2 +R3 2} / {2(a2×R3)} (14B)

[0131] Next, the CPU 121 (signal generation unit 121b) calculates the center C1' of the first frame used to generate the virtual focus detection signal as C1' = C3 - (R1' - a1 × R3) from the radius R1' of the first frame, the first aperture parameter a1, and the center C3 and radius R3 of the third frame F3. Similarly, the CPU 121 (signal generation unit 121b) calculates the center C2' of the second frame used to generate the virtual focus detection signal as C2' = C3 + (R2' - a2 × R3) from the radius R2' of the second frame, the second aperture parameter a2, and the center C3 and radius R3 of the third frame F3.

[0132] When the above parameters are acquired and calculated, the CPU 121 (signal generating unit 121b) generates a first virtual pupil partial region VP1601 and a second virtual pupil partial region VP2602 shown in the upper left diagram of FIG.

[0133] The first virtual pupil partial region VP1601 and the second virtual pupil partial region VP2602 are A first pupil intensity distribution 501 and a second pupil intensity distribution 502 of the image sensor 107, Focus detection position coordinates (x AF ,y AF ), Pupil decentering amount dX(x AF ,y AF ), Entrance pupil distance Z+dZ(x AF ,y AF ), First to third frames F1 to F3 of the imaging optical system It corresponds to.

[0134] Next, as shown in the upper center diagram of FIG. 17, the CPU 121 (signal generating unit 121b) projects the first virtual pupil partial region VP1 in a direction perpendicular to the pupil division direction, and calculates a virtual defocus amount d vir The size of |d vir Scale conversion is performed according to | to generate a first virtual line image A. Similarly, the CPU 121 (signal generating unit 121b) projects the second virtual pupil partial region VP2602 in a direction perpendicular to the pupil division direction, and calculates a virtual defocus amount d vir The size of |d vir Scale conversion is performed according to | to generate a second virtual line image B.

[0135] Virtual defocus amount d vir When the image is in front focus (<0), the CPU 121 (signal generating unit 121b) flips the first virtual linear image A and the second virtual linear image B left and right, centering on the center of gravity of the linear image A+B, which is the sum of the first virtual linear image A and the second virtual linear image B.

[0136] The CPU 121 (signal generating unit 121b) performs convolution of the first virtual line image A and the second virtual line image B with the object signal, and generates a first virtual object image VA I and second virtual subject image VB I This corresponds to the process shown in the upper center to upper right diagrams in FIG.

[0137] Next, the CPU 121 (signal generating unit 121b) performs the processes shown in the upper right diagram to the lower right diagram in Fig. 17. The CPU 121 (signal generating unit 121b) generates the first virtual subject image VA I and second virtual subject image VB I The signal processing is then applied virtually to the input signals in sequence. This signal processing includes optical low-pass filter processing, pixel aperture low-pass filter processing, and pixel sampling processing in the RGB Bayer array (Figure 2).

[0138] Then, the CPU 121 (signal generating unit 121b) First virtual focus detection signal VI1(j,i|d vir ,dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) and, Second virtual focus detection signal VI2(j,i|d vir ,dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) and,are generated.

[0139] 16 (and the lower part of FIG. 17), the CPU 121 (focus detection unit 121c) performs virtual focus detection processing using the first virtual focus detection signal VI1 and the second virtual focus detection signal VI2. The virtual focus detection processing includes processing from calculation of the correlation amount shown in equations (13A) and (13B) in S103 to calculation of the shift amount of real values ​​and their average value in S104.

[0140] Then, the CPU 121 (focus detection unit 121c) calculates the virtual image shift amount q vir (d vir ,dX(x AF ,y AF ),Z+dZ(x AF ,y AF),x AF ,y AF ,F,LPO,a1,a2) are calculated.

[0141] In addition, by using multiple types of subject signals for convolution with the virtual line image, the virtual image shift amount q vir For example, by using a plurality of subject signals having different spatial frequency bands for convolution with a virtual line image and averaging or weighting the individual results, the virtual image shift amount q vir The calculation accuracy can be improved.

[0142] In S204 of FIG. 16, the CPU 121 (focus detection unit 121c) calculates the virtual defocus amount d vir The calculated virtual image shift amount q vir This gives the conversion factor K(dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) are calculated.

[0143] As described above, the virtual defocus amount d vir Alternatively, the accuracy of the conversion coefficient K can be improved by calculating the conversion coefficient K using a plurality of object signals. However, since the calculation load or calculation time increases, the virtual defocus amount d vir Alternatively, it may be determined whether or not to use a plurality of object signals.

[0144] Returning to FIG. 14, in S106, the CPU 121 (focus detection unit 121c) calculates the image shift amount q det Conversion factor K(dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF ,F,LPO,a1,a2) and the defocus amount d det The defocus amount ddet Although the example in which is calculated for pixel signals output from the image sensor 107 has been described, it may also be calculated for pixel signals already recorded on the recording medium 133.

[0145] The image capture device 100 of this embodiment is a focus detection device that performs focus detection using pixel signals obtained by individually photoelectrically converting light passing through different partial regions of a pupil region of an imaging optical system. The image capture device 100 generates focus detection signals corresponding to each partial region using the pixel signals and calculates an image shift amount based on the focus detection signals. The image capture device 100 also calculates a defocus amount by applying a conversion coefficient to the image shift amount. The image capture device 100 calculates a virtual image shift amount from the correlation amount of the virtual focus detection signals generated by setting a virtual defocus amount, and calculates a conversion coefficient from the virtual image shift amount and the virtual defocus amount. The conversion coefficient is based on the pupil decentering amount of the image sensor, the entrance pupil distance of the image sensor, and aperture information for multiple frames of the imaging optical system.

[0146] In the present embodiment, an example has been described in which the CPU 121 of the imaging device 100 calculates the conversion coefficient K. However, the parameter (dX(x AF ,y AF ),Z+dZ(x AF ,y AF ),x AF ,y AF , F, LPO, a1, a2) can be prepared in advance. In this case, the conversion coefficient K can be stored in a non-volatile storage device such as the ROM 135 of the image pickup device 100 and used when performing focus detection processing.

[0147] FIG. 18 shows an example of changes in the first virtual pupil partial region VP1601 and the second virtual pupil partial region VP2602 according to the values ​​of the first aperture parameter a1 (0≦a1≦1) and the second aperture parameter a2 (0≦a2≦1).

[0148] (Variation) The reciprocal of the conversion coefficient 1 / K(dX(x AF ,y AF ), Z+dZ(x AF ,yAF ),x AF ,y AF , F, LPO, a1, a2) by a polynomial function of variables (1-a1) and (1-a2), the amount of data stored in the ROM 135 of the image pickup device 100 can be reduced. Specifically, the coefficients PD of each degree of the polynomial function αβ (dX,Z,x AF ,y AF , F, LPO) can be stored in the ROM 135.

[0149] In this case, in S105, the CPU 121 can calculate the conversion coefficient K by the following equation (15) instead of the process shown in FIG.

number

[0150] Coefficient PD αβ is stored, and the CPU 121 calculates the conversion coefficient K by applying the focus detection position, imaging conditions, etc. at the time of imaging. In this way, by calculating the conversion coefficient K using the coefficients prepared in advance and equation (15), the calculation load on the CPU 121 can be reduced and high-speed focus detection can be achieved.

[0151] Coefficient PD to remember αβ does not use the first aperture parameter a1 or the second aperture parameter a2. Therefore, if the range of settable aperture values ​​F and the range of possible exit pupil distances LPO of the aperture frame of the imaging optical system are determined, the conversion coefficient K can be calculated independently of the imaging optical system.

[0152] As another method for reducing the load on the CPU 121 associated with calculating the conversion coefficient K, the conversion coefficient k may be calculated by hardware separate from the CPU 121. A circuit for calculating the conversion coefficient K may be realized using, for example, an ASCI or an FPGA, and provided in the imaging device 100. Alternatively, the conversion coefficient K may be calculated in advance for various imaging conditions and focus detection positions using equation (15), and stored in at least one of the ROM 153 and the nonvolatile memory in the interchangeable lens. The bias coefficient K stored in the nonvolatile memory in the interchangeable lens is acquired by the CPU 121 (lens information acquisition unit 121d) via the lens communication circuit 130.

[0153] 13, it can be seen that when the aperture value F is sufficiently large (the aperture opening is sufficiently small), the first pupil partial region 601 and the second pupil partial region 602 are hardly influenced by the first frame F1 and the second frame F2, and are essentially determined by the third frame F3. This means that when the aperture value F is sufficiently large, the influence of the first aperture parameter a1 and the second aperture parameter a2 on the conversion coefficient K becomes small.

[0154] Therefore, the calculation method of the conversion coefficient K described in this embodiment is particularly suitable when the aperture value F is small (the aperture opening is large). On the other hand, when the aperture value F is large (the aperture opening is small), the conversion coefficient may be obtained by a conventional method depending on the load on the CPU 121. For example, by calculating the conversion coefficient by the conventional method when the aperture value F is equal to or greater than a threshold, and by calculating the conversion coefficient by the method of this embodiment when the aperture value F is less than the threshold, it is possible to achieve both high-precision focus detection and reduced calculation load.

[0155] In this embodiment, a focus detection device that performs image plane phase-difference AF using focus detection signals obtained from a two-dimensional image sensor performs focus detection by taking into account at least one of pupil decentration and entrance pupil distance changes according to image height, which occur due to manufacturing errors, etc. Specifically, a focus detection signal is generated and a deviation amount is calculated by taking into account at least one of pupil decentration and entrance pupil distance at the focus detection position. Furthermore, a conversion coefficient that converts the deviation amount into a defocus amount is also calculated by taking into account at least one of pupil decentration and entrance pupil distance at the focus detection position.

[0156] This makes it possible to calculate with high accuracy a conversion coefficient suitable for the focus detection position for any combination of the image sensor actually installed and the interchangeable lens (imaging optical system) attached in an interchangeable lens imaging device, thereby achieving high focus detection accuracy.

[0157] ●(Second embodiment) Next, a second embodiment of the present invention will be described. In the first embodiment, the accuracy of the conversion coefficient K is improved by calculating the conversion coefficient K taking into account the pupil decentering amount dX and the entrance pupil distance error dZ, which change depending on the image height. In this embodiment, the accuracy of the conversion coefficient K is further improved by correcting the decrease in accuracy of the conversion coefficient K caused by differences in the shape of the pupil intensity distribution.

[0158] 19 shows the X-axis cross section of the pupil plane of a two-dimensional reference pupil intensity distribution that serves as a reference, such as design characteristics or average characteristics, and a one-dimensional correction pupil intensity distribution that is specific to the camera body of image capture device 100. The horizontal axis represents the angle of incidence [deg]. A two-dimensional pupil intensity distribution has a wider angular range and a larger amount of data than a one-dimensional pupil intensity distribution. Therefore, by calculating a reference conversion coefficient K in advance using the two-dimensional reference pupil intensity distribution and relatively correcting the reference conversion coefficient K based on the one-dimensional correction pupil intensity distribution that is specific to the camera body, it is possible to obtain a highly accurate conversion coefficient K that is tailored to the characteristics of each individual camera body.

[0159] In Figure 19(a), reference numeral 1801 denotes a pupil plane X-axis cross section of a two-dimensional reference pupil intensity distribution serving as a reference corresponding to first pupil intensity distribution 501 in Figure 4, and 1802 denotes a pupil plane X-axis cross section of a two-dimensional reference pupil intensity distribution serving as a reference corresponding to second pupil intensity distribution 502 in Figure 4. Meanwhile, reference numeral 1803 denotes a one-dimensional correction pupil intensity distribution specific to the camera body corresponding to first pupil intensity distribution 501 in Figure 4, and 1804 denotes a one-dimensional correction pupil intensity distribution specific to the camera body corresponding to second pupil intensity distribution 502 in Figure 4. The one-dimensional correction pupil intensity distribution specific to a camera body differs for each individual camera body (each camera body). Intersection 1805 denotes the intersection of pupil plane X-axis cross sections 1801 and 1802 of a pair of two-dimensional reference pupil intensity distributions, and the intersection of a pair of one-dimensional correction pupil intensity distributions 1803 and 1804.

[0160] 19(b), reference numeral 1806 denotes a corrected two-dimensional pupil intensity distribution obtained by scaling two-dimensional reference pupil intensity distribution 1801 of FIG. 19(a) using one-dimensional correction pupil intensity distribution 1803 so as to increase the degree of match in shape in the region near intersection 1805. Similarly, reference numeral 1807 denotes the X-axis cross section of the pupil plane of the corrected two-dimensional pupil intensity distribution obtained by scaling two-dimensional reference pupil intensity distribution 1802 of FIG. 19(a) using one-dimensional correction pupil intensity distribution 1804 so as to increase the degree of match in shape in the region near intersection 1805.

[0161] 20(a) to 20(c), a method for calculating a correction value for the conversion coefficient K in this embodiment will be described. In Fig. 20(a), a dotted line 1901 extending vertically indicates the aperture frame of the imaging optical system.

[0162] In FIG. 20(b), a dashed line 1902 indicates a line image derived by taking into account the influence of vignetting by the aperture frame 1901 on the pupil plane X-axis cross section 1801 of the two-dimensional reference pupil intensity distribution. A dashed line 1903 indicates a line image derived by taking into account the influence of vignetting by the aperture frame 1901 on the pupil plane X-axis cross section 1802 of the two-dimensional reference pupil intensity distribution. A solid line 1904 indicates a line image derived by taking into account the influence of vignetting by the aperture frame 1901 on the pupil plane X-axis cross section 1806 of the corrected two-dimensional pupil intensity distribution. A solid line 1905 (solid line) indicates a line image derived by taking into account the influence of vignetting by the aperture frame 1901 on the pupil plane X-axis cross section 1807 of the pupil intensity distribution after two-dimensional correction. Also, 1906 (BL1) indicates the base line length which is the difference in center of gravity between the line images 1902 and 1903, and 1907 (BL2) indicates the base line length which is the difference in center of gravity between the line images 1904 and 1905.

[0163] In this embodiment, a highly accurate correction value of the conversion coefficient K is calculated based on a comparison of these base line lengths BL1 and BL2. Specifically, the correction value Correstion_K of the conversion coefficient K is calculated by the following equation (16) using the base line lengths 1906 (BL1) and 1907 (BL2) shown in FIG. 20(b). Correction_K=BL1 / BL2 (16)

[0164] Since the conversion coefficient is the reciprocal of the baseline length, the correction value Correction_K to be applied to the conversion coefficient is calculated as the ratio of the baseline length 1906 (BL1) to the baseline length 1907 (BL2). When correcting the baseline length, the correction value is BL2 / BL1.

[0165] The correction value Correction_K of the conversion coefficient may be in a form other than equation (16) as long as it is derived based on a comparison between the baseline length information of the designed two-dimensional reference pupil intensity distribution and the baseline length information of the one-dimensional correction pupil intensity distribution, which is the solid-state information of the imaging device.

[0166] The accuracy of the conversion coefficients can be improved by applying the correction value Correction_K of the conversion coefficients derived by equation (16). Furthermore, when band-pass filtering is applied to the focus detection signals during focus detection, the accuracy of the conversion coefficients can be further improved by calculating a correction value that takes into account the spatial frequency band of the focus detection signals.

[0167] In FIG. 20(c), The dashed line 1908 is the result (absolute value) of applying bandpass filtering to the line image 1902. The dashed line 1909 is the result (absolute value) of applying band-pass filtering to the line image 1903. The solid line 1910 is the result of applying bandpass filtering to the line image 1904 (absolute value). The solid line 1911 is the result (absolute value) of applying band-pass filtering to the line image 1905. are shown respectively.

[0168] Also, 1912 (BL1') indicates the base line length, which is the difference in the centers of gravity of the line images (absolute values) after band-pass filtering, shown by dashed lines 1908 and 1909. Similarly, 1913 (BL2') indicates the base line length, which is the difference in the centers of gravity of the line images (absolute values) after band-pass filtering, shown by solid lines 1910 and 1911.

[0169] The CPU 121 (focus detection unit 121c) applies a band-pass filter similar to that applied to focus detection signals in focus detection processing to each of the line images 1902-1905, and derives post-filtering line images (absolute values) 1908-1911. The CPU 121 (focus detection unit 121c) then calculates a base line length 1912 (BL1'), which is the difference between the centers of gravity of the post-filtering line images (absolute values) 1908 and 1909. Similarly, the CPU 121 (focus detection unit 121c) calculates a base line length 1913 (BL2'), which is the difference between the centers of gravity of the post-filtering line images (absolute values) 1910 and 1911, and calculates a correction value Correstion_K for the conversion coefficient K using equation (17). Correction_K=BL1' / BL2' (17)

[0170] Equation (16) and equation (17) are essentially the same except for the difference in the processing of the line image to calculate the base line length. Therefore, even when band-pass filtering is applied to the line image, the correction value Correction_K of the conversion coefficient may be in a format other than equation (17).

[0171] The correction value Correction_K of the conversion coefficient may be calculated for each coordinate of the focus detection position, or may be calculated only for a plurality of discrete representative coordinates, and other coordinates may be calculated by interpolation or the like.

[0172] The shape of the curve in the region near intersection 1805 of first pupil intensity distribution 501 and second pupil intensity distribution 502 changes little depending on the coordinates on the image sensor 107. Therefore, the conversion coefficient correction value Correction_K also changes little depending on the coordinates on the image sensor 107. Using this characteristic, for example, the conversion coefficient correction value Correction_K may be calculated for one coordinate near the central coordinate (0,0) and used as a correction value common to all conversion coefficients.

[0173] The coordinates used to calculate the commonly used correction value Correction_K can be the central coordinate (0,0) or a coordinate close to it, which has the shortest maximum distance from the focus detection position. Note that the correction value Correction_K of the conversion coefficient may also be calculated using a method that does not use the baseline length. For example, the correction value Correction_K of the conversion coefficient can also be calculated based on the relative relationship between the reference two-dimensional reference pupil intensity distribution and the one-dimensional correction pupil intensity distribution specific to the camera body.

[0174] Note that the correction value Correction_K of the conversion coefficient may be stored in advance instead of being calculated by the CPU 121 (focus detection unit 121c). For example, the conversion coefficient K for each image height and the corresponding correction value Correction_K may be stored in advance in the ROM 135, and the conversion coefficient K and the correction value Correction_K of the conversion coefficient may be acquired according to the lens information and the image height lens information of the focus detection position.

[0175] The CPU 121 (focus detection unit 121c) calculates the final conversion coefficient K' used in the focus detection process from the conversion coefficient K calculated in the first embodiment and the correction value Correction_K using equation (18). K' = Correction_K·K (18)

[0176] The CPU 121 (focus detection unit 121c) uses the conversion coefficient K' calculated in this way to calculate the defocus amount in S106 of Fig. 14. That is, the CPU 121 (focus detection unit 121c) uses the image shift amount q calculated in S104. det The conversion coefficient K' obtained by correcting the conversion coefficient K obtained in S105 is applied to the defocus amount d det is calculated as shown in equation (19). d det =K' q det (19)

[0177] In this embodiment, the defocus amount d det Although the example in which is calculated for pixel signals output from the image sensor 107 has been described, it may also be calculated for pixel signals already recorded on the recording medium 133.

[0178] In this embodiment, the conversion coefficient K calculated using the two-dimensional reference pupil intensity distribution is corrected according to the difference in shape between the two-dimensional reference pupil intensity distribution that serves as the reference and the one-dimensional correction pupil intensity distribution that is specific to the image pickup device. This corrects the effect on the accuracy of the conversion coefficient that is caused by differences in the shape of the pupil intensity distribution due to, for example, variations in the height of microlenses of the image pickup element attached to the image pickup device, and makes it possible to calculate a conversion coefficient with higher accuracy. As a result, in addition to the effects of the first embodiment, further improvement in focus detection accuracy can be achieved.

[0179] ●(Third embodiment) Next, a third embodiment of the present invention will be described. This embodiment differs from the first and second embodiments in the configuration of the image sensor 107, specifically the method of dividing the sub-pixels. The other configurations are the same as those of the first and second embodiments, so a duplicated description will be omitted.

[0180] 21 and 22, in the image sensor 107 of this embodiment, the photoelectric conversion unit of each pixel is divided into two in the row direction as well as the column direction, and the image sensor 107 has first to fourth sub-pixels 201 to 204. An image signal (captured image) is generated by adding up the signals of the first to fourth sub-pixels 201 to 204 for each pixel.

[0181] Fig. 21 is a diagram showing the pixel (imaging pixel) arrangement of the image sensor 107. Fig. 22 is a diagram showing the pixel structure of the image sensor 107, with Fig. 22(a) showing a plan view of pixel 200G of the image sensor 107 (viewed from the +z direction), and Fig. 22(b) showing a cross-sectional view of line aa in Fig. 22(a) (viewed from the -y direction). In Figs. 21 and 22, components common to Figs. 2 and 3 are given the same reference numerals and descriptions thereof will be omitted.

[0182] FIG. 21 shows the pixel (unit pixel) array of the image sensor 107 in this embodiment in a range of 4 columns x 4 rows, and the sub-pixel array in a range of 8 columns x 8 rows. The 4 column by 4 row pixel group 200 is provided with color filters in a primary color Bayer array, as in Fig. 2. Furthermore, each pixel (unit pixel) has a photoelectric conversion region divided into two in the x direction and two in the y direction, and is composed of a plurality of sub-pixels, first to fourth sub-pixels 201 to 204.

[0183] The first to fourth subpixels 201 to 204 share one microlens. The first subpixel 201 receives a light beam that has passed through a first pupil partial region of the imaging optical system. The second subpixel 202 receives a light beam that has passed through a second pupil partial region of the imaging optical system. The third subpixel 203 receives a light beam that has passed through a third pupil partial region of the imaging optical system. The fourth subpixel 204 receives a light beam that has passed through a fourth pupil partial region of the imaging optical system.

[0184] In the image sensor 107 of this embodiment, for example, the pixel period P is 6 μm, and the number of pixels N is 6000 columns horizontally×4000 rows vertically=24 million pixels. Therefore, in the image sensor 107, the sub-pixel periods PSUB in the column and row directions are each 3 μm, and the number of sub-pixels NSUB is 12000 columns horizontally×8000 rows vertically=96 million pixels.

[0185] As shown in Fig. 22(b), pixel 200G of this embodiment is provided with a microlens 305 for focusing incident light on the light-receiving surface side of the pixel. A plurality of microlenses 305 are arranged two-dimensionally, and are disposed at positions a predetermined distance away from the light-receiving surface in the z-axis direction (direction of optical axis OA). Pixel 200G also has first photoelectric conversion units 301, 302, 303, and 304 formed therein, which are divided into N-H divisions (two divisions) in the x direction and N-V divisions (two divisions) in the y direction. The first photoelectric conversion units 301 to 304 correspond to subpixels 201 to 204, respectively.

[0186] In this embodiment, a first focus detection signal is generated based on pixel signals from the first sub-pixel 201 and the third sub-pixel 203 of each pixel of the image sensor 107, and a second focus detection signal is generated based on pixel signals from the second sub-pixel 202 and the fourth sub-pixel 204 of each pixel. In addition, by adding together the pixel signals from the first to fourth sub-pixels 201 to 204 of each pixel of the image sensor 107, an image signal (captured image) having a resolution of N effective pixels can be generated.

[0187] The first focus detection signal may be generated based on the pixel signals of the first subpixel 201 and the second subpixel 202, and the second focus detection signal may be generated based on the pixel signals of the third subpixel 203 and the fourth subpixel 204. Alternatively, the first focus detection signal may be generated based on the pixel signals of the first subpixel 201 and the fourth subpixel 204, and the second focus detection signal may be generated based on the pixel signals of the second subpixel 202 and the third subpixel 203.

[0188] Except for the fact that the pixel signals used to generate the first focus detection signal and the second focus detection signal are different, focus detection can be performed according to either the first or second embodiment. In addition to the effects of the first or second embodiment, this embodiment has the effect of being able to dynamically change the pupil division direction.

[0189] (Other embodiments) The imaging unit and imaging device described in each embodiment can be applied to various applications. For example, the imaging unit can be used to sense light other than visible light, such as infrared light, ultraviolet light, and X-rays. The imaging device is typically used in digital cameras, but can also be applied to camera-equipped mobile phones such as smartphones, surveillance cameras, game consoles, and the like. Furthermore, the imaging device can be applied to medical devices that perform endoscopy and vascular imaging, beauty devices that observe the skin and scalp, and video cameras that capture sports and action videos. Furthermore, the imaging unit and imaging device can also be applied to traffic cameras such as those used for traffic and ship monitoring and dashcams, academic cameras for astronomical observation and specimen observation, camera-equipped home appliances, machine vision, and the like. In particular, the machine vision technology can be used not only in robots used in factories, but also in agriculture and fishing.

[0190] Furthermore, the configuration of the imaging device shown in the above-described embodiment is merely an example, and imaging devices to which the present invention can be applied are not limited to the configuration shown in Fig. 1. Furthermore, the circuit configurations of the various parts of the imaging device are not limited to the configurations shown in the figures.

[0191] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0192] The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to clarify the scope of the invention. [Explanation of symbols]

[0193] 100...imaging device, 107...imaging element, 121...CPU, 121a...image signal acquisition unit, 121b...signal generation unit, 121c...focus detection unit, 121d...lens information acquisition unit, 130...lens communication circuit

Claims

1. a generating means for generating a first focus detection signal and a second focus detection signal based on a signal obtained from the image sensor; an image shift amount calculation unit that calculates an image shift amount between the first focus detection signal and the second focus detection signal; a coefficient calculation means for calculating a conversion coefficient for converting the image shift amount into a defocus amount of the imaging optical system; a detection unit that detects a defocus amount by applying the conversion coefficient to the image shift amount, The focus detection device according to claim 1, wherein the coefficient calculation means calculates the conversion coefficient based on at least one of an amount of pupil decentration corresponding to an image height at a focus detection position and an entrance pupil distance.

2. 2. The focus detection device according to claim 1, wherein the coefficient calculation means calculates the amount of pupil decentration at the focus detection position from amounts of pupil decentration stored in advance for a plurality of image heights, and uses the amount of pupil decentration at the focus detection position to calculate the conversion coefficient.

3. 3. The focus detection device according to claim 1, wherein the coefficient calculation means calculates the entrance pupil distance at the focus detection position from entrance pupil distances stored in advance for a plurality of image heights, and uses the calculated entrance pupil distance to calculate the conversion coefficient.

4. 4. The focus detection device according to claim 1, wherein the coefficient calculation means calculates the conversion coefficient based on aperture information of a plurality of frames of the imaging optical system, in addition to at least one of the amount of pupil decentration and the entrance pupil distance corresponding to the image height at the focus detection position.

5. 5. The focus detection device according to claim 1, wherein the coefficient calculation means calculates the conversion coefficient by dividing the virtual defocus amount by the image shift amount of the first focus detection signal and the second focus detection signal calculated by the image shift amount calculation means for the virtual defocus amount.

6. The focus detection device according to any one of claims 1 to 5, characterized in that the coefficient calculation means uses an aperture parameter obtained from information on a lens frame of the imaging optical system as a variable and calculates the conversion coefficient using the coefficient of a polynomial function that approximates the inverse of the conversion coefficient.

7. The focus detection device according to any one of claims 1 to 6, characterized in that, when the aperture value at the time of image capture is equal to or greater than a threshold value, the coefficient calculation means calculates a conversion coefficient that is not based on the amount of pupil decentration corresponding to the image height at the focus detection position and the entrance pupil distance.

8. the coefficient calculation means corrects the conversion coefficient based on a difference in shape between a reference two-dimensional pupil intensity distribution and a one-dimensional pupil intensity distribution specific to the focus detection device; the detection means detects the defocus amount by applying the corrected conversion coefficient to the image shift amount.

8. The focus detection device according to claim 1, wherein the focus detection device is a focus detection device having a focus detection function.

9. The focus detection device of claim 8, wherein the coefficient calculation means corrects the conversion coefficient using the ratio between the base length of a pair of line images obtained from a pair of the two-dimensional pupil intensity distributions corresponding to different partial regions of the exit pupil of the imaging optical system and the base length of a pair of line images obtained from a two-dimensional pupil intensity distribution obtained by correcting the pair of the two-dimensional pupil intensity distributions based on the corresponding one-dimensional pupil intensity distribution.

10. 10. The focus detection device according to claim 1, wherein the pixel of the image sensor has a plurality of sub-pixels, and the generation means generates the first focus detection signal and the second focus detection signal based on signals obtained from different sub-pixels.

11. An imaging element; a focus detection device according to any one of claims 1 to 10; an imaging apparatus that adjusts the focus of an imaging optical system based on the amount of defocus detected by the focus detection device;

12. A focus detection method executed by a focus detection device, comprising: a generating step of generating a first focus detection signal and a second focus detection signal based on a signal obtained from the image sensor; an image shift amount calculation step of calculating an image shift amount between the first focus detection signal and the second focus detection signal; a coefficient calculation step of calculating a conversion coefficient for converting the image shift amount into a defocus amount of the imaging optical system; a detection step of detecting a defocus amount by applying the conversion coefficient to the image shift amount, a focus detection method, wherein the coefficient calculation step calculates a conversion coefficient based on at least one of an amount of pupil decentration corresponding to an image height at a focus detection position and an entrance pupil distance;

13. A program for causing a computer to function as each of the means included in the focus detection device according to any one of claims 1 to 10.

Images