Focus detection device and method, electronic device, program, and storage medium

Abstract

To enable focus detection signals having parallax in multiple different directions to be processed using a single arithmetic circuit for focus detection.SOLUTION: A focus detection device includes: acquisition means that acquires, for each pixel, a pair of focus detection signals having parallax in one of multiple different directions from signals of multiple pixels output by imaging means; separation means that separates the focus detection signals for each direction of the parallax; detection means that detects a focus state on the basis of a first focus detection signal which is separated by the separation means and in which the direction of the parallax is a predetermined first direction; and rearrangement means that rearranges a sequence of second focus detection signals which are separated by the separation means and in which the direction of the parallax is a second direction different from the first direction so that the direction of the parallax becomes the first direction. The detection means further detects the focus state on the basis of the second focus detection signals rearranged by the rearrangement means.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a focus detection device and method, an electronic device, a program, and a storage medium. [Background technology]

[0002] One focus detection method used in imaging devices is a so-called image-plane phase-difference method, in which at least a portion of the pixels of an image sensor are configured as focus detection pixels capable of acquiring signals with phase differences, thereby performing phase-difference focus detection. As an example of focus detection pixels for such image-plane phase-difference focus detection (hereinafter referred to as "image-plane phase-difference AF"), Patent Document 1 discloses an imaging device using an image sensor in which each pixel is provided with one microlens and a photoelectric conversion unit divided into multiple sections, and multiple pixels having this configuration are arranged two-dimensionally. The multiple photoelectric conversion units are configured to receive light transmitted through different exit pupil regions of the imaging lens via a single microlens, thereby performing pupil division. Image-plane phase-difference AF can be performed by calculating the amount of image shift based on the phase difference between signals obtained from the multiple photoelectric conversion units. Furthermore, an image can be acquired from a sum signal obtained by adding together signals from the multiple photoelectric conversion units for each pixel.

[0003] In such an image sensor, in a configuration in which multiple photoelectric conversion units are arranged horizontally within a pixel and the pupil division direction is horizontal, if the subject has no horizontal brightness change, such as horizontal stripes, there is little change in the phase difference between signals, which can result in reduced focus detection accuracy.

[0004] Therefore, Patent Document 2 discloses a technology for improving focus detection accuracy by providing two types of arrangement directions for the photoelectric conversion units of focus detection pixels and two types of pupil division directions. Furthermore, Patent Document 3 discloses a method for performing focus detection calculations for signals having different pupil division directions. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 58-24105 [Patent Document 2] Patent No. 5810196 [Patent Document 3] U.S. Patent No. 9,485,442 Summary of the Invention [Problem to be solved by the invention]

[0006] Here, a focus detection arithmetic circuit configured on the premise of processing a pair of focus detection signals obtained from a photoelectric conversion unit divided in one pupil division direction (for example, the horizontal direction) cannot process a pair of focus detection signals obtained from a photoelectric conversion unit divided in a different pupil division direction (for example, the vertical direction).

[0007] The present invention has been made in consideration of the above problems, and aims to enable processing of focus detection signals having parallax in multiple different directions using a single focus detection arithmetic circuit. [Means for solving the problem]

[0008] In order to achieve the above object, the focus detection device of the present invention comprises: an image sensor capable of acquiring a first pair of focus detection signals having a parallax in a first direction and a second pair of focus detection signals having a parallax in a second direction different from the first direction from within an image pickup surface that captures an object image formed by an optical system, the image sensor having a first pair of photoelectric conversion units pupil-divided in the first direction and a second pair of photoelectric conversion units pupil-divided in the second direction different from the first direction within an image pickup surface that captures an object image formed by an optical system; For each direction of the parallax, Read out from the image sensor a separation means for separating the focus detection signals; Record number 1 versus focus detection signal Calculate the correlation value of , detect the focus state focus a detecting means and the separating means No. 2 versus a rearrangement unit that rearranges the arrangement of the focus detection signals so that the direction of the parallax is the first direction, focus The detection means further detects the second versus focus detection signal Calculate the correlation value of , detect focus state and reading and outputting signals in a first readout mode in which signals are read out from the image sensor without being thinned out in the vertical direction to detect focus states in the first direction and the second direction, and in a second readout mode in which signals are read out from the image sensor with being thinned out in the vertical direction to not detect focus states in the second direction, and the separating means separates focus detection signals for each of the directions of parallax when the signals are read out in the first readout mode. do. [Effects of the Invention]

[0009] According to the present invention, it is possible to process focus detection signals having parallax in a plurality of different directions using a single focus detection arithmetic circuit. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing an example of a pixel array according to the first embodiment. [Figure 3] 1A and 1B are a schematic plan view and a schematic cross-sectional view of a pixel according to a first embodiment. [Figure 4] FIG. 2 is a schematic explanatory diagram of a pixel structure and pupil division according to the first embodiment. [Figure 5] FIG. 3 is a schematic diagram illustrating pupil division of an exit pupil according to the first embodiment. [Figure 6] 5A and 5B are diagrams showing the relationship between the defocus amount and the image shift amount in the first embodiment. [Figure 7] FIG. 1 is a block diagram showing the configuration of an image processing circuit according to a first embodiment. [Figure 8] FIG. 3 is a conceptual diagram of a focus detection signal according to the first embodiment. [Figure 9] FIG. 4 is a conceptual diagram of a line interpolation process according to the first embodiment. [Figure 10] FIG. 4 is a conceptual diagram of a reallocation process according to the first embodiment. [Figure 11] 10A and 10B are diagrams showing an example of processing in which the array of vertical focus detection signals is transposed to match the horizontal focus detection signal array with the pupil division direction. [Figure 12] 5 is a flowchart showing focus detection processing in the first embodiment. [Figure 13] 10 is a flowchart showing focus detection processing in the second embodiment. [Figure 14] FIG. 10 is a schematic diagram showing an example of a pixel array according to a third embodiment. [Figure 15] FIG. 10 is a conceptual diagram of a focus detection signal generated in the third embodiment. [Figure 16] 10 is a conceptual diagram of a reallocation process according to the third embodiment. [Figure 17] FIG. 10 is a block diagram showing the configuration of an image processing circuit according to a third embodiment. [Figure 18] 10 is a flowchart showing focus detection processing in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment [Overall configuration] 1 is a block diagram showing the schematic configuration of a single-lens reflex digital camera with interchangeable lenses as an imaging device having an image sensor according to an embodiment of the present invention. While this embodiment has a specific and specific configuration to facilitate understanding and explanation of the invention, the present invention is not limited to such a specific configuration. Furthermore, the present invention can also be implemented in, for example, a digital camera with an integrated lens, a video camera, or any electronic device equipped with a camera, such as a mobile phone, a personal computer (laptop, tablet, desktop, etc.), or a game console.

[0013] In Figure 1, a first lens group 101 is disposed at the tip of the imaging optical system and is held so as to be movable back and forth along the optical axis. Aperture / shutter (aperture) 102 adjusts its aperture diameter to adjust the amount of light during photography, and also functions as a shutter for adjusting the exposure time during still photography. A second lens group 103 moves back and forth along the optical axis together with aperture / shutter 102, and can achieve a variable magnification effect (zoom function) in conjunction with the movement of the first lens group 101 forward and backward.

[0014] The third lens group 105 (focus lens) adjusts the focus by moving forward and backward along the optical axis. The optical low-pass filter 106 is an optical element that reduces false colors and moiré in the captured image. The image sensor 107 consists of a two-dimensional CMOS photosensor and its peripheral circuitry, and is placed on the imaging plane of the imaging optical system.

[0015] The zoom actuator 111 rotates a cam barrel (not shown) to drive the first lens group 101 to the second lens group 103 forward and backward in the optical axis direction, thereby performing a magnification change operation. The aperture / shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the amount of light for shooting, and also controls the exposure time when shooting a still image. The focus actuator 114 drives the third lens group 105 forward and backward in the optical axis direction, thereby performing focus adjustment.

[0016] The electronic flash 115 used to illuminate the subject during photography is preferably a flashlight device using a xenon tube, but a lighting device equipped with a continuous-light LED may also be used. The AF assist light emitter 116 projects an image of a mask with a predetermined aperture pattern onto the subject field via a projection lens, improving focus detection performance for dark or low-contrast subjects.

[0017] The camera's internal CPU 121 controls various aspects of the camera body and includes a calculation unit, ROM, RAM, A / D converter, D / A converter, communication interface circuit, etc. Based on a predetermined program stored in the ROM, the CPU 121 drives the various circuits of the camera and executes a series of operations such as AF, shooting, image processing and recording.

[0018] An electronic flash control circuit 122 controls the lighting of the electronic flash 115 in synchronization with the shooting operation. An assist light drive circuit 123 controls the lighting of the AF assist light emitter 116 in synchronization with the focus detection operation. An image sensor drive circuit 124 controls the imaging operation of the image sensor 107, and also A / D converts signals read from the image sensor 107 and sends them to the CPU 121. An image processing circuit 125 performs processes such as gamma conversion, color interpolation, and JPEG compression on image signals obtained based on signals read from the image sensor 107.

[0019] A focus driving circuit 126 drives and controls the focus actuator 114 based on the correction value calculated by the CPU 121, and drives the third lens group 105 forward and backward in the optical axis direction to perform focus adjustment. An aperture shutter driving circuit 128 drives and controls the aperture shutter actuator 112 to control the opening of the aperture / shutter 102. A zoom driving circuit 129 drives the zoom actuator 111 in response to the zoom operation by the photographer.

[0020] A display 131 such as an LCD displays information about the camera's shooting mode, a preview image before shooting and a confirmation image after shooting, an in-focus state display image when focus is detected, etc. The operation switch group 132 is made up of a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, etc. A removable flash memory 133 records captured images.

[0021] [Image sensor] Next, an outline of the arrangement of imaging pixels and focus detection pixels of the image sensor 107 in the first embodiment is shown in Fig. 2. Fig. 2 shows the pixel (imaging pixel) arrangement of a two-dimensional CMOS sensor as the image sensor 107 in the first embodiment, in an area of ​​4 columns x 4 rows.

[0022] The pixel group 200 is composed of pixels arranged in 2 columns and 2 rows, with pixel 200R having R (red) spectral sensitivity arranged in the upper left, pixel 200Ga having G (green) spectral sensitivity arranged in the upper right, pixel 200Gb, which is pixel 200Ga rotated 90 degrees, arranged in the lower left, and pixel 200B having B (blue) spectral sensitivity arranged in the lower right. Furthermore, pixels 200R, 200Ga, and 200B are each composed of first focus detection pixels 201 and second focus detection pixels 202 arranged in 2 columns and 1 row, and pixel 200Gb is composed of first focus detection pixels 201 and second focus detection pixels 202 arranged in 1 column and 2 rows.

[0023] By arranging a large number of pixel groups 200 of 2 columns x 2 rows shown in FIG. 2 on a surface, it is possible to acquire a captured image and a focus detection signal.

[0024] 3(a) shows a plan view of one pixel 200Ga of the image sensor 107 shown in FIG. 2, as seen from the light receiving surface side (+z side) of the image sensor 107, and FIG. 3(b) shows a cross-sectional view of the aa cross section of FIG. 3(a) as seen from the -y side. As shown in FIG. 3, in the pixel 200Ga of this embodiment, a microlens 305 for focusing incident light is formed on the light receiving side of each pixel, and a photoelectric conversion unit 301 and a photoelectric conversion unit 302 are formed that are N-H divided (divided into two) in the x direction and N-V divided (divided into one) in the y direction. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 are included in the first focus detection pixel 201 and the second focus detection pixel 202, respectively.

[0025] The photoelectric conversion units 301 and 302 may each be a pin structure photodiode with an intrinsic layer sandwiched between a p-type layer and an n-type layer, or, if necessary, the intrinsic layer may be omitted and a pn junction photodiode may be used.

[0026] In the pixel 200Ga, a color filter 306 is provided between the microlens 305 and the photoelectric conversion unit 301 and the photoelectric conversion unit 302. Furthermore, the spectral transmittance of the color filter 306 can be changed for each pixel or each photoelectric conversion unit as necessary. Furthermore, the color filter may be omitted.

[0027] 3 is collected by a microlens 305, dispersed by a color filter 306, and then received by a photoelectric conversion unit 301 and a photoelectric conversion unit 302. In the photoelectric conversion unit 301 and the photoelectric conversion unit 302, pairs of electrons and holes are generated according to the amount of received light, and after being separated by a depletion layer, the negatively charged electrons are accumulated in an n-type layer (not shown), while the holes are discharged to the outside of the image sensor 107 through a p-type layer connected to a constant voltage source (not shown). The electrons accumulated in the n-type layers (not shown) of the photoelectric conversion unit 301 and the photoelectric conversion unit 302 are transferred to a capacitance unit (FD) via a transfer gate, converted into a voltage signal, and output.

[0028] 2 have the same configuration as pixel 200Ga in Fig. 3, and like pixel 200Ga, output voltage signals corresponding to light separated into each color by color filter 306. Pixel 200Gb shown in Fig. 2 has a configuration obtained by rotating pixel 200Ga in Fig. 3 by 90 degrees, and like pixel 200Ga, output voltage signals corresponding to light separated into each color by color filter 306.

[0029] The correspondence between the pixel structure and pupil division in this embodiment shown in Fig. 3 will be described with reference to Fig. 4. Note that pixels 200R, 200Ga, and 200B have the same configuration, and pixel 200Gb has a configuration obtained by rotating pixel 200Ga by 90 degrees, so the following description will be made using pixel 200Ga as a representative.

[0030] 4 shows a cross-sectional view of the aa cross section of pixel 200Ga shown in FIG. 3 as seen from the +y side, and also shows the pupil plane (exit pupil distance Ds) at a position a distance Z in the z-axis direction (direction of the optical axis) from the imaging surface of image sensor 107. Note that in FIG. 4, the x-axis and y-axis of the cross-sectional view are reversed compared to FIG. 3 in order to correspond to the coordinate axes of the pupil plane.

[0031] The first pupil partial region 501 is in a generally conjugate relationship with the light receiving surface of the photoelectric conversion unit 301, whose center of gravity is decentered in the -x direction, via the microlens 305, and represents the pupil region through which light that can be received by the photoelectric conversion unit 301 passes. The center of gravity of the first pupil partial region 501 is decentered in the +X direction on the pupil plane.

[0032] Furthermore, the second pupil partial region 502 is in a generally conjugate relationship with the light receiving surface of the photoelectric conversion unit 302, whose center of gravity is decentered in the +x direction, via the microlens 305, and represents the pupil region through which light that can be received by the photoelectric conversion unit 302 passes. The center of gravity of the second pupil partial region 502 is decentered in the -X direction on the pupil plane.

[0033] The pupil region 500 is a pupil region through which light that can be received by the entire pixel 200Ga when the photoelectric conversion units 301 and 302 are all combined passes.

[0034] 5 is a schematic diagram showing the correspondence between the image sensor 107 and pupil division in this embodiment. A pair of light beams that pass through different pupil partial regions of the imaging optical system, the first pupil partial region 501 and the second pupil partial region 502, are incident on each pixel of the image sensor 107 at different angles corresponding to the image height of each pixel, and are received by the 2×1 divided photoelectric conversion unit 301 and the photoelectric conversion unit 302. Note that while FIG. 5 shows the correspondence between the pupil region divided into two horizontally and the horizontally divided photoelectric conversion units 301 and 302 of pixel 200Ga, pixel 200R, and pixel 200B, the pupil region divided into two vertically and the vertically divided photoelectric conversion units 301 and 302 of pixel 200Gb also have a similar correspondence, although rotated by 90 degrees.

[0035] In the following description, pixels having photoelectric conversion units 301 and 302 divided in the horizontal direction, such as pixel 200Ga, pixel 200R, and pixel 200B, are referred to as "horizontally divided pixels." Furthermore, pixels having photoelectric conversion units 301 and 302 divided in the vertical direction, such as pixel 200Gb, are referred to as "vertically divided pixels."

[0036] 4 and 5, the photoelectric conversion unit 301 of each pixel receives a light beam that passes through a first pupil partial region 501 of the imaging optical system, and the photoelectric conversion unit 302 of each pixel receives a light beam that passes through a second pupil partial region 502 of the imaging optical system that is different from the first pupil partial region 501. Furthermore, when the photoelectric conversion unit 301 and the photoelectric conversion unit 302 of each pixel are combined, they receive a light beam that passes through a pupil region that is the combined pupil region of the first pupil partial region 501 and the second pupil partial region 502 of the imaging optical system.

[0037] Note that, although the image sensor 107 of this embodiment is configured with first focus detection pixels 201, each including a photoelectric conversion unit 301, and second focus detection pixels 202, each including a photoelectric conversion unit 302, the present invention is not limited to this. For example, an image pickup pixel having an undivided photoelectric conversion unit, the first focus detection pixel, and the second focus detection pixel may each be configured as separate pixels, and the first focus detection pixel and the second focus detection pixel may be discretely arranged in part of the array of image pickup pixels.

[0038] 2 to 5, a signal (signal A+B) obtained by adding together the signals obtained from the first focus detection pixel 201 and the second focus detection pixel 202 is used as an imaging signal, and the signals (signal A, signal B) from the first focus detection pixel 201 and the second focus detection pixel 202 are used as a focus detection signal pair. Note that the imaging signal and focus detection signal may be read out separately, but in consideration of the processing load, they may also be obtained as follows: That is, the imaging signal (signal A+B) and the focus detection signal (for example, signal A) from either the first focus detection pixel 201 or the second focus detection pixel 202 are read out, and the difference is taken to obtain the other focus detection signal (signal B) having parallax.

[0039] Then, a signal (hereinafter referred to as a "horizontal A image") is generated by collecting the A signals (hereinafter referred to as a "horizontal A signal") of each horizontally divided pixel, and a signal (hereinafter referred to as a "horizontal B image") is generated by collecting the B signals (hereinafter referred to as a "horizontal B signal") of each horizontally divided pixel. Focus detection is then performed based on the phase difference between the generated horizontal A image and horizontal B image. Also, a signal (hereinafter referred to as a "vertical A image") is generated by collecting the A signals (hereinafter referred to as a "vertical A signal") of each vertically divided pixel, and a signal (hereinafter referred to as a "vertical B image") is generated by collecting the B signals (hereinafter referred to as a "vertical B signal") of each vertically divided pixel, and focus detection is performed based on the phase difference between the generated vertical A image and vertical B image. In the following explanation, when describing processing common to both the horizontal and vertical directions, the terms "A signal" and "B signal" and "A image" and "B image" are simply used.

[0040] [Relationship between defocus amount and image shift amount] Next, the relationship between the defocus amount and the amount of image shift between images A and B acquired by the image sensor 107 of this embodiment will be described. Fig. 6 is a diagram showing the relationship between the defocus amount and the amount of image shift. The image sensor 107 of this embodiment is arranged on an imaging plane 800, and similar to Figs. 4 and 5, the pupil region 500 of the imaging optical system is divided into two, a first pupil partial region 501 and a second pupil partial region 502, in the horizontal or vertical direction.

[0041] The defocus amount d is defined as the distance from the subject's imaging position to the imaging plane 800, where |d| is the magnitude. A front-focus state in which the subject's imaging position is on the subject side of the imaging plane is negative (d<0), and a back-focus state in which the subject is on the opposite side of the imaging plane 800 is positive (d>0). The in-focus state in which the subject's imaging position is on the imaging plane 800 (focus position) is d=0. In FIG. 6, subject 801 shows an example of an in-focus state (d=0), and subject 802 shows an example of a front-focus state (d<0). The front-focus state (d<0) and back-focus state (d>0) are collectively referred to as the defocus state (|d|>0).

[0042] In a front-focus state (d<0), the subject light that passes through the first pupil partial region 501 from the light beam of the subject 802 is first collected and then spreads to a width Γ1 with the center of gravity G1 of the light beam as the center, forming a blurred image on the imaging surface 800. The same is true for the subject light that passes through the second pupil partial region 502, forming a blurred image that spreads to a width Γ2 with the center of gravity G2 as the center. The blurred image is received by the first focus detection pixel 201 and the second focus detection pixel 202 that constitute the pixels arranged on the image sensor 107, and images A and B are generated from the obtained light reception signals. Therefore, images A and B are recorded at the centers of gravity G1 and G2 on the imaging surface 800 as subject images in which the subject 802 is blurred to widths Γ1 and Γ2.

[0043] The blur widths Γ1 and Γ2 of the subject images increase roughly in proportion to the increase in the magnitude |d| of the defocus amount d. Similarly, the magnitude |p| of the image shift amount p (= the difference G1 - G2 in the center of gravity positions of the light beams) between image A and image B also increases roughly in proportion to the increase in the magnitude |d| of the defocus amount d. The same is true in the back-focus state (d>0), although the direction of the image shift of the subject image between the first focus detection signal and the second focus detection signal is opposite to that in the front-focus state.

[0044] In this way, the image shift amount p calculated from the images A and B is roughly proportional to the defocus amount d. Therefore, by using a conversion coefficient calculated in advance based on the base line length, the defocus amount d can be calculated from the image shift amount p.

[0045] [Image processing circuit] Next, the configuration of the image processing circuit 125 in this embodiment will be described with reference to Fig. 7. Here, the configuration for performing processing related to correlation calculations for focus detection, among the functions of the image processing circuit 125, will be described.

[0046] First, the A signal and the B signal obtained by reading out the signal from the image sensor 107 are stored in the memory 1251 in the image processing circuit 125 . Fig. 8(a) is a conceptual diagram showing signals obtained when the pixels shown in Fig. 2 are arranged in m columns x n rows. As described above, because horizontally divided pixels and vertically divided pixels are arranged in a mixed manner, horizontal A signal 601 and horizontal B signal 602, and vertical A signal 603 and vertical B signal 604, which are divided in different directions, are obtained. The A signal and the B signal may be stored in a memory (not shown) that is normally provided in the imaging device, and may be read out by the image processing circuit 125 as needed.

[0047] Next, the separation circuit 1252 separates the first and second focus detection signals arranged as shown in FIG. 8( a ) into a horizontal A signal 601 and a horizontal B signal 602 , and a vertical A signal 603 and a vertical B signal 604 .

[0048] Fig. 8(b) is a conceptual diagram in which the horizontal A signals 601 and horizontal B signals 602 of the signals shown in Fig. 8(a) are separated, and Fig. 8(c) is a conceptual diagram in which the vertical A signals 603 and vertical B signals 604 are separated. In the case of the image sensor 107 of this embodiment having the above-mentioned configuration, the number of vertical A signals 603 and vertical B signals 604 is half the number of horizontal A signals 601 and horizontal B signals 602 in both the row and column directions.

[0049] 8B, the pixel values ​​(shaded pixels) of the addresses where the vertical A signal 603 and vertical B signal 604 were stored are interpolated from the pixel values ​​of adjacent pixels. For example, in the pixel array shown in FIG. 2, the pixel values ​​of the addresses where the vertical A signal 603 and vertical B signal 604 from pixel 200Gb were stored are replaced with the horizontal A signal 601 and horizontal B signal 602 from pixel 200Ga. Alternatively, in different color mixing processing described later, the values ​​of the horizontal A signal 601 and horizontal B signal 602 from pixel 200Ga may be doubled and saved as a Y signal obtained by adding the signals of each color.

[0050] Then, the horizontal A signal 601 and the horizontal B signal 602 are output to a shading correction circuit 1253 , and the vertical A signal 603 and the vertical B signal 604 are output to a shading correction circuit 1255 .

[0051] The shading correction circuit 1253 performs shading correction on the input horizontal A signal 601 and horizontal B signal 602 .

[0052] When the exit pupil distance and the pupil distance of the image sensor differ, a difference in signal amount between the A signal and the B signal depending on the image height is called shading. It is known that performing shading correction and reducing the difference between the A signal and the B signal enables good focus detection. Note that in this embodiment, a known method such as the method described in Japanese Patent Application Laid-Open No. 2016-57474 can be used as shading correction, and therefore a detailed description thereof will be omitted.

[0053] The different color mixing processing circuit 1254 performs different color mixing processing on the shading-corrected horizontal A signal 601 and horizontal B signal 602. In this embodiment, smoothing processing in the correlation direction (horizontal direction) and smoothing processing in the vertical direction relative to the correlation direction are performed. Na First, smoothing the correlation direction (for example, using an 11 filter or a 121 filter) mixes the colors in the correlation direction, and then the colors perpendicular to the correlation direction are mixed. Na Adding two or more pixels in the Y direction blends the colors vertically, A Signal, horizontal Y B This process leaves the horizontal number of pixels in the horizontally divided pixel array unchanged, but reduces the vertical number of pixels to half or less.

[0054] Horizontal Y generated by the different color mixing processing circuit 1254 A Signal and horizontal Y B The signal is sent to a correlation calculation circuit 1258. The correlation calculation circuit 1258 calculates the horizontal Y A Signal and horizontal Y BA horizontal image A and a horizontal image B are generated from the signal, and a correlation value is calculated using a known method while shifting the relative positions of the horizontal images A and B. The amount of shift when the correlation value showing the highest correlation is obtained is then taken as the image shift amount, and the calculated image shift amount is converted into a defocus amount using a conversion coefficient. The image shift amount and defocus amount calculated here indicate the focus state. Then, the CPU 121 calculates a correction value for driving the focus actuator 114 based on the obtained defocus amount, and sends the correction value to the focus drive circuit 126 .

[0055] Meanwhile, the separated vertical A signal 603 and vertical B signal 604 are subjected to shading correction in a shading correction circuit 1255 in the same manner as in the shading correction circuit 1253. The shading-corrected vertical A signal 603 and vertical B signal 604 are sent to a row interpolation circuit 1256.

[0056] The row interpolation circuit 1256 performs interpolation processing in the column direction, and sends the interpolated vertical A signal 603 and vertical B signal 604 to the rearrangement processing circuit 1257 . In the first embodiment, the vertical A signal and vertical B signal are only signals output from pixel 200Gb, which is a vertically divided pixel, and in the vertical direction, which is the correlation direction, the signal of the row of pixel 200R is missing, which raises concerns about a decrease in detection accuracy. Therefore, in order to improve focus detection accuracy in the vertical direction, it is necessary to align the pixel spacing of the focus detection signals with the pixel spacing in the horizontal direction. Therefore, the vertical A signal and vertical B signal corresponding to the position of the R pixel are output from the adjacent G b The vertical A signal and vertical B signal of the pixel are interpolated in the vertical direction.

[0057] This interpolation method is shown in Figure 9. After separating the 200 Gb pixel signal as shown in Figure 8(c), zeros are inserted between the 200 Gb pixels (the positions of the signal from pixel 200R before the 200 Gb pixel signal was separated) as shown in Figure 9(a). Then, as shown in Figure 9(b), the rows of zeros are interpolated with the average value of the 200 Gb signals from the adjacent pixels above and below.

[0058] Note that since the vertical A signal and the vertical B signal are only green (G) signals, there is no need to generate a Y signal. However, if the vertical A signal and the vertical B signal are output from pixels of multiple colors due to the color filter arrangement, a Y signal is generated.

[0059] The vertical A signal and vertical B signal that have undergone row interpolation are sent to a rearrangement processing circuit 1257, where they are rearranged.

[0060] Although it is sufficient to perform array rearrangement processing so that the pupil division direction matches for either the horizontal A signal and horizontal B signal or the vertical A signal and vertical B signal, it is more efficient to perform array rearrangement processing on the focus detection signal array with a smaller number of arrays in the pupil division direction, and therefore in this embodiment, array rearrangement processing is performed on the vertical A signal and vertical B signal. Because the pupil division direction is orthogonal to the horizontal and vertical directions, transposition processing is performed as the rearrangement processing.

[0061] FIG. 10 is a diagram for explaining the transposition process. FIG. 10(a) shows the concept of arrangement before transposition, and FIG. 10(b) shows the concept of arrangement after transposition. The transposition process is a process in which the signal a of the address (i, j) is ij to address (j, i) and rearrange the data. The transposition rearrangement process is not limited to the transposition of an array, but may also be a combination of rotation and inversion of the array.

[0062] 11 shows an example in which the arrangement of vertical focus detection signals is transposed to match the horizontal focus detection signal arrangement with the pupil division direction for processing purposes. A Signal and horizontal Y B 11(b) shows the vertical A signal 603 and vertical B signal 604 before transposition, including signals row-interpolated by the row interpolation circuit 1256, and FIG. 11(c) shows the vertical A signal 603 and vertical B signal 604 after transposition. [Focus detection processing] 12 is a flowchart showing the focus detection process in the first embodiment. The process shown in FIG.

[0063] First, in S101, the A signal output from the first focus detection pixel 201 and the B signal output from the second focus detection pixel 202 of the image sensor 107 are acquired and stored in the memory 1251 of the image processing circuit 125. Next, in S102, the A and B signals stored in the memory 1251 are separated by the separation circuit 1252 into horizontal A and B signals and vertical A and B signals.

[0064] Next, in S103, the horizontal A signal and horizontal B signal are output to a shading correction circuit 1253, and the vertical A signal and vertical B signal are output to a shading correction circuit 1255, where they are subjected to shading correction processing in S104 and S106, respectively.

[0065] Then, in S105, the different color mixing circuit 1254 performs the above-described different color mixing process to reduce the data amount of the horizontal A signal and the horizontal B signal. A Signal and horizontal Y B However, if the detected signal is only a single color signal, this process is not necessary. Also, if a column of a specific color signal is thinned out during readout, the thinned column signal may be interpolated using the average value of the left and right columns to improve detection accuracy in the horizontal direction. After the different color mixing process, proceed to S109.

[0066] Meanwhile, in S107, the row interpolation circuit 1256 performs row interpolation processing on the vertical A signal and the vertical B signal as described with reference to Fig. 9. Note that if rows are not thinned out when reading out the focus detection signals in the vertical direction (for example, if the division direction of the pixels 200R, 200Gb, and 200B is vertical), a circuit equivalent to the different color mixing processing circuit 1254 may be provided instead of the row interpolation circuit 1256 to generate the Y signal.

[0067] Thereafter, in S108, the rearrangement processing circuit 1257 performs the rearrangement processing of the array on the interpolated vertical A signal and vertical B signal as described with reference to Fig. 11. By the processing of S108, in the subsequent processing, the horizontal YA Signal and horizontal Y B This makes it possible to process the vertical A signal and the vertical B signal using the same processing circuit.

[0068] In S109, in order to increase the correlation (degree of signal coincidence) and improve focus detection accuracy, the correlation calculation circuit 1258 performs band-pass filtering with a specific pass frequency band on the A and B signals. Examples of band-pass filters include differential filters such as {1, 4, 4, 4, 0, -4, -4, -4, -1} that cut DC components and perform edge extraction, and additive filters such as {1, 2, 1} that suppress high-frequency noise components.

[0069] Next, in S109, a shift process is performed to relatively shift the first focus detection signal and the second focus detection signal after the filter process in the pupil division direction, and a correlation amount indicating the degree of coincidence of the signals is calculated.

[0070] The kth A signal after filtering is A(k), the B signal is B(k), and the range of number k corresponding to the focus detection area is W , Shi When the shift amount by the soft processing is s and the shift range of the shift amount s is Γ, the correlation amount COR is calculated by the following equation (1).

[0071] TIFF0007781116000001.tif10103By shifting the shift amount s, the kth A signal (k) is matched to the ksth B signal (ks) and subtracted to generate a shift subtraction signal.The absolute value of the generated shift subtraction signal is calculated, and the sum of the number k is taken within the range W corresponding to the focus detection area to calculate the correlation amount COR(s).If necessary, the correlation amount calculated for each row can be added across multiple rows for each shift amount.

[0072] Then, in S110, the amount of shift of the real value that minimizes the amount of correlation is calculated from the amount of correlation by sub-pixel calculation, and the amount of image shift p is obtained. The amount of image shift p is then multiplied by a conversion coefficient K to obtain the amount of defocus d.

[0073] In this way, by performing an array rearrangement process on the focus detection signals output from the vertically divided pixels, it is possible to use the same circuit to perform processes from filtering to calculation of the defocus amount for focus detection signals that have been pupil-divided in the horizontal and vertical directions.

[0074] As described above, according to the first embodiment, it is possible to process focus detection signals divided in different pupil directions using a common circuit, thereby making it possible to reduce the circuit scale.

[0075] <Second embodiment> Next, a second embodiment of the present invention will be described. In a read mode in which rows are thinned and read out from the image sensor 107 (for example, an image for flicker detection), if focus detection is to be performed using focus detection signals output from vertically divided pixels, there is a concern that focus detection performance may be reduced because image shift in the thinned signals cannot be detected.

[0076] Therefore, in the second embodiment, when focus detection is performed while switching between multiple readout modes, vertical focus detection is performed in a readout mode that reads without vertical thinning, and vertical focus detection is not performed in a readout mode that reads signals by thinning them vertically.

[0077] 1 to 4 and 7 in the first embodiment, so a description thereof will be omitted. However, in the second embodiment, it is assumed that signals can be read from the image sensor 107 under the control of the image sensor drive circuit 124 in an all-pixel readout mode in which signals are read without thinning out, and in a thinning-out readout mode in which signals are read out by thinning out rows.

[0078] [Focus detection processing] Fig. 13 is a flowchart showing focus detection processing in the second embodiment. The processing shown in Fig. 13 is executed by the image sensor 107, image processing circuit 125, and CPU 121. Of the processing shown in Fig. 13, the same steps as those in Fig. 12 described in the first embodiment are assigned the same step numbers, and their description will be omitted.

[0079] First, in S201, it is determined whether or not the thinning readout mode is in effect. If the thinning readout mode is in effect, the process proceeds to S202. If the thinning readout mode is not in effect (if the all-pixel readout mode is in effect), the process proceeds to S101. 2 The process described above is performed with reference to the above.

[0080] In S202, the horizontal A signal output from the first focus detection pixel 201 and the horizontal B signal output from the second focus detection pixel 202 of the image sensor 107 are acquired and stored in the memory 1251 of the image processing circuit 125.

[0081] In S203, the shading correction circuit 1253 performs shading correction processing on the horizontal A signal and the horizontal B signal to make the intensities uniform.

[0082] In step S204, the different color mixing circuit 1254 performs the above-described different color mixing process to reduce the data amount of the horizontal A signal and the horizontal B signal, and A Signal and horizontal Y B If the horizontal A signal and the horizontal B signal are only single color signals, different color mixing processing is not required. Also, if a column of a specific color signal is thinned out, in order to improve the detection accuracy in the horizontal direction, the thinned column signal is converted into the horizontal A signal and the horizontal B signal of the left and right columns, respectively. No. Interpolation may be performed using the average value.

[0083] In this way, the same circuit is commonly used for filtering and calculating the defocus amount for the horizontal focus detection signal in the thinning readout mode and the horizontal and vertical focus detection signals in the all-pixel readout mode.

[0084] As described above, according to the second embodiment, even when focus detection is performed while switching between a readout mode that performs vertical focus detection and a readout mode that does not, it is possible to process the focus detection using a common circuit, thereby reducing the circuit size.

[0085] <Third embodiment> Next, a third embodiment of the present invention will be described. In the third embodiment, focus detection is performed using focus detection signals obtained by dividing the pupil in a third direction (for example, a direction tilted by 45 degrees) in addition to the horizontal and vertical pupil division directions.

[0086] The configuration of the imaging device is the same as that described with reference to Figures 1 and 7 in the first embodiment, so a description thereof will be omitted here. However, the configuration of the imaging element 107 is different from that described with reference to Figures 2 to 4, so it will be described below.

[0087] [Image sensor] An outline of the arrangement of imaging pixels and focus detection pixels of the image sensor 107 in the third embodiment is shown in Fig. 14. Fig. 14 shows the pixel (imaging pixel) arrangement of a two-dimensional CMOS sensor as the image sensor 107 in the third embodiment, in an area of ​​4 columns x 4 rows.

[0088] The pixel group 700 is composed of 2 rows and 2 columns of pixels, with pixel 700R having R (red) spectral sensitivity arranged in the upper left, pixel 700Ga having G (green) spectral sensitivity arranged in the upper right, pixel 700Gb arranged in the lower left, and pixel 700B having B (blue) spectral sensitivity arranged in the lower right. Furthermore, the pixels are composed of a first focus detection pixel 701, a second focus detection pixel 702, a third focus detection pixel 703, and a fourth focus detection pixel 704.

[0089] By arranging a large number of pixel groups 700 of 2 columns x 2 rows shown in FIG. 14 on a surface, it is possible to acquire a captured image and a focus detection signal.

[0090] When generating a focus detection signal when the pupil division direction is horizontal, the outputs from the first focus detection pixel 701 and the second focus detection pixel 702 of each pixel are added together to generate a horizontal A signal, and the outputs from the third focus detection pixel 703 and the fourth focus detection pixel 704 are added together to generate a horizontal B signal.

[0091] Furthermore, when generating a focus detection signal when the pupil division direction is vertical, the outputs from the second focus detection pixel 702 and the third focus detection pixel 703 of each pixel are added together to generate a vertical A signal, and the outputs from the first focus detection pixel 701 and the fourth focus detection pixel 704 are added together to generate a vertical B signal.

[0092] Furthermore, when generating focus detection signals having a pupil division direction at a 45-degree angle, the output from the first focus detection pixel 701 of each pixel is an A signal (hereinafter referred to as a "diagonal A signal"), and the output from the third focus detection pixel 703 is a B signal (hereinafter referred to as a "diagonal B signal"). Alternatively, the output from the second focus detection pixel 702 may be the diagonal A signal, and the output from the fourth focus detection pixel 704 may be the diagonal B signal.

[0093] Furthermore, for each pixel of the image sensor 107, the outputs from the first to fourth focus detection pixels 701 to 704 are added together to generate an image signal (captured image) with a resolution of N effective pixels.

[0094] An example of m×n focus detection signals having a pupil division direction at an oblique 45° angle, generated in the third embodiment, is shown in Fig. 15. Fig. 15 shows the oblique A signal output from the second focus detection pixel 702 and the oblique B signal output from the fourth focus detection pixel 703. 7 1 is a conceptual diagram showing a diagonal B signal output from the .

[0095] [Relocation process] In Fig. 16, the pupil division direction is at a 45-degree angle. hot 16(a) shows the state before the rearrangement process, and as shown in FIG. 16(b), the rearrangement process rotates the array by 45 degrees.

[0096] [Image processing circuit] Next, the configuration of the image processing circuit 125 in the third embodiment will be described with reference to Fig. 17. Here, the configuration that performs processing related to correlation calculations for focus detection, among the functions of the image processing circuit 125, will be described. Note that the same reference numerals are used to designate components that are the same as those shown in Fig. 7.

[0097] First, the A and B signals obtained by reading out signals from the image sensor 107 are stored in memory 1251 in the image processing circuit 125. In the third embodiment, horizontal A and horizontal B signals, or vertical A and vertical B signals, or diagonal A and diagonal B signals are obtained by combining the outputs of the first to fourth focus detection pixels 701 to 704 during readout.

[0098] The shading correction circuit 1253 performs shading correction on the input A and B signals. After that, the different color mixing processing circuit 1254 performs different color mixing processing on the shading-corrected A and B signals, and outputs Y A Signal and Y B Generate the signal. Generated Y A Signal and Y B is sent to a switching unit 1260, and when the focus detection direction is the horizontal direction, it is output to a correlation calculation circuit 1258. On the other hand, when the focus detection direction is the vertical direction or a 45-degree oblique direction, Y A Signal and Y B The signal is output to the rearrangement processing circuit 1257.

[0099] The rearrangement processing circuit 1257 converts the signal arrangement as described in FIG. 11 when focus detection is performed in the vertical direction, and converts the signal arrangement as described in FIG. 16 when focus detection is performed in a 45-degree diagonal direction, and outputs the converted signal to the correlation calculation circuit 1258.

[0100] [Focus detection processing] Fig. 18 is a flowchart showing focus detection processing in the third embodiment. The processing shown in Fig. 18 is executed by the image processing circuit 125 and the CPU 121. Of the processing shown in Fig. 18, the same steps as those in Fig. 12 described in the first embodiment are given the same step numbers, and their description will be omitted. First, in S301, signals A and B are generated from the signals obtained from the first to fourth focus detection pixels 701 to 704 of each pixel of the image sensor 107 by pupil division in the horizontal direction, vertical direction, and 45-degree diagonal direction as described above, and are stored in the memory 1251 of the image processing circuit 125. Then, in S302, the shading correction circuit 1253 performs shading correction on the A and B signals, and in S303, the different color mixing processing circuit 1254 performs Y A Signal and Y B In the third embodiment, first and second focus detection signals are obtained for all pixels by pupil-splitting in the horizontal direction, vertical direction, or diagonal 45-degree direction, and therefore, unlike the first and second embodiments, there is no need to interpolate signals lost due to the splitting.

[0101] Next, in S304, the direction in which focus detection is to be performed is determined, and if the determined direction is horizontal, the process proceeds to S108; if it is vertical, the process proceeds to S305; and if it is a 45-degree diagonal direction, the process proceeds to S306.

[0102] In S305, the rearrangement processing circuit 1257 performs the rearrangement processing of the arrays of the A signal and the B signal as explained with reference to FIG. 11, and the process proceeds to S108. In S306, the rearrangement processing circuit 1257 performs the rearrangement processing of the arrays of the A signal and the B signal as explained with reference to FIG. 17, and the process proceeds to S108.

[0103] In this way, by performing the array rearrangement process for focus detection signals having pupil division directions other than the horizontal and vertical directions, it becomes possible to perform focus detection calculations using a common circuit.

[0104] As described above, according to the third embodiment, it is possible to process focus detection signals divided in different pupil directions using a common circuit, thereby making it possible to reduce the circuit scale.

[0105] In the third embodiment described above, the image sensor 107 having the configuration shown in Fig. 14 generates focus detection signals that are pupil-divided only in one of the horizontal, vertical, and 45-degree diagonal directions, but the present invention is not limited to this. For example, it is also possible to generate pupil-divided focus detection signals that include a mixture of directions by changing the signal combination for each pixel, in which case focus detection can be performed using the image processing circuit 125 shown in Fig. 7 described in the first embodiment.

[0106] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0107] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0108] <Summary> The disclosure of this embodiment includes the following configuration.

[0109] (Item 1) an acquisition means for acquiring, for each pixel from signals of a plurality of pixels output from the imaging means, a pair of focus detection signals having parallax in any one of a plurality of different directions; a separation unit that separates the focus detection signal for each direction of the parallax; a detection means for detecting a focus state based on a first focus detection signal separated by the separation means, the first focus detection signal indicating that the direction of the parallax is a predetermined first direction; a rearrangement means for rearranging an arrangement of second focus detection signals separated by the separation means, the arrangement of which is such that the direction of the parallax is a second direction different from the first direction, so that the direction of the parallax is the first direction; The focus detection device is characterized in that the detection means further detects a focus state based on the second focus detection signals rearranged by the rearrangement means. (Item 2) The focus detection device described in item 1, characterized in that the number of the first focus detection signals arranged in the first direction is greater than the number of the second focus detection signals arranged in the second direction. (Item 3) 3. The focus detection device according to item 1 or 2, wherein the first direction and the second direction are orthogonal to each other. (Item 4) 4. The focus detection device according to any one of items 1 to 3, wherein the rearrangement means rearranges the pixels by transposing the array or by combining rotation and inversion of the array. (Item 5) 5. The focus detection device according to any one of items 1 to 4, further comprising a processing unit that performs shading correction on the second focus detection signal before rearrangement by the rearrangement unit. (Item 6) The focus detection device described in any one of items 1 to 5 is characterized in that it further comprises an interpolation means that interpolates using the second focus detection signals adjacent in the second direction so as to increase the number of the second focus detection signals arranged in the second direction before rearrangement by the rearrangement means. (Item 7) The focus detection device described in any one of items 1 to 4 is characterized in that it further comprises processing means for performing shading correction on the first focus detection signals and processing for adding the first focus detection signals so as to reduce the number of the first focus detection signals arranged in the second direction. (Item 8) the imaging means reads and outputs signals in an all-pixel readout mode in which signals are read out from all of the plurality of pixels constituting the imaging means, and in a thinning-out readout mode in which some of the plurality of pixels are thinned out and read out; 8. The focus detection device according to any one of items 1 to 7, wherein when the signals are read out in the thinning readout mode, the separation means separates the first focus detection signal from the focus detection signals of the plurality of pixels. (Item 9) a generating means for generating a pair of first focus detection signals having parallax in any one of a plurality of different directions for each pixel from signals of the plurality of pixels output from the imaging means; a detection means for detecting a focus state based on the first focus detection signal when the direction of the parallax of the first focus detection signal is a predetermined first direction; a rearrangement unit that, when the direction of the parallax of the first focus detection signal is a second direction different from the first direction, rearranges the first focus detection signal so that the direction of the parallax becomes the first direction, and generates a second focus detection signal; The focus detection device is characterized in that the detection means further detects the focus state based on the second focus detection signal. (Item 10) 10. The focus detection device according to item 9, wherein the first direction and the second direction are orthogonal to each other. (Item 11) 10. The focus detection device according to item 9, wherein the second direction is inclined at 45 degrees with respect to the first direction. (Item 12) An imaging means; A focus detection device according to any one of items 1 to 11, a focus adjustment means for adjusting the focus based on the focus state; An electronic device comprising: (Item 13) an acquisition step in which an acquisition means acquires, for each pixel, a pair of focus detection signals having parallax in any one of a plurality of different directions from signals of the plurality of pixels output from the imaging means; a separating step in which a separating unit separates the focus detection signal for each direction of the parallax; a first detection step in which a detection means detects a focus state based on a first focus detection signal separated in the separation step, the first focus detection signal indicating that the direction of the parallax is a predetermined first direction; a rearrangement step in which a rearrangement unit rearranges an array of second focus detection signals separated by the separation unit, the second focus detection signals having a parallax direction in a second direction different from the first direction, so that the parallax direction becomes the first direction; a second detection step in which the detection means detects a focus state based on the second focus detection signals rearranged in the rearrangement step; A focus detection method characterized by: (Item 14) a generation step in which a generation means generates a pair of first focus detection signals having parallax in one of a plurality of different directions for each pixel from signals of a plurality of pixels output from an imaging means; a rearrangement means, when the direction of the parallax of the first focus detection signals is a second direction different from a predetermined first direction, rearranges the first focus detection signals so that the direction of the parallax becomes the first direction, and generates a second focus detection signal; a detection step in which a detection means detects a focus state based on the first focus detection signal when the direction of the parallax of the first focus detection signal is a predetermined first direction, and detects a focus state based on the second focus detection signal when the direction of the parallax of the first focus detection signal is the second direction; A focus detection method comprising: (Item 15) 12. A program for causing a computer to function as each of the means of the focus detection device according to any one of items 1 to 11. (Item 16) Item 16. A computer-readable storage medium storing the program described in item 15.

[0110] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0111] 101: first lens group, 103: second lens group, 105: third lens group, 107: image sensor, 200R, 200G, 200B: pixels, 201, 701: first focus detection pixel, 202, 702: second focus detection pixel, 703: third focus detection pixel, 704: fourth focus detection pixel, 301, 302: photoelectric conversion unit, 1251: memory, 1252: separation circuit, 1253, 1255: shading correction circuit, 1254: different color mixing processing circuit, 1256: row interpolation circuit, 1257: rearrangement processing circuit, 1258: correlation calculation circuit, 1260: switching unit

Claims

1. An imaging element capable of acquiring a first pair of focus detection signals having parallax in a first direction and a second pair of focus detection signals having parallax in a second direction different from the first direction from within an imaging plane that captures an image of a subject formed by an optical system, and having a first pair of photoelectric conversion units whose pupils are divided in the first direction within the imaging plane that captures an image of a subject formed by an optical system and a second pair of photoelectric conversion units whose pupils are divided in a second direction different from the first direction; a separation unit that separates the focus detection signals read out from the image sensor for each of the directions of parallax; a focus detection unit that calculates a correlation value of the first pair of focus detection signals separated by the separation unit and detects a focus state; a rearrangement unit that rearranges the arrangement of the second pair of focus detection signals separated by the separation unit so that the direction of the parallax is the first direction, the focus detection means further calculates a correlation value of the second pair of focus detection signals rearranged by the rearrangement means to detect a focus state; reading and outputting signals in a first readout mode in which signals are read out from the image sensor without being thinned out in the vertical direction to detect focus states in the first direction and the second direction, and in a second readout mode in which signals are read out from the image sensor with being thinned out in the vertical direction to not detect focus states in the second direction; The focus detection device, wherein the separating means separates the focus detection signals for each of the directions of parallax when the signals are read out in the first readout mode.

2. 2. The focus detection device according to claim 1, wherein the number of the first pair of focus detection signals arranged in the first direction is greater than the number of the second pair of focus detection signals arranged in the second direction.

3. 2. The focus detection device according to claim 1, wherein the first direction and the second direction are orthogonal to each other.

4. 2. The focus detection device according to claim 1, wherein the rearrangement means rearranges the pixels by transposing the array or by combining rotation and inversion of the array.

5. 2. The focus detection device according to claim 1, further comprising processing means for performing shading correction on the second pair of focus detection signals before rearrangement by the rearrangement means.

6. 2. The focus detection device according to claim 1, further comprising an interpolation means for interpolating using the second pairs of focus detection signals adjacent in the second direction so as to increase the number of the second pairs of focus detection signals arranged in the second direction before rearrangement by the rearrangement means.

7. 2. The focus detection device according to claim 1, further comprising a processing means for performing shading correction on the first pair of focus detection signals and a process of adding the first pair of focus detection signals so as to reduce the number of the first pair of focus detection signals arranged in the second direction.

8. A focus detection device according to any one of claims 1 to 7; a focus adjustment means for adjusting the focus based on the focus state; An electronic device comprising:

9. A focus detection method in a focus detection device having an image sensor capable of acquiring a first pair of focus detection signals having a parallax in a first direction and a second pair of focus detection signals having a parallax in a second direction different from the first direction from within an imaging plane that captures an image of a subject formed by an optical system, and having a first pair of photoelectric conversion units whose pupils are divided in the first direction within the imaging plane that captures an image of a subject formed by an optical system and a second pair of photoelectric conversion units whose pupils are divided in a second direction different from the first direction, a separation step of separating the focus detection signals read out from the image sensor for each direction of the parallax; a first detection step of inputting the first pair of focus detection signals separated in the separation step to a correlation calculation means to find a correlation value and detect a focus state; a rearrangement step of rearranging the arrangement of the second pair of focus detection signals separated in the separation step so that the direction of the parallax is the first direction; a second detection step of inputting the second pair of focus detection signals rearranged in the rearrangement step to the correlation calculation means to find a correlation value and detect a focus state, signals are read out in a first readout mode in which signals are read out from the image sensor without being thinned out in the vertical direction to detect focus states in the first direction and the second direction, and in a second readout mode in which signals are read out from the image sensor with being thinned out in the vertical direction to not detect focus states in the second direction; The focus detection method, wherein the separating step separates focus detection signals for each direction of parallax when the signals are read out in the first readout mode.

10. A program for causing a computer to function as each of the means of the focus detection device according to any one of claims 1 to 7.

11. A computer-readable storage medium storing the program according to claim 10.

Images