Image pickup element and image pickup device

Abstract

To provide an imaging element with which it is possible to read out a signal used for focus detection and a signal used for image formation at high speed.SOLUTION: An imaging element 212 comprises: a first photoelectric conversion unit 14 and a second photoelectric conversion unit 13 that photoelectrically convert light having passed through a first micro-lens to generate an electric charge; a first signal line 22a that outputs a signal based on the electric charge generated by the first photoelectric conversion unit 14; a second signal line 22b that outputs a signal based on the electric charge generated by the second photoelectric conversion unit 13; and a first A / D conversion unit 23a that is electrically connected to the first signal line 22a, and converts an analog signal based on the electric charge generated by the first photoelectric conversion unit 14 into a first digital signal; a second A / D conversion unit 23b that is electrically connected to the second signal line 22b, and converts an analog signal based on the electric charge generated by the second photoelectric conversion unit 13 into a second digital signal; and a digital addition circuit 26 that adds up the first and second digital signals.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an imaging element and an imaging device. [Background technology]

[0002] An imaging device is known in which an image sensor having an array of focus detection pixels, each of which is made up of a microlens and a pair of photoelectric conversion units arranged behind it, is arranged on the intended focal plane of the photographing lens, whereby a pair of image signals corresponding to a pair of images formed by a pair of focus detection light beams passing through the optical system are generated as analog signals in a pair of photoelectric conversion units, the pair of analog signals are read out independently from the image sensor and the amount of image shift (phase difference) between the pair of image signals is detected, thereby detecting the focus adjustment state (defocus amount) of the photographing lens, and the analog signals generated by the pair of photoelectric conversion units of the focus detection pixels are analog-added within the focus detection pixel, and the analog signal after addition is read out of the image sensor as an image signal to generate image information (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-83407 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the image capture device as described above, analog addition processing of a pair of analog signals is performed within the focus detection pixel, so during focus detection, it is necessary to read out a pair of analog signals independently from the image sensor, and when generating image information, it is necessary to add and read out the pair of analog signals from the image sensor. This creates the problem that focus detection and image information generation cannot be performed simultaneously when reading out one frame's worth of signals from the image sensor. [Means for solving the problem]

[0005] The imaging device according to the first aspect of the present invention comprises: 1M Light transmitted through a microlens Power Convert to load do No. 1 light Electric conversion unit and a photoelectric conversion unit that converts light transmitted through the first microlens into an electric charge, the photoelectric conversion unit being disposed adjacent to the first photoelectric conversion unit in the first direction; No. 2 light An electric conversion unit; a third photoelectric conversion unit that converts light transmitted through a second microlens into an electric charge; a fourth photoelectric conversion unit that converts light transmitted through the second microlens into an electric charge and is disposed adjacent to the third photoelectric conversion unit in a second direction intersecting the first direction; and a control unit that outputs a first summed signal obtained by performing summation processing using a first signal based on the electric charges converted by the first photoelectric conversion unit and a second signal based on the electric charges converted by the second photoelectric conversion unit, and a second summed signal obtained by performing summation processing using a third signal based on the electric charges converted by the third photoelectric conversion unit and a fourth signal based on the electric charges converted by the fourth photoelectric conversion unit. Equipped with. According to the second aspect of the present invention Imaging device is the above Image sensor Equipped with. Effect of the Invention

[0006] According to the present invention, for example, signals used for focus detection and signals used for image generation can be read out at high speed. [Brief description of the drawings]

[0007] [Figure 1] 1 is a cross-sectional view showing a configuration of a lens-interchangeable digital still camera equipped with an image sensor according to an embodiment of the present invention. [Diagram 2] FIG. 4 is a diagram showing focus detection positions on a shooting screen of an interchangeable lens. [Diagram 3] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 4] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Diagram 5] FIG. 4 is a diagram showing the spectral sensitivity characteristics of each color filter. [Figure 6] FIG. 2 is a diagram showing a configuration of a focus detection pixel. [Figure 7] FIG. 2 is a cross-sectional view of a focus detection pixel. [Figure 8] FIG. 1 is a diagram showing a configuration of a split-pupil phase difference detection type focus detection optical system. [Figure 9] FIG. 2 is a block diagram showing in detail the relationship between an image sensor and a body drive control device. [Figure 10] FIG. 2 is a block diagram showing a configuration of an imaging element. [Figure 11]13 is a timing chart when an individual readout operation of output signals from a pair of photoelectric conversion units in a focus detection pixel and a readout operation of a sum signal obtained by adding together the output signals from the pair of photoelectric conversion units are performed in parallel during one frame period. [Figure 12] 13 is a timing chart when an individual readout operation of output signals from a pair of photoelectric conversion units in a focus detection pixel and a readout operation of a sum signal obtained by adding together the output signals from the pair of photoelectric conversion units are performed in parallel during one frame period. [Figure 13] 11 is a flowchart showing the operation of a CPUa for focus detection in a body drive control device provided in a digital still camera. [Figure 14] 4 is a flowchart showing the operation of an image processing CPU of a body drive control device provided in a digital still camera. [Figure 15] FIG. 13 is a diagram showing the results of a correlation calculation between a pair of data strings. [Figure 16] 13 is a timing chart for performing partial row readout. [Figure 17] FIG. 2 is a diagram showing a configuration of a focus detection pixel. [Figure 18] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 19] FIG. 2 is a block diagram showing a configuration of an imaging element. [Figure 20] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 21] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 22] FIG. 2 is a diagram showing a configuration of an imaging pixel. [Figure 23] FIG. 2 is a cross-sectional view of an imaging pixel. [Figure 24] FIG. 2 is a diagram for explaining the state of a photographing light beam. [Diagram 25] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 26] FIG. 2 is a block diagram showing a configuration of an imaging element. [Figure 27] 11 is a diagram illustrating a selection operation of switches provided in two adjacent pixel columns. FIG. [Figure 28]11 is a diagram illustrating a selection operation of switches provided in two adjacent pixel columns. FIG. [Figure 29] 13 is a timing chart showing a case in which an individual readout operation of output signals from a pair of photoelectric conversion units in a focus detection pixel and a readout operation of an output signal corresponding to an output signal from an imaging pixel are performed in parallel during one frame period. [Diagram 30] 13 is a timing chart for performing partial row readout. [Diagram 31] FIG. 2 is a block diagram showing a configuration of an imaging element. [Diagram 32] 11 is a diagram illustrating a selection operation of switches provided in two adjacent pixel columns. FIG. [Diagram 33] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Diagram 34] FIG. 2 is a block diagram showing a configuration of an imaging element. [Diagram 35] FIG. 2 is a block diagram showing a configuration of an imaging element. [Diagram 36] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 37] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Figure 38] FIG. 2 is a block diagram showing a configuration of an imaging element. [Figure 39] FIG. 2 is a front view showing a detailed configuration of an imaging element. [Diagram 40] FIG. 2 is a block diagram showing a configuration of an imaging element. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] First Embodiment An image sensor and an image sensor device according to an embodiment of the present invention will be described. Fig. 1 is a cross-sectional view showing the configuration of a lens-interchangeable digital still camera equipped with an image sensor according to an embodiment. A digital still camera 201 according to an embodiment is composed of an interchangeable lens 202 and a camera body 203, and various interchangeable lenses 202 are attached to the camera body 203 via a mount unit 204.

[0009] The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, and a lens drive control device 206. The lens drive control device 206 is composed of a microcomputer, a memory, a drive control circuit, and the like (not shown), and performs drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, detects the states of the zooming lens 208, the focusing lens 210, and the aperture 211, and transmits lens information and receives camera information by communicating with a body drive control device 214 (described later). The aperture 211 forms an aperture with a variable aperture centered on the optical axis to adjust the amount of light and the amount of blur.

[0010] The camera body 203 includes an image sensor 212, a body drive control device 214, a liquid crystal display device drive circuit 215, a liquid crystal display device 216, an eyepiece lens 217, and a memory card 219. The image sensor 212 has pixels that function as both imaging pixels and focus detection pixels arranged two-dimensionally. The image sensor 212 will be described in detail later.

[0011] The body drive control device 214 is composed of a microcomputer, memory, drive control circuit, etc., and repeatedly performs drive control of the image sensor 212, readout of the output signal from the image sensor 212, focus detection calculation based on the output signal, and focus adjustment of the interchangeable lens 202, as well as image processing calculation and recording based on the output signal, camera operation control, etc. The body drive control device 214 also communicates with the lens drive control device 206 via electrical contacts 213, receiving lens information and transmitting camera information (defocus amount, aperture value, etc.).

[0012] The liquid crystal display element 216 functions as an electronic viewfinder (EVF). The liquid crystal display element drive circuit 215 displays a through image captured by the image sensor 212 on the liquid crystal display element 216, and the photographer can observe the through image through an eyepiece 217. The memory card 219 is an image storage that stores images captured by the image sensor 212.

[0013] A subject image is formed on the light receiving surface of the image sensor 212 by a light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by each pixel of the image sensor 212, and the output signals of each pixel are sent to a body drive control device 214.

[0014] The body drive controller 214 calculates a defocus amount based on an output signal from each pixel of the image sensor 212, and sends this defocus amount to the lens drive controller 206. The body drive controller 214 also processes the output signal from each pixel of the image sensor 212 to generate image data, which is stored in a memory card 219, and also sends a through-image signal from the image sensor 212 to a liquid crystal display element drive circuit 215, which displays the through-image on the liquid crystal display element 216. Furthermore, the body drive controller 214 sends aperture control information to the lens drive controller 206 to control the aperture of the aperture 211.

[0015] The lens drive control device 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture maximum aperture, etc. Specifically, it detects the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211, and calculates the lens information according to these lens positions and aperture value, or selects the lens information according to the lens position and aperture value from a lookup table prepared in advance.

[0016] The lens drive control device 206 calculates the lens drive amount based on the received defocus amount, and drives the focusing lens 210 to a focus position according to the lens drive amount. The lens drive control device 206 also drives the aperture 211 according to the received aperture value.

[0017] 2 is a diagram showing focus detection positions (set by the user by operating an operating member not shown in FIG. 1) on the shooting screen of the interchangeable lens 202, and shows an example of an area (focus detection area, focus detection position) where a pixel row on the image sensor 212 (described later) samples an image on the shooting screen during focus detection. In this example, a focus detection area 101 is placed in the center of a rectangular shooting screen 100. The rectangular focus detection area 101 extends in the horizontal direction on the shooting screen 100, and output signals from pixels linearly arranged along the longitudinal direction of the focus detection area 101 are used for focus detection.

[0018] 3 and 4 are front views showing a detailed configuration of the image sensor 212, and are enlarged views showing the vicinity of the focus detection area 101 on the image sensor 212. Fig. 3 shows the layout of pixels 311 (hereinafter referred to as focus detection pixels 311) that serve as imaging pixels and focus detection pixels, and the focus detection pixels 311 are densely arranged in a two-dimensional square lattice in the row direction (horizontal light direction) and column direction (vertical direction). Fig. 4 shows the arrangement of color filters in the arrangement of focus detection pixels 311 shown in Fig. 3, and color filters (R: red filter, G: green filter, B: blue filter) are arranged in the focus detection pixels 311 according to the Bayer array rules, and the spectral sensitivity of each color filter has the characteristics shown in Fig. 5.

[0019] As shown in Fig. 6, the focus detection pixel 311 is composed of a rectangular microlens 10 and a pair of photoelectric conversion units 13, 14 divided into two by an element isolation region 15 extending in the vertical direction. When the pair of photoelectric conversion units 13, 14 are integrated, the size becomes the same as the photoelectric conversion unit of a normal imaging pixel. For simplicity, color filters are not shown in Fig. 6. When the outputs of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 are added, it is desirable to make the width of the element isolation region 15 as narrow as possible and to place the pair of photoelectric conversion units 13, 14 close to each other so that the added output becomes equivalent to the output of the photoelectric conversion unit of a normal imaging pixel.

[0020] FIG. 7 is a cross-sectional view of the focus detection pixel 311 shown in FIG. 6. A light-shielding mask 30 is formed on and adjacent to the photoelectric conversion units 13 and 14, and the photoelectric conversion units 13 and 14 receive light that passes through an opening 30d of the light-shielding mask 30. A planarization layer 31 is formed on the light-shielding mask 30, and a color filter 38 is formed thereon. A planarization layer 32 is formed on the color filter 38, and a microlens 10 is formed thereon. The shape of the photoelectric conversion units 13 and 14 limited to the opening 30d by the microlens 10 is projected forward to form a pair of ranging pupils. The photoelectric conversion units 13 and 14 are formed on a semiconductor circuit substrate 29. In addition, an element isolation region 15 is formed to isolate the photoelectric conversion units 13 and 14. With the above configuration, the photoelectric conversion units 13 and 14 receive a pair of focus detection light beams that pass through a pair of ranging pupils of the exit pupil of the interchangeable lens.

[0021] FIG. 8 shows the configuration of a split-pupil phase difference detection type focus detection optical system using a microlens. A portion of the focus detection pixel array in the focus detection area 101 is shown enlarged. In FIG. 8, an exit pupil 90 is set at a position a distance d forward from a microlens 10 arranged on a planned imaging plane of an interchangeable lens 202 (see FIG. 1). This distance d is determined according to the curvature and refractive index of the microlens 10, the distance between the microlens 10 and the photoelectric conversion units 13 and 14, and is referred to as the ranging pupil distance in this specification. FIG. 11 also shows an optical axis 91 of the interchangeable lens, the microlens 10, the photoelectric conversion units 13 and 14, the focus detection pixel 311, and the focus detection light beams 73 and 74.

[0022] The ranging pupil 93 is obtained by projecting the photoelectric conversion unit 13 limited by the opening 30d through the microlens 10. Similarly, the ranging pupil 94 is obtained by projecting the photoelectric conversion unit 14 limited by the opening 30d through the microlens 10. The ranging pupils 93 and 94 are different partial regions of the exit pupil 90, are aligned in the horizontal direction, and are shaped line-symmetrical with respect to a vertical line passing through the optical axis 91.

[0023] 8 shows a schematic example of five adjacent focus detection pixels 311 in the focus detection area 101 near the photographing optical axis 91, but even in the focus detection pixels 311 arranged on the periphery of the screen, each photoelectric conversion unit is configured to receive a light beam reaching each microlens from the corresponding ranging pupil 93, 94. The microlens 10 causes the pair of photoelectric conversion units 13 and 14 and the above-mentioned different partial regions, i.e., the pair of ranging pupils 93 and 94, to be in a conjugate relationship with each other.

[0024] With the above-mentioned configuration, the photoelectric conversion unit 13 outputs a signal corresponding to the intensity of an image formed on the microlens 10 by the light beam 73 that passes through the ranging pupil 93 and heads toward the microlens 10 of the focus detection pixel 311. In addition, the photoelectric conversion unit 14 outputs a signal corresponding to the intensity of an image formed on the microlens 10 by the light beam 74 that passes through the ranging pupil 94 and heads toward the microlens 10 of the focus detection pixel 311.

[0025] By grouping the outputs of the photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged horizontally in the focus detection area 101 described above into output groups corresponding to the ranging pupil 93 and ranging pupil 94, information is obtained about the intensity distribution of a pair of images formed on the array of focus detection pixels 311 by the focus detection light beams 73, 74 passing through the ranging pupil 93 and ranging pupil 94, respectively. By subjecting this information to image shift detection calculation processing (correlation calculation processing, phase difference detection processing) described later, the amount of image shift of the pair of images is detected by a so-called split-pupil phase difference detection method. Furthermore, by performing a conversion calculation on the amount of image shift according to the proportional relationship between the center-of-gravity interval of the pair of ranging pupils 93, 94 and the ranging pupil distance, the deviation (defocus amount) of the current imaging plane (imaging plane at the focus detection position corresponding to the position of the microlens array on the expected imaging plane) from the expected imaging plane is calculated. Specifically, the defocus amount (deviation between the imaging plane and the intended imaging plane in the direction of the optical axis 91) is calculated by multiplying the image shift amount (amount in a plane perpendicular to the optical axis 91) by a predetermined conversion coefficient (the value obtained by dividing the focusing pupil distance d by the distance between the centers of gravity of the focusing pupils 93, 94).

[0026] Furthermore, by obtaining an output signal by adding the outputs of the photoelectric conversion units 13 and 14 of each focus detection pixel 311 across the entire screen, it is possible to obtain an image signal equivalent to that obtained when normal imaging pixels are arranged in a Bayer array.

[0027] 9 is a block diagram showing in detail the relationship between the image sensor 212 and the body drive control device 214, which are related to the present invention. The body drive control device 214 houses an image sensor control unit 220, a buffer memory 221, a CPUa (microcomputer) 222, and a CPUb (microcomputer) 223. The image sensor 212 performs charge accumulation control (charge accumulation time and charge accumulation timing) and signal output control of the focus detection pixel 311 under the control of the image sensor control unit 220. As described later, the image sensor 212 performs AD conversion of the output signal of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 and outputs it as digital data (data for focus detection) from channel 1, and at the same time outputs digital data (a signal equivalent to the output signal of a normal image sensor) obtained by digitally adding the digital data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 as digital data from channel 2. The digital data output from channel 1 and channel 2 is temporarily stored in the buffer memory 221 as one frame's worth of digital data. The CPUa 222 performs focus detection by performing processing, described later, on digital data (data for focus detection) from a pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 in the focus detection area stored in the buffer memory 221. The CPUb 223 performs well-known image processing on one frame's worth of digital data (image data) stored in the buffer memory 221 to display or record an image.

[0028] As described above, the digital data for focus detection and the digital data for an image are output in a time-overlapping manner via different channels from the image sensor 212. In addition, since the digital data for focus detection and the digital data for an image are processed in separate CPUs 222 and 223, there is no need to separate the focus detection process and the image process in time, and they can be performed simultaneously and independently.

[0029] Next, the configuration of an image sensor 212 capable of outputting digital data (a signal equivalent to the output signal of a normal imaging pixel) obtained by digitally adding the digital data of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 and the digital data of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 simultaneously from two channels will be described with reference to FIG. 10.

[0030] 10 is a block diagram showing the configuration of the image sensor 212 (CMOS image sensor). The image sensor 212 includes a pixel array section 40 in which a large number of focus detection pixels 311, each of which includes a pair of photoelectric conversion sections 13 and 14, are arranged two-dimensionally in a matrix, as well as a row scanning circuit 41, a column AD conversion device 42, a second line memory 44, a second column scanning circuit 51, a second horizontal output circuit 45, a column digital addition device 46, a first line memory 48, a first column scanning circuit 52, a first horizontal output circuit 49, and a timing control circuit 50.

[0031] In this system configuration, the timing control circuit 50 generates clock signals and control signals that serve as the basis for the operation of the row scanning circuit 41, the column AD conversion device 42, the column digital adding device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc., based on a master clock input from outside and a control signal input from the image sensor control unit 220, and provides these signals to the row scanning circuit 41, the column AD conversion device 42, the column digital adding device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc.

[0032] Furthermore, the peripheral drive system and signal processing system that drive and control each focus detection pixel 311 of the pixel array section 40, i.e., the row scanning circuit 41, the column AD conversion device 42, the column digital addition device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, the first horizontal output circuit 49, the second horizontal output circuit 45, and the timing control circuit 50, are integrated on the same chip (semiconductor substrate) as the pixel array section 40. The chip on which these are integrated is stacked on the chip of the pixel array section 40.

[0033] The focus detection pixel 311 may be a three-transistor configuration (not shown) having a pair of photoelectric conversion elements 13, 14 (e.g., photodiodes) as well as a transfer transistor that transfers the charge obtained by photoelectric conversion in the photoelectric conversion elements 13, 14 to an FD (floating diffusion) section, a reset transistor that controls the potential of the FD section, and an amplification transistor that outputs a signal corresponding to the potential of the FD section, or a four-transistor configuration that further has a selection transistor for pixel selection.

[0034] In the pixel array section 40, focus detection pixels 311 are arranged two-dimensionally in 2N rows and 2M columns. In other words, the pixel array section 40 has a focus detection pixel group in which 2M focus detection pixels 311 are arranged in the horizontal direction in each row, and the focus detection pixel groups are arranged in 2N rows in the vertical direction intersecting the horizontal direction. In FIG. 10, the focus detection pixel 311 in the upper left is the pixel in the first row and first column, and a green filter in a Bayer array is arranged in this pixel, and a green filter and a blue filter are arranged in the focus detection pixels arranged as the pixel group in the first row. For this pixel arrangement of 2N rows and 2M columns, one row control line 21 (21(1) to 21(2N)) is wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are wired for each column. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to an output terminal of the row scanning circuit 41 corresponding to each row, and control signals R(1) to R(2N) are output to each of the row control lines 21. The row scanning circuit 41 is composed of a shift register or the like, and controls the row addresses and row scanning of the pixel array section 40 via the row control lines 21 (21(1) to 21(2N)).

[0035] The pair of photoelectric conversion units 13, 14 of each focus detection pixel 311 in the same row are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to control signals R(1),...,R(L),...,R(2N). One of the pair of photoelectric conversion units 13, 14 of each focus detection pixel 311 is connected to one of two column signal lines 22(m)b provided for each column, and an output signal (analog signal) of the photoelectric conversion unit 13 is output to the column signal line 22(m)b. The other of the pair of photoelectric conversion units 13, 14 of each focus detection pixel 311 is connected to the other of two column signal lines 22(m)a provided for each column, and an output signal (analog signal) of the photoelectric conversion unit 14 is output to the column signal line 22(m)a. For example, when a focus detection pixel 311 constituting a focus detection pixel group in the Lth row of the pixel array section 40 is selected by a control signal R(L) provided from the row scanning circuit 41, the output signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 in the Lth row is output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b).

[0036] The column AD conversion device 42 has ADCs (analog-digital conversion circuits) 23(1)a, 23(1)b to 23(2M)a, 23(2M)b provided for each column signal line 22(1)a, 22(1)b to 22(2M)a, 22(2M)b provided corresponding to the pixel columns of the pixel array section 40, and converts a pair of analog signals output for each column from each focus detection pixel 311 of the pixel array section 40 into an H-bit digital signal in response to a control signal TA1 provided by the timing control circuit 50, and outputs the digital signal. "H bits" represents the number of bits, and is, for example, 10 bits, 12 bits, 14 bits, etc.

[0037] The second line memory 44 has memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) provided for each ADC (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) constituting the column AD conversion device 42, and stores the digital signal output from each ADC (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) as an H-bit digital signal in response to a control signal TM2 provided from the timing control circuit 50. Here, the output signals of pairs of photoelectric conversion units 13, 14 for one row of focus detection pixels are stored as digital signals in each memory (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of the second line memory 44.

[0038] Column digital adding device 46 has digital adding circuits (26(1) to 26(2M)) provided for each pair of ADCs ((23(1)a, 23(1)b) to (23(2M)a, 23(2M)b)) constituting column AD conversion device 42, and adds up the digital signals output from the pairs of ADCs ((23(1)a, 23(1)b) to (23(2M)a, 23(2M)b)) in response to a control signal TD1 provided from timing control circuit 50, and outputs an H-bit added digital signal.

[0039] First line memory 48 has memories (28(1) to 28(2M)) provided for each digital adder circuit (26(1) to 26(2M)) constituting column digital adder 46, and stores the added digital signal output from each digital adder circuit (26(1) to 26(2M)) as an H-bit digital signal in response to a control signal TM1 provided from timing control circuit 50. Here, each memory (28(1) to 28(2M)) of first line memory 48 stores, as a digital signal, a sum signal (corresponding to the output signal of an imaging pixel) obtained by adding together the output signals of a pair of photoelectric conversion units 13, 14 for one row of focus detection pixels.

[0040] The second column scanning circuit 51 is configured with a shift register and the like, and controls the column addresses and column scanning of the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) in the second line memory 44 under the control of the timing control circuit 50. The second line memory 44 operates in response to a scanning signal TS2 provided from the second column scanning circuit 51, and the H-bit digital signals stored in each of the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) are sequentially read out to the second horizontal output circuit 45 and serially output to the outside via the second horizontal output circuit 45 as output signals (digital signals) of the pair of photoelectric conversion units 13, 14 for focus detection.

[0041] The first column scanning circuit 52 is configured with a shift register and the like, and controls the column addresses and column scanning of the memories (28(1) to 28(2M)) in the first line memory 48 under the control of the timing control circuit 50. The first line memory 48 operates in response to a scanning signal TS1 provided from the first column scanning circuit 52, and the H-bit added digital signals stored in each of the memories (28(1) to 28(2M)) are sequentially read out to the first horizontal output circuit 49 and serially output to the outside via the first horizontal output circuit 49 as output signals (digital signals) equivalent to the output signals of the imaging pixels.

[0042] Next, a case in which an individual readout operation of output signals from a pair of photoelectric conversion units in a focus detection pixel and a readout operation of a sum signal obtained by adding together output signals from the pair of photoelectric conversion units are performed in parallel during one frame period in the image sensor configuration shown in Fig. 10 will be described using the timing charts of Fig. 11 and Fig. 12. In Fig. 11 and Fig. 12, VS is a vertical synchronization signal that indicates one frame period, and HS is a horizontal synchronization signal that indicates one horizontal scanning period.

[0043] In the operation shown in FIG. 11, control signals R(1), R(2), R(3) to R(2n+1), R(2n+2), R(2n+3) to R(N) are sequentially issued from the row scanning circuit 41 to the pixel array section 40 in synchronization with the horizontal synchronization signal HS, and analog signals from pairs of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in a row corresponding to the control signals R(1), R(2), R(3) to R(2n+1), R(2n+2), R(2n+3) to R(N) are sequentially output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b).

[0044] Fig. 12 is an enlarged view of the operating portions of rows (2n+1), (2n+2), and (2n+3) in Fig. 11. When row (2n+1) of the pixel array section 40 is selected by the control signal R(2n+1), analog signals from a pair of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in row (2n+1) are output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b). Analog signals from a pair of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in the (2n+1) row output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are converted into digital signals by ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of a column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b to 22(2M)a, 22(2M) in response to a control signal TA1.

[0045] The digital signals of the pair of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in the (2n+1) rows that have been digitally converted are stored in memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of a second line memory 44 connected to the ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of a column AD conversion device 42 in response to a control signal TM2.

[0046] At the same time, the digitally converted digital signals of a pair of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in the (2n+1) rows are added together by digital addition circuits (26(1) to 26(2M)) of a column digital addition device 46 provided for each pair of ADCs ((23(1)a, 23(1)b) to (23(2M)a, 23(2M)b)) constituting a column AD conversion device 42, in response to a control signal TD1.

[0047] The summed digital signal of one line of focus detection pixels 311 in a (2n+1) row, obtained by adding together the output signals of a pair of photoelectric conversion units 13, 14, is stored in memories (28(1) to 28(2M)) of a first line memory 48 connected to digital addition circuits (26(1) to 26(2M)) of a column digital addition device 46 in response to a control signal TM1.

[0048] Digital signals from pairs of photoelectric conversion units 13, 14 of focus detection pixels 311 for one line of (2n+1) rows stored in memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 in response to the scanning signal TS2 during a period until the next horizontal synchronization signal HS is generated. Based on the digital signals output from the second horizontal output circuit 45, the focus detection CPUa 222 of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system) and adjusts the focus state, as shown in FIG. 13, which will be described later.

[0049] Similarly, the sum digital signal of one line of focus detection pixels 311 in the (2n+1) row, obtained by adding the output signals of a pair of photoelectric conversion units 13, 14 stored in the memories ((28(1) to 28(2M)) of the first line memory 48, is sequentially serially output to the outside from the first horizontal output circuit 49 in response to the scanning signal TS1 during the period until the next horizontal synchronization signal HS is generated. Based on the sum digital signal output from the first horizontal output circuit 49, the image processing CPUb 223 of the body drive control device 214 generates image data, as shown in FIG. 14, which will be described later.

[0050] When a control signal R(2n+2) is issued in synchronization with the next horizontal synchronization signal HS and row (2n+2) of the pixel array section 40 is selected, a similar operation is repeated for the analog signals of the pair of photoelectric conversion units 13, 14 of one line of focus detection pixels 311 in row (2n+2). Furthermore, a similar process is repeated under a control signal R(2n+3) synchronized with the next horizontal synchronization signal HS.

[0051] 13 and 14 are flowcharts showing the operation of the digital still camera (imaging device) 201 according to an embodiment. The processes according to these flowcharts are performed in parallel. FIG. 13 is a flowchart showing the operation of the CPUa 222 for focus detection of the body drive control device 214. When the power supply of the digital still camera 201 is turned on in step S100, the focus detection operation from step S110 onwards is started. In step S110, data of a pair of photoelectric conversion units of focus detection pixels arranged in a focus detection area selected in frame synchronization is read out. The data of the pair of photoelectric conversion units is a digital signal output from the above-mentioned second horizontal output circuit 45. In the following step S120, image shift detection calculation processing (correlation calculation processing, phase difference detection processing) described later is performed based on the data of the focus detection pixels to calculate the amount of image shift. It is assumed that the position of the focus detection area is selected in advance by the photographer using an operation member (not shown).

[0052] In step S130, the image shift amount is converted into a defocus amount.

[0053] In step S140, it is detected whether the focus state of the interchangeable lens 202 (optical system) is near the in-focus state, that is, whether the absolute value of the calculated defocus amount is within a predetermined value. If it is determined that the focus state is not near the in-focus state, the process proceeds to step S150, where the defocus amount is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the in-focus position to adjust the focus state of the interchangeable lens 202 (optical system).

[0054] Note that even if focus detection is not possible, the process branches to this step, and a scan drive command is sent to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to perform a scan drive between infinity and close range. After that, the process proceeds to step S160.

[0055] If it is determined in step S140 that the image is close to being in focus, the process proceeds to step S160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that a shutter release has not been performed, the process returns to step S110, and the above-mentioned operations are repeated. On the other hand, if it is determined that a shutter release has been performed, the process proceeds to step S170, where the process waits for the shooting operation corresponding to the shutter release to end, and when the shooting operation is completed, the process returns to step S110, and the above-mentioned operations are repeated.

[0056] The image shift detection calculation process (correlation calculation process, phase difference detection process) in steps S120 and S130 in Fig. 13 will be described in detail below. Note that a pair of data from the focus detection pixels 311 is sorted for each of the same colors in the Bayer array.

[0057] Since the light quantity balance of the pair of images detected by the focus detection pixels 311 may be disrupted due to the focus detection pupils 93, 94 being vignetted by the lens aperture, a correlation calculation is performed that can maintain the accuracy of image shift detection for the light quantity balance. If the pair of data strings read out from the array of focus detection pixels 311 are generalized as A1n (A11,...,A1j:j is the number of data) and A2n (A21,...,A2j) without distinction due to color differences, the following correlation calculation formula (1) disclosed in JP2007-333720A is performed on the pair of data strings A1n and A2n to calculate the correlation amount C(k). C(k)=Σ|A1n·A2n+1+k-A2n+k·A1n+1| ···(1)

[0058] In equation (1), the Σ calculation is accumulated for n. The range of n is limited to the range where data A1n, A1n+1, A2n+k, and A2n+1+k exist according to the image shift amount k. The shift amount k is an integer, and is the relative shift amount in units of the data interval of the data string. As shown in Figure 15(a), the calculation result of equation (1) is the shift amount at which the correlation between a pair of data is high (in Figure 15(a), k = kj = 2, where the correlation amount C(k) is minimal·BR>I, the smaller the correlation, the higher the correlation).

[0059] Next, the shift amount X that gives the minimum value C(X) for continuous correlation amounts is found using the three-point interpolation method of equations (2) to (5). X = kj + D / SLOP (2) C(X)=C(kj)-|D| (3) D={C(kj-1)-C(kj+1)} / 2 ···(4) SLOP=MAX{C(kj+1)-C(kj),C(kj-1)-C(kj)}···(5)

[0060] Whether the shift amount X calculated by formula (2) is reliable is determined as follows. As shown in FIG. 15(b), when the correlation between a pair of data is low, the value of the minimum value C(X) of the interpolated correlation amount becomes large. Therefore, when C(X) is equal to or greater than a predetermined threshold value, it is determined that the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled. Alternatively, in order to normalize C(X) by the contrast of the data, when the value obtained by dividing C(X) by SLOP, which is a value proportional to the contrast, is equal to or greater than a predetermined value, it is determined that the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled. Alternatively, when SLOP, which is a value proportional to the contrast, is equal to or less than a predetermined value, it is determined that the subject has low contrast, and the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled.

[0061] As shown in FIG. 15(c), when the correlation between a pair of data is low and there is no drop in the correlation amount C(k) within the shift range kmin to kmax, the minimum value C(X) cannot be obtained, and in such a case it is determined that focus detection is impossible.

[0062] If it is determined that the calculated shift amount X is reliable, it is converted into the image shift amount shft by equation (6). shft = PY·X·(6)

[0063] In equation (6), PY is twice the pixel pitch of the focus detection pixels 311 (the pixel pitch of focus detection pixels of the same color).

[0064] The image shift amount shft calculated by equation (6) is multiplied by a predetermined conversion coefficient k to convert it into a defocus amount def. def = k·shft1·(7)

[0065] In equation (7), the conversion coefficient k is a conversion coefficient according to the proportional relationship between the distance between the centers of gravity of the pair of focus detection pupils 93, 94 and the focus detection pupil distance, and changes according to the aperture F-number of the optical system.

[0066] In this way, three defocus amounts are calculated for the three colors of the Bayer array, and then averaging processing such as simple averaging or weighted averaging is performed to calculate the final defocus amount in the selected focus detection area.

[0067] 14 is an operational flowchart of the image processing CPUb 223 of the body drive control device 214. When the power supply of the digital still camera 201 is turned on in step S200, the image processing operation from step S210 onwards is started. In step S210, added digital data (corresponding to the data of the imaging pixel) obtained by adding the output data of a pair of photoelectric conversion units of the focus detection pixel in frame synchronization is read out, and the data is subjected to image processing for display before being displayed on the electronic viewfinder. The added digital data read out in step S210 is the added digital signal output from the first horizontal output circuit 49 described above.

[0068] In step S220, it is determined whether or not the shutter has been released by operating the shutter button (not shown). If it is determined that the shutter has not been released, the process returns to step S210 and the above-mentioned operations are repeated. On the other hand, if it is determined that the shutter has been released, the process proceeds to step S230 and the photographing operation according to the shutter release is performed. First, an aperture adjustment command is sent to the lens drive control device 206, and the aperture value of the interchangeable lens 202 is set to the control F-value (the F-value set by the photographer or automatically). When the aperture control is completed, summed digital data (corresponding to the data of the Bayer-arranged imaging pixels) obtained by adding the output data of a pair of photoelectric conversion units of the focus detection pixels is read out, and the summed digital data is subjected to well-known image processing (demosaic processing, noise processing, gradation processing, white balance processing, etc.) to generate image data, and the image data is stored in the memory card in step S240. When a series of photographing operations is completed, the process returns to step S210 and the above-mentioned operations are repeated.

[0069] In the first embodiment described above, focus detection is performed only in a selected focus detection area, but since focus detection data for the entire screen is stored in the buffer memory, if the processing power of the focus detection CPU a 222 is high, focus detection can be performed in a plurality of focus detection areas across the entire screen and lens focus adjustment can be performed based on the results.

[0070] In the first embodiment described above, it has been explained that the data obtained by adding up the data from a pair of photoelectric conversion units of an image focus detection pixel is read out in full for each frame. However, instead of reading out all the data, a circuit configuration for performing thinning readout (row / column) or pixel addition readout (row / column) may be further added to the configuration of the present invention, and the read out image data may be used for display, etc.

[0071] In the first embodiment described above, it has been explained that data from pairs of photoelectric conversion units of all focus detection pixels used for focus detection is read out every frame. However, reading out data from pairs of photoelectric conversion units of all focus detection pixels imposes a heavy load and requires a large amount of memory capacity for storing the data, so frame thinning (reading out once every few frames), row thinning (reading out one row out of several rows), partial row readout (reading out only some of the rows), column thinning (reading out one column out of several columns), or partial column readout (reading out only some of the columns) may also be performed as necessary.

[0072] Figure 16 is a timing chart corresponding to Figure 12 when partial row readout is performed (reading out data from a pair of photoelectric conversion units of a focus detection pixel in only the (2n+2)th row), and is an enlarged diagram of the operating portions of the (2n+1)th, (2n+2)th, and (2n+3)th rows in Figure 11.

[0073] The operation when the (2n+2) row of the pixel array unit 40 is selected by the control signal R(2n+2) is the same as that in Fig. 12. On the other hand, when a row other than the (2n+2) row is selected (operation according to the control signal R(2n+1) and the control signal R(2n+3) in Fig. 16), the control signal TM2 is not generated, and the digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 for one line, which have been converted into digital signals by the ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of the column AD conversion device 42, are not stored in the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of the second line memory 44 connected to the ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of the column AD conversion device 42. Furthermore, since the scanning signal TS2 is not generated, the second horizontal output circuit 45 does not sequentially output the signals in series to the outside until the next horizontal synchronizing signal HS is generated.

[0074] The rows to be read out from the row portion and the columns to be read out from the column portion can be changed by sending information from the body drive control device 214 to the image sensor 212 according to the position of the selected focus detection area.

[0075] In the first embodiment described above, a column A / D conversion device 42 having a number of ADCs corresponding to the number of pairs of photoelectric conversion units in one row of focus detection pixels is provided, and a column digital addition device 46 having digital addition circuits 26 for digitally adding digital output signals of the pairs of ADCs is provided in the same number as the number of focus detection pixels in one row, so that it is possible to perform in parallel an individual readout operation of output signals of pairs of photoelectric conversion units in focus detection pixels and a readout operation of a sum signal (corresponding to an output signal of an imaging pixel) obtained by adding output signals of the pairs of photoelectric conversion units during one frame period. This solves the problem of the conventional technology (providing an analog adder device for each focus detection pixel) (individual readout operation of output signals of pairs of photoelectric conversion units in focus detection pixels and a readout operation of a sum signal (corresponding to an output signal of an imaging pixel) obtained by adding output signals of the pair of photoelectric conversion units during one frame period cannot be performed in parallel).

[0076] As a method for solving the problems of the conventional technology, it is also possible to read the output data of the pair of photoelectric conversion units of all focus detection pixels from the image sensor during one frame period individually, temporarily store the data in an external buffer memory, and perform an addition process on the output data of the pair of photoelectric conversion units stored in the buffer memory. In that case, however, the processing time increases by the time for the addition process, and the external processing load also increases. According to the configuration and operation of the image sensor of the present application, the image sensor can be handled in the same way as a normal image sensor in terms of image data reading / image processing. In addition, it is also possible to read the output data of the pair of photoelectric conversion units of some focus detection pixels individually, so that the load of the read process can be reduced and the capacity of the buffer memory for storing data can be saved.

[0077] As a method for solving the problems of the conventional technology, instead of providing the column digital adder 46, when the output data of the pair of photoelectric conversion units of the focus detection pixels is horizontally scanned and sequentially output serially, a data holding memory (which holds data with a delay of one data output time) and a digital adder circuit can be provided in parallel to the output terminal of the second horizontal output circuit 45, and the output data of the pair of photoelectric conversion units of the focus detection pixels can be added in synchronization with the individual data output to generate and output added data. However, this slows down the data transfer rate by the addition processing time (addition time for one focus detection pixel x total number of focus detection pixels), making it impossible to read data at high speed. The image sensor 212 of the present application is provided with the column digital adder 46, and addition processing is performed independently and simultaneously for each column, making it possible to read data at high speed at almost the same data transfer rate as an image sensor consisting of only normal image sensors.

[0078] <Second embodiment> In the first embodiment, the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 are arranged side by side in the horizontal direction (row direction), but by changing the direction in which the pair of photoelectric conversion units of the focus detection pixel are arranged to a direction other than the horizontal direction (row direction), it is possible to detect image shift in a direction other than the horizontal direction. Fig. 17 is a diagram showing a focus detection pixel 312 having a configuration in which the focus detection pixel 311 shown in Fig. 6 is rotated by 90 degrees, and the focus detection pixel 312 is composed of a rectangular microlens 10 and a pair of photoelectric conversion units 16, 17 divided into two by an element isolation region 18 extending in the horizontal direction. When the pair of photoelectric conversion units 16, 17 are combined, the size becomes the same as that of the photoelectric conversion unit of a normal imaging pixel.

[0079] Figure 18 corresponds to the pixel layout diagram of Figure 3 (the filter array corresponds to Figure 4) and is a front view showing a detailed configuration of the image sensor 212 in which focus detection pixels 311 and focus detection pixels 312 are arranged, and shows an enlarged view of the vicinity of the focus detection area 101 on the image sensor 212. The focus detection pixels 311 and focus detection pixels 312 are arranged alternately in every other row.

[0080] Fig. 19 is a block diagram showing the configuration of an image sensor 212 having the pixel layout shown in Fig. 18, and only the characteristic parts will be described while omitting the description of the parts that are the same as the configuration in Fig. 10. The difference between the pixel array section 40 in Fig. 10 is that focus detection pixels 312 each having a pair of photoelectric conversion units 16, 17 separated in the vertical direction are arranged in the even-numbered rows.

[0081] The pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 arranged in an even row are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to a control signal R(L) (L is an even number). One of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 arranged in an even row is connected to one of two column signal lines 22(m)a provided for each column, and an output signal (analog signal) of the photoelectric conversion unit 16 is output to the column signal line 22(m)a. The other of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 is connected to the other of two column signal lines 22(m)b provided for each column, and an output signal (analog signal) of the photoelectric conversion unit 17 is output to the column signal line 22(m)b. For example, when the focus detection pixel 312 in the Lth row of the pixel array section 40 is selected by a control signal R(L) provided from the row scanning circuit 41, the output signals of the pair of photoelectric conversion units 16, 17 of the focus detection pixel 312 in the Lth row are output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b).

[0082] When an image sensor 212 having the above-described configuration is used, phase difference detection is possible for an image of a subject whose contrast varies in the horizontal direction, using a pair of data obtained by grouping the data of pairs of photoelectric conversion units 13, 14 of focus detection pixels 311 of the same color arranged in odd-numbered rows in the horizontal direction for each photoelectric conversion unit, and phase difference detection is possible for an image of a subject whose contrast varies in the vertical direction, using a pair of data obtained by grouping the data of pairs of photoelectric conversion units 16, 17 of focus detection pixels 312 of the same color arranged in even-numbered rows in the vertical direction for each photoelectric conversion unit.

[0083] Fig. 20 is a modification of Fig. 18, in which focus detection pixels 311 and focus detection pixels 312 are arranged alternately in every other column. When an image sensor 212 with such a configuration is used, phase difference detection is possible for an image of a subject whose contrast varies in the horizontal direction, using a pair of data obtained by grouping in the horizontal direction for each photoelectric conversion unit data of a pair of photoelectric conversion units 13, 14 of focus detection pixels 311 of the same color arranged in odd-numbered columns, and phase difference detection is possible for an image of a subject whose contrast varies in the vertical direction, using a pair of data obtained by grouping in the vertical direction for each photoelectric conversion unit data of a pair of photoelectric conversion units 16, 17 of focus detection pixels 312 of the same color arranged in even-numbered columns.

[0084] Fig. 21 is a modification of Fig. 18, in which focus detection pixels 311 and focus detection pixels 312 are arranged in an alternating staggered manner. That is, focus detection pixels 311 are arranged in positions (odd rows and odd columns) or (even rows and even columns), and focus detection pixels 312 are arranged in positions (odd rows and even columns) or (even rows and odd columns). In terms of color filters in the Bayer array, the focus detection pixels 311 are equipped with green filters, and the focus detection pixels 312 are equipped with red filters or blue filters.

[0085] When an image sensor 212 having such a configuration is used, phase difference detection is possible for an image of a subject whose contrast varies in the horizontal direction, using a pair of data obtained by grouping in the horizontal direction for each photoelectric conversion unit the data of the pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 having a green filter arranged in odd-numbered rows and odd-numbered columns or even-numbered rows and even-numbered columns, and phase difference detection is possible for an image of a subject whose contrast varies in the vertical direction, using a pair of data obtained by grouping in the vertical direction for each photoelectric conversion unit the data of the pair of photoelectric conversion units 16, 17 of a focus detection pixel 312 having a blue filter arranged in odd-numbered rows and even-numbered columns or a focus detection pixel 312 having a red filter arranged in odd-numbered rows and odd-numbered columns.

[0086] <Third embodiment> In the first embodiment, all pixels were composed of focus detection pixels, but by mixing normal imaging pixels whose photoelectric conversion unit is not divided and focus detection pixels whose photoelectric conversion unit is divided, the number of focus detection pixels in the entire image sensor can be reduced and the configuration of the image sensor can be simplified, and by reducing the amount of focus detection data output externally from the image sensor, it is possible to make the data transfer rate of focus detection data comparable to the data transfer rate of image processing data.

[0087] As shown in FIG. 22, the imaging pixel 310 has a rectangular microlens 10 and a photoelectric conversion section 11 whose light receiving area is limited by a light shielding mask, which will be described later.

[0088] Fig. 23 is a cross-sectional view of the imaging pixel 310 shown in Fig. 22. In the imaging pixel 310, a light-shielding mask 30 is formed adjacent to and above a photoelectric conversion unit 11 for imaging, and the photoelectric conversion unit 11 receives light that has passed through an opening 30a of the light-shielding mask 30. A planarizing layer 31 is formed on the light-shielding mask 30, and a color filter 38 is formed thereon. A planarizing layer 32 is formed on the color filter 38, and a microlens 10 is formed thereon. The shape of the opening 30a is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on a semiconductor circuit substrate 29.

[0089] FIG. 24 is a diagram for explaining the state of the imaging light beam received by the imaging pixel 310 shown in FIG. 22 in comparison with FIG. 8, and a description of the parts that overlap with FIG. 8 will be omitted.

[0090] The imaging pixel 310 is composed of a microlens 10 and a photoelectric conversion unit 11 arranged behind it, and the shape of an opening 30a (see Figure 23) arranged close to the photoelectric conversion unit 11 is projected onto an exit pupil 90 spaced from the microlens 10 by the ranging pupil distance d, and the projected shape forms a region 95 that approximately circumscribes the ranging pupils 93, 94.

[0091] The photoelectric conversion section 11 outputs a signal corresponding to the intensity of an image formed on the microlens 10 by the imaging light beam 71 that passes through the region 95 and travels toward the microlens 10 .

[0092] FIG. 25 corresponds to the pixel layout diagram of FIG. 3 (the filter array corresponds to FIG. 4), and the imaging pixels 310 and the focus detection pixels 311 are arranged in a staggered pattern. That is, the focus detection pixels 311 are arranged in positions (odd rows and odd columns) or (even rows and even columns), and the imaging pixels 310 are arranged in positions (odd rows and even columns) or (odd rows and even columns). In terms of color filters in the Bayer array, the focus detection pixels 311 are provided with green filters, and the imaging pixels 310 are provided with red or blue filters. In terms of focus detection performance, the spectral sensitivity characteristics of the green filters are intermediate between the spectral sensitivity characteristics of the red filters and the spectral sensitivity characteristics of the blue filters, as shown in FIG. 5, so it is preferable to provide the focus detection pixels 311 with green filters. 6 and 22, the sum of the surface areas of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 is smaller than the surface area of ​​the photoelectric conversion unit 11 of the imaging pixel 310 due to the presence of the element isolation region 15. Therefore, in terms of imaging performance, since the sum of the photoelectric conversion signal values ​​output by the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 is smaller than the photoelectric conversion signal value of the photoelectric conversion unit 11 of the imaging pixel 310, it is preferable to provide the focus detection pixel 311 with more green filters than red and blue filters.

[0093] Fig. 26 is a block diagram showing the configuration of an image sensor 212 having the pixel layout shown in Fig. 25, and only the characteristic parts will be described while omitting the description of the parts that are the same as the configuration in Fig. 10. The main difference from Fig. 10 is that the number of memories constituting the second line memory 44 and the number of digital addition circuits constituting the column digital addition device 46 are reduced by providing a second column switch device 43 between the column AD conversion device 42 and the second line memory 44 and the column digital addition device 46.

[0094] The image sensor 212 has a pixel array section 40 in which a large number of focus detection pixels 311, each of which includes a pair of photoelectric conversion sections 13, 14, are arranged two-dimensionally in a matrix, as well as a row scanning circuit 41, a column AD conversion device 42, a second column switching device 43, a second line memory 44, a second column scanning circuit 51, a second horizontal output circuit 45, a column digital adding device 46, a first column switching device 47, a first line memory 48, a first column scanning circuit 52, a first horizontal output circuit 49 and a timing control circuit 50.

[0095] In this system configuration, the timing control circuit 50 generates clock signals and control signals that serve as references for the operation of the row scanning circuit 41, the column AD conversion device 42, the first column switch device 47, the second column switch device 43, the column digital adder 46, the first line memory 48, the first line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc., based on a master clock input from outside and a control signal input from the image sensor control unit 220, and provides these signals to the row scanning circuit 41, the column AD conversion device 42, the first column switch device 47, the second column switch device 43, the column digital adder 46, the first line memory 48, the first line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc.

[0096] In the pixel array section 40, imaging pixels 310 and focus detection pixels 311 are two-dimensionally arranged in 2N rows and 2M columns. In FIG. 26, the focus detection pixel 311 in the upper left is the pixel in the first row and first column, and a green filter in a Bayer array is arranged in this pixel. For this pixel arrangement of 2N rows and 2M columns, row control lines 21 (21(1) to 21(2N)) are wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are wired for each column. The total number of row control lines is 2N, and the total number of column signal lines is 4M. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to each output terminal corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each row control line 21.

[0097] The photoelectric conversion units of the imaging pixels 310 arranged in the same row and the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to a control signal R(L). The photoelectric conversion unit 11 of the imaging pixel 310 is connected to one of two column signal lines provided for each column, a column signal line 22(m)a, and an output signal (analog signal) of the photoelectric conversion unit 11 is output to the column signal line 22(m)a. In addition, one of the photoelectric conversion units 13 of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 is connected to one of two column signal lines provided for each column, a column signal line 22(m)b, and an output signal (analog signal) of the photoelectric conversion unit 13 is output to the column signal line 22(m)b. Furthermore, the other photoelectric conversion unit 14 of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 is connected to the other column signal line 22(m)a of the two column signal lines provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 14 is output to the column signal line 22(m)a. For example, when the Lth row of the pixel array unit 40 is selected by a control signal R(L) provided from the row scanning circuit 41, the output signal of the photoelectric conversion unit 11 of the imaging pixel 310 of the Lth row is output to the column signal lines (22(1)a to 22(2M)a), and the output signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 of the Lth row is output to the column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b). In this case, if L is an odd number, the imaging pixels 310 are arranged in the even columns of that row, and therefore the signal on the column signal line 22(2m)b corresponding to the even columns is an invalid signal. Conversely, if L is an even number, the imaging pixels 310 are arranged in the odd columns of that row, and therefore the signal on the column signal line 22(2m+1)b corresponding to the odd columns is an invalid signal.

[0098] The column AD conversion device 42 has 4M ADCs (analog-to-digital conversion circuits) 23(1)a, 23(1)b to 23(2M)a, 23(2M)b provided for each column signal line 22(1)a, 22(1)b to 22(2M)a, 22(2M)b provided corresponding to the pixel columns of the pixel array section 40, and converts analog signals output for each column from each pixel of the pixel array section 40 into H-bit digital signals (S(1)a, S(1)b to S(2M)a, S(2M)b) in response to a control signal TA1 provided by the timing control circuit 50, and outputs the digital signals.

[0099] The second column switch device 43 has M switches 24(1, 2) to 24(2M-1, 2M) provided for each two adjacent pixel columns, and selects and outputs the digital signals outputted from each ADC (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) in response to a control signal TW2 provided from the timing control circuit 50.

[0100] Figures 27(a) and (b) are diagrams explaining the selection operation of switches 24 (2m+1, 2m+2) provided in two adjacent pixel columns (the (2m+1) column and the (2m+2) column), in which four digital signals S(2m+1)a, S(2m+1)b, ​​S(2m+2)a, and S(2m+2)b are input to the switches 24 (2m+1, 2m+2) from four ADCs (23(2m+1)a, 23(2m+1)b, ​​23(2m+2)a, and 23(2m+2)b) corresponding to the two pixel columns (the (2m+1) column and the (2m+2) column). A multiple of four, for example, eight digital signals S(2m+1)a, S(2m+1)b, ​​S(2m+2)a, S(2m+2)b, S(2m+3)a, S(2m+3)b, S(2m+4)a, and S(2m+4)b may be input to switch 24(2m+1, 2m+2) from a multiple of four, for example, eight ADCs (23(2m+1)a, 23(2m+1)b, ​​23(2m+2)a, 23(2m+2)b, 23(2m+3)a, 23(2m+3)b, 23(2m+4)a, and 23(2m+4)b) corresponding to a multiple of two columns, for example, four pixel columns ((2m+1) columns, (2m+2) columns, (2m+3) columns, and (2m+4) columns). In this case, for example, a focus detection pixel 311 is arranged in the (2m+1)th column, and image sensors 310 are arranged in the (2m+2)th, (2m+3)th, and (2m+4)th columns.

[0101] 27(a) shows the selection operation of the switch 24(2m+1, 2m+2) when an odd-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, and four digital signals S(2m+1)a, S(2m+1)b, ​​S(2m+2)a, and S(2m+2)b corresponding to the imaging pixel 310 arranged in the odd-numbered row and even-numbered column and the focus detection pixel 311 in the odd-numbered row and odd-numbered column are input to the switch 24(2m+1, 2m+2). Of these, the signal S(2m+2)b is an invalid signal.

[0102] The switch 24(2m+1, 2m+2) selects and outputs digital signals S(2m+1)a, S(2m+1)b corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a pair of signals (Q(2m+1, 2m+2)a, Q(2m+1, 2m+2)b) for digital addition in accordance with a control signal TW2 (identification information for odd or even rows) input to the second column switch device 43.

[0103] 27(b) shows the selection operation of the switch 24(2m+1, 2m+2) when an even-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, and four digital signals S(2m+1)a, S(2m+1)b, ​​S(2m+2)a, and S(2m+2)b corresponding to the imaging pixel 310 arranged in the odd-numbered column of the even row and the focus detection pixel 311 in the even-numbered column of the even row are input to the switch 24(2m+1, 2m+2). Of these, the signal S(2m+1)b is an invalid signal.

[0104] The switch 24(2m+1, 2m+2) selects and outputs digital signals S(2m+2)a, S(2m+2)b corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 as a pair of signals (Q(2m+1, 2m+2)a, Q(2m+1, 2m+2)b) corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 in accordance with a control signal TW2 (identification information for odd or even rows) input to the second column switch device 43.

[0105] The second line memory 44 has a total of 2M memories (25(1,2)a, 25(1,2)b to 25(2M-1,2M)a, 25(2M-1,2M)b) provided as a pair for each of the M switches 24(1,2) to 24(2M-1,2M) of the second column switch device 43, and stores pairs of digital signals (Q(1,2)a, Q(1,2)b to Q(2M-1,2M)a, Q(2M-1,2M)b) corresponding to the pairs of photoelectric conversion units 13, 14 of the focus detection pixels 311 output for each of the M switches 24(1,2) to 24(2M-1,2M) as H-bit digital signals in response to a control signal TM2 provided from the timing control circuit 50. Here, each memory (25(1,2)a, 25(1,2)b to 25(22M-1,2M)a, 25(22M-1,2M)b) of the second line memory 44 stores the output signals of a pair of photoelectric conversion units 13, 14 for one row of M focus detection pixels as digital signals.

[0106] The column digital addition device 46 has a total of M digital addition circuits (26(1,2) to 26(2M-1,2M)) provided for each of the M switches 24(1,2) to 24(2M-1,2M) of the second column switch device 43, and adds pairs of digital signals (Q(1,2)a, Q(1,2)b to Q(2M-1,2M)a, Q(2M-1,2M)b) corresponding to a pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 outputted for each of the M switches 24(1,2) to 24(2M-1,2M) in accordance with a control signal TD1 provided from the timing control circuit 50, and outputs an H-bit sum digital signal (P(1,2) to P(2M-1,2M)).

[0107] The first column switch device 47 has M switches 27(1, 2) to 27(2M-1, 2M) provided for each two adjacent pixel columns, and selects and outputs digital signals (S(1)a, S(2)a to S(2M-1)a, S(2M)a) outputted for each of the 2M ADCs (23(1)a to 23(2M)a) and added digital signals (P(1, 2) to P(2M-1, 2M)) outputted for each of the M digital adder circuits (26(1, 2) to 26(2M-1, 2M)) in response to a control signal TW1 (identification information for odd or even rows) provided from a timing control circuit 50.

[0108] Figures 28(a) and (b) are diagrams explaining the selection operation of switch 27 (2m+1, 2m+2) provided for two adjacent pixel columns (the (2m+1) column and the (2m+2) column), in which two digital signals S(2m+1)a, S(2m+2)a are input to switch 27 (2m+1, 2m+2) from two ADCs (23(2m+1)a, 23(2m+2)a) corresponding to the two pixel columns (the (2m+1) column and the (2m+2) column), and one added digital signal P(2m+1, 2m+2) is input from digital adder circuit 26 (2m+1, 2m+2).

[0109] Figure 28(a) shows the selection operation of the switch 27(2m+1, 2m+2) when an odd-numbered row of the pixel array section 40 is selected by the row scanning circuit 41. The switch 27(2m+1, 2m+2) receives one digital signal S(2m+2)a corresponding to the photoelectric conversion unit 11 of the imaging pixel 310 arranged in the odd-numbered row and even-numbered column, one digital signal S(2m+1)a corresponding to the photoelectric conversion unit 14 of the focus detection pixel 311 arranged in the odd-numbered row and odd-numbered column, and a sum digital signal P(2m+1, 2m+2) (corresponding to the signal of the imaging pixel) obtained by adding together a pair of signals S(2m+1)a, (2m+1)b corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 arranged in the odd-numbered row and odd-numbered column from the digital addition circuit 26(2m+1, 2m+2). Of these, the signal S(2m+1)a input from the ADC 23(2m+1)a does not correspond to a signal from an imaging pixel.

[0110] In response to a control signal TW1 (identification information for odd or even rows) input to the first column switch device 47, the switch 27(2m+1, 2m+2) selects and outputs a digital sum signal P(2m+1, 2m+2) obtained by adding together digital signals corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a signal U(2m+1) equivalent to the signal of a virtual imaging pixel arranged in an odd-numbered column, and also selects and outputs a digital signal S(2m+2)a corresponding to the imaging pixel 310 as the signal U(2m+1) of an imaging pixel arranged in an even-numbered column.

[0111] 28(b) shows the selection operation of the switch 27(2m+1, 2m+2) when an even-numbered row of the pixel array section 40 is selected by the row scanning circuit 41. The switch 27(2m+1, 2m+2) receives one digital signal S(2m+1)a corresponding to the photoelectric conversion unit 11 of the imaging pixel 310 arranged in an odd-numbered column of an even-numbered row, one digital signal S(2m+2)a corresponding to the photoelectric conversion unit 14 of the focus detection pixel 311 arranged in an even-numbered column of an even-numbered row, and a sum digital signal P(2m+1, 2m+2) (corresponding to the signal of the imaging pixel) obtained by adding a pair of signals S(2m+2)a, (2m+2)b corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 arranged in an even-numbered column of an even-numbered row from the digital addition circuit 26(2m+1, 2m+2). Of these signals, the signal S(2m+2)a does not correspond to the signal of the imaging pixel.

[0112] In response to a control signal TW1 (identification information for odd or even rows) input to the first column switch device 47, the switch 27(2m+1, 2m+2) selects and outputs a digital sum signal P(2m+1, 2m+2), which is obtained by adding together digital signals corresponding to a pair of photoelectric conversion units of the focus detection pixel 311, as a signal U(2m+2) equivalent to the signal of a virtual imaging pixel arranged in an even column, and also selects and outputs a digital signal S(2m+1)a corresponding to the imaging pixel 310 as the signal U(2m+1) of the imaging pixel arranged in an odd column.

[0113] The first line memory 48 has 2M memories (28(1) to 28(2M)) provided in pairs for each of the M switches (27(1,2) to 27(2M-1,2M)) constituting the column switch device 47, and stores a pair of digital signals output for each switch (27(1,2) to 27(2M-1,2M)) as an H-bit digital signal in response to a control signal TM1 provided from the timing control circuit 50. Here, in each memory (28(1) to 28(2M)) of the first line memory 48, a sum signal (corresponding to an output signal of an imaging pixel) obtained by adding together the output signals of a pair of photoelectric conversion units 13, 14 for one row of focus detection pixels and the output signals of the photoelectric conversion units of the imaging pixels are stored as digital signals in accordance with the arrangement order of the focus detection pixels.

[0114] The second column scanning circuit 51 is configured with a shift register and the like, and controls the column addresses and column scanning of the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) in the second line memory 44 under the control of the timing control circuit 50. The second line memory 44 operates in response to a scanning signal TS2 provided from the second column scanning circuit 51, and the H-bit digital signals stored in each of the memories (25(1,2)a, 25(1,2)b to 25(2M-1,2M)a, 25(2M-1,2M)b) are sequentially read out to the second horizontal output circuit 45, and are serially output (the number of data is 2M) to the outside as output signals (digital signals) of the pair of photoelectric conversion units 13, 14 for focus detection via the second horizontal output circuit 45.

[0115] The first column scanning circuit 52 is configured with a shift register and the like, and controls the column addresses and column scanning of the memories (28(1) to 28(2M)) in the first line memory 48 under the control of the timing control circuit 50. The first line memory 48 operates in response to a scanning signal TS1 provided from the first column scanning circuit 52, and the H-bit digital signals and the sum digital signal stored in each of the memories (28(1) to 28(2M)) are sequentially read out to the first horizontal output circuit 49 and serially output to the outside via the first horizontal output circuit 49 as an output signal (digital signal) equivalent to the output signal of the imaging pixel array.

[0116] Next, using the timing chart of FIG. 29, a case will be described in which, in the configuration of the image sensor shown in FIG. 26, an individual read operation of the output signals (signals for focus detection) of a pair of photoelectric conversion units of a focus detection pixel and a read operation of an output signal (signal for image processing) corresponding to the output signal of the image pickup pixel are performed in parallel during one frame period.

[0117] In the configuration of the image sensor shown in FIG. 26, the outline of the row scanning selection operation by the row scanning circuit 41 is the same as the operation shown in FIG.

[0118] Fig. 29 is an enlarged view of the operating portions of rows (2n+1), (2n+2), and (2n+3) in Fig. 11. When row (2n+1) of the pixel array section 40 is selected by the control signal R(2n+1), analog signals of one line of focus detection pixels 311 and imaging pixels 310 in row (2n+1) are output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b). Analog signals from a pair of photoelectric conversion units 13, 14 of focus detection pixels 311 arranged in an odd-numbered column for one line of (2n+1) rows output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) and an analog signal from a photoelectric conversion unit 11 of an image pickup signal 310 arranged in an even-numbered column are converted into digital signals by ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of a column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b to 22(2M)a, 22(2M) in response to a control signal TA1.

[0119] The digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd columns for one line of (2n+1) rows and the digital signals of the photoelectric conversion units 11 of the image pickup signal 310 arranged in even columns are input from the column AD conversion device 42 to the second column switch device 43, and the digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd columns are selected and output by the second column switch device 43 (24(1, 2) to 24(2M-1, 2M)) in response to a control signal TW2.

[0120] The digital signals of a pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd-numbered columns output from the second column switch device 43 (24(1, 2) to 24(2M-1, 2M)) are stored in 2M memories (25(1, 2)a, 25(1, 2)b to 25(2M-1, 2M)a, 25(2M-1, 2M)b) of the second line memory 44 in response to the control signal TM2.

[0121] At the same time, the digital signals from a pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd-numbered columns are added together and output by M digital addition circuits (26(1, 2) to 26(2M-1, 2M)) of the column digital addition device 46 in response to a control signal TD1.

[0122] The sum digital signal obtained by adding together the digital signal corresponding to one photoelectric conversion unit 14 of the pair of photoelectric conversion units of the focus detection pixels 311 arranged in odd-numbered columns, the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in even-numbered columns, and the digital signal corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd-numbered columns, which is input to the first column switch device 47, is selected by the first column switch device 47 (27(1,2) to 27(2M-1,2M)) in response to a control signal TW1, and the sum digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in odd-numbered columns is output as the output signal of the image pickup pixels in the odd-numbered columns, and the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in the even-numbered columns is output as the output signal of the image pickup pixels in the even-numbered columns.

[0123] The summed digital signal of a pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 arranged in an odd-numbered column selected and output by the first column switch device 47 and the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in an even-numbered column are stored in memories (28(1) to 28(2M)) of the first line memory 48 in response to a control signal TM1.

[0124] Digital signals from pairs of photoelectric conversion units 13, 14 of M focus detection pixels 311 arranged in odd columns of (2n+1) rows stored in 2M memories (25(1,2)a, 25(1,2)b to 25(2M-1,2M)a, 25(2M-1,2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 in response to the scanning signal TS2 during the period until the next horizontal synchronization signal HS is generated.

[0125] Similarly, 2M digital signals (the summed digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the odd columns and the digital signal of the photoelectric conversion unit 11 of the image pickup pixel 310 arranged in the even columns) corresponding to the output signals of the (2n+1) rows stored in the memories ((28(1) to 28(2M)) of the first line memory 48 are sequentially serially output to the outside from the first horizontal output circuit 49 in accordance with the scanning signal TS1 during the period until the next horizontal synchronization signal HS is generated.

[0126] When a control signal R(2n+2) is issued in synchronization with the next horizontal synchronization signal HS and the (2n+2)th row of the pixel array section 40 is selected, analog signals from the focus detection pixels 311 and imaging pixels 310 for one line of the (2n+2)th row are output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b). Analog signals from a pair of photoelectric conversion units 13, 14 of focus detection pixels 311 arranged in an even column for one line of (2n+2) rows output to column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) and an analog signal from a photoelectric conversion unit 11 of an image pickup signal 310 arranged in an odd column are converted into digital signals by ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of a column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b to 22(2M)a, 22(2M)b, in response to a control signal TA1.

[0127] The digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns of one line of (2n+2) rows and the digital signals of the photoelectric conversion units 11 of the image pickup signal 310 arranged in the odd columns are input from the column AD conversion device 42 to the second column switch device 43, and the digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns are selected and output by the second column switch device 43 (24(1, 2) to 24(2M-1, 2M)) in response to a control signal TW2.

[0128] The digital signals of a pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns output from the second column switch device 43 (24(1, 2) to 24(2M-1, 2M)) are stored in 2M memories (25(1, 2)a, 25(1, 2)b to 25(2M-1, 2M)a, 25(2M-1, 2M)b) of the second line memory 44 in response to the control signal TM2.

[0129] At the same time, the digital signals from a pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns are added together and output by M digital addition circuits (26(1, 2) to 26(2M-1, 2M)) of the column digital addition device 46 in response to the control signal TD1.

[0130] The sum digital signal obtained by adding together the digital signal corresponding to one photoelectric conversion unit 14 of the pair of photoelectric conversion units of the focus detection pixels 311 arranged in the even columns, the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in the odd columns, and the digital signal corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns, which is input to the first column switch device 47, is selected by the first column switch device 47 (27(1,2) to 27(2M-1,2M)) in response to a control signal TW1, and the sum digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns is output as the output signal of the image pickup pixels in the even columns, and the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in the odd columns is output as the output signal of the image pickup pixels in the odd columns.

[0131] The summed digital signal of a pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 arranged in an even-numbered column selected and output by the first column switch device 47 and the digital signal of the photoelectric conversion unit 11 of the image pickup signal 310 arranged in an odd-numbered column are stored in memories (28(1) to 28(2M)) of the first line memory 48 in response to a control signal TM1.

[0132] Digital signals from pairs of photoelectric conversion units 13, 14 of M focus detection pixels 311 arranged in even columns of (2n+2) rows stored in 2M memories (25(1,2)a, 25(1,2)b to 25(2M-1,2M)a, 25(2M-1,2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 in response to the scanning signal TS2 during a period until the next horizontal synchronization signal HS is generated. Based on the digital signals output from the second horizontal output circuit 45, the focus detection CPUa 222 of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system) and adjusts the focus state, as shown in FIG. 13.

[0133] Similarly, 2M digital signals (the summed digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 arranged in the odd columns) corresponding to the output signals of the imaging pixels on the (2n+2) rows stored in the memories ((28(1) to 28(2M)) of the first line memory 48 are sequentially serially output to the outside from the first horizontal output circuit 49 in response to the scanning signal TS1 during the period until the next horizontal synchronization signal HS is generated. Based on the 2M digital signals (the summed digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 on the even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 on the odd columns) output from the first horizontal output circuit 49, the image processing CPUb 223 of the body drive control device 214 generates image data as shown in FIG. 14. However, in this embodiment, 14, data of 2M digital signals (addition digital signals of pairs of photoelectric conversion units 13, 14 of focus detection pixels 311 on even columns and digital signals of photoelectric conversion units 11 of imaging pixels 310 on odd columns) output from the first horizontal output circuit 49 is read out, and the read out data is subjected to image processing for display and then displayed on the electronic viewfinder. Also, in step S230 in Fig. 14, data of 2M digital signals (addition digital signals of pairs of photoelectric conversion units 13, 14 of focus detection pixels 311 on even columns and digital signals of photoelectric conversion units 11 of imaging pixels 310 on odd columns) output from the first horizontal output circuit 49 is read out, and the read out data is subjected to well-known image processing (demosaic processing, noise processing, gradation processing, white balance processing, etc.) to generate image data.

[0134] When a control signal R(2n+3) is issued in synchronization with the next horizontal synchronization signal HS and the (2n+3)th row of the pixel array section 40 is selected, processing is repeated for one line of focus detection pixels 311 and imaging pixels 310 in the (2n+3)th row in the same manner as in the case of the control signal R(2n+1).

[0135] As described above, in the third embodiment, the imaging pixels 310 and the focus detection pixels 311 are mixed in the pixel array section 40, the focus detection pixels 311 are arranged at the position of the green filter in the Bayer array, and the imaging pixels 310 are arranged at the positions of the red and blue filters. In addition, the selection process of the first column switch device 47 and the second column switch device 43 is switched depending on whether the odd or even row of the pixel array section 40 is scanned by the row scanning circuit 41. Therefore, compared to the configuration of the image sensor in FIG. 10, it is possible to reduce the number of digital circuits constituting the column digital addition device 46, which has a larger circuit scale than the switch circuit (reduced from 2M to M), and to reduce the number of memories constituting the second line memory 44, which also has a larger circuit scale than the switch circuit (from 4M to 2M), thereby simplifying the configuration of the image sensor. At the same time, the amount of data output from the second horizontal output circuit 45 during a horizontal scanning period is halved (reduced from 4M to 2M) compared to the configuration of the image sensor in Fig. 10 and becomes the same as the amount of data output from the first horizontal output circuit 49 during a horizontal scanning period, making it possible to reduce the data transfer rate. Also, the focus detection data read out from the second horizontal output circuit 45 is unified into data from focus detection pixels equipped with green filters, which is convenient for focus detection (there are many subjects in nature that have green contrast, and generally when a photographing lens has chromatic aberration, the focal position for green is the in-focus position).

[0136] In the third embodiment described above, the number of data for focus detection corresponding to the horizontal scanning of the second column scanning circuit 51 in one row is the same as the number of data for the image corresponding to the horizontal scanning of the first column scanning circuit 52. Therefore, by commonizing the second column scanning circuit 51 and the first column scanning circuit 52 (for example, using the scanning signal TS1 of the first column scanning circuit 52 as the scanning signal TS2 of the second line memory 44), it is possible to further simplify the configuration of the image sensor.

[0137] In the third embodiment described above, it has been explained that data from pairs of photoelectric conversion units of all focus detection pixels is read out every frame for focus detection purposes. However, reading out data from pairs of photoelectric conversion units of all focus detection pixels is a heavy load and requires a large amount of memory capacity for storing the data, so frame thinning (reading out once every few frames), row thinning (reading out one row out of several rows), partial row readout (reading out only some of the rows), column thinning (reading out one column out of several columns), or partial column readout (reading out only some of the columns) may be performed as necessary.

[0138] FIG. 30 is a timing chart corresponding to FIG. 29 when performing row partial readout (reading out data from a pair of photoelectric conversion units of focus detection pixels in only the (2n+2)th row). The operation when the (2n+2)th row of the pixel array unit 40 is selected by the control signal R(2n+2) is the same as that in FIG. 29. On the other hand, when a row other than the (2n+2)th row is selected (operation according to the control signal R(2n+1) and control signal R(2n+3) in FIG. 30), the control signal TM2 is not generated, and data from the pair of photoelectric conversion units of the focus detection pixels is not stored in the memories (25(1,2)a, 25(1,2)b to 25(2M-1,2M)a, 25(2M-1,2M)b) of the second line memory 44. In addition, since the scanning signal TS2 is not generated, the data is not sequentially serially output from the second horizontal output circuit 45 to the outside during the period until the next horizontal synchronization signal HS is generated.

[0139] The rows to be read out from the row portion and the columns to be read out from the column portion can be changed by sending information from the body drive control device 214 to the image sensor 212 according to the position of the selected focus detection area.

[0140] In the third embodiment described above, as shown in FIG. 28, each switch constituting the first column switch device 47 selects two signals corresponding to the output signals of imaging pixels depending on whether an odd-numbered row or an even-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, and distributes the two selected signals so as to match the pixel arrangement in the selected row and stores them in the memories (28(1) to 28(2M)) of the first line memory 48. Alternatively, the signals may be stored fixedly in the memories ((28(1) to 28(2M)) of the first line memory 48 without being allocated in the above manner, and the scanning signal TS1 supplied by the first column scanning circuit 52 to the memories ((28(1) to 28(2M)) of the first line memory 48 may be changed depending on whether an odd-numbered row or an even-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, so that the scanning signal TS1 is consistent with the pixel arrangement order in the selected row, and the memories ((28(1) to 28(2M)) of the first line memory 48 are scanned.

[0141] <Fourth embodiment> The fourth embodiment is a modification of the third embodiment, and in the configuration of the image sensor 212 of the fourth embodiment shown in FIG. 31, the description of the same parts as those in FIG. 26 will be omitted, and only the characteristic parts will be described. The difference between the pixel array section 40 and FIG. 26 is that in the pixel array section 40 of FIG. 26, two column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are wired for each column, and the number of column signal lines is 4M in total, whereas in FIG. 31, the number of column signal lines is reduced to one in even columns, and the column signal lines of odd columns are shared, thereby reducing the number of column signal lines to 3M in total. In the fourth embodiment, the number of column signal lines is reduced, thereby alleviating the overcrowding of the wiring layout in the pixel array section 40 and enabling an increase in the area of ​​the photoelectric conversion section, enabling higher quality image acquisition and high-precision focus detection.

[0142] Furthermore, in the fourth embodiment, with the reduction in the number of column signal lines, the number of ADCs constituting the column AD conversion device 42 can also be reduced (from 4M to 3M), and the configuration of the imaging element can be simplified.

[0143] In FIG. 31, one end of each of the row control lines 21 (21(1) to 21(2N)) is connected to an output terminal corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each of the row control lines 21.

[0144] The photoelectric conversion units of the imaging pixels 310 arranged in the same row and the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to the control signal R(L). In the pixel array section 40, two column signal lines 22(2m+1)a, 22(2m+1)b are arranged in the odd columns, and one column signal line 22(2m+2)a is arranged in the even columns. The photoelectric conversion units 11 of the imaging pixels 310 arranged in the odd columns and the photoelectric conversion units 14 of the focus detection pixels 311 arranged in the odd columns are connected to one of the two column signal lines arranged in the odd columns, the column signal line 22(2m+1)a, and the photoelectric conversion units 13 of the focus detection pixels 311 arranged in the odd columns are connected to the other column signal line 22(2m+1)b arranged in the odd columns. In addition, the photoelectric conversion units 11 of the imaging pixels 310 provided in the even columns and the photoelectric conversion units 14 of the focus detection pixels 311 provided in the even columns are connected to a column signal line 22(2m+2)a provided in the even columns, and the photoelectric conversion units 13 of the focus detection pixels 311 provided in the even columns are connected to a column signal line 22(2m+1)b provided in the odd columns.

[0145] For example, when an odd-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, the output signal of the photoelectric conversion unit 11 of the imaging pixel 310 of the odd-numbered row is output to the column signal line 22(2m+2)a, and the output signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 of the odd-numbered row are output to the column signal lines 22(2m+1)a and 22(2m+1)b. When an even-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, the output signal of the photoelectric conversion unit 11 of the imaging pixel 310 of the even-numbered row is output to the column signal line 22(2m+1)a, the output signal of the photoelectric conversion unit 14 of the focus detection pixel 311 of the even-numbered row is output to the column signal line 22(2m+2)a, and the output signal of the photoelectric conversion unit 13 of the focus detection pixel 311 of the even-numbered row is output to the column signal line 22(2m+1)b.

[0146] The column AD conversion device 42 has 3M ADCs (analog-to-digital conversion circuits) 23(1)a, 23(1)b to 23(2M)a provided for each of the 3M column signal lines 22(1)a, 22(1)b to 22(2M)a provided corresponding to the pixel columns of the pixel array section 40, and converts analog signals output for each column from each pixel of the pixel array section 40 into H-bit digital signals (S(1)a, S(1)b to S(2M)a) in response to a control signal TA1 provided by the timing control circuit 50, and outputs the converted signals.

[0147] The second column switch device 43 has M switches 24(1, 2) to 24(2M-1, 2M) provided for each two adjacent pixel columns, and selects and outputs the digital signals outputted from each ADC (23(1)a, 23(1)b to 23(2M)a) in response to a control signal TW2 provided from the timing control circuit 50.

[0148] Figures 32(a) and (b) are diagrams explaining the selection operation of switches 24 (2m+1, 2m+2) provided in two adjacent pixel columns (columns (2m+1) and (2m+2)), in which three digital signals S(2m+1)a, S(2m+1)b, ​​and S(2m+2)a are input to the switches 24 (2m+1, 2m+2) from two ADCs (23(2m+1)a, 23(2m+1)b) corresponding to the odd-numbered column (2m+1) and one ADC (23(2m+2)a) corresponding to the even-numbered column (2m+2).

[0149] Figure 32(a) shows the selection operation of switch 24(2m+1, 2m+2) when an odd-numbered row of the pixel array section 40 is selected by the row scanning circuit 41, and three digital signals S(2m+1)a, S(2m+1)b, ​​S(2m+2)a corresponding to an imaging pixel 310 arranged in an odd-numbered row and even-numbered column and a focus detection pixel 311 in an odd-numbered row and odd-numbered column are input to switch 24(2m+1, 2m+2).

[0150] In response to a control signal TW2 (indicating an odd-numbered row) input to the second column switch device 43, the switch 24(2m+1, 2m+2) selects and outputs digital signals S(2m+1)a, S(2m+1)b corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a pair of signals (Q(2m+1, 2m+2)a, Q(2m+1, 2m+2)b) for digital addition.

[0151] Figure 32(b) shows the selection operation of switch 24(2m+1, 2m+2) when an even row of the pixel array section 40 is selected by the row scanning circuit 41, and three digital signals S(2m+1)a, S(2m+1)b, ​​and S(2m+2)a corresponding to the imaging pixel 310 arranged in the odd column of the even row and the focus detection pixel 311 in the even column of the even row are input to switch 24(2m+1, 2m+2).

[0152] In response to a control signal TW2 (indicating an even-numbered row) input to the second column switch device 43, the switch 24(2m+1, 2m+2) selects and outputs digital signals S(2m+2)a, S(2m+1)b corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 as a pair of signals (Q(2m+1, 2m+2)a, Q(2m+1, 2m+2)b) corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311.

[0153] <Fifth embodiment> The fifth embodiment is a modified example of the configuration of focus detection pixels in the pixel array unit in the third embodiment. Fig. 33 corresponds to the pixel layout diagram in Fig. 25 (the filter arrangement corresponds to Fig. 4), and focus detection pixels 311 arranged on even-numbered rows in Fig. 25 are replaced with focus detection pixels 312 having a pair of photoelectric conversion units 16, 17 arranged side by side in the vertical direction.

[0154] That is, in odd-numbered rows, focus detection pixels 311 are arranged in odd-numbered columns and imaging pixels 310 are arranged in even-numbered columns, and in even-numbered rows, imaging pixels 310 are arranged in odd-numbered columns and focus detection pixels 312 are arranged in even-numbered columns. In terms of color filters in the Bayer array, the focus detection pixels 311 and focus detection pixels 312 are equipped with green filters, and the imaging pixel 310 is equipped with a red filter or a blue filter.

[0155] Fig. 34 is a block diagram showing the configuration of an image sensor 212 having the pixel layout shown in Fig. 33, and only characteristic parts will be described while omitting explanations of parts that are the same as the configuration in Fig. 26. The pixel array section 40 differs from Fig. 26 in that focus detection pixels 312 each having a pair of photoelectric conversion units 16, 17 separated in the vertical direction are arranged in an even-numbered row and an even-numbered column.

[0156] The pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 arranged in an even-numbered row and an even-numbered column are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to a control signal R(L) (L is an even number). One of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 in an even-numbered row and an even-numbered column is connected to one of two column signal lines, column signal line 22(2m+2)a, and an output signal (analog signal) of the photoelectric conversion unit 16 is output to column signal line 22(2m+2)a. The other of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 is connected to the other of two column signal lines, column signal line 22(2m+2)b, and an output signal (analog signal) of the photoelectric conversion unit 17 is output to column signal line 22(2m+2)b.

[0157] When an image sensor 212 having the above-described configuration is used, phase difference detection is possible for a subject image having contrast that varies in the horizontal direction, using a pair of data obtained by grouping the data of a pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 having a green filter arranged in an odd-numbered row and even-numbered column in the horizontal direction for each photoelectric conversion unit, and phase difference detection is possible for a subject image having contrast that varies in the vertical direction, using a pair of data obtained by grouping the data of a pair of photoelectric conversion units 16, 17 of a focus detection pixel 312 having a green filter arranged in an even-numbered row and even-numbered column in the vertical direction for each photoelectric conversion unit.

[0158] Sixth embodiment The sixth embodiment is a modified example of the configuration of the focus detection pixels in the pixel array of the fourth embodiment. The pixel layout in the sixth embodiment is the same as that in Fig. 33, and the focus detection pixels 311 arranged in the even-numbered rows of Fig. 25 are replaced with focus detection pixels 312 having a pair of photoelectric conversion units 16, 17 arranged side by side in the vertical direction.

[0159] That is, in odd-numbered rows, focus detection pixels 311 are arranged in odd-numbered columns and imaging pixels 310 are arranged in even-numbered columns, and in even-numbered rows, imaging pixels 310 are arranged in odd-numbered columns and focus detection pixels 312 are arranged in even-numbered columns. In terms of color filters in the Bayer array, the focus detection pixels 311 and focus detection pixels 312 are equipped with green filters, and the imaging pixel 310 is equipped with a red filter or a blue filter.

[0160] Fig. 35 is a block diagram showing the configuration of an image sensor 212 having the pixel layout shown in Fig. 33, and only the characteristic parts will be described while omitting a description of the parts that are the same as the configuration in Fig. 31. The pixel array section 40 differs from Fig. 31 in that focus detection pixels 312 each having a pair of photoelectric conversion units 16, 17 separated in the vertical direction are arranged in an even-numbered row and an even-numbered column.

[0161] The pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 arranged in an even-numbered row and an even-numbered column are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are performed simultaneously in response to a control signal R(L) (L is an even number). One of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 is connected to one of the column signal lines 22(2m+2)a of one column signal line provided in an even-numbered column, and an output signal (analog signal) of the photoelectric conversion unit 16 is output to the column signal line 22(2m+2)a. The other of the pair of photoelectric conversion units 16, 17 of each focus detection pixel 312 is connected to one of the column signal lines 22(2m+1)b of two column signal lines provided in odd-numbered columns, and an output signal (analog signal) of the photoelectric conversion unit 17 is output to the column signal line 22(2m+1)b.

[0162] When an image sensor 212 having the above-described configuration is used, phase difference detection is possible for a subject image having contrast that varies in the horizontal direction, using a pair of data obtained by grouping the data of a pair of photoelectric conversion units 13, 14 of a focus detection pixel 311 having a green filter arranged in an odd-numbered row and even-numbered column in the horizontal direction for each photoelectric conversion unit, and phase difference detection is possible for a subject image having contrast that varies in the vertical direction, using a pair of data obtained by grouping the data of a pair of photoelectric conversion units 16, 17 of a focus detection pixel 312 having a green filter arranged in an even-numbered row and even-numbered column in the vertical direction for each photoelectric conversion unit.

[0163] Seventh embodiment In the first to sixth embodiments, the pixels (focus detection pixels, imaging pixels) were arranged in a square lattice pattern in the pixel array section 40, but the present invention can also be applied to pixel arrays other than a square lattice pattern.

[0164] Fig. 36 shows a pixel array known as a honeycomb array, which is a pixel array obtained by rotating a square lattice array by 45 degrees. Fig. 37 shows a filter array corresponding to the pixel layout of Fig. 36. The filter array shown in Fig. 37 is a filter array obtained by tilting a Bayer array by 45 degrees. In the honeycomb array shown in Figs. 36 and 37, focus detection pixels 411 are arranged, each having a pair of photoelectric conversion units 33, 34 arranged side by side in the horizontal direction.

[0165] The matrix in such a honeycomb arrangement is defined as follows: the horizontal pixel array in which green filters are arranged is the odd-numbered rows, the horizontal pixel array in which red or blue filters are arranged is the even-numbered rows, the vertical pixel array in which green filters are arranged is the odd-numbered columns, and the vertical pixel array in which red or blue filters are arranged is the even-numbered columns.

[0166] Fig. 38 is a block diagram showing the configuration of an image sensor 212 having a pixel layout (2N rows and 2M columns) in the honeycomb arrangement shown in Fig. 36, in which every other focus detection pixel 311 arranged in the pixel array section 40 in the configuration of the image sensor shown in Fig. 10 is thinned out and replaced with focus detection pixels 411. That is, in odd-numbered rows, focus detection pixels 411 are arranged only in odd-numbered columns, and in even-numbered rows, focus detection pixels 411 are arranged only in even-numbered columns.

[0167] In the pixel array section 40, focus detection pixels 411 are arranged two-dimensionally in 2N rows and 2M columns. In FIG. 38, the focus detection pixel 411 in the upper left is the pixel in the first row and first column, and a green filter is arranged in this pixel. For this pixel arrangement of 2N rows and 2M columns, row control lines 21 (21(1) to 21(2N)) are wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M-1)a, 22(2M-1)b) are wired in the odd columns. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to an output terminal of the row scanning circuit 41 corresponding to each row, and control signals R(1) to R(2N) are output to each row control line 21.

[0168] The pair of photoelectric conversion units 33, 34 of each focus detection pixel 411 are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to a control signal R(L). Furthermore, one photoelectric conversion unit 33 of the pair of photoelectric conversion units 33, 34 of the focus detection pixel 411 arranged in the odd-numbered rows and odd-numbered columns (2m+1 columns) is connected to one column signal line 22(2m+1)b of two column signal lines provided in the odd-numbered columns (2m+1 columns), and the output signal (analog signal) of the photoelectric conversion unit 33 is output to the column signal line 22(2m+1)b. In addition, the other photoelectric conversion unit 34 of the pair of photoelectric conversion units 33, 34 of each focus detection pixel 411 is connected to the other column signal line 22(2m+1)a of the two column signal lines provided in the odd-numbered columns (2m+1 columns), and the output signal (analog signal) of the photoelectric conversion unit 34 is output to the column signal line 22(2m+1)a.

[0169] Moreover, one photoelectric conversion unit 33 of a pair of photoelectric conversion units 33, 34 of a focus detection pixel 411 arranged in an even-numbered row and an even-numbered column (2m+2 column) is connected to one column signal line 22(2m+1)b of two column signal lines provided in an odd-numbered column (2m+1 column), and an output signal (analog signal) of the photoelectric conversion unit 33 is output to the column signal line 22(2m+1)b. Moreover, the other photoelectric conversion unit 34 of the pair of photoelectric conversion units 33, 34 of each focus detection pixel 411 is connected to the other column signal line 22(2m+1)a of the two column signal lines provided in an odd-numbered column (2m+1 column), and an output signal (analog signal) of the photoelectric conversion unit 34 is output to the column signal line 22(2m+1)a.

[0170] The column AD conversion device 42 has ADCs (analog-to-digital conversion circuits) 23(1)a, 23(1)b to 23(2M-1)a, 23(2M-1)b provided for each column signal line 22(1)a, 22(1)b to 22(2M-1)a, 22(2M-1)b provided corresponding to the pixel columns of the pixel array section 40, and converts a pair of analog signals output for each column from each focus detection pixel 411 of the pixel array section 40 into an H-bit digital signal in response to a control signal TA1 provided by the timing control circuit 50, and outputs the digital signal.

[0171] The second line memory 44 has memories (25(1)a, 25(1)b to 25(2M-1)a, 25(2M-1)b) provided for each ADC (23(1)a, 23(1)b to 23(2M-1)a, 23(2M-1)b) constituting the column AD conversion device 42, and stores the digital signal output from each ADC (23(1)a, 23(1)b to 23(2M-1)a, 23(2M-1)b) as an H-bit digital signal in response to a control signal TM2 provided from the timing control circuit 50. Here, the output signals of the pairs of photoelectric conversion units 33, 34 for one row of focus detection pixels are stored as digital signals in each memory (25(1)a, 25(1)b to 25(2M-1)a, 25(2M-1)b) of the second line memory 44.

[0172] The column digital adding device 46 has digital adding circuits (26(1) to 26(2M-1)) provided for each pair of ADCs ((23(1)a, 23(1)b) to (23(2M-1)a, 23(2M-1)b)) that constitute the column AD conversion device 42, and adds up the digital signals output from the pairs of ADCs ((23(1)a, 23(1)b) to (23(2M-1)a, 23(2M-1)b)) in response to a control signal TD1 provided from the timing control circuit 50, and outputs an H-bit added digital signal.

[0173] The first line memory 48 has memories (28(1) to 28(2M-1)) provided for each digital adder circuit (26(1) to 26(2M-1)) constituting the column digital adder 46, and stores the added digital signal output from each digital adder circuit (26(1) to 26(2M-1)) as an H-bit digital signal in response to a control signal TM1 provided from the timing control circuit 50. Here, each memory (28(1) to 28(2M-1)) of the first line memory 48 stores, as a digital signal, a sum signal (corresponding to the output signal of an imaging pixel) obtained by adding together the output signals of a pair of photoelectric conversion units 33, 34 for one row of focus detection pixels.

[0174] The second line memory 44 operates in response to a scanning signal TS2 provided from the second column scanning circuit 51, and the H-bit digital signals stored in each of the memories (25(1)a, 25(1)b to 25(2M-1)a, 25(2M-1)b) are sequentially read out to the second horizontal output circuit 45 and serially output to the outside via the second horizontal output circuit 45 as output signals (digital signals) of a pair of photoelectric conversion units 33, 34 for focus detection.

[0175] The first line memory 48 operates in response to a scanning signal TS1 provided from the first column scanning circuit 52, and the H-bit summed digital signals stored in each of the memories (28(1) to 28(2M-1)) are sequentially read out to the first horizontal output circuit 49 and serially output to the outside via the first horizontal output circuit 49 as output signals (digital signals) equivalent to the output signals of the imaging pixels.

[0176] Eighth embodiment The eighth embodiment is a modified example of the configuration of focus detection pixels in the pixel array unit in the seventh embodiment. Fig. 39 corresponds to the pixel layout diagram in Fig. 36 (the filter arrangement corresponds to Fig. 37), and focus detection pixels 411 arranged on even-numbered rows in Fig. 36 are replaced with focus detection pixels 412 having a pair of photoelectric conversion units 36, 37 arranged side by side in the vertical direction.

[0177] That is, in the odd-numbered rows, focus detection pixels 411 are arranged in odd-numbered columns, and in the even-numbered rows, focus detection pixels 412 are arranged in even-numbered columns.

[0178] Figure 40 is a block diagram showing the configuration of an image sensor 212 having the pixel layout shown in Figure 39, and only the characteristic parts will be described while omitting a description of the parts that are the same as the configuration in Figure 38. The pixel array section 40 differs from Figure 38 in that focus detection pixels 412 each having a pair of photoelectric conversion units 36, 37 separated in the vertical direction are arranged in an even-numbered row and an even-numbered column.

[0179] The pair of photoelectric conversion units 36, 37 of each focus detection pixel 412 arranged in the even columns (2m+2 columns) of even rows are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to a control signal R(L) (L is an even number). One photoelectric conversion unit 36 ​​of the pair of photoelectric conversion units 36, 37 of each focus detection pixel 412 is connected to one column signal line 22(2m+1)a of two column signal lines provided in the odd columns (2m+1 columns), and the output signal (analog signal) of the photoelectric conversion unit 36 ​​is output to the column signal line 22(2m+1)a. In addition, the other photoelectric conversion unit 37 of the pair of photoelectric conversion units 36, 37 of each focus detection pixel 412 is connected to the other column signal line 22(2m+1)b of the two column signal lines provided in the odd-numbered columns (2m+1 columns), and the output signal (analog signal) of the photoelectric conversion unit 37 is output to the column signal line 22(2m+1)b.

[0180] When the image sensor 212 configured as described above is used, phase difference detection is possible for a subject image having contrast that varies in the horizontal direction, using a pair of data obtained by grouping horizontally the data of the pair of photoelectric conversion units 33, 34 of a focus detection pixel 411 having green filters arranged in odd-numbered rows and odd-numbered columns for each photoelectric conversion unit, and phase difference detection is possible for a subject image having contrast that varies in the vertical direction, using a pair of same-color data obtained by grouping vertically the data of the pair of photoelectric conversion units 36, 37 of a focus detection pixel 412 having red and blue filters arranged in even-numbered rows and even-numbered columns for each photoelectric conversion unit.

[0181] <Other embodiments> In the present invention, the number of photoelectric conversion units in the focus detection pixel is not limited to two, and the present invention can be applied to a configuration in which the focus detection pixel has two or more photoelectric conversion units. For example, the focus detection pixel 311 shown in FIG. 6 has two photoelectric conversion units 13, 14 obtained by dividing a square in half in the horizontal direction, but the present invention can also be applied to a configuration in which the focus detection pixel has four photoelectric conversion units obtained by dividing the two photoelectric conversion units 13, 14 in half in the vertical direction. For example, by providing four column signal lines in each column to independently read out the analog signals of the four photoelectric conversion units, and providing a column AD conversion device composed of an ADC that individually converts the analog signals output from the four photoelectric conversion units into digital signals and a column digital addition device having a digital addition circuit that digitally adds the digital signals of the four photoelectric conversion units output from the column AD conversion device, the same effect as the configuration shown in FIG. 10 can be obtained.

[0182] Also, instead of providing four column signal lines corresponding to the four photoelectric conversion units, it is possible to reduce the number of column signal lines to two by constructing an image sensor with two of the four photoelectric conversion units and the other two photoelectric conversion units regarded as the photoelectric conversion units of two virtually adjacent rows of focus detection pixels. For example, if four photoelectric conversion units are arranged in two rows of two in each focus detection pixel 311, the two photoelectric conversion units in the upper row and the two photoelectric conversion units in the lower row are regarded as the photoelectric conversion units of two virtually adjacent rows of focus detection pixels, and the number of column signal lines is reduced to two.

[0183] In the image sensor of the above-described embodiment, an example is shown in which the entire image array section is provided with one data output channel for focus detection and one data output channel for images, but in order to increase the readout speed, it is also possible to divide the image array section into multiple regions and provide each region with one data output channel for focus detection and one data output channel for images.

[0184] In the image sensor 212 of the embodiment described above, the CPUa 222 of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system) based on a digital signal obtained by the column AD conversion device 42 converting a pair of analog signals. However, the digital signal obtained by the column AD conversion device 42 converting a pair of analog signals may be used as a signal for a 3D camera.

[0185] The present invention is applicable not only to an imaging element of the type in which a wiring layer exists between a microlens and a photoelectric conversion unit as shown in Fig. 7, but also to a back-illuminated imaging element in which a wiring layer does not exist between a microlens and a photoelectric conversion unit, and a wiring layer is arranged on the opposite side of the photoelectric conversion unit to the direction of the microlens. An imaging element that requires column signal lines 22, such as the imaging element of the present invention, is provided with more wiring layers than a conventional imaging element. In a back-illuminated imaging element, the wiring layers can be arranged without being restricted by the layout of the photoelectric conversion unit, and therefore flexibility in terms of an increase in the number of column signal lines is improved.

[0186] In the above embodiment, the image sensor 212 has an image sensor pixel with a Bayer array color filter, but the configuration and array of the color filter are not limited to this, and the present invention can be applied to an array of complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan: Cy) and an array other than the Bayer array. The present invention can also be applied to a monochrome image sensor.

[0187] The image sensor 212 in the above-described embodiment has a pixel array section 40 and other portions. The pixel array section 40 and other portions are provided on separate substrates, and these separate substrates are stacked on top of each other, but the pixel array section 40 and other portions may be provided on the same substrate.

[0188] The imaging device is not limited to the digital still camera having an interchangeable lens attached to a camera body as described above. For example, the present invention can be applied to a digital still camera or video camera with an integrated lens. Furthermore, the present invention can be applied to a small camera module built into a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, etc. [Explanation of symbols]

[0189] 10 microlens, 11,13,14,16,17,33,34,36,37 photoelectric conversion section, 15,18 element isolation region, 21 row control line, 22 column signal line, 23 ADC (analog-to-digital conversion circuit), 24,27 switch, 25,28 memory, 26 digital addition circuit, 29 semiconductor substrate, 30 light shielding mask, 31,32 planarization layer, 38 color filter, 40 pixel array section, 41 row scanning circuit, 42 column AD conversion device, 43 second column switch device, 44 second line memory, 45 second horizontal output circuit, 46 column digital addition circuit, 47 first column switch device, 48 first line memory, 49 first horizontal output circuit, 50 timing control circuit, 51 second column scanning circuit, 52 first column scanning circuit, 71 shooting light beam, 73,74 focus detection light beam, 90 Exit pupil, 91 Optical axis, 93, 94 Focusing pupil, 95 Area, 100 Shooting screen, 101 Focus detection area, 201 Digital still camera, 202 Interchangeable lens, 203 Camera body, 204 Mount unit, 206 Lens drive control device, 208 Zooming lens, 209 Lens, 210 Focusing lens, 211 Aperture, 212 Image sensor, 213 Electrical contact, 214 Body drive control device, 215 Liquid crystal display element drive circuit, 216 Liquid crystal display element, 217 Eyepiece, 219 Memory card, 220 Image sensor control unit, 221 Buffer memory, 222 CPUa, 223 CPUb, 310 Imaging pixel, 311, 312, 411, 412 Focus detection pixel

Claims

1. a first photoelectric conversion unit that converts light transmitted through the first microlens into an electric charge; a second photoelectric conversion unit that converts light transmitted through the first microlens into an electric charge and is disposed adjacent to the first photoelectric conversion unit in the first direction; a third photoelectric conversion unit that converts the light transmitted through the second microlens into an electric charge; a fourth photoelectric conversion unit that converts light transmitted through the second microlens into an electric charge, the fourth photoelectric conversion unit being disposed adjacent to the third photoelectric conversion unit in a second direction intersecting the first direction; a control unit that outputs a first sum signal obtained by performing addition processing using a first signal based on the charges converted by the first photoelectric conversion unit and a second signal based on the charges converted by the second photoelectric conversion unit, and a second sum signal obtained by performing addition processing using a third signal based on the charges converted by the third photoelectric conversion unit and a fourth signal based on the charges converted by the fourth photoelectric conversion unit; An imaging element comprising:

2. 2. The imaging device according to claim 1, A first conversion unit that converts the first signal into a digital signal; A second conversion unit that converts the second signal into a digital signal; a third conversion unit that converts the third signal into a digital signal; a fourth conversion unit that converts the fourth signal into a digital signal; Equipped with the control unit outputs the first sum signal obtained by performing addition processing using the first signal converted into a digital signal by the first conversion unit and the second signal converted into a digital signal by the second conversion unit, and the second sum signal obtained by performing addition processing using the third signal converted into a digital signal by the third conversion unit and the fourth signal converted into a digital signal by the fourth conversion unit. Image sensor.

3. The imaging element according to claim 2, a first signal line through which the first signal is output; a second signal line through which the second signal is output; a third signal line through which the third signal is output; a fourth signal line through which the fourth signal is output; Equipped with the first conversion unit converts the first signal output to the first signal line into a digital signal; the second conversion unit converts the second signal output to the second signal line into a digital signal; the third conversion unit converts the third signal output to the third signal line into a digital signal; The fourth conversion unit converts the fourth signal output to the fourth signal line into a digital signal. Image sensor.

4. 2. The imaging device according to claim 1, the control unit outputs a first signal based on the charges converted by the first photoelectric conversion unit, a second signal based on the charges converted by the second photoelectric conversion unit, a third signal based on the charges converted by the third photoelectric conversion unit, and a fourth signal based on the charges converted by the fourth photoelectric conversion unit. Image sensor.

5. 5. The imaging device according to claim 4, A first conversion unit that converts the first signal into a digital signal; A second conversion unit that converts the second signal into a digital signal; a third conversion unit that converts the third signal into a digital signal; a fourth conversion unit that converts the fourth signal into a digital signal; Equipped with the control unit outputs the first sum signal obtained by performing addition processing using the first signal converted into a digital signal by the first conversion unit and the second signal converted into a digital signal by the second conversion unit, the second sum signal obtained by performing addition processing using the third signal converted into a digital signal by the third conversion unit and the fourth signal converted into a digital signal by the fourth conversion unit, the first signal converted into a digital signal by the first conversion unit, the second signal converted into a digital signal by the second conversion unit, the third signal converted into a digital signal by the third conversion unit, and the fourth signal converted into a digital signal by the fourth conversion unit. Image sensor.

6. 6. The imaging device according to claim 5, a first signal line through which the first signal is output; a second signal line through which the second signal is output; a third signal line through which the third signal is output; a fourth signal line through which the fourth signal is output; Equipped with the first conversion unit converts the first signal output to the first signal line into a digital signal; the second conversion unit converts the second signal output to the second signal line into a digital signal; the third conversion unit converts the third signal output to the third signal line into a digital signal; The fourth conversion unit converts the fourth signal output to the fourth signal line into a digital signal. Image sensor.

7. 2. The imaging device according to claim 1, a first conversion unit that converts the first signal and the third signal into digital signals; a second conversion unit that converts the second signal and the fourth signal into digital signals; Equipped with the control unit outputs the first sum signal obtained by performing addition processing using the first signal converted into a digital signal by the first conversion unit and the second signal converted into a digital signal by the second conversion unit, and the second sum signal obtained by performing addition processing using the third signal converted into a digital signal by the first conversion unit and the fourth signal converted into a digital signal by the second conversion unit. Image sensor.

8. 8. The imaging device according to claim 7, a first signal line through which the first signal and the third signal are output; a second signal line through which the second signal and the fourth signal are output; Equipped with the first conversion unit converts the first signal output to the first signal line and the third signal output to the first signal line into digital signals; the second conversion unit converts the second signal output to the second signal line and the fourth signal output to the second signal line into digital signals. Image sensor.

9. 8. The imaging device according to claim 7, the control unit outputs the first signal based on the charges converted by the first photoelectric conversion unit, the second signal based on the charges converted by the second photoelectric conversion unit, the third signal based on the charges converted by the third photoelectric conversion unit, and the fourth signal based on the charges converted by the fourth photoelectric conversion unit. Image sensor.

10. 10. The imaging device according to claim 9, the control unit outputs the first sum signal obtained by performing addition processing using the first signal converted into a digital signal by the first conversion unit and the second signal converted into a digital signal by the second conversion unit, the second sum signal obtained by performing addition processing using the third signal converted into a digital signal by the first conversion unit and the fourth signal converted into a digital signal by the second conversion unit, the first signal converted into a digital signal by the first conversion unit, the second signal converted into a digital signal by the second conversion unit, the third signal converted into a digital signal by the first conversion unit, and the fourth signal converted into a digital signal by the second conversion unit. Image sensor.

11. The imaging device according to claim 10, a first signal line through which the first signal and the third signal are output; a second signal line through which the second signal and the fourth signal are output; Equipped with the first conversion unit converts the first signal output to the first signal line and the third signal output to the first signal line into digital signals; the second conversion unit converts the second signal output to the second signal line and the fourth signal output to the second signal line into digital signals. Image sensor.

12. An imaging device comprising the imaging element according to claim 1 .

13. 13. The imaging device according to claim 12, an imaging device comprising a generation section that generates image data based on the first sum signal and the second sum signal;

Images