Imaging element

Abstract

To provide an imaging element capable of reading out a signal used for focus detection and a signal used for image generation at high speed.SOLUTION: The imaging element includes: a first photoelectric conversion part 14 and a second photoelectric conversion part 13 for converting light transmitted through a first microlens into charges; a third photoelectric conversion part and a fourth photoelectric conversion part for converting the light transmitted through a second microlens into the charges; and a digital addition circuit 26 for performing addition processing for adding a first digital signal based on the charges converted by the first photoelectric conversion part and a second digital signal based on the charges converted by the second photoelectric conversion part and performing the addition processing for adding a third digital signal based on the charges converted by the third photoelectric conversion part and a fourth digital signal based on the charges converted by the fourth photoelectric conversion part.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] An imaging device in which focus detection pixels each composed of a microlens and a pair of photoelectric conversion units disposed behind the microlens are arranged on a planned focal plane of a photographing lens, and a pair of image signals corresponding to a pair of images formed by a pair of focus detection light beams passing through an optical system are generated as analog signals in the pair of photoelectric conversion units, the pair of analog signals are independently read out from the imaging device, and the amount of image shift (phase difference) between the pair of image signals is detected to detect the focus adjustment state (defocus amount) of the photographing lens, and the analog signals generated by the pair of photoelectric conversion units of the focus detection pixels are subjected to analog addition within the focus detection pixels, and the added analog signal is read out from the imaging device as an image signal to generate image information is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the imaging device as described above, since analog addition processing of a pair of analog signals is performed within the focus detection pixels, it is necessary to independently read out the pair of analog signals from the imaging device at the time of focus detection and to read out the pair of analog signals after addition from the imaging device at the time of image information generation, and there is a problem that focus detection and image information generation cannot be performed simultaneously in reading out signals for one frame from the imaging device.

Means for Solving the Problems

[0005] The image sensor according to the first aspect of the present invention includes a first photoelectric conversion unit that converts light transmitted through a first microlens into electric charges, a photoelectric conversion unit that converts light transmitted through the first microlens into electric charges, and a second photoelectric conversion unit that is disposed adjacent to the first photoelectric conversion unit in a first direction, a third photoelectric conversion unit that converts light transmitted through a second microlens into electric charges, a photoelectric conversion unit that converts light transmitted through the second microlens into electric charges, and a fourth photoelectric conversion unit that is disposed adjacent to the third photoelectric conversion unit in a second direction intersecting the first direction, a control unit that outputs a first addition signal obtained by performing an addition process using a first signal based on the electric charges converted by the first photoelectric conversion unit and a second signal based on the electric charges converted by the second photoelectric conversion unit, and a second addition signal obtained by performing an addition process using a third signal based on the electric charges converted by the third photoelectric conversion unit and a fourth signal based on the electric charges converted by the fourth photoelectric conversion unit.

Advantages of the Invention

[0006] According to the present invention, for example, signals used for focus detection and signals used for image generation can be read out at high speed.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Embodiments for Carrying Out the Invention

[0008] <First Embodiment> An imaging device and an imaging apparatus according to an embodiment of the present invention will be described. FIG. 1 is a cross-sectional view showing the configuration of a lens interchangeable digital still camera equipped with the imaging device according to an embodiment. The digital still camera 201 according to an embodiment is composed of an interchangeable lens 202 and a camera body 203, and various interchangeable lenses 202 are attached to the camera body 203 via a mount portion 204.

[0009] The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, a diaphragm 211, a lens drive control device 206, and the like. The lens drive control device 206 is composed of a microcomputer, a memory, a drive control circuit, etc., and performs drive control for focus adjustment of the focusing lens 210 and aperture adjustment of the diaphragm 211, state detection of the zooming lens 208, the focusing lens 210, and the diaphragm 211, etc. In addition, it transmits lens information and receives camera information through communication with a body drive control device 214 described later. The diaphragm 211 forms an aperture with a variable aperture diameter at the center of the optical axis for adjusting the light amount and the amount of blur.

[0010] The camera body 203 includes an imaging device 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging device 212, pixels that function as imaging pixels and focus detection pixels are two-dimensionally arranged. Details of this imaging device 212 will be described later.

[0011] The body drive control device 214 is composed of a microcomputer, a memory, a drive control circuit, etc., and repeatedly performs drive control of the imaging device 212, reading of the output signal from the imaging device 212, focus detection calculation based on the output signal, and focus adjustment of the interchangeable lens 202, and also performs image processing calculation, recording, and operation control of the camera based on the output signal. In addition, the body drive control device 214 communicates with the lens drive control device 206 via an electrical contact 213, receives lens information, and transmits camera information (such as defocus amount and aperture value).

[0012] The liquid crystal display element 216 functions as an electronic viewfinder (EVF). The liquid crystal display element driving circuit 215 displays the through-image captured by the imaging device 212 on the liquid crystal display element 216, and the photographer can observe the through-image through the eyepiece lens 217. The memory card 219 is an image storage that stores the images captured by the imaging device 212.

[0013] The subject image is formed on the light-receiving surface of the imaging device 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by each pixel of the imaging device 212, and the output signal of each pixel is sent to the body drive control device 214.

[0014] The body drive control device 214 calculates the defocus amount based on the output signals from each pixel of the imaging device 212, and sends this defocus amount to the lens drive control device 206. Also, the body drive control device 214 processes the output signals from each pixel of the imaging device 212 to generate image data, stores it in the memory card 219, and sends the through-image signal from the imaging device 212 to the liquid crystal display element driving circuit 215 to display the through-image on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to perform aperture control of the aperture 211.

[0015] The lens drive control device 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture open F value, etc. Specifically, it detects the positions of the zoom lens 208 and the focusing lens 210 and the aperture value of the aperture 211, calculates the lens information according to these lens positions and aperture values, or selects the lens information corresponding to the lens positions and aperture values from a pre-prepared look-up table.

[0016] The lens drive control device 206 calculates the lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Also, the lens drive control device 206 drives the aperture 211 according to the received aperture value.

[0017] FIG. 2 is a diagram showing a focus detection position on the imaging screen of the interchangeable lens 202 (set by the user by operating an operation member not shown in FIG. 1), and shows an example of a region (focus detection area, focus detection position) where a pixel column on the imaging element 212 to be described later samples an image on the imaging screen during focus detection. In this example, a focus detection area 101 is arranged at the center of a rectangular imaging screen 100. The focus detection area 101 shown as a rectangle extends in the horizontal direction on the imaging screen 100, and output signals of pixels linearly arranged along the longitudinal direction of the focus detection area 101 are used for focus detection.

[0018] FIGS. 3 and 4 are front views showing a detailed configuration of the imaging element 212, and show an enlarged view of the vicinity of the focus detection area 101 on the imaging element 212. FIG. 3 is a diagram showing the layout of pixels 311 (hereinafter referred to as focus detection pixels 311) that serve as both imaging pixels and focus detection pixels, and the focus detection pixels 311 are densely arranged in a two-dimensional square lattice in the row direction (horizontal direction) and the column direction (vertical direction). FIG. 4 is a diagram showing the arrangement of color filters in the arrangement of the focus detection pixels 311 shown in FIG. 3. Color filters (R: red filter, G: green filter, B: blue filter) are arranged in the focus detection pixels 311 according to the rules of the Bayer array, and the spectral sensitivities of the respective color filters have the characteristics shown in FIG. 5.

[0019] As shown in FIG. 6, the focus detection pixel 311 is composed of a pair of photoelectric conversion units 13 and 14 divided into two by a rectangular microlens 10 and an element separation region 15 extending in the vertical direction. When the pair of photoelectric conversion units 13 and 14 are integrated, the size becomes equivalent to that of the photoelectric conversion unit of a normal imaging pixel. For simplicity, the color filter is not shown in FIG. 6. When the outputs of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 are added, in order to make the added output equivalent to the output of the photoelectric conversion unit of a normal imaging pixel, it is desirable to make the width of the element separation region 15 as narrow as possible and bring the pair of photoelectric conversion units 13 and 14 close to each other.

[0020] FIG. 7 is a cross-sectional view of the focus detection pixel 311 shown in FIG. 6. A light-shielding mask 30 is formed in proximity to the photoelectric conversion units 13 and 14, and the photoelectric conversion units 13 and 14 receive light that has passed through the opening 30d of the light-shielding mask 30. A planarization layer 31 is formed on the light-shielding mask 30, and a color filter 38 is formed thereon. A planarization layer 32 is formed on the color filter 38, and a microlens 10 is formed thereon. The shape of the photoelectric conversion units 13 and 14 restricted by the opening 30d is projected forward by the microlens 10 to form a pair of distance measurement pupils. The photoelectric conversion units 13 and 14 are formed on a semiconductor circuit board 29. Also, an element isolation region 15 is formed to separate the photoelectric conversion units 13 and 14. With the above configuration, the photoelectric conversion units 13 and 14 receive a pair of focus detection light beams that pass through a pair of distance measurement pupils of the exit pupil of the interchangeable lens, respectively.

[0021] FIG. 8 shows the configuration of a focus detection optical system of a pupil division type phase difference detection method using a microlens. Note that a part of the focus detection pixel array in the focus detection area 101 is shown enlarged. In FIG. 8, the exit pupil 90 is set at a position that is a distance d forward from a microlens 10 disposed on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature, refractive index of the microlens 10, the distance between the microlens 10 and the photoelectric conversion units 13 and 14, etc., and is referred to as the distance measurement pupil distance in this specification. FIG. 11 also shows the optical axis 91 of the interchangeable lens, the microlens 10, the photoelectric conversion units 13, 14, the focus detection pixel 311, and the focus detection light beams 73 and 74.

[0022] The distance measurement pupil 93 is the projection of the photoelectric conversion unit 13 restricted by the opening 30d by the microlens 10. Similarly, the distance measurement pupil 94 is the projection of the photoelectric conversion unit 14 restricted by the opening 30d by the microlens 10. The distance measurement pupils 93 and 94 are different partial regions of the exit pupil 90, are arranged horizontally, and have a shape that is line-symmetric with respect to a vertical line passing through the optical axis 91.

[0023] FIG. 8 schematically illustrates five adjacent focus detection pixels 311 in the focus detection area 101 near the imaging optical axis 91. Even for the focus detection pixels 311 arranged at the periphery of the screen, each photoelectric conversion unit is configured to receive the light beam arriving from the corresponding ranging pupils 93 and 94 through each microlens. The microlens 10 causes a pair of photoelectric conversion units 13 and 14 and the above-described different partial regions, that is, a pair of ranging pupils 93 and 94, to be in a conjugate relationship with each other.

[0024] With the above configuration, the photoelectric conversion unit 13 passes through the ranging pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the light beam 73 directed toward the microlens 10 of the focus detection pixel 311. Further, the photoelectric conversion unit 14 passes through the ranging pupil 94 and outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the light beam 74 directed toward the microlens 10 of the focus detection pixel 311.

[0025] By collecting the outputs of the photoelectric conversion units 13 and 14 of the plurality of focus detection pixels 311 arranged horizontally in the above-described focus detection area 101 into output groups corresponding to the ranging pupil 93 and the ranging pupil 94, information regarding the intensity distribution of a pair of images formed by the focus detection light beams 73 and 74 passing through the ranging pupil 93 and the ranging pupil 94, respectively, on the arrangement of the focus detection pixels 311 is obtained. By performing the image shift detection arithmetic processing (correlation arithmetic processing, phase difference detection processing) described later on this information, the image shift amount of the pair of images is detected by a so-called pupil division type phase difference detection method. Further, by performing a conversion arithmetic operation on the image shift amount according to the proportional relationship between the center-of-gravity interval of the pair of ranging pupils 93 and 94 and the ranging pupil distance, the deviation (defocus amount) of the current imaging surface with respect to the planned imaging surface (the imaging surface at the focus detection position corresponding to the position of the microlens array on the planned imaging surface) is calculated. Specifically, the defocus amount (the deviation between the imaging surface and the planned imaging surface in the direction of the optical axis 91) is calculated by multiplying the image shift amount (the amount in the plane perpendicular to the optical axis 91) by a predetermined conversion coefficient (the value obtained by dividing the ranging pupil distance d by the center-of-gravity interval of the ranging pupils 93 and 94).

[0026] Further, by obtaining an output signal obtained by adding the outputs of the photoelectric conversion units 13 and 14 of each focus detection pixel 311 over the entire screen, an image signal equivalent to the case where normal imaging pixels are arranged in a Bayer array can be obtained.

[0027] FIG. 9 is a block diagram showing in detail the relationship between the imaging device 212 and the body drive control device 214 of a part related to the present invention. In the body drive control device 214, an imaging device control unit 220, a buffer memory 221, a CPUa (microcomputer) 222, and a CPUb (microcomputer) 223 are housed. The imaging device 212 performs charge accumulation control (charge accumulation time and charge accumulation timing) of the focus detection pixel 311 and signal output control according to the control of the imaging device control unit 220. As will be described later, the imaging device 212 AD-converts the output signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 and outputs them as digital data (data for focus detection) from channel 1. At the same time, the imaging device 212 outputs digital data (a signal equivalent to the output signal of a normal imaging pixel) obtained by digitally adding the digital data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 as digital data from channel 2. The digital data output from channel 1 and channel 2 is temporarily stored in the buffer memory 221 as digital data for one frame. The CPUa 222 performs the processes described later on the digital data (data for focus detection) of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in the focus detection area stored in the buffer memory 221 to perform focus detection. The CPUb 223 performs well-known image processing on the digital data (image data) for one frame stored in the buffer memory 221 to perform image display and image recording.

[0028] As described above, digital data for focus detection and digital data for imaging are output from the imaging device 212 overlapping in time via separate channels. Further, since the digital data for focus detection and the digital data for imaging are processed by individual CPUs 222 and 223, there is no need to separate the focus detection process and the image process in time, and they can be performed simultaneously and independently.

[0029] Next, with reference to FIG. 10, the configuration of the imaging device 212 capable of outputting digital data of a pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 from two channels simultaneously and digital data obtained by digitally adding the digital data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 (a signal equivalent to the output signal of a normal imaging pixel) will be described.

[0030] FIG. 10 is a block diagram showing the configuration of the imaging device 212 (CMOS image sensor). The imaging device 212 includes, in addition to a pixel array unit 40 in which a large number of focus detection pixels 311 each including a pair of photoelectric conversion units 13 and 14 are two-dimensionally arranged in a matrix (matrix) form, a row scanning circuit 41, a column AD conversion device 42, a second line memory 44, a second column scanning circuit 51, a second horizontal output circuit 45, a column digital addition device 46, a first line memory 48, a first column scanning circuit 52, a first horizontal output circuit 49, and a timing control circuit 50.

[0031] In this system configuration, based on the master clock input from the outside and the control signal input from the imaging device control unit 220, the timing control circuit 50 generates a clock signal, a control signal, etc., which serve as the basis for the operations of the row scanning circuit 41, the column AD conversion device 42, the column digital addition device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc., and supplies them to the row scanning circuit 41, the column AD conversion device 42, the column digital addition device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc.

[0032] Also, the peripheral drive system and signal processing system for driving and controlling each focus detection pixel 311 of the pixel array unit 40, that is, the row scanning circuit 41, the column AD conversion device 42, the column digital addition device 46, the first line memory 48, the second line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, the first horizontal output circuit 49, the second horizontal output circuit 45, and the timing control circuit 50, etc., are integrated on the same chip (semiconductor substrate) as the pixel array unit 40. The chip on which these are integrated is laminated on the chip of the pixel array unit 40.

[0033] As the focus detection pixel 311, although not shown here, in addition to a pair of photoelectric conversion element portions 13 and 14 (for example, photodiodes), for example, a transfer transistor that transfers the charges obtained by photoelectric conversion in the photoelectric conversion portions 13 and 14 to the FD (floating diffusion) portion, a reset transistor that controls the potential of the FD portion, and an amplification transistor that outputs a signal corresponding to the potential of the FD portion, a three-transistor configuration having these components can be used, or a four-transistor configuration having a selection transistor for pixel selection separately can also be used.

[0034] In the pixel array portion 40, 2N rows and 2M columns of focus detection pixels 311 are two-dimensionally arranged. In other words, the pixel array portion 40 has, in each row, a focus detection pixel group in which 2M focus detection pixels 311 are arranged horizontally, and the focus detection pixel groups are arranged in 2N rows in the vertical direction intersecting the horizontal direction. In FIG. 10, the upper left focus detection pixel 311 is the pixel in the first row and the first column, and a green filter of the Bayer array is arranged in this pixel. Green filters and blue filters are arranged in the focus detection pixels arranged as the pixel group in the first row. For this 2N-row and 2M-column pixel arrangement, one system of row control lines 21 (21(1) to 21(2N)) is wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are wired for each column. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to each output end corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each row control line 21. The row scanning circuit 41 is constituted by a shift register or the like, and controls the row address and row scanning of the pixel array portion 40 via the row control lines 21 (21(1) to 21(2N)).

[0035] The pair of photoelectric conversion units 13 and 14 of each focus detection pixel 311 in the same row are connected to the horizontal scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed according to the control signals R(1), ···, R(L), ···, R(2N). Also, one of the pair of photoelectric conversion units 13 and 14 of each focus detection pixel 311, i.e., the photoelectric conversion unit 13, is connected to one of the two column signal lines 22(m)b provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 13 is output to the column signal line 22(m)b. Further, the other photoelectric conversion unit 14 of the pair of photoelectric conversion units 13 and 14 of each focus detection pixel 311 is connected to the other column signal line 22(m)a of the two column signal lines provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 14 is output to the column signal line 22(m)a. For example, when the focus detection pixel 311 constituting the focus detection pixel group in the L-th row of the pixel array unit 40 is selected by the control signal R(L) given from the horizontal scanning circuit 41, the output signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in the L-th row are output to the column signal lines (22(1)a, 22(1)b ~ 22(2M)a, 22(2M)b).

[0036] The column AD conversion device 42 has ADCs (analog-digital conversion circuits) 23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b provided for each of the column signal lines 22(1)a, 22(1)b ~ 22(2M)a, 22(2M)b provided corresponding to the pixel columns of the pixel array unit 40, and converts a pair of analog signals output from each focus detection pixel 311 of the pixel array unit 40 for each column into H-bit digital signals and outputs them according to the control signal TA1 given from the timing control circuit 50. "H bits" represents the number of bits, for example, 10 bits, 12 bits, 14 bits, etc.

[0037] The second line memory 44 has memories (25(1)a, 25(1)b ~ 25(2M)a, 25(2M)b) provided for each ADC (23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b) that constitutes the column AD conversion device 42, and stores the digital signals output for each ADC (23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b) as H-bit digital signals according to the control signal TM2 given from the timing control circuit 50. Here, the output signals of the pair of photoelectric conversion units 13 and 14 for one row of focus detection pixels are stored as digital signals in each memory (25(1)a, 25(1)b ~ 25(2M)a, 25(2M)b) of the second line memory 44.

[0038] The column digital addition device 46 has digital addition circuits (26(1) ~ 26(2M)) provided for each pair of ADCs ((23(1)a, 23(1)b) ~ (23(2M)a, 23(2M)b)) that constitutes the column AD conversion device 42, adds the digital signals output from the pair of ADCs ((23(1)a, 23(1)b) ~ (23(2M)a, 23(2M)b)) according to the control signal TD1 given from the timing control circuit 50, and outputs the result as an H-bit added digital signal.

[0039] The first line memory 48 has memories (28(1) ~ 28(2M)) provided for each digital addition circuit (26(1) ~ 26(2M)) that constitutes the column digital addition device 46, and stores the added digital signals output for each digital addition circuit (26(1) ~ 26(2M)) as H-bit digital signals according to the control signal TM1 given from the timing control circuit 50. Here, the added signals (corresponding to the output signals of the imaging pixels) obtained by adding the output signals of the pair of photoelectric conversion units 13 and 14 for one row of focus detection pixels are stored as digital signals in each memory (28(1) ~ 28(2M)) of the first line memory 48.

[0040] The second column scanning circuit 51 is composed of a shift register or the like, and controls the column addresses and column scanning of the memories (25(1)a, 25(1)b~25(2M)a, 25(2M)b) in the second line memory 44 under the control of the timing control circuit 50. The second line memory 44 operates in response to the scanning signal TS2 given from the second column scanning circuit 51, and the H-bit digital signals stored in each of the memories (25(1)a, 25(1)b~25(2M)a, 25(2M)b) are sequentially read out to the second horizontal output circuit 45, and are serially output to the outside as output signals (digital signals) of the pair of photoelectric conversion units 13 and 14 for focus detection via the second horizontal output circuit 45.

[0041] The first column scanning circuit 52 is composed of a shift register or the like, and controls the column addresses and column scanning of the memories (28(1)~28(2M)) in the first line memory 48 under the control of the timing control circuit 50. The first line memory 48 operates in response to the scanning signal TS1 given from the first column scanning circuit 52, and the H-bit addition digital signals stored in each of the memories (28(1)~28(2M)) are sequentially read out to the first horizontal output circuit 49, and are serially output to the outside as output signals (digital signals) equivalent to the output signals of the imaging pixels via the first horizontal output circuit 49.

[0042] Next, in the configuration of the imaging device shown in FIG. 10, the case where the individual readout operation of the output signals of the pair of photoelectric conversion units of the focus detection pixels and the readout operation of the addition signal obtained by adding the output signals of the pair of photoelectric conversion units are performed in parallel during one frame period will be described using the timing charts of FIGS. 11 and 12. In FIGS. 11 and 12, VS is a vertical synchronization signal indicating one frame period, and HS is a horizontal synchronization signal indicating one horizontal scanning period.

[0043] In the operation shown in FIG. 11, in synchronization with the horizontal synchronization signal HS, control signals R(1), R(2), R(3)~R(2n + 1), R(2n + 2), R(2n + 3)~R(N) are sequentially issued from the horizontal scanning circuit 41 to the pixel array unit 40, and analog signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of the row corresponding to the control signals R(1), R(2), R(3)~R(2n + 1), R(2n + 2), R(2n + 3)~R(N) are sequentially output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b).

[0044] FIG. 12 is an enlarged view of the operation portions of the (2n + 1)-th row, (2n + 2)-th row, and (2n + 3)-th row in FIG. 11. When the (2n + 1)-th row of the pixel array unit 40 is selected by the control signal R(2n + 1), analog signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of the (2n + 1)-th row are output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b). The analog signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of the (2n + 1)-th row output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b) are converted into digital signals by the ADCs (23(1)a, 23(1)b~23(2M)a, 23(2M)b) of the column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b~22(2M)a, 22(2M) according to the control signal TA1.

[0045] The digital signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of the (2n + 1)-th row after digital conversion are stored in the memories (25(1)a, 25(1)b~25(2M)a, 25(2M)b) of the second line memory 44 connected to the ADCs (23(1)a, 23(1)b~23(2M)a, 23(2M)b) of the column AD conversion device 42 according to the control signal TM2.

[0046] At the same time, the digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of (2n + 1) lines that have been digitally converted are added by the digital addition circuits (26(1) to 26(2M)) of the column digital addition device 46 provided for each pair of ADCs ((23(1)a, 23(1)b) to (23(2M)a, 23(2M)b)) that make up the column AD conversion device 42 according to the control signal TD1.

[0047] The added digital signal of the focus detection pixels 311 for one line of (2n + 1) lines, where the output signals of the pair of photoelectric conversion units 13 and 14 are added, is stored in the memories ((28(1) to 28(2M))) of the first line memory 48 connected to the digital addition circuits (26(1) to 26(2M)) of the column digital addition device 46 according to the control signal TM1.

[0048] The digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line of (2n + 1) lines stored in the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 during the period until the next horizontal synchronization signal HS is generated according to the scanning signal TS2. Based on the digital signal output from the second horizontal output circuit 45, the CPUa222 for focus detection of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system) as shown in FIG. 13 described later and adjusts the focus state.

[0049] Similarly, the added digital signal of the focus detection pixels 311 for one line of (2n + 1) lines, where the output signals of the pair of photoelectric conversion units 13 and 14 stored in the memories ((28(1) to 28(2M))) of the first line memory 48 are added, is sequentially serially output to the outside from the first horizontal output circuit 49 during the period until the next horizontal synchronization signal HS is generated according to the scanning signal TS1. Based on the added digital signal output from the first horizontal output circuit 49, the CPUb223 for image processing of the body drive control device 214 generates image data as shown in FIG. 14 described later.

[0050] When the control signal R(2n + 2) is issued in synchronization with the next horizontal synchronization signal HS and the (2n + 2)-th row of the pixel array unit 40 is selected, the same operation is repeated for the analog signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels for one line of the (2n + 2)-th row. Further, the same process is repeated under the control signal R(2n + 3) synchronized with the next horizontal synchronization signal HS.

[0051] FIGS. 13 and 14 are flowcharts showing the operations of a digital still camera (imaging device) 201 according to an embodiment. The processes according to these flowcharts are performed in parallel. FIG. 13 is a flowchart of the operation of the CPUa222 for focus detection of the body drive control device 214. When the power of the digital still camera 201 is turned on in step S100, the focus detection operation after step S110 is started. In step S110, data of a pair of photoelectric conversion units of the focus detection pixels arranged in the focus detection area selected in frame synchronization is read out. The data of this pair of photoelectric conversion units is the digital signal output from the second horizontal output circuit 45 described above. In the subsequent step S120, based on the data of the focus detection pixels, an image shift detection arithmetic process (correlation arithmetic process, phase difference detection process) described later is performed to calculate the amount of image shift. Note that the position of the focus detection area is assumed to be previously selected by the photographer using an operation member (not shown).

[0052] In step S130, the amount of image shift is converted into the amount of defocus.

[0053] In step S140, it is detected whether the focus state of the interchangeable lens 202 (optical system) is near the in-focus state, that is, whether the absolute value of the calculated amount of defocus is within a predetermined value. If it is determined that the state is not near the in-focus state, the process proceeds to step S150, the amount of defocus is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the in-focus position to adjust the focus state of the interchangeable lens 202 (optical system).

[0054] If focus detection is impossible, the process also branches to this step, sends a scan drive command to the lens drive control device 206, and scan drives the focusing lens 210 of the interchangeable lens 202 from infinity to the closest distance. Then, the process proceeds to step S160.

[0055] If it is determined in step S140 that the subject is near the in-focus state, the process proceeds to step S160, and it is determined whether or not the shutter has been released by operating a shutter button (not shown). If it is determined that the shutter has not been released, the process returns to step S110 and repeats the above-described operations. On the other hand, if it is determined that the shutter has been released, the process proceeds to step S170, waits for the shooting operation corresponding to the shutter release to end, and when the shooting operation ends, returns to step S110 and repeats the above-described operations.

[0056] Details of the image shift detection arithmetic processing (correlation arithmetic processing, phase difference detection processing) in steps S120 and S130 of FIG. 13 will be described below. Note that a pair of data of the focus detection pixels 311 are separated for each color of the same color in the Bayer array.

[0057] Since there is a possibility that the pair of images detected by the focus detection pixels 311 has the light quantity balance disrupted due to the photometric pupils 93 and 94 being restricted by the aperture opening of the lens, a type of correlation arithmetic that can maintain the image shift detection accuracy with respect to the light quantity balance is performed. When the pair of data columns read from the array of the focus detection pixels 311 are represented as A1n (A11, ···, A1j: j is the number of data) and A2n (A21, ···, A2j) and are generalized without distinction based on the color difference, the following correlation arithmetic formula (1) disclosed in Japanese Patent Application Laid-Open No. 2007-333720 is performed on the pair of data columns A1n and A2n to calculate the correlation quantity C(k). C(k)=Σ|A1n·A2n+1+k - A2n+k·A1n+1| ···(1)

[0058] (1) In the formula, the Σ operation is accumulated with respect to n. The range that n takes is limited to the range where the data of A1n, A1n+1, A2n+k, and A2n+1+k exist according to the image shift amount k. The shift amount k is an integer and is the relative shift amount with the data interval of the data sequence as the unit. The calculation result of formula (1) is that, as shown in Fig. 15(a), the correlation amount C(k) is minimized at the shift amount (k = kj = 2 in Fig. 15(a)) where the correlation between a pair of data is high (the smaller the correlation amount C(k), the higher the correlation degree).

[0059] Next, the shift amount X that gives the minimum value C(X) for the continuous correlation amount is obtained by using the three-point interpolation method from formula (2) to formula (5). X = kj + D / SLOP ···(2) C(X) = C(kj) - |D| ···(3) D = {C(kj - 1) - C(kj + 1)} / 2 ···(4) SLOP = MAX{C(kj + 1) - C(kj), C(kj - 1) - C(kj)}···(5)

[0060] Whether the calculated shift amount X in formula (2) is reliable is determined as follows. As shown in Fig. 15(b), when the correlation degree between a pair of data is low, the value of the minimum value C(X) of the interpolated correlation amount becomes large. Therefore, when C(X) is greater than or equal to a predetermined threshold value, it is determined that the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled. Alternatively, in order to normalize C(X) with the contrast of the data, when the value obtained by dividing C(X) by SLOP, which is a value proportional to the contrast, is greater than or equal to a predetermined value, it is determined that the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled. Or, when SLOP, which is a value proportional to the contrast, is less than or equal to a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount is low, and the calculated shift amount X is canceled.

[0061] As shown in Fig. 15(c), when the correlation degree of a pair of data is low and there is no drop in the correlation amount C(k) between the shift ranges kmin to kmax, the minimum value C(X) cannot be obtained, and in such a case, it is determined that focus detection is impossible.

[0062] When it is determined that the calculated shift amount X is reliable, it is converted into the image shift amount shft by equation (6). shft = PY·X ···(6)

[0063] In equation (6), PY is a value twice the pixel pitch of the focus detection pixel 311 (the pixel pitch of the focus detection pixels of the same color).

[0064] The image shift amount shft calculated by equation (6) is multiplied by a predetermined conversion coefficient k to be converted into the defocus amount def. def = k·shft1 ···(7)

[0065] In equation (7), the conversion coefficient k is a conversion coefficient according to the proportional relationship between the center-of-gravity interval of the pair of ranging pupils 93, 94 and the ranging pupil distance, and it changes according to the aperture F value of the optical system.

[0066] In this way, three defocus amounts are calculated for the three colors of the Bayer array, so an averaging process such as simple averaging or weighted averaging is performed to calculate the final defocus amount in the selected focus detection area.

[0067] Fig. 14 is an operation flowchart of the CPU b223 for image processing of the body drive control device 214. When the power of the digital still camera 201 is turned on in step S200, the image processing operations after step S210 are started. In step S210, the added digital data (corresponding to the data of the imaging pixels) obtained by adding the output data of the pair of photoelectric conversion units of the focus detection pixels in frame synchronization is read out, and after performing image processing for display on the data, it is displayed on the electronic viewfinder. The added digital data read out in step S210 is the added digital signal output from the first horizontal output circuit 49 described above.

[0068] In step S220, it is determined whether or not a shutter release has been made by operating a shutter button (not shown). If it is determined that the shutter release has not been made, the process returns to step S210 and the above-described operations are repeated. On the other hand, if it is determined that the shutter release has been made, the process proceeds to step S230 and a shooting operation corresponding to the shutter release is performed. First, an aperture adjustment command is sent to the lens drive control device 206 to set the aperture value of the interchangeable lens 202 to the controlled F value (the F value set by the photographer or automatically). When the aperture control is completed, the added digital data (corresponding to the data of the imaging pixels arranged in a Bayer array) obtained by adding the output data of the pair of photoelectric conversion units of the focus detection pixels is read out, and well-known image processing (demosaicing processing, noise processing, gradation processing, white balance processing, etc.) is performed on the added digital data to generate image data, and the image data is stored in the memory card in step S240. When a series of shooting operations is completed, the process returns to step S210 and the above-described operations are repeated.

[0069] In the first embodiment described above, the focus detection was performed only in the selected focus detection area. However, since the focus detection data for the entire screen is stored in the buffer memory, if the processing ability of the focus detection CPU a222 is high, the focus detection may be performed in a plurality of focus detection areas of the entire screen, and the focus adjustment of the lens may be performed according to the result.

[0070] In the first embodiment described above, the description was made on the assumption that the data obtained by adding the data of the pair of photoelectric conversion units of the focus detection pixels for images is read out in full data for each frame. However, instead of reading out all the data, a circuit configuration for performing decimated reading (row / column) or pixel addition reading (row / column) may be further added to the configuration of the present invention, and the read image data may be used for display or the like.

[0071] In the first embodiment described above, the description was given assuming that the data of the pair of photoelectric conversion units of all the focus detection pixels for focus detection is read out for each frame. However, reading out the data of the pair of photoelectric conversion units of all the focus detection pixels involves a large load and also requires a large amount of memory capacity for data storage. Therefore, it is also possible to perform decimation (read out once every several frames) / interleaving (read out one row every several rows) / partial row readout (read out only some rows) / column decimation (read out one column every several columns) / partial column readout (read out only some columns) as necessary.

[0072] FIG. 16 is a timing chart corresponding to FIG. 12 in the case of performing partial row readout (reading out the data of the pair of photoelectric conversion units of the focus detection pixels only in the (2n + 2)-th row), and is an enlarged view of the operation portions of the (2n + 1)-th row, (2n + 2)-th row, and (2n + 3)-th row in FIG. 11.

[0073] The operation when the (2n + 2)-th row of the pixel array unit 40 is selected by the control signal R(2n + 2) is the same as that in FIG. 12. On the other hand, when rows other than the (2n + 2)-th row are selected (operations corresponding to the control signal R(2n + 1) and the control signal R(2n + 3) in FIG. 16), the control signal TM2 is not generated, and the digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 for one line, which are converted into digital signals by the ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of the column AD conversion device 42, are not stored in the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) of the second line memory 44 connected to the ADCs (23(1)a, 23(1)b to 23(2M)a, 23(2M)b) of the column AD conversion device 42. Also, since the scanning signal TS2 is not generated, serial output is not sequentially performed to the outside from the second horizontal output circuit 45 during the period until the next horizontal synchronization signal HS is generated.

[0074] The rows for partial row readout and the columns for partial column readout can be made changeable by sending information from the body drive control device 214 to the imaging device 212 according to the position of the selected focus detection area.

[0075] In the first embodiment described above, a column AD conversion device 42 having the number of ADCs corresponding to the number of pairs of photoelectric conversion units of the focus detection pixels for one line is provided, and a digital addition circuit 26 for digitally adding the digital output signals of the pair of ADCs is provided for each line of the focus detection pixels. Since the column digital addition device 46 is provided, it is possible to perform in parallel the individual readout operation of the output signals of the pair of photoelectric conversion units of the focus detection pixels and the readout operation of the addition signal (corresponding to the output signal of the imaging pixel) obtained by adding the output signals of the pair of photoelectric conversion units during one frame period. As a result, the problem of the conventional technique (providing an analog addition device for each focus detection pixel) (the individual readout operation of the output signals of the pair of photoelectric conversion units of the focus detection pixels and the readout operation of the addition signal (corresponding to the output signal of the imaging pixel) obtained by adding the output signals of the pair of photoelectric conversion units cannot be performed in parallel during one frame period) can be solved.

[0076] As a method for solving the problems of the conventional technique, it is also conceivable to individually read out the output data of the pair of photoelectric conversion units of all the focus detection pixels from the imaging device during one frame period, temporarily store them in an external buffer memory, and perform an addition process on the output data of the pair of photoelectric conversion units stored in the buffer memory. However, in that case, the processing time increases by the amount of the addition processing time, and the external processing load also increases. According to the configuration and operation of the imaging device of the present application, the image data can be handled in the same manner as a normal imaging device with respect to readout / image processing. In addition, since it is possible to partially individually read out the output data of the pair of photoelectric conversion units of the focus detection pixels, the load of the readout process can be reduced, and the capacity of the buffer memory for data storage can also be saved.

[0077] As a method for solving the problems of the conventional technology, when serially outputting the output data of a pair of photoelectric conversion units of the focus detection pixels by horizontally scanning instead of providing the column digital adder 46, a data holding memory (which holds the data with a delay for one data output time) and a digital addition circuit are provided in parallel at the output end of the second horizontal output circuit 45, and the output data of the pair of photoelectric conversion units of the focus detection pixels are added in synchronization with the individual data output to generate and output addition data. However, the data transfer rate is reduced by the amount of the addition processing time (the addition time for one focus detection pixel × the total number of focus detection pixels), and high-speed data reading becomes impossible. The imaging device 212 of the present application is provided with a column digital adder 46, and the addition processing is performed independently and simultaneously for each column, so that high-speed reading is possible at almost the same data transfer rate as an imaging device composed only of normal imaging pixels.

[0078] <Second Embodiment> In the first embodiment, the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 are juxtaposed in the horizontal direction (row direction). However, by setting the direction in which the pair of photoelectric conversion units of the focus detection pixel are juxtaposed to be other than the horizontal direction (row direction), image shift detection can be performed in a direction other than the horizontal direction. FIG. 17 is a diagram showing a focus detection pixel 312 having a configuration in which the focus detection pixel 311 shown in FIG. 6 is rotated by 90 degrees. The focus detection pixel 312 is composed of a rectangular microlens 10 and a pair of photoelectric conversion units 16 and 17 divided into two by an element separation region 18 extending in the horizontal direction. When the pair of photoelectric conversion units 16 and 17 are integrated, the size becomes equivalent to that of the photoelectric conversion unit of a normal imaging pixel.

[0079] FIG. 18 is a diagram corresponding to the pixel layout diagram of FIG. 3 (the filter array corresponds to FIG. 4), and is a front view showing a detailed configuration of the imaging device 212 in which the focus detection pixel 311 and the focus detection pixel 312 are arranged, and shows an enlarged view of the vicinity of the focus detection area 101 on the imaging device 212. The focus detection pixel 311 and the focus detection pixel 312 are alternately arranged every other row.

[0080] FIG. 19 is a block diagram showing the configuration of the imaging device 212 having the pixel layout shown in FIG. 18. The same parts as those in the configuration of FIG. 10 will not be described, and only the characteristic parts will be described. The difference from FIG. 10 in the pixel array unit 40 is that in the even rows, the focus detection pixels 312 each including a pair of photoelectric conversion units 16 and 17 separated in the vertical direction are arranged.

[0081] The pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 arranged in the even rows are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to the control signal R(L) (L is an even number). Also, one of the pair of photoelectric conversion units 16 of each focus detection pixel 312 arranged in the even rows is connected to one of the two column signal lines 22(m)a provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 16 is output to the column signal line 22(m)a. Further, the other photoelectric conversion unit 17 of the pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 is connected to the other column signal line 22(m)b of the two column signal lines provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 17 is output to the column signal line 22(m)b. For example, when the focus detection pixel 312 in the L-th row of the pixel array unit 40 is selected by the control signal R(L) given from the row scanning circuit 41, the output signals of the pair of photoelectric conversion units 16 and 17 of the focus detection pixel 312 in the L-th row are output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b).

[0082] When the imaging device 212 having the above configuration is used, it is possible to perform phase difference detection on a subject image with a contrast change in the horizontal direction using a pair of data obtained by horizontally grouping the data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 of the same color arranged in the odd rows for each photoelectric conversion unit, and it is also possible to perform phase difference detection on a subject image with a contrast change in the vertical direction using a pair of data obtained by vertically grouping the data of the pair of photoelectric conversion units 16 and 17 of the focus detection pixels 312 of the same color arranged in the even rows for each photoelectric conversion unit.

[0083] FIG. 20 is a modification of FIG. 18, in which the focus detection pixels 311 and the focus detection pixels 312 are arranged alternately with one pixel in between. When the imaging device 212 having such a configuration is used, phase difference detection can be performed on a subject image with a contrast change in the horizontal direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 of the same color arranged in odd-numbered columns in the horizontal direction for each photoelectric conversion unit. At the same time, phase difference detection can be performed on a subject image with a contrast change in the vertical direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 16 and 17 of the focus detection pixels 312 of the same color arranged in even-numbered columns in the vertical direction for each photoelectric conversion unit.

[0084] FIG. 21 is a modification of FIG. 18, in which the focus detection pixels 311 and the focus detection pixels 312 are arranged in a staggered pattern. That is, the focus detection pixels 311 are arranged at positions of (odd rows and odd columns) or (even rows and even columns), and the focus detection pixels 312 are arranged at positions of (odd rows and even columns) or (even rows and odd columns). From the viewpoint of the color filter of the Bayer array, the focus detection pixel 311 is provided with a green filter, and the focus detection pixel 312 is provided with a red filter or a blue filter.

[0085] When the imaging device 212 having such a configuration is used, phase difference detection can be performed on a subject image with a contrast change in the horizontal direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 provided with green filters arranged in odd-numbered columns of odd-numbered rows or even-numbered columns of even-numbered rows in the horizontal direction for each photoelectric conversion unit. At the same time, phase difference detection can be performed on a subject image with a contrast change in the vertical direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 16 and 17 of the focus detection pixels 312 provided with blue filters arranged in even-numbered columns of odd-numbered rows or the focus detection pixels 312 provided with red filters arranged in odd-numbered columns of even-numbered rows in the vertical direction for each photoelectric conversion unit.

[0086] <Third Embodiment> In the first embodiment, all pixels were constituted by focus detection pixels. However, by mixing normal imaging pixels in which the photoelectric conversion unit is not divided and focus detection pixels in which the photoelectric conversion unit is divided, the number of focus detection pixels in the entire imaging device is reduced, the configuration of the imaging device is simplified, and the data transfer rate of the focus detection data can be made the same as that of the image processing data by reducing the number of focus detection data output from the imaging device to the outside.

[0087] As shown in FIG. 22, the imaging pixel 310 has a rectangular microlens 10 and a photoelectric conversion unit 11 whose light receiving area is restricted by a light shielding mask described later.

[0088] FIG. 23 is a cross-sectional view of the imaging pixel 310 shown in FIG. 22. In the imaging pixel 310, a light shielding mask 30 is formed in proximity to the photoelectric conversion unit 11 for imaging, and the photoelectric conversion unit 11 receives light that has passed through the opening 30a of the light shielding mask 30. A planarization layer 31 is formed on the light shielding mask 30, and a color filter 38 is formed thereon. A planarization layer 32 is formed on the color filter 38, and a microlens 10 is formed thereon. The shape of the opening 30a is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit board 29.

[0089] FIG. 24 is a diagram for explaining the state of the imaging light beam received by the imaging pixel 310 shown in FIG. 22 in comparison with FIG. 8, and the description of the overlapping portion with FIG. 8 is omitted.

[0090] The imaging pixel 310 is composed of a microlens 10 and a photoelectric conversion unit 11 disposed behind it, etc. The shape of the opening 30a (see FIG. 23) disposed in proximity to the photoelectric conversion unit 11 is projected onto the exit pupil 90 that is separated from the microlens 10 by the ranging pupil distance d, and the projected shape forms a region 95 that substantially circumscribes the ranging pupils 93 and 94.

[0091] The photoelectric conversion unit 11 outputs a signal corresponding to the intensity of the image formed on the microlens 11 by the imaging light beam 71 that passes through the region 95 and travels toward the microlens 10.

[0092] FIG. 25 is a diagram corresponding to the pixel layout diagram of FIG. 3 (the filter array corresponds to FIG. 4), in which the imaging pixels 310 and the focus detection pixels 311 are arranged alternately in a staggered manner. That is, the focus detection pixels 311 are arranged at positions of (odd rows and odd columns) or (even rows and even columns), and the imaging pixels 310 are arranged at positions of (odd rows and even columns) or (odd rows and even columns). From the perspective of the color filters of the Bayer array, the focus detection pixels 311 are provided with green filters, and the imaging pixels 310 are provided with red filters or blue filters. In terms of focus detection performance, since the spectral sensitivity characteristics of the green filter are located between the spectral sensitivity characteristics of the red filter and the spectral sensitivity characteristics of the blue filter as shown in FIG. 5, it is preferable to provide the focus detection pixels 311 with green filters. Also, as shown in FIGS. 6 and 22, since there is an element isolation region 15, the sum of the surface areas of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 is smaller than the surface area of the photoelectric conversion unit 11 of the imaging pixels 310. Therefore, in terms of imaging performance, since the sum of the photoelectric conversion signal values output by the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 is smaller than the photoelectric conversion signal value of the photoelectric conversion unit 11 of the imaging pixels 310, it is preferable to provide the focus detection pixels 311 with green filters that are arranged more than the red filters and the blue filters.

[0093] FIG. 26 is a block diagram showing the configuration of the imaging device 212 having the pixel layout shown in FIG. 25. The same parts as those in FIG. 10 will not be described, and only the characteristic parts will be described. The main difference from FIG. 10 is that by providing a second column switch device 43 between the column AD conversion device 42, the second line memory 44, and the column digital addition device 46, the number of memories constituting the second line memory 44 and the number of digital addition circuits constituting the column digital addition device 46 are reduced.

[0094] The imaging element 212 includes a pixel array unit 40 in which a large number of focus detection pixels 311 each including a pair of photoelectric conversion units 13 and 14 are two-dimensionally arranged in a matrix (matrix) form, and further includes a row scanning circuit 41, a column AD conversion device 42, a second column switch device 43, a second line memory 44, a second column scanning circuit 51, a second horizontal output circuit 45, a column digital addition device 46, a first column switch device 47, a first line memory 48, a first column scanning circuit 52, a first horizontal output circuit 49, and a timing control circuit 50.

[0095] In this system configuration, the timing control circuit 50 generates a clock signal, a control signal, etc. serving as a reference for the operations of the row scanning circuit 41, the column AD conversion device 42, the first column switch device 47, the second column switch device 43, the column digital addition device 46, the first line memory 48, the first line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc. based on a master clock input from the outside and a control signal input from the imaging element control unit 220, and supplies them to the row scanning circuit 41, the column AD conversion device 42, the first column switch device 47, the second column switch device 43, the column digital addition device 46, the first line memory 48, the first line memory 44, the first column scanning circuit 52, the second column scanning circuit 51, etc.

[0096] In the pixel array unit 40, 2N rows and 2M columns of imaging pixels 310 and focus detection pixels 311 are two-dimensionally arranged. In FIG. 26, the upper left focus detection pixel 311 is the pixel in the first row and the first column, and a green filter of a Bayer array is arranged in this pixel. For this 2N-row and 2M-column pixel arrangement, row control lines 21 (21(1) to 21(2N)) are wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b) are wired for each column. The total number of row control lines is 2N, and the total number of column signal lines is 4M. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to each output end corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each row control line 21.

[0097] The photoelectric conversion units of the imaging pixels 310 and the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the same row are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to the control signal R(L). The photoelectric conversion unit 11 of the imaging pixel 310 is connected to one of the two column signal lines 22(m)a provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 11 is output to the column signal line 22(m)a. Also, one of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311, i.e., the photoelectric conversion unit 13, is connected to one of the two column signal lines 22(m)b provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 13 is output to the column signal line 22(m)b. Further, the other photoelectric conversion unit 14 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 is connected to the other column signal line 22(m)a of the two column signal lines provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 14 is output to the column signal line 22(m)a. For example, when the L-th row of the pixel array unit 40 is selected by the control signal R(L) given from the row scanning circuit 41, the output signal of the photoelectric conversion unit 11 of the imaging pixel 310 in the L-th row is output to the column signal lines (22(1)a to 22(2M)a), and the output signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in the L-th row are output to the column signal lines (22(1)a, 22(1)b to 22(2M)a, 22(2M)b). At this time, when L is odd, since the imaging pixels 310 are arranged in the even columns of this row, the signals on the column signal lines 22(2m)b corresponding to the even columns become invalid signals. Also, when L is even, since the imaging pixels 310 are arranged in the odd columns of this row, the signals on the column signal lines 22(2m + 1)b corresponding to the odd columns become invalid signals.

[0098] The column AD conversion device 42 has 4M ADCs (analog-digital conversion circuits) 23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b provided for each of the column signal lines 22(1)a, 22(1)b ~ 22(2M)a, 22(2M)b provided corresponding to the pixel columns of the pixel array unit 40, and converts the analog signals output column by column from each pixel of the pixel array unit 40 into H-bit digital signals (S(1)a, S(1)b ~ S(2M)a, S(2M)b) according to the control signal TA1 given from the timing control circuit 50 and outputs them.

[0099] The second column switch device 43 has M switches 24(1, 2) ~ 24(2M - 1, 2M) provided for every two adjacent pixel columns, and selects and outputs the digital signals output for each of the ADCs (23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b) according to the control signal TW2 given from the timing control circuit 50.

[0100] Figs. 27(a) and (b) are diagrams for explaining the selection operation of the switches 24(2m + 1, 2m + 2) provided for two adjacent columns of pixel columns ((2m + 1) - th column and (2m + 2) - th column). Four digital signals S(2m + 1)a, S(2m + 1)b, S(2m + 2)a, and S(2m + 2)b are input to the switches 24(2m + 1, 2m + 2) from four ADCs (23(2m + 1)a, 23(2m + 1)b, 23(2m + 2)a, 23(2m + 2)b) corresponding to two columns of pixel columns ((2m + 1) - th column and (2m + 2) - th column). Four times the number of multiples, for example, eight digital signals S(2m + 1)a, S(2m + 1)b, S(2m + 2)a, S(2m + 2)b, S(2m + 3)a, S(2m + 3)b, S(2m + 4)a, and S(2m + 4)b may be input to the switches 24(2m + 1, 2m + 2) from four times the number of multiples, for example, eight ADCs (23(2m + 1)a, 23(2m + 1)b, 23(2m + 2)a, 23(2m + 2)b, 23(2m + 3)a, 23(2m + 3)b, 23(2m + 4)a, 23(2m + 4)b) corresponding to two - multiple columns, for example, four columns of pixel columns ((2m + 1) - th column, (2m + 2) - th column, (2m + 3) - th column, and (2m + 4) - th column). In that case, for example, the focus - detection pixel 311 is arranged in the (2m + 1) - th column, and the imaging elements 310 are arranged in all of the (2m + 2) - th column, (2m + 3) - th column, and (2m + 4) - th column.

[0101] Fig. 27(a) shows the selection operation of the switches 24(2m + 1, 2m + 2) when the odd - numbered rows of the pixel array unit 40 are selected by the horizontal scanning circuit 41. Four digital signals S(2m + 1)a, S(2m + 1)b, S(2m + 2)a, and S(2m + 2)b corresponding to the imaging pixels 310 arranged in the even - numbered columns of the odd - numbered rows and the focus - detection pixel 311 arranged in the odd - numbered columns of the odd - numbered rows are input to the switches 24(2m + 1, 2m + 2). Among these, the signal S(2m + 2)b becomes an invalid signal.

[0102] Switch 24(2m + 1, 2m + 2) selects and outputs digital signals S(2m + 1)a and S(2m + 1)b corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a pair of signals (Q(2m + 1, 2m + 2)a, Q(2m + 1, 2m + 2)b) for digital addition according to the control signal TW2 (identification information of whether it is an odd row or an even row) input to the second column switch device 43.

[0103] Figure 27(b) shows the selection operation of switch 24(2m + 1, 2m + 2) when the even rows of the pixel array unit 40 are selected by the horizontal scanning circuit 41. Four digital signals S(2m + 1)a, S(2m + 1)b, S(2m + 2)a, and S(2m + 2)b corresponding to the imaging pixel 310 arranged in the odd-numbered columns of the even rows and the focus detection pixel 311 in the even-numbered columns of the even rows are input to switch 24(2m + 1, 2m + 2). Among these, signal S(2m + 1)b becomes an invalid signal.

[0104] Switch 24(2m + 1, 2m + 2) selects and outputs digital signals S(2m + 2)a and S(2m + 2)b corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a pair of signals (Q(2m + 1, 2m + 2)a, Q(2m + 1, 2m + 2)b) according to the control signal TW2 (identification information of whether it is an odd row or an even row) input to the second column switch device 43.

[0105] The second line memory 44 has a total of 2M memories (25(1,2)a, 25(1,2)b ~ 25(2M-1,2M)a, 25(2M-1,2M)b) provided in pairs for each of the M switches 24(1,2) to 24(2M-1,2M) of the second column switch device 43, and stores, as H-bit digital signals, a pair of digital signals (Q(1,2)a, Q(1,2)b ~ Q(2M-1,2M)a, Q(2M-1,2M)b) corresponding to a pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 output for each of the M switches 24(1,2) to 24(2M-1,2M) in accordance with a control signal TM2 given from the timing control circuit 50. Here, the output signals of the pair of photoelectric conversion units 13, 14 for M focus detection pixels for one line are stored as digital signals in each memory (25(1,2)a, 25(1,2)b ~ 25(22M-1,2M)a, 25(22M-1,2M)b) of the second line memory 44.

[0106] The column digital addition device 46 has a total of M digital addition circuits (26(1,2) ~ 26(2M-1,2M)) provided for each of the M switches 24(1,2) to 24(2M-1,2M) of the second column switch device 43, adds a pair of digital signals (Q(1,2)a, Q(1,2)b ~ Q(2M-1,2M)a, Q(2M-1,2M)b) corresponding to a pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 output for each of the M switches 24(1,2) to 24(2M-1,2M) in accordance with a control signal TD1 given from the timing control circuit 50, and outputs the result as an H-bit addition digital signal (P(1,2) ~ P(2M-1,2M)).

[0107] The first column switch device 47 has M switches 27(1, 2) to 27(2M - 1, 2M) provided for every two adjacent columns of pixel columns, and selects and outputs the digital signals (S(1)a, S(2)a to S(2M - 1)a, S(2M)a) output for every 2M ADCs (23(1)a to 23(2M)a) and the added digital signals (P(1, 2) to P(2M - 1, 2M)) output for every M digital addition circuits (26(1, 2) to 26(2M - 1, 2M)) according to the control signal TW1 (identification information of whether it is an odd row or an even row) given from the timing control circuit 50.

[0108] FIGS. 28(a) and (b) are diagrams for explaining the selection operation of the switch 27(2m + 1, 2m + 2) provided for two adjacent columns of pixel columns ((2m + 1) - th column and (2m + 2) - th column). Two digital signals S(2m + 1)a and S(2m + 2)a are input to the switch 27(2m + 1, 2m + 2) from two ADCs (23(2m + 1)a, 23(2m + 2)a) corresponding to the two columns of pixel columns ((2m + 1) - th column and (2m + 2) - th column), and one added digital signal P(2m + 1, 2m + 2) is input from the digital addition circuit 26(2m + 1, 2m + 2).

[0109] FIG. 28(a) shows the selection operation of the switch 27(2m + 1, 2m + 2) when the odd - numbered rows of the pixel array unit 40 are selected by the horizontal scanning circuit 41. One digital signal S(2m + 2)a corresponding to the photoelectric conversion unit 11 of the imaging pixel 310 arranged in the even - numbered column of the odd - numbered row and one digital signal S(2m + 1)a corresponding to the photoelectric conversion unit 14 of the focus detection pixel 311 arranged in the odd - numbered column of the odd - numbered row are input to the switch 27(2m + 1, 2m + 2), and the added digital signal P(2m + 1, 2m + 2) (corresponding to the signal of the imaging pixel), which is the sum of a pair of signals S(2m + 1)a, (2m + 1)b corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixel 311 arranged in the odd - numbered column of the odd - numbered row, is input from the digital addition circuit 26(2m + 1, 2m + 2). Among these, the signal S(2m + 1)a input from the ADC 23(2m + 1)a does not correspond to the signal of the imaging pixel.

[0110] Switch 27(2m + 1, 2m + 2) selects and outputs, as signal U(2m + 1) corresponding to the signal of a virtual imaging pixel arranged in an odd column, the digital addition signal P(2m + 1, 2m + 2) obtained by adding digital signals corresponding to a pair of photoelectric conversion units of focus detection pixel 311, in accordance with the control signal TW1 (identification information of whether it is an odd row or an even row) input to the first column switch device 47. At the same time, as signal U(2m + 1) of the imaging pixel arranged in an even column, it selects and outputs the digital signal S(2m + 2)a corresponding to imaging pixel 310.

[0111] Figure 28(b) shows the selection operation of switch 27(2m + 1, 2m + 2) when the even rows of pixel array unit 40 are selected by the row scanning circuit 41. One digital signal S(2m + 1)a corresponding to the photoelectric conversion unit 11 of imaging pixel 310 arranged in the odd column of the even row, one digital signal S(2m + 2)a corresponding to the photoelectric conversion unit 14 of focus detection pixel 311 arranged in the even column of the even row, and the addition digital signal P(2m + 1, 2m + 2) (corresponding to the signal of the imaging pixel) obtained by adding a pair of signals S(2m + 2)a, (2m + 2)b corresponding to a pair of photoelectric conversion units 13, 14 of focus detection pixel 311 arranged in the even column of the even row from digital addition circuit 26(2m + 1, 2m + 2) are input to switch 27(2m + 1, 2m + 2). Among these, signal S(2m + 2)a does not correspond to the signal of the imaging pixel.

[0112] Switch 27(2m + 1, 2m + 2) selects and outputs, as signal U(2m + 2) corresponding to the signal of a virtual imaging pixel arranged in an even column, the digital addition signal P(2m + 1, 2m + 2) obtained by adding digital signals corresponding to a pair of photoelectric conversion units of focus detection pixel 311, in accordance with the control signal TW1 (identification information of whether it is an odd row or an even row) input to the first column switch device 47. At the same time, as signal U(2m + 1) of the imaging pixel arranged in an odd column, it selects and outputs the digital signal S(2m + 1)a corresponding to imaging pixel 310.

[0113] The first line memory 48 has 2M memories (28(1) to 28(2M)) provided in pairs for each of the M switches (27(1, 2) to 27(2M - 1, 2M)) that make up the column switch device 47, and stores the digital signals output in pairs for each switch (27(1, 2) to 27(2M - 1, 2M)) as H-bit digital signals according to the control signal TM1 given from the timing control circuit 50. Here, in each memory (28(1) to 28(2M)) of the first line memory 48, an addition signal (corresponding to the output signal of the imaging pixel) obtained by adding the output signals of the pair of photoelectric conversion units 13 and 14 for the focus detection pixels in one row and the output signal of the photoelectric conversion unit of the imaging pixel are stored as digital signals according to the arrangement order of the focus detection pixels as imaging pixels.

[0114] The second column scanning circuit 51 is composed of a shift register or the like, and controls the column addresses and column scanning of the memories (25(1)a, 25(1)b to 25(2M)a, 25(2M)b) in the second line memory 44 under the control of the timing control circuit 50. The second line memory 44 operates according to the scanning signal TS2 given from the second column scanning circuit 51, and the H-bit digital signals stored in each of the memories (25(1, 2)a, 25(1, 2)b to 25(2M - 1, 2M)a, 25(2M - 1, 2M)b) are sequentially read out to the second horizontal output circuit 45, and are serially output (the number of data is 2M) to the outside as the output signals (digital signals) of the pair of photoelectric conversion units 13 and 14 for focus detection via the second horizontal output circuit 45.

[0115] The first column scanning circuit 52 is composed of a shift register or the like, and controls the column addresses and column scanning of the memories (28(1) to 28(2M)) in the first line memory 48 under the control of the timing control circuit 50. The first line memory 48 operates according to the scanning signal TS1 given from the first column scanning circuit 52, and the H-bit digital signals and addition digital signals stored in each of the memories (28(1) to 28(2M)) are sequentially read out to the first horizontal output circuit 49, and are serially output to the outside as output signals (digital signals) equivalent to the output signals of the imaging pixel array via the first horizontal output circuit 49.

[0116] Next, in the configuration of the imaging device shown in FIG. 26, a case where the individual readout operation of the output signals (signals for focus detection) of the pair of photoelectric conversion units of the focus detection pixels and the readout operation of the output signals (signals for image processing) corresponding to the output signals of the imaging pixels are performed in parallel during one frame period will be described with reference to the timing chart of FIG. 29.

[0117] In the configuration of the imaging device shown in FIG. 26, the outline of the row scanning selection operation by the row scanning circuit 41 is the same as the operation shown in FIG. 11.

[0118] FIG. 29 is an enlarged view of the operation portions of the (2n + 1)-th row, (2n + 2)-th row, and (2n + 3)-th row in FIG. 11. When the (2n + 1)-th row of the pixel array unit 40 is selected by the control signal R(2n + 1), the analog signals of the focus detection pixels 311 and the imaging pixels 310 for one line of the (2n + 1)-th row are output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b). The analog signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the odd-numbered columns and the analog signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even-numbered columns for one line of the (2n + 1)-th row output to the column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b) are converted into digital signals by the ADCs (23(1)a, 23(1)b~23(2M)a, 23(2M)b) of the column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b~22(2M)a, 22(2M).

[0119] The digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the odd-numbered columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even-numbered columns for one line of the (2n + 1)-th row input from the column AD conversion device 42 to the second column switch device 43 are selected and output by the second column switch device 43(24(1, 2)~24(2M - 1, 2M)) according to the control signal TW2 so that the digital signals of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the odd-numbered columns are selected.

[0120] The digital signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns output from the second column switch device 43 (24(1, 2) to 24(2M - 1, 2M)) are stored in 2M memories (25(1, 2)a, 25(1, 2)b to 25(2M - 1, 2M)a, 25(2M - 1, 2M)b) of the second line memory 44 according to the control signal TM2.

[0121] At the same time, the digital signals of a pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns are added by M digital addition circuits (26(1, 2) to 26(2M - 1, 2M)) of the column digital addition device 46 and output according to the control signal TD1.

[0122] The addition digital signal obtained by adding the digital signal corresponding to one of the pair of photoelectric conversion units of the focus detection pixels 311 arranged in the odd columns input to the first column switch device 47, the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even columns, and the digital signals corresponding to the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns is selected by the first column switch device 47 (27(1, 2) to 27(2M - 1, 2M)) according to the control signal TW1. The addition digital signal of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even columns are selected. The addition digital signal of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns is output as the output signal of the imaging pixels in the odd columns, and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even columns is output as the output signal of the imaging pixels in the even columns.

[0123] The addition digital signal of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even columns, which are selectively output by the first column switch device 47, are stored in the memory ((28(1) to 28(2M)) of the first line memory 48 according to the control signal TM1.

[0124] The digital signals of the pair of photoelectric conversion units 13 and 14 of the M focus detection pixels 311 arranged in the odd-numbered columns of the (2n + 1)th row stored in the 2M memories (25(1, 2)a, 25(1, 2)b ~ 25(2M - 1, 2M)a, 25(2M - 1, 2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 during the period until the next horizontal synchronization signal HS is generated in accordance with the scanning signal TS2.

[0125] Similarly, the 2M digital signals (the addition digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the odd-numbered columns and the digital signals of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the even-numbered columns) corresponding to the output signals of the imaging pixels of the (2n + 1)th row stored in the memories ((28(1) ~ 28(2M)) of the first line memory 48 are sequentially serially output to the outside from the first horizontal output circuit 49 during the period until the next horizontal synchronization signal HS is generated in accordance with the scanning signal TS1.

[0126] When the control signal R(2n + 2) is issued in synchronization with the next horizontal synchronization signal HS and the (2n + 2)th row of the pixel array unit 40 is selected, the analog signals of the focus detection pixels 311 and the imaging pixels 310 for one line of the (2n + 2)th row are output to the column signal lines (22(1)a, 22(1)b ~ 22(2M)a, 22(2M)b). The analog signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the even-numbered columns and the analog signals of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd-numbered columns for one line of the (2n + 2)th row output to the column signal lines (22(1)a, 22(1)b ~ 22(2M)a, 22(2M)b) are converted into digital signals by the ADCs (23(1)a, 23(1)b ~ 23(2M)a, 23(2M)b) of the column AD conversion device 42 connected to the column signal lines 22(1)a, 22(1)b ~ 22(2M)a, 22(2M) in accordance with the control signal TA1.

[0127] The digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the even columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd columns, which are input from the column AD conversion device 42 to the second column switch device 43 for one line of (2n + 2) rows, are selected and output by the second column switch device 43 (24(1, 2) to 24(2M - 1, 2M)) according to the control signal TW2. The digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the even columns are selected and output.

[0128] The digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the even columns, which are output from the second column switch device 43 (24(1, 2) to 24(2M - 1, 2M)), are stored in 2M memories (25(1, 2)a, 25(1, 2)b to 25(2M - 1, 2M)a, 25(2M - 1, 2M)b) of the second line memory 44 according to the control signal TM2.

[0129] At the same time, the digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the even columns are added and output by M digital addition circuits (26(1, 2) to 26(2M - 1, 2M)) of the column digital addition device 46 according to the control signal TD1.

[0130] The digital signal corresponding to one of the pair of photoelectric conversion units of the focus detection pixels 311 arranged in the even columns input to the first column switch device 47, the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd columns, and the digital signals corresponding to the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns are added. The added digital signal is, according to the control signal TW1, selected by the first column switch device 47 (27(1, 2)~27(2M - 1, 2M)) to be the added digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd columns. As the output signal of the imaging pixels in the even columns, the added digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns is output, and as the output signal of the imaging pixels in the odd columns, the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd columns is output.

[0131] The added digital signal of the pair of photoelectric conversion units 13, 14 of the focus detection pixels 311 arranged in the even columns and the digital signal of the photoelectric conversion unit 11 of the imaging signal 310 arranged in the odd columns, which are selectively output by the first column switch device 47, are stored in the memory of the first line memory 48 ((28(1)~28(2M)) according to the control signal TM1.

[0132] The digital signals of the pair of photoelectric conversion units 13, 14 of the M focus detection pixels 311 arranged in the even columns of the (2n + 2) rows stored in the 2M memories (25(1, 2)a, 25(1, 2)b~25(2M - 1, 2M)a, 25(2M - 1, 2M)b) of the second line memory 44 are sequentially serially output to the outside from the second horizontal output circuit 45 during the period until the next horizontal synchronization signal HS is generated according to the scanning signal TS2. Based on the digital signal output from the second horizontal output circuit 45, the CPUa222 for focus detection of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system) as shown in FIG. 13 and adjusts the focus state.

[0133] Similarly, 2M digital signals corresponding to the output signals of the imaging pixels of (2n + 2) lines stored in the memory ((28(1) to 28(2M)) of the first line memory 48) (the addition digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 arranged in odd columns) are sequentially serially output to the outside from the first horizontal output circuit 49 during the period until the next horizontal synchronization signal HS is generated in accordance with the scanning signal TS1. Based on the 2M digital signals (the addition digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 in even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 in odd columns) output from the first horizontal output circuit 49, the CPUb223 for image processing of the body drive control device 214 generates image data as shown in FIG. 14. However, in the present embodiment, in step S210 of FIG. 14, the data of the 2M digital signals (the addition digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 in even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 in odd columns) output from the first horizontal output circuit 49 are read out, and after the read data is subjected to image processing for display, it is displayed on the electronic viewfinder. Also, in step S230 of FIG. 14, the data of the 2M digital signals (the addition digital signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 in even columns and the digital signal of the photoelectric conversion unit 11 of the imaging pixels 310 in odd columns) output from the first horizontal output circuit 49 are read out, and well-known image processing (demosaic processing, noise processing, gradation processing, white balance processing, etc.) is performed on the read data to generate image data.

[0134] When the control signal R(2n + 3) is issued in synchronization with the next horizontal synchronization signal HS and the (2n + 3)th row of the pixel array unit 40 is selected, the same operation as in the case of the control signal R(2n + 1) is repeated for the focus detection pixels 311 and the imaging pixels 310 for one line of the (2n + 3)th row.

[0135] As described above, in the third embodiment, the imaging pixels 310 and the focus detection pixels 311 are mixed in the pixel array unit 40. The focus detection pixels 311 are arranged at the positions of the green filters in the Bayer array, and the imaging pixels 310 are arranged at the positions of the red and blue filters. Also, depending on whether the odd rows or the even rows of the pixel array unit 40 are scanned by the row scanning circuit 41, the selection processes of the first column switch device 47 and the second column switch device 43 are switched. Therefore, compared with the configuration of the imaging device in FIG. 10, the number of digital circuits constituting the column digital addition device 46, which has a larger circuit scale compared to the switch circuit, can be reduced (from 2M to M), and similarly, the number of memories constituting the second line memory 44, which has a larger circuit scale compared to the switch circuit, can be reduced (from 4M to 2M), and the configuration of the imaging device can be simplified. At the same time, the number of data output during the horizontal scanning period from the second horizontal output circuit 45 is halved (reduced from 4M to 2M) compared to the configuration of the imaging device in FIG. 10, becoming the same as the number of data output during the horizontal scanning period from the first horizontal output circuit 49, and the data transfer rate can be lowered. Also, the data for focus detection read from the second horizontal output circuit 45 is unified to the data of the focus detection pixels provided with green filters, which is convenient for focus detection (there are many subjects with green contrast in nature, and generally when there is chromatic aberration in the imaging lens, the focus position for green is set as the in-focus position).

[0136] In the third embodiment described above, since the number of data for focus detection corresponding to the horizontal scanning of the second column scanning circuit 51 and the number of data for imaging corresponding to the horizontal scanning of the first column scanning circuit 52 in one row are the same, by sharing the second column scanning circuit 51 and the first column scanning circuit 52 (for example, using the scanning signal TS1 of the first column scanning circuit 52 as the scanning signal TS2 of the second line memory 44), it is also possible to further simplify the configuration of the imaging device.

[0137] In the third embodiment described above, the description was made on the assumption that the data of the pair of photoelectric conversion units of all the focus detection pixels is read out for each frame for focus detection. However, reading out the data of the pair of photoelectric conversion units of all the focus detection pixels involves a large load and also requires a large memory capacity for data storage. Therefore, it is also possible to perform frame thinning (reading out once every several frames) / line thinning (reading out one line every several lines) / partial line reading (reading out only some lines) / column thinning (reading out one column every several columns) / partial column reading (reading out only some columns) as needed.

[0138] FIG. 30 is a timing chart corresponding to FIG. 29 in the case of performing partial line reading (reading out the data of the pair of photoelectric conversion units of the focus detection pixels only for the (2n + 2)-th line). The operation when the (2n + 2)-th line of the pixel array unit 40 is selected by the control signal R(2n + 2) is the same as that in FIG. 29. On the other hand, when lines other than the (2n + 2)-th line are selected (operations according to the control signal R(2n + 1) and the control signal R(2n + 3) in FIG. 30), the control signal TM2 is not generated, and the data of the pair of photoelectric conversion units of the focus detection pixels is not stored in the memories (25(1, 2)a, 25(1, 2)b to 25(2M - 1, 2M)a, 25(2M - 1, 2M)b) of the second line memory 44. Also, since the scanning signal TS2 is not generated, serial output is not sequentially performed from the second horizontal output circuit 45 to the outside during the period until the next horizontal synchronization signal HS is generated.

[0139] The lines for partial line reading and the columns for partial column reading can be made changeable by sending information from the body drive control device 214 to the imaging device 212 according to the position of the selected focus detection area.

[0140] In the third embodiment described above, as shown in FIG. 28, each switch constituting the first column switch device 47 selects two signals corresponding to the output signals of the imaging pixels according to whether odd rows or even rows of the pixel array unit 40 are selected by the row scanning circuit 41, and distributes the two selected signals to the memories ((28(1) to 28(2M)) of the first line memory 48 so as to match the pixel arrangement in the selected row. However, without distributing the two selected signals in accordance with the pixel arrangement order in the selected row, they are fixedly stored in the memories ((28(1) to 28(2M)) of the first line memory 48, and the scanning signal TS1 supplied by the first column scanning circuit 52 to the memories ((28(1) to 28(2M)) of the first line memory 48 is changed according to whether odd rows or even rows of the pixel array unit 40 are selected by the row scanning circuit 41, and the memories ((28(1) to 28(2M)) of the first line memory 48 may be scanned by making the scanning signal TS1 match the pixel arrangement order in the selected row.

[0141] <Fourth Embodiment> The fourth embodiment is a modification of the third embodiment. In the configuration of the image sensor 212 of the fourth embodiment shown in FIG. 31, the same parts as those in the configuration of FIG. 26 will not be described, and only the characteristic parts will be described. The difference between FIG. 31 and FIG. 26 in the pixel array unit 40 is that in the pixel array unit 40, in FIG. 26, two column signal lines (22(1)a, 22(1)b~22(2M)a, 22(2M)b) are wired for each column, and the total number of column signal lines is 4M. In contrast, in FIG. 31, the number of column signal lines in the even columns is reduced to 1, and by sharing the column signal lines in the odd columns, the total number of column signal lines is reduced to 3M. In the fourth embodiment, by reducing the number of column signal lines, the overcrowded state of the wiring layout in the pixel array unit 40 is alleviated, and the area of the photoelectric conversion unit can be increased, enabling higher-quality image acquisition and higher-precision focus detection.

[0142] Also, in the fourth embodiment, as the number of column signal lines decreases, the number of ADCs constituting the column AD conversion device 42 can also be reduced (from 4M to 3M), and the configuration of the imaging device can be simplified.

[0143] In FIG. 31, one end of each of the row control lines 21 (21(1) to 21(2N)) is connected to each output end corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each of the row control lines 21.

[0144] The photoelectric conversion units of the imaging pixels 310 arranged in the same row and the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to the control signal R(L). In the pixel array unit 40, two column signal lines 22(2m + 1)a and 22(2m + 1)b are arranged in odd-numbered columns, and one column signal line 22(2m + 2)a is arranged in even-numbered columns. The photoelectric conversion unit 11 of the imaging pixel 310 provided in the odd-numbered column and the photoelectric conversion unit 14 of the focus detection pixel 311 provided in the odd-numbered column are connected to one of the two column signal lines 22(2m + 1)a provided in the odd-numbered column, and the photoelectric conversion unit 13 of the focus detection pixel 311 provided in the odd-numbered column is connected to the other column signal line 22(2m + 1)b provided in the odd-numbered column. Also, the photoelectric conversion unit 11 of the imaging pixel 310 provided in the even-numbered column and the photoelectric conversion unit 14 of the focus detection pixel 311 provided in the even-numbered column are connected to the column signal line 22(2m + 2)a provided in the even-numbered column, and the photoelectric conversion unit 13 of the focus detection pixel 311 provided in the even-numbered column is connected to the column signal line 22(2m + 1)b provided in the odd-numbered column.

[0145] For example, when the odd-numbered rows of the pixel array unit 40 are selected by the row scanning circuit 41, the output signals of the photoelectric conversion units 11 of the imaging pixels 310 in the odd-numbered rows are output to the column signal lines 22(2m + 2)a, and the output signals of the pair of photoelectric conversion units 13 and 14 of the focus detection pixels 311 in the odd-numbered rows are output to the column signal lines 22(2m + 1)a and 22(2m + 1)b. Also, when the even-numbered rows of the pixel array unit 40 are selected by the row scanning circuit 41, the output signal of the photoelectric conversion unit 11 of the imaging pixels 310 in the even-numbered rows is output to the column signal line 22(2m + 1)a, the output signal of the photoelectric conversion unit 14 of the focus detection pixels 311 in the even-numbered rows is output to the column signal line 22(2m + 2)a, and the output signal of the photoelectric conversion unit 13 of the focus detection pixels 311 in the even-numbered rows is output to the column signal line 22(2m + 1)b.

[0146] The column AD conversion device 42 has 3M ADCs (analog-digital conversion circuits) 23(1)a, 23(1)b ~ 23(2M)a provided for each of the 3M column signal lines 22(1)a, 22(1)b ~ 22(2M)a provided corresponding to the pixel columns of the pixel array unit 40, and converts the analog signals output column by column from each pixel of the pixel array unit 40 into H-bit digital signals (S(1)a, S(1)b ~ S(2M)a) according to the control signal TA1 given from the timing control circuit 50 and outputs them.

[0147] The second column switch device 43 has M switches 24(1, 2) ~ 24(2M - 1, 2M) provided for each pair of adjacent pixel columns, and selects and outputs the digital signals output for each ADC (23(1)a, 23(1)b ~ 23(2M)a) according to the control signal TW2 given from the timing control circuit 50.

[0148] FIG. 32 is a diagram for explaining the selection operation of the switches 24(2m + 1, 2m + 2) provided in two adjacent columns of pixel columns ((2m + 1)-th column and (2m + 2)-th column). Three digital signals S(2m + 1)a, S(2m + 1)b, and S(2m + 2)a are input to the switches 24(2m + 1, 2m + 2) from two ADCs (23(2m + 1)a, 23(2m + 1)b) corresponding to the odd-numbered column (2m + 1)-th column and one ADC (23(2m + 2)a) corresponding to the even-numbered column (2m + 2)-th column.

[0149] FIG. 32(a) shows the selection operation of the switches 24(2m + 1, 2m + 2) when the odd-numbered rows of the pixel array unit 40 are selected by the row scanning circuit 41. Three digital signals S(2m + 1)a, S(2m + 1)b, and S(2m + 2)a corresponding to the imaging pixels 310 arranged in the even-numbered columns of the odd-numbered rows and the focus detection pixels 311 arranged in the odd-numbered columns of the odd-numbered rows are input to the switches 24(2m + 1, 2m + 2).

[0150] The switches 24(2m + 1, 2m + 2) select and output the digital signals S(2m + 1)a and S(2m + 1)b corresponding to the pair of photoelectric conversion units of the focus detection pixels 311 as a pair of signals for digital addition (Q(2m + 1, 2m + 2)a, Q(2m + 1, 2m + 2)b) according to the control signal TW2 (indicating odd-numbered rows) input to the second column switch device 43.

[0151] FIG. 32(b) shows the selection operation of the switches 24(2m + 1, 2m + 2) when the even-numbered rows of the pixel array unit 40 are selected by the row scanning circuit 41. Three digital signals S(2m + 1)a, S(2m + 1)b, and S(2m + 2)a corresponding to the imaging pixels 310 arranged in the odd-numbered columns of the even-numbered rows and the focus detection pixels 311 arranged in the even-numbered columns of the even-numbered rows are input to the switches 24(2m + 1, 2m + 2).

[0152] Switch 24(2m + 1, 2m + 2) selects and outputs digital signals S(2m + 2)a and S(2m + 1)b corresponding to a pair of photoelectric conversion units of the focus detection pixel 311 as a pair of signals (Q(2m + 1, 2m + 2)a, Q(2m + 1, 2m + 2)b) corresponding to the pair of photoelectric conversion units of the focus detection pixel 311 in response to the control signal TW2 (indicating even rows) input to the second column switch device 43.

[0153] <Fifth Embodiment> The fifth embodiment is a modified example of the configuration of the focus detection pixel in the pixel array unit in the third embodiment. FIG. 33 is a diagram corresponding to the pixel layout diagram of FIG. 25 (the filter array corresponds to FIG. 4), and the focus detection pixel 311 arranged in the even rows of FIG. 25 is replaced by a focus detection pixel 312 having a pair of vertically juxtaposed photoelectric conversion units 16, 17.

[0154] That is, in the odd rows, the focus detection pixel 311 is arranged in the odd columns and the imaging pixel 310 is arranged in the even columns, and in the even rows, the imaging pixel 310 is arranged in the odd columns and the focus detection pixel 312 is arranged in the even columns. From the perspective of the color filter of the Bayer array, the focus detection pixel 311 and the focus detection pixel 312 are provided with green filters, and the imaging pixel 310 is provided with a red filter or a blue filter.

[0155] FIG. 34 is a block diagram showing the configuration of the imaging device 212 having the pixel layout shown in FIG. 33. The parts identical to those in FIG. 26 are not described, and only the characteristic parts will be described. The difference from FIG. 26 in the pixel array unit 40 is that a focus detection pixel 312 having a pair of vertically separated photoelectric conversion units 16, 17 is arranged in the even columns of the even rows.

[0156] A pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 arranged in the even-numbered columns of the even-numbered rows are connected to the horizontal scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed according to the control signal R(L) (L is an even number). Also, one of the pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 in the even-numbered columns of the even-numbered rows, i.e., the photoelectric conversion unit 16, is connected to one of the two column signal lines 22(2m + 2)a provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 16 is output to the column signal line 22(2m + 2)a. Further, the other photoelectric conversion unit 17 of the pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 is connected to the other column signal line 22(2m + 2)b of the two column signal lines provided for each column, and the output signal (analog signal) of the photoelectric conversion unit 17 is output to the column signal line 22(2m + 2)b.

[0157] When the image sensor 212 having the above-described configuration is used, it is possible to perform phase difference detection on a subject image with a contrast change in the horizontal direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 provided with the green filter arranged in the even-numbered columns of the odd-numbered rows for each photoelectric conversion unit in the horizontal direction. Also, it is possible to perform phase difference detection on a subject image with a contrast change in the vertical direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 16 and 17 of the focus detection pixel 312 provided with the green filter arranged in the even-numbered columns of the even-numbered rows for each photoelectric conversion unit in the vertical direction.

[0158] <Sixth Embodiment> The sixth embodiment is a modification of the configuration of the focus detection pixels in the pixel array unit in the fourth embodiment. The pixel layout in the sixth embodiment is the same as that in FIG. 33, and the focus detection pixel 311 arranged in the even-numbered rows in FIG. 25 is replaced with a focus detection pixel 312 having a pair of vertically juxtaposed photoelectric conversion units 16 and 17.

[0159] That is, in odd rows, focus detection pixels 311 are arranged in odd columns, and imaging pixels 310 are arranged in even columns. In even rows, imaging pixels 310 are arranged in odd columns, and focus detection pixels 312 are arranged in even columns. From the perspective of the color filter of the Bayer array, the focus detection pixels 311 and the focus detection pixels 312 are equipped with green filters, and the imaging pixels 310 are equipped with red filters or blue filters.

[0160] FIG. 35 is a block diagram showing the configuration of the imaging device 212 having the pixel layout shown in FIG. 33. The same parts as those in the configuration of FIG. 31 will be omitted from the description, and only the characteristic parts will be described. The difference from FIG. 31 in the pixel array unit 40 is that focus detection pixels 312 each including a pair of vertically separated photoelectric conversion units 16 and 17 are arranged in the even columns of the even rows.

[0161] The pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 arranged in the even columns of the even rows are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed in response to the control signal R(L) (L is an even number). One of the pair of photoelectric conversion units 16 of each focus detection pixel 312 is connected to one of the column signal lines 22(2m + 2)a of the one column signal line provided in the even column, and the output signal (analog signal) of the photoelectric conversion unit 16 is output to the column signal line 22(2m + 2)a. The other photoelectric conversion unit 17 of the pair of photoelectric conversion units 16 and 17 of each focus detection pixel 312 is connected to one of the two column signal lines 22(2m + 1)b provided in the odd column, and the output signal (analog signal) of the photoelectric conversion unit 17 is output to the column signal line 22(2m + 1)b.

[0162] When the imaging device 212 configured as described above is used, it is possible to perform phase difference detection on a subject image with a contrast change in the horizontal direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 having a green filter arranged in the even columns of the odd rows in the horizontal direction for each photoelectric conversion unit. Also, it is possible to perform phase difference detection on a subject image with a contrast change in the vertical direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 16 and 17 of the focus detection pixel 312 having a green filter arranged in the even columns of the even rows in the vertical direction for each photoelectric conversion unit.

[0163] <Seventh Embodiment> In the first to sixth embodiments, the pixels (focus detection pixels, imaging pixels) in the pixel array unit 40 are arranged in a square lattice, but the present invention can also be applied to pixel arrays other than the square lattice pixel array.

[0164] FIG. 36 shows a pixel array called a so-called honeycomb array, which is a pixel array obtained by rotating a square lattice array by 45 degrees. Also, a filter array corresponding to the pixel layout of FIG. 36 is shown in FIG. 37. The filter array shown in FIG. 37 is a filter array obtained by tilting a Bayer array by 45 degrees. In the honeycomb array shown in FIGS. 36 and 37, focus detection pixels 411 in which a pair of photoelectric conversion units 33 and 34 are juxtaposed in the horizontal direction are arranged.

[0165] The matrix in such a honeycomb array is defined as follows. That is, the horizontal pixel array in which the green filter is arranged is the odd rows, the horizontal pixel array in which the red filter or the blue filter is arranged is the even rows, the vertical pixel array in which the green filter is arranged is the odd columns, and the vertical pixel array in which the red filter or the blue filter is arranged is the even columns.

[0166] FIG. 38 is a block diagram showing the configuration of the imaging device 212 having the pixel layout (2N rows and 2M columns) of the honeycomb array shown in FIG. 36. In the configuration of the imaging device shown in FIG. 10, the focus detection pixels 311 arranged in the pixel array unit 40 are thinned out every other pixel and replaced with the focus detection pixels 411. That is, in the odd rows, the focus detection pixels 411 are arranged only in the odd columns, and in the even rows, the focus detection pixels 411 are arranged only in the even columns.

[0167] In the pixel array unit 40, 2N rows and 2M columns of the focus detection pixels 411 are two-dimensionally arranged. In FIG. 38, the upper left focus detection pixel 411 is the pixel in the first row and the first column, and a green filter is arranged on this pixel. For this 2N-row and 2M-column pixel arrangement, row control lines 21 (21(1) to 21(2N)) are wired for each row, and two column signal lines (22(1)a, 22(1)b to 22(2M−1)a, 22(2M-1)b) are wired in the odd columns. One end of each of the row control lines 21 (21(1) to 21(2N)) is connected to each output end corresponding to each row of the row scanning circuit 41, and control signals R(1) to R(2N) are output to each of the row control lines 21.

[0168] A pair of photoelectric conversion units 33, 34 of each focus detection pixel 411 are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed according to the control signal R(L). Also, one of the pair of photoelectric conversion units 33, 34 of the focus detection pixel 411 arranged in the odd columns (2m + 1 columns) of the odd rows is connected to one of the two column signal lines 22(2m + 1)b provided in the odd columns (2m + 1 columns), and the output signal (analog signal) of the photoelectric conversion unit 33 is output to the column signal line 22(2m + 1)b. The other photoelectric conversion unit 34 of the pair of photoelectric conversion units 33, 34 of each focus detection pixel 411 is connected to the other column signal line 22(2m + 1)a of the two column signal lines provided in the odd columns (2m + 1 columns), and the output signal (analog signal) of the photoelectric conversion unit 34 is output to the column signal line 22(2m + 1)a.

[0169] Also, one of the pair of photoelectric conversion units 33 and 34 of the focus detection pixel 411 arranged in the even-numbered columns (2m + 2 columns) of the even-numbered rows is connected to one of the two column signal lines 22(2m + 1)b provided in the odd-numbered column (2m + 1 column), and the output signal (analog signal) of the photoelectric conversion unit 33 is output to the column signal line 22(2m + 1)b. Further, the other photoelectric conversion unit 34 of the pair of photoelectric conversion units 33 and 34 of each focus detection pixel 411 is connected to the other column signal line 22(2m + 1)a of the two column signal lines provided in the odd-numbered column (2m + 1 column), and the output signal (analog signal) of the photoelectric conversion unit 34 is output to the column signal line 22(2m + 1)a.

[0170] The column AD conversion device 42 has ADCs (analog-digital conversion circuits) 23(1)a, 23(1)b ~ 23(2M - 1)a, 23(2M - 1)b provided for each of the column signal lines 22(1)a, 22(1)b ~ 22(2M - 1)a, 22(2M - 1)b provided corresponding to the pixel columns of the pixel array unit 40, and converts a pair of analog signals output from each focus detection pixel 411 of the pixel array unit 40 column by column into H-bit digital signals in accordance with a control signal TA1 given from the timing control circuit 50 and outputs them.

[0171] The second line memory 44 has memories (25(1)a, 25(1)b ~ 25(2M - 1)a, 25(2M - 1)b) provided for each of the ADCs (23(1)a, 23(1)b ~ 23(2M - 1)a, 23(2M - 1)b) constituting the column AD conversion device 42, and stores the digital signals output for each of the ADCs (23(1)a, 23(1)b ~ 23(2M - 1)a, 23(2M - 1)b) as H-bit digital signals in accordance with a control signal TM2 given from the timing control circuit 50. Here, the output signals of the pair of photoelectric conversion units 33 and 34 for the focus detection pixels for one row are stored as digital signals in each of the memories (25(1)a, 25(1)b ~ 25(2M - 1)a, 25(2M - 1)b) of the second line memory 44.

[0172] The column digital adder 46 has digital adder circuits (26(1) to 26(2M - 1)) provided for each pair of ADCs ((23(1)a, 23(1)b) to (23(2M - 1)a, 23(2M - 1)b)) that make up the column AD converter 42. It adds the digital signals output from the pair of ADCs ((23(1)a, 23(1)b) to (23(2M - 1)a, 23(2M - 1)b)) according to the control signal TD1 given from the timing control circuit 50, and outputs it as an H-bit added digital signal.

[0173] The first line memory 48 has memories (28(1) to 28(2M - 1)) provided for each of the digital adder circuits (26(1) to 26(2M - 1)) that make up the column digital adder 46. It stores the added digital signals output for each of the digital adder circuits (26(1) to 26(2M - 1)) as H-bit digital signals according to the control signal TM1 given from the timing control circuit 50. Here, in each of the memories (28(1) to 28(2M - 1)) of the first line memory 48, an added signal (corresponding to the output signal of the imaging pixel) obtained by adding the output signals of the pair of photoelectric conversion units 33, 34 for one row of focus detection pixels is stored as a digital signal.

[0174] The second line memory 44 operates according to the scanning signal TS2 given from the second column scanning circuit 51. The H-bit digital signals stored in each of the memories (25(1)a, 25(1)b to 25(2M - 1)a, 25(2M - 1)b) are sequentially read out to the second horizontal output circuit 45, and are serially output to the outside as output signals (digital signals) of the pair of photoelectric conversion units 33, 34 for focus detection via the second horizontal output circuit 45.

[0175] The first line memory 48 operates according to the scanning signal TS1 given from the first column scanning circuit 52. The H-bit added digital signals stored in each of the memories (28(1) to 28(2M - 1)) are sequentially read out to the first horizontal output circuit 49, and are serially output to the outside as output signals (digital signals) equivalent to the output signals of the imaging pixels via the first horizontal output circuit 49.

[0176] <Eighth Embodiment> The eighth embodiment is a modification of the configuration of the focus detection pixels in the pixel array unit in the seventh embodiment. FIG. 39 is a diagram corresponding to the pixel layout diagram of FIG. 36 (the filter array corresponds to FIG. 37), and the focus detection pixel 411 arranged in the even rows of FIG. 36 is replaced with a focus detection pixel 412 having a pair of photoelectric conversion units 36 and 37 juxtaposed in the vertical direction.

[0177] That is, in the odd rows, the focus detection pixels 411 are arranged in the odd columns, and in the even rows, the focus detection pixels 412 are arranged in the even columns.

[0178] FIG. 40 is a block diagram showing the configuration of the imaging device 212 having the pixel layout shown in FIG. 39. The parts identical to those in FIG. 38 are not described, and only the characteristic parts will be described. The difference from FIG. 38 in the pixel array unit 40 is that in the even columns of the even rows, the focus detection pixels 412 including a pair of vertically separated photoelectric conversion units 36 and 37 are arranged.

[0179] The pair of photoelectric conversion units 36 and 37 of each focus detection pixel 412 arranged in the even columns (2m + 2 columns) of the even rows are connected to the row scanning circuit 41 by the same row control line 21, and charge accumulation control and signal readout control are simultaneously performed according to the control signal R(L) (L is an even number). One of the pair of photoelectric conversion units 36 of each focus detection pixel 412 is connected to one of the two column signal lines 22(2m + 1)a provided in the odd columns (2m + 1 columns), and the output signal (analog signal) of the photoelectric conversion unit 36 is output to the column signal line 22(2m + 1)a. The other photoelectric conversion unit 37 of the pair of photoelectric conversion units 36 and 37 of each focus detection pixel 412 is connected to the other column signal line 22(2m + 1)b of the two column signal lines provided in the odd columns (2m + 1 columns), and the output signal (analog signal) of the photoelectric conversion unit 37 is output to the column signal line 22(2m + 1)b.

[0180] When the imaging device 212 configured as described above is used, it becomes possible to perform phase difference detection on a subject image with a contrast change in the horizontal direction using a pair of data obtained by grouping the data of the pair of photoelectric conversion units 33 and 34 of the focus detection pixel 411 having a green filter arranged in the odd-numbered columns of the odd-numbered rows for each photoelectric conversion unit in the horizontal direction. Also, it becomes possible to perform phase difference detection on a subject image with a contrast change in the vertical direction using a pair of data of the same color obtained by grouping the data of the pair of photoelectric conversion units 36 and 37 of the focus detection pixel 412 having a red filter and a blue filter arranged in the even-numbered columns of the even-numbered rows for each photoelectric conversion unit in the vertical direction.

[0181] <Other Embodiments> In the present invention, the number of photoelectric conversion units in the focus detection pixel is not limited to two, and the present invention can also be applied to a configuration in which the focus detection pixel includes two or more photoelectric conversion units. For example, the focus detection pixel 311 shown in FIG. 6 includes two photoelectric conversion units 13 and 14 obtained by dividing a square into two equal parts in the horizontal direction. However, the present invention can also be applied to a configuration including a focus detection pixel having four photoelectric conversion units obtained by further dividing the two photoelectric conversion units 13 and 14 into two equal parts in the vertical direction. For example, in order to independently read out the analog signals of the four photoelectric conversion units, four column signal lines are provided in each column, and a column AD conversion device including an ADC that outputs a digital signal obtained by individually AD-converting the analog signals output from the four photoelectric conversion units, and a column digital addition device including a digital addition circuit that digitally adds the digital signals of the four photoelectric conversion units output from the column AD conversion device are provided, whereby an effect similar to the configuration shown in FIG. 10 can be obtained.

[0182] Instead of providing four column signal lines corresponding to the four photoelectric conversion units, by regarding two of the four photoelectric conversion units and the other two photoelectric conversion units as the photoelectric conversion units of the focus detection pixels in two virtual adjacent rows, it is also possible to reduce the number of column signal lines to two. For example, when two photoelectric conversion units are arranged in two stages for each of the four photoelectric conversion units in each focus detection pixel 311, the two photoelectric conversion units in the upper stage and the two photoelectric conversion units in the lower stage are regarded as the photoelectric conversion units of the focus detection pixels in two virtual adjacent rows, and the number of column signal lines is reduced to two.

[0183] In the image pickup device of the above-described embodiment, an example is shown in which one data output channel for focus detection and one data output channel for an image are provided for the entire image array unit. However, in order to increase the readout speed, the image array unit may be divided into a plurality of regions, and one data output channel for focus detection and one data output channel for an image may be provided for each region.

[0184] In the image pickup device 212 of the above-described embodiment, based on the digital signal obtained by the column AD conversion device 42 converting a pair of analog signals, the CPUa222 of the body drive control device 214 detects the focus state of the interchangeable lens 202 (optical system). However, the digital signal obtained by the column AD conversion device 42 converting a pair of analog signals may be used as a signal for a 3D camera.

[0185] The present invention is applicable not only to an image pickup device in which a wiring layer exists between a microlens and a photoelectric conversion unit as shown in FIG. 7, but also to a back-illuminated type image pickup device in which no wiring layer exists between the microlens and the photoelectric conversion unit and the wiring layer is arranged on the side opposite to the direction of the microlens with respect to the photoelectric conversion unit. In an image pickup device that requires column signal lines 22 like the image pickup device according to the present invention, more wiring layers are provided than in a conventional image pickup device. In a back-illuminated type image pickup device, the wiring layer can be arranged without being limited by the layout of the photoelectric conversion unit, so the flexibility with respect to an increase in the number of column signal lines is improved.

[0186] In the imaging device 212 in the above-described embodiment, an example in which the imaging pixels include color filters in a Bayer array is shown. However, the configuration and arrangement of the color filters are not limited to this, and the present invention can also be applied to arrangements of complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan: Cy) and arrangements other than the Bayer array. The present invention can also be applied to monochrome imaging devices.

[0187] The imaging device 212 in the above-described embodiment has a pixel array section 40 and other parts. Although the pixel array section 40 and other parts are provided on separate substrates and these separate substrates are laminated on each other, the pixel array section 40 and other parts may be provided on the same substrate.

[0188] Note that the imaging device is not limited to a digital still camera configured such that an interchangeable lens is attached to the camera body as described above. For example, the present invention can also be applied to a lens-integrated digital still camera or a video camera. Furthermore, the present invention can also be applied to a small camera module incorporated in a mobile phone or the like, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

Explanation of Reference Numerals

[0189] 10 Microlenses, 11, 13, 14, 16, 17, 33, 34, 36, 37 Photoelectric conversion units, 15, 18 Element separation regions, 21 Row control lines, 22 Column signal lines, 23 ADC (Analog-to-Digital conversion circuit), 24, 27 Switches, 25, 28 Memories, 26 Digital addition circuit, 29 Semiconductor substrate, 30 Light-shielding mask, 31, 32 Planarization layers, 38 Color filter, 40 Pixel array section, 41 Row scanning circuit, 42 Column AD conversion device, 43 Second column switch device, 44 Second line memory, 45 Second horizontal output circuit, 46 Column digital addition circuit, 47 First column switch device, 48 First line memory, 49 First horizontal output circuit, 50 Timing control circuit, 51 Second column scanning circuit, 52 First column scanning circuit, 71 Imaging light beam, 73, 74 Focus detection light beam, 90 Exit pupil, 91 Optical axis, 93, 94 Rangefinding pupil, 95 Region, 100 Imaging screen, 101 Focus detection area, 201 Digital still camera, 202 Interchangeable lens, 203 Camera body, 204 Mounting section, 206 Lens drive control device, 208 Zooming lens, 209 Lens, 210 Focusing lens, 211 Diaphragm, 212 Imaging element, 213 Electrical contact, 214 Body drive control device, 215 Liquid crystal display element drive circuit, 216 Liquid crystal display element, 217 Eyepiece lens, 219 Memory card, 220 Imaging element control section, 221 Buffer memory, 222 CPUa, 223 CPUb, 310 Imaging pixel, 311, 312, 411, 412 Focus detection pixel

Claims

[Claim 1] a first photoelectric conversion unit that converts light transmitted through the first microlens into an electric charge; a second photoelectric conversion unit that converts light transmitted through the first microlens into an electric charge and is disposed adjacent to the first photoelectric conversion unit in the first direction; a third photoelectric conversion unit that converts the light transmitted through the second microlens into an electric charge; a fourth photoelectric conversion unit that converts light transmitted through the second microlens into an electric charge, the fourth photoelectric conversion unit being disposed adjacent to the third photoelectric conversion unit in a second direction intersecting the first direction; a control unit that outputs a first sum signal obtained by performing addition processing using a first signal based on the charges converted by the first photoelectric conversion unit and a second signal based on the charges converted by the second photoelectric conversion unit, and a second sum signal obtained by performing addition processing using a third signal based on the charges converted by the third photoelectric conversion unit and a fourth signal based on the charges converted by the fourth photoelectric conversion unit; An imaging element comprising:

Images