Image sensor and imaging device

Abstract

To achieve both high-simultaneity signal acquisition from the image sensor and high-speed readout. [Solution] The image sensor has a pixel array in which a plurality of pixels are arranged, and a first memory for each of the signals output from the pixel array. Each of the plurality of pixels includes a plurality of photoelectric conversion units that photoelectrically convert light that has passed through different pupil regions of the imaging optical system to output an analog signal, analog-to-digital conversion means that converts the analog signals output from the plurality of photoelectric conversion units into digital signals, and latch means that latches the digital signals converted by the analog-to-digital conversion means, and the first memory holds the digital signals latched by the latch means.

Description

[Technical Field]

[0001] The present invention relates to an image sensor having a two-dimensional arrangement of pixel sections, each having a plurality of photoelectric conversion units, and an imaging device equipped with the image sensor. [Background technology]

[0002] One of the focus detection methods used in imaging devices is the so-called image plane phase difference method, which uses focus detection pixels formed on the image sensor to acquire a pair of pupil division signals and perform phase difference focus detection. As an example of such image plane phase difference focus detection (hereinafter referred to as "image plane phase difference AF"), Patent Document 1 discloses an imaging device using an image sensor in which pixels are arranged in two dimensions, with each pixel having multiple photoelectric conversion units formed on one microlens (ML). The multiple photoelectric conversion units are configured to receive light transmitted through different regions of the exit pupil of the imaging lens via one ML, thereby performing pupil division. Image plane phase difference AF can then be performed by calculating the amount of image shift from the phase difference signals, which are the signals of each photoelectric conversion unit. Alternatively, an image can be acquired from an imaging signal obtained by summing the signals of the individual photoelectric conversion units for each pixel.

[0003] In such image sensors, where multiple photoelectric conversion units are arranged horizontally within a pixel and pupil division is horizontal, the focus detection accuracy may decrease when the subject is horizontal stripes or otherwise exhibits minimal horizontal parallax. Patent Document 2 discloses a technique for improving focus detection accuracy by using two different arrangement directions for the photoelectric conversion units of the focus detection pixel and two different directions for pupil division.

[0004] In addition, among the electronic shutter methods in an image sensor, the global electronic shutter (GS) method can perform exposure simultaneously for all pixels, and it is a method that enables shooting without causing motion distortion that occurs in the rolling shutter method that performs sequential exposure for each row. Patent Document 3 discloses a configuration that realizes a global shutter by arranging a light receiving part and an ADC (Analog-to-Digital Converter) in a pixel and performing AD conversion for all pixels simultaneously. Also, in this method, since the signal read out from the pixel is a digital signal, high-speed signal readout is possible compared to an image sensor that reads out an analog signal from a pixel.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0006] In Patent Document 3, a configuration including phase difference detection pixels is disclosed in an image sensor having high simultaneous performance due to the all-pixel simultaneous exposure function and high-speed readout performance due to digital signal readout, but there is no disclosure regarding the appropriate arrangement of vertical split pixels and horizontal split pixels. Therefore, in vertical phase difference detection and horizontal phase difference detection, there is a possibility that a large bias in detection performance or a difference in the area where focus detection is possible may occur.

[0007] The present invention has been made in view of the above problems, and an object thereof is to achieve both acquisition of a signal with high simultaneity from an image sensor and high-speed readout.

Means for Solving the Problems

[0008] To achieve the above objective, the image sensor of the present invention includes a pixel array in which a plurality of pixels are arranged, and a first memory for each of the signals output from the pixel array, wherein each of the plurality of pixels includes a plurality of photoelectric conversion units that photoelectrically convert light that has passed through different pupil regions of an imaging optical system to output an analog signal, analog-to-digital conversion means that converts the analog signals output from the plurality of photoelectric conversion units into digital signals, and latch means that latches the digital signals converted by the analog-to-digital conversion means, and the first memory holds the digital signals latched by the latch means. [Effects of the Invention]

[0009] According to the present invention, it is possible to achieve both high-simultaneity signal acquisition from the image sensor and high-speed readout. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram schematically showing the overall configuration of the image sensor according to Embodiment 1. [Figure 2] A diagram schematically showing the circuit configuration of a pixel according to Embodiment 1. [Figure 3] A timing diagram schematically showing the operation timing according to Embodiment 1. [Figure 4] A diagram showing the basic layout of the elements constituting the pixel portion on the PD substrate side according to Embodiment 1. [Figure 5] A schematic diagram showing the basic cross-sectional structure of a partial pixel on the PD substrate side according to Embodiment 1. [Figure 6] A schematic diagram showing xy cross-sections of the PDA and PDB at different depths according to Embodiment 1. [Figure 7] A schematic diagram showing the cross-section along D-D' in Figure 4 according to Embodiment 1. [Figure 8] A diagram schematically showing the horizontally divided pixels according to Embodiment 1. [Figure 9] A diagram schematically showing the vertically divided pixels according to Embodiment 1. [Figure 10]A diagram schematically showing the arrangement of horizontally divided pixels and vertically divided pixels and the arrangement of a color filter in a 2x2 pixel area according to Embodiment 1. [Figure 11] A block diagram showing the schematic configuration of the imaging device according to Embodiment 1. [Figure 12] A diagram illustrating the correspondence between pixels of an image sensor according to Embodiment 1 and pupil intensity distribution. [Figure 13] A diagram schematically showing the pupil intensity distribution according to Embodiment 1. [Figure 14] A diagram illustrating the correspondence between the image sensor and pupil intensity distribution according to Embodiment 1. [Figure 15] A diagram illustrating the imaging optical system and pupil division in the image sensor according to Embodiment 1. [Figure 16] A diagram schematically showing the overall configuration of the image sensor according to Embodiment 2. [Figure 17] A diagram schematically showing the live view signal readout operation according to Embodiment 2. [Figure 18] A diagram schematically showing the image signal readout operation according to Embodiment 2. [Figure 19] A diagram schematically showing the overall configuration of the image sensor according to Embodiment 2. [Figure 20] A diagram schematically showing the imaging operation and live view operation according to Embodiment 2. [Figure 21] A schematic diagram showing the arrangement of color filters in a modified example within a 2x2 pixel area. [Modes for carrying out the invention]

[0011] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0012] <Embodiment 1> [Image sensor configuration] Figure 1 is a schematic diagram showing the overall configuration of the image sensor 100 according to Embodiment 1 of the present invention.

[0013] The image sensor 100 is constructed by stacking a substrate 101 (hereinafter referred to as the "PD substrate") located on the light incident side and a substrate 102 (hereinafter referred to as the "TAD substrate") located on the opposite side of the light incident side. The PD substrate 101 contains the PD substrate side portion 111 of the pixel array, a vertical scanning circuit 115, and a lamp circuit 116, while the TAD substrate 102 contains the TAD substrate side portion 112 of the pixel array. Multiple pixels are arranged in a matrix in the pixel array, and each pixel consists of a pixel portion 113 configured on the PD substrate 101 and a pixel portion 114 configured on the TAD substrate 102. In Figure 1, the vertical scanning circuit 115 and the lamp circuit 116 are located to the left of the PD substrate side portion 111 of the pixel array, but this is not the only option; they can be placed in any available area of ​​the PD substrate 101 excluding the PD substrate side portion 111.

[0014] Furthermore, the TAD board 102 is equipped with an N-signal frame memory 121, a subtraction circuit 131, an A / A+B-signal frame memory 122, and an output circuit 141. The number of bits in each memory of the N-signal frame memory 121 is smaller than the number of bits in each memory 122 of the A / A+B-signal frame memory. However, the number of bits in each memory of the N-signal frame memory 121 and the number of bits in each memory of the A / A+B-signal frame memory 122 may be the same. In addition, the N-signal frame memory 121 and the A / A+B-signal frame memory 122 each have memory equal to the number of pixels in the pixel array.

[0015] [Pixel circuit] Figure 2 is a schematic diagram showing the circuit configuration of one pixel in this embodiment, and shows the pixel portion 113 configured on the PD substrate 101 and the pixel portion 114 configured on the TAD substrate 102 side by side.

[0016] The pixel portion 113 consists of a photoelectric conversion unit PDA201, PDB202, a charge-voltage conversion unit (FD)208 that converts signal charge to voltage, a transfer transistor (TXA)203 for transferring signal charge accumulated in PDA201 to FD208, a transfer transistor (TXB)204 for transferring signal charge accumulated in PDB202 to FD208, an overflow gate (OFGA)205 for draining charge from the PDA and resetting it, an overflow gate (OFGB)206 for draining charge from the PDB and resetting it, a reset transistor (RES)207 for draining signal charge from the FD208 and resetting it, a current source transistor 211 for the differential amplifier, a differential amplifier input transistor 212 on the FD208 side, and a differential amplifier input transistor 213 on the ramp signal side, which will be described later.

[0017] The on / off states of TXA203, TXB204, OFGA205, OFGB206, and RES207 are controlled by the vertical scanning circuit 115, and the gate of the differential amplifier input transistor 213 receives a ramp signal V from the ramp circuit 116. RAMP It will be supplied.

[0018] The pixel portion 114 consists of transistors 214 and 215, a common-source transistor 216, a voltage-limiting transistor 221, a positive feedback circuit 230, and latch circuits 241 corresponding to the number of bits, which are active loads for a differential amplifier. The positive feedback circuit 230 consists of transistors 231, 232, 233, 234, and 235. The latch circuit 241 latches a count signal corresponding to the time when the positive feedback circuit 230 inverts. This count signal latched by the latch circuit 241 is a digital signal corresponding to the voltage (analog signal) corresponding to the charge input from the pixel portion 113, thereby performing analog-to-digital (AD) conversion. This operation will be described later with reference to Figure 3.

[0019] TXA203, TXB204, OFGA205, OFGB206, RES207, transistors 211, 212, 213, 221, 231, and 234 are NMOS transistors, while transistors 214, 215, 216, 232, 233, and 235 are PMOS transistors. The connections between the transistors are shown in Figure 2.

[0020] The wiring 251 connecting the drain of transistor 212 and the drain of transistor 214, and the wiring 252 connecting the drain of transistor 213 and the drain of transistor 215, span both the PD board 101 and the TAD board 102, and include inter-board connections such as hybrid bonding. PDA201, PDB202 to transistor 221 are relatively high-voltage transistors, while transistors 231, 232, 233, 234, 235 and the transistors constituting the latch circuit 241 are relatively low-voltage transistors. Furthermore, the power supply voltage supplied to the low-voltage transistors is lower than the power supply voltage supplied to the high-voltage transistors. By using a low power supply voltage, power consumption can be kept low, and by using low-voltage transistors, it is possible to use smaller transistors and reduce the pixel pitch.

[0021] [Operation timing] Figure 3 is an operation timing diagram of this embodiment. The operation of this embodiment will be explained using Figure 3. In this specification, each timing is referred to as "t 301 It is indicated as follows. Also, φOFGA indicates the on / off state of OFGA205, with the top side indicating on and the bottom side indicating off. The other switches are treated similarly. V FDIt shows an example of the voltage change of FD208, and the same applies to other nodes. In this embodiment, the reset of the photoelectric conversion unit, N signal conversion, N+A signal conversion, and N+A+B signal conversion described later are performed simultaneously for all pixels. Therefore, a global shutter operation is achieved where the accumulation time is equal for all pixels.

[0022] ·Reset of the photoelectric conversion unit Before starting the accumulation operation, turn on OFGA205 and OFGB206 (t 301 ), turn off (t 302 ), discharge the charges of PDA201 and PDB202, and start the accumulation.

[0023] ·AD conversion of the N signal After that, turn on RES207 (t 303 ), turn off (t 304 ) to reset the voltage of FD208 to a high voltage, and initialize the positive feedback circuit 2,30 by turning on and off φINI. When the positive feedback circuit 230 is initialized, the positive feedback circuit output V 262 is set to a high voltage.

[0024] After that, lower the voltage of the ramp signal V RAMP input to the differential amplifier input transistor 213 on the ramp signal side over time, and input a count signal to the latch circuit 241. During the ramp signal input period (t 305 ~t 307 ), when V RAMP is lower than V FD , the differential amplifier output reverses from a high voltage to a low voltage. In response to the reversal of the differential amplifier output, the output of the source-grounded transistor 216 reverses from a low voltage to a high voltage, but at this time, the reversal speed of the output V 261 from the source-grounded transistor 216 is faster than the reversal speed of the differential amplifier output. Among the reversals of the output V 261 of the source-grounded transistor 216, the low voltage region is input to the positive feedback circuit 230 via the voltage limiting transistor 221, but voltages above a certain level are blocked by the voltage limiting transistor 221 and thus not input to the positive feedback circuit 230.

[0025] The positive feedback circuit 230 responds to the increase in the output voltage of the common-source transistor 216, and the output of the positive feedback circuit 230 V 262 This inverts the voltage from high to low. Of the positive feedback circuit 230, transistors 234 and 235 form an inverter (INV), and the output of the common-source transistor 216 V 261 The voltage reverses as it rises, but the output of the positive feedback circuit 230, which is also the output of INV consisting of transistors 234 and 235, is V 261 Since this is also input to transistor 232, the output V of the positive feedback circuit 230 261 As the voltage decreases, transistor 232 turns on and the output of common-source transistor 216 V 261 This increases the output V of the positive feedback circuit 230. 261 The inversion time is equal to the output V of the common-source transistor 216. 261 Faster than the inversion time. Output V of this positive feedback circuit 230 261 Upon inversion, the latch circuit 241 holds the count signal at that point in time. Also, in the positive feedback circuit 230, when φINI is at a low voltage, transistor 231 is off and transistor 233 is on. Therefore, when φINI is at a low voltage, the output of the positive feedback circuit 230 V 262 After the voltage drops to a low level, the output of the positive feedback circuit 230 V 262 In response, transistor 232 turns on, and the output of the positive feedback circuit 230 V continues until φINI turns on again. 262 The voltage remains low.

[0026] In this way, V after reset FD A count signal (hereinafter referred to as the "N signal"), which is a noise signal corresponding to the above, is obtained. Also, a ramp signal V RAMP The time (t) from when the decrease begins until the decrease is complete. 305 ~t 307 This is called the N-signal AD conversion time.

[0027] After the AD conversion of the N signal, the N signals held in the latch circuit 241 of each pixel in the pixel array are sequentially read out into the N signal frame memory 121 using a digital readout circuit (not shown). 307 ~t 308 ), hold.

[0028] • AD conversion of N+A signals Next, turn the TXA203 on and off (t 309 ) and transfer the charge stored in PDA201 to FD208. The voltage of FD208 is V FD The voltage drops by an amount corresponding to the amount of charge stored in the PDA201.

[0029] After turning φINI on and off, the ramp signal V is used, similar to the AD conversion of the N signal. RAMP The voltage is dropped over time, and a count signal is input to the latch circuit 241 (t 310 ~t 312 ). Similar to when converting N signals to AD, V RAMP ga V FD time below (t 311 The count signal corresponding to ) is held in the latch circuit 241. The signal obtained in this way is called the "N+A signal". Also, the ramp signal V RAMP The time (t) from when the decrease begins until the decrease is complete. 310 ~t 312 This is called the AD conversion time for an N+A signal.

[0030] After the AD conversion of the N+A signal, the N+A signals held in the latch circuit 241 of each pixel in the pixel array are sequentially read out to the subtraction circuit 131 using a digital readout circuit (not shown). 312 ~t 313 ). Simultaneously, the N signals of the corresponding pixels are sequentially read from the N signal frame memory 121 to the subtraction circuit 131. The subtraction circuit 131 calculates the difference between the input N signal and the N+A signal (hereinafter referred to as the "A signal"), sequentially outputs it to the A / A+B signal frame memory 122, and holds the A signals in the A / A+B signal frame memory 122.

[0031] • AD conversion of N+A+B signals Next, turn TXB204 on and off (t 314 ) and transfer the charge accumulated in PDB202 to FD208. The voltage of FD208 is V FD The voltage drops by an amount corresponding to the amount of charge stored in PDB202.

[0032] After turning φINI on and off, the ramp signal V is used during AD conversion of the N signal, just as it is during AD conversion of the N+A signal. RAMP The voltage is dropped over time, and a count signal is input to the latch circuit 241 (t 315 ~t 317 ). When converting the N signal to AD, V RAMP ga V FD time below (t 316 The count signal corresponding to ) is held in the latch circuit 241. The signal obtained in this way is called the N+A+B signal. Also, the ramp signal V RAMP The time (t) from when the decrease begins until the decrease is complete. 315 ~t 317 This is called the AD conversion time for the N+A+B signal.

[0033] Furthermore, simultaneously with the AD conversion of the N+A+B signal, the A signal held in the A / A+B signal frame memory 122 is output to the outside of the chip via the output circuit 141. 315 ~t 318 In this embodiment, the start of signal readout to the outside of the chip (t 315 ) was performed simultaneously with the start of AD conversion of the N+A+B signal, but it is sufficient if the signal readout from the pixel array of the N+A signal is completed, for example, t 313 You can start from there.

[0034] The output of signal A to the outside of the chip is complete (t 318After this, the N+A+B signals held in the latch circuit 241 of each pixel in the pixel array are sequentially read out to the subtraction circuit 131 using a digital readout circuit (not shown) (t318~t319). Simultaneously, the N signals of the corresponding pixels are sequentially read out from the N signal frame memory 121 to the subtraction circuit 131. The subtraction circuit 131 calculates the difference between the input N signal and the N+A+B signal (hereinafter referred to as the "A+B signal"), sequentially outputs it to the A / A+B signal frame memory 122, and holds the A+B signals in the A / A+B signal frame memory 122.

[0035] Readout of N+A+B signals from the pixel array is complete (t 319 ) Afterwards, the A+B signal held in the A / A+B signal frame memory 122 is output to the outside of the chip via the output circuit 141 (t 319 ~t 320 ).

[0036] [Structure of the light-receiving part] • Basic structure of the light-receiving section Next, with reference to Figures 4 to 7, the basic configuration of the pixel portion 113 on the PD substrate 101 side in this embodiment will be described.

[0037] Figure 4 is a schematic diagram showing the basic layout of the elements constituting the pixel portion 113 on the PD substrate 101 side according to this embodiment. In Figure 4, the horizontal direction of the paper is the x-direction, the downward direction of the paper is the y-direction, and the direction toward the back relative to the paper is the z-direction. In this embodiment, "planar view" refers to viewing from the z-direction or -z-direction with respect to a plane (xy-plane) that is roughly parallel to the side of the semiconductor substrate where the transistor gates are located. In this embodiment, the "horizontal" direction refers to the x-direction, the "vertical" direction refers to the y-direction, and the "depth" direction refers to the z-direction.

[0038] In Figure 4, the same reference numerals are used for components identical to those in Figure 2, and detailed explanations are omitted. In Figure 4, 421 is a microlens (ML), 403 is the gate electrode of TXA203, 404 is the gate electrode of TXB204, 405 is the gate electrode of OFGA205, 406 is the gate electrode of OFGB206, 407 is the gate electrode of RES207, 411 is the gate electrode of transistor 211, 412 is the gate electrode of transistor 212, and 413 is the gate electrode of transistor 213.

[0039] PDA201 includes a storage region 431, a sensitivity region 433, and an N-type connection region 435, while PDB202 includes a storage region 432, a sensitivity region 434, and an N-type connection region 436. These storage regions 431 and 432, sensitivity regions 433 and 434, and N-type connection regions 435 and 436 are composed of N-type semiconductors. Sensitivity regions 433 and 434 have a larger area than storage regions 431 and 432. As will be described in detail later with reference to Figure 5, storage regions 431 and 432 are configured at a first depth, while sensitivity regions 433 and 434 are configured at a second depth different from the first depth. Also, for the sake of clarity, the region where charge is generated in response to incident light is mainly referred to as the "sensitivity region," and the region where the generated charge is mainly stored is mainly referred to as the "storage region," but there is no clear distinction between the charge generation region and the charge storage region. In the storage regions 431 and 432, charge is generated in response to the incoming light, and in the sensitivity regions 433 and 434, a portion of the generated charge remains.

[0040] Figure 5 is a schematic diagram showing the basic cross-sectional structure of the pixel portion 113 on the PD substrate 101 side. Figure 5(a) is a schematic diagram of the A-A' cross section in Figure 4, Figure 5(b) is a schematic diagram of the B-B' cross section in Figure 4, and Figure 5(c) is a schematic diagram of the C-C' cross section in Figure 4. The PD substrate 101 has a first surface and a second surface opposite the first surface. The first surface is the front surface of the PD substrate 101, and the second surface is the back surface of the PD substrate 101. The direction from the first surface to the second surface is defined as the positive Z direction. The first surface (front side) of the PD substrate 101 is equipped with transistor gate electrodes, multilayer wiring structures, etc. The second surface (back side) of the PD substrate 101 is equipped with optical structures such as a color filter 501 and ML421 that cover the two photodiodes of each pixel, and light is incident from the second surface (back side).

[0041] As shown in Figure 5(a), the PD substrate 101 has a P-type semiconductor region 500, and storage regions 431 and 432 and sensitivity regions 433 and 434 surrounded by the P-type semiconductor region 500. The storage region 431 and sensitivity region 433, and the storage region 432 and sensitivity region 434 each have different shapes in plan view, and some overlap in plan view. Also, as described above, the storage regions 431 and 432 and the sensitivity regions 433 and 434 are located at different depths, with the storage regions 431 and 432 on the first side (first depth) and the sensitivity regions 433 and 434 on the second side (second depth). Within the P-type semiconductor region 500, the storage isolation region 502 separates the storage region 431 and the storage region 432, and the sensitivity isolation region 503 separates the sensitivity region 433 and the sensitivity region 434.

[0042] As shown in Figure 5(b), the storage region 431 and the sensitivity region 433 are connected in the depth direction via an N-type connection region 435. Also, as shown in Figure 5(c), the storage region 432 and the sensitivity region 434 are connected in the depth direction via an N-type connection region 436.

[0043] Furthermore, in Figure 5(b), region 504 has a shape that is recessed in the Z direction by the P-type semiconductor in the storage region 431. This recessed region 504 suppresses the generation of charge accumulation in the region on the first surface side of the storage region 431 that overlaps with the N-type connection region 435 in a plan view. This suppresses the generation of residual signal charge when transferring the signal charge stored in the storage region 431 in the PDA 201 to the FD 208. Note that if the generation of residual signal charge can be suppressed, other methods such as lowering the impurity concentration in a part of the storage region 431 may be used, rather than being limited to the recessed region 504.

[0044] Furthermore, as shown in Figure 5(c), the length of the storage regions 431 and 432 in the Z direction is shorter than the length of the storage regions 431 and 432 in the Z direction in the cross-sections shown in Figures 5(a) and 5(b), and the shortened region 505 is formed of a P-type semiconductor. This suppresses the generation of residual signal charge when transferring the signal charge stored in the storage regions 431 and 432 to the FD208.

[0045] Figure 6 schematically shows xy cross-sections of PDA201 and PDB202 at different depths in the z direction. Figure 6(a) is the E-E' cross-section of Figure 5, Figure 6(b) is the F-F' cross-section, Figure 6(c) is the G-G' cross-section of Figure 5, and Figure 6(d) is the H-H' cross-section of Figure 5. As shown in Figure 6(d), in the partial regions of storage areas 431 and 432 located away from gate electrodes 403, 404, 405, and 406, the storage areas 431 and 432 disappear and are replaced by the notched region 505 of the P-type semiconductor, as explained with reference to Figure 5(c).

[0046] Figure 7 is a schematic diagram showing the cross-section indicated by D-D' in Figure 4. The storage regions 431 and 432, sensitivity regions 433 and 434, and N-type connection regions 435 and 436 are shown unfolded along the D-D' line in Figure 4 in the xy plane so that they are all included in the same figure. During the storage period, when light is incident on the second surface of the PD substrate 101 via ML421, electrons (signal charge) are generated mainly in the sensitivity regions 433 and 434 by photoelectric conversion. Most of the signal charge generated in the sensitivity region 433 moves to the storage region 431 via the N-type connection region 435 and is stored there. Also, most of the signal charge generated in the sensitivity region 434 moves to the storage region 432 via the N-type connection region 436 and is stored there. In order to realize the transfer of signal charge from the sensitivity region to the storage region, it is desirable that the potential experienced by electrons decreases monotonically along the charge transfer path from the sensitivity region to the storage region.

[0047] • Horizontally divided pixels and vertically divided pixels Since the storage region and the sensitivity region are located at different depths, the direction in which the sensitivity regions of the PDA201 and PDB202 extend can be made different while keeping the arrangement of transistors in the pixel portion 113 on the PD substrate 101 the same. Alternatively, the direction in which the sensitivity regions of the PDA201 and PDB202 extend can be made different by changing the arrangement of transistors in the pixel portion 113 on the PD substrate 101.

[0048] Figure 8 shows a pixel 800 in the pixel portion 113 on the PD substrate 101 side, in which the sensitivity region is divided horizontally (hereinafter referred to as the "horizontal division arrangement"), and corresponds to one of the R pixels, B pixels, and G pixels of the Bayer array described later. Figure 8(a) is an exploded perspective view of the storage regions 431 and 432, the sensitivity regions 433 and 434, the N-type connection regions 435 and 436, the gate electrode 403 of TXA203, the gate electrode 404 of TXB204, the gate electrode 405 of OFGA205, the gate electrode 406 of OFGB206, and FD208 in the lateral split configuration. In the lateral split configuration, the storage regions 431 and 432 and the sensitivity regions 433 and 434 all extend in the y direction, i.e., in the same direction.

[0049] Figure 8(b) is a schematic plan view showing the positional relationship in a plan view of the storage regions 431 and 432, sensitivity regions 433 and 434, N-type connection regions 435 and 436, gate electrode 403 of TXA203, gate electrode 404 of TXB204, gate electrode 405 of OFGA205, gate electrode 406 of OFGB206, and FD208 in a horizontal split configuration. In the horizontal split configuration, since the sensitivity regions 433 and 434, where charge is generated by photoelectric conversion, are aligned in the x-direction, a phase difference signal with pupil splitting in the x-direction can be acquired. 801 indicates the splitting direction of the phase difference signal.

[0050] Figure 8(c) is a schematic plan view showing the positional relationship between the accumulation separation region 502 and the sensitivity separation region 503 in a horizontally divided configuration. In the horizontally divided configuration, both the accumulation separation region 502 and the sensitivity separation region 503 extend in the y direction.

[0051] Figure 9 shows a pixel 900 in the pixel portion 113 on the PD substrate 101 side, in which the sensitivity region is divided vertically (hereinafter referred to as the "vertical division arrangement"), and corresponds to one of the G pixels of the Bayer array described later. Figure 9(a) is an exploded perspective view of the storage regions 431 and 432, the sensitivity regions 433 and 434, the N-type connection regions 435 and 436, the gate electrode 403 of TXA203, the gate electrode 404 of TXB204, the gate electrode 405 of OFGA205, the gate electrode 406 of OFGB206, and FD208 in the vertical split configuration. In the vertical split configuration, the storage regions 431 and 432 extend in the y direction, and the sensitivity regions 433 and 434 extend in the x direction, extending in orthogonal directions, i.e., in different directions, in a plan view.

[0052] Figure 9(b) is a schematic plan view showing the positional relationship in a plan view of the storage regions 431 and 432, sensitivity regions 433 and 434, N-type connection regions 435 and 436, gate electrode 403 of TXA203, gate electrode 404 of TXB204, gate electrode 405 of OFGA205, gate electrode 406 of OFGB206, and FD208 in a vertical split configuration. In the vertical split configuration, since the sensitivity regions 433 and 434, where charge is generated by photoelectric conversion, are aligned in the y direction, a phase difference signal with pupil splitting in the y direction can be obtained. 901 indicates the separation direction of the phase difference signal.

[0053] Figure 9(c) is a schematic plan view showing the positional relationship between the storage separation region 502 and the sensitivity separation region 503 in a vertically divided configuration. In the vertically divided configuration, the storage separation region 502 extends in the y direction, and the sensitivity separation region 503 extends in the x direction.

[0054] [Arrangement of horizontally divided pixels, vertically divided pixels, and color filters] Figure 10 is a schematic diagram showing the sensitivity regions and the arrangement of the color filter 501 within a 2x2 pixel area for horizontally divided pixels 800 (hereinafter referred to as "horizontally divided pixels") and vertically divided pixels 900 (hereinafter referred to as "vertically divided pixels") in this embodiment.

[0055] The image consists of 800 horizontally divided pixels with a color filter having R (red) spectral sensitivity in the upper left, 800 horizontally divided pixels with a color filter having G (green) spectral sensitivity in the upper right, 900 vertically divided pixels with a color filter having G (green) spectral sensitivity in the lower left, and 800 horizontally divided pixels with B (blue) spectral sensitivity in the lower right. The color filters are arranged in a Bayer array.

[0056] By applying the 2x2 arrangement (unit) shown in Figure 10 to the entire pixel array, it is possible to acquire phase difference signals with the pupil division direction in the horizontal direction and phase difference signals with the pupil division direction in the vertical direction across the entire area of ​​the pixel array.

[0057] Furthermore, since horizontal phase difference signals are acquired for each of the R, G, and B color filters, horizontal phase difference signals can be acquired regardless of the color of the subject. In addition, since the vertical phase difference signals are acquired at the G pixel, which has the highest transmittance among the R, G, and B color filters, the accuracy of the obtained phase difference signals is improved compared to when the vertical division pixels are R pixels or B pixels.

[0058] [Overall configuration of the imaging system] Figure 11 is a block diagram showing the schematic configuration of the imaging device according to this embodiment. The imaging device of this embodiment includes an image sensor 100 having the above-described configuration, an overall control and calculation unit 2, an instruction unit 3, a timing generation unit 4, a shooting lens unit 5, a lens drive unit 6, a signal processing unit 7, a display unit 8, and a recording unit 9.

[0059] The imaging lens unit 5 forms an optical image of the subject onto the image sensor 100. Although it is shown as a single lens in the diagram, the imaging lens unit 5 may include multiple lenses, such as a focus lens and a zoom lens, as well as an aperture, and may be detachable from the main body of the imaging device or may be integrally configured with the main body.

[0060] The image sensor 100 has the configuration described in the above embodiment and converts the light incident via the imaging lens unit 5 into an electrical signal and outputs it. Signals are read out from each pixel of the image sensor 100 so that a pupil-splitting signal, which can be used for phase-difference focus detection, and an image signal, which is a signal for each pixel, can be acquired.

[0061] The signal processing unit 7 performs predetermined signal processing, such as correction processing, on the signal output from the image sensor 100, and outputs a pupil division signal used for focus detection and an image signal used for recording.

[0062] The overall control and calculation unit 2 performs the overall drive and control of the entire imaging device. It also performs calculations for focus detection using the pupil division signal processed by the signal processing unit 7, and performs predetermined signal processing on the image signal, such as calculation processing for exposure control, and development and compression to generate images for recording and playback.

[0063] The lens drive unit 6 drives the shooting lens unit 5 and performs focus control, zoom control, aperture control, etc., on the shooting lens unit 5 according to the control signals from the overall control and calculation unit 2.

[0064] The instruction unit 3 receives inputs from the outside, such as instructions to execute shooting, settings for the drive mode of the imaging device, and various other settings and selections, via user operations, and transmits them to the overall control and calculation unit 2.

[0065] The timing generation unit 4 generates timing signals to drive the image sensor 100 and the signal processing unit 7 according to the control signals from the overall control and calculation unit 2. The display unit 8 displays information such as preview images, playback images, and the drive mode settings of the imaging device.

[0066] The recording unit 9 is equipped with a recording medium (not shown) on which recording image signals are recorded. Examples of recording media include semiconductor memory such as flash memory. The recording medium may be detachable from the recording unit 9 or may be built into it.

[0067] [Calculation of defocus amount] Next, referring to Figures 12 to 15, the calculation method for calculating the defocus amount from the pupil division signal in the overall control and calculation unit 2 will be explained. Note that the calculation for calculating the defocus amount from the lateral phase difference signal and the calculation for calculating the defocus amount from the vertical phase difference signal are fundamentally the same; therefore, the case of calculation from the lateral phase difference signal will be explained here.

[0068] Figure 12 shows a cross-sectional view in the lateral direction of a horizontally divided pixel 800, where the division direction is horizontal, and the pupil plane of the image sensor 100 at a distance Ds in the negative z-axis direction from the imaging surface 1200.

[0069] Through ML421, the pupil surface and the light-receiving surface (second surface) of the image sensor 100 are in a roughly conjugate relationship. Therefore, the light beam that passes through partial pupil region 1201 is generally received in sensitivity region 433 (PDA201). Similarly, the light beam that passes through partial pupil region 1202 is generally received in sensitivity region 434 (PDB202). The signal charge converted photoelectrically near the boundary between sensitivity region 433 and sensitivity region 434 is probabilistically transported to storage region 431 or storage region 432. Therefore, as the x-coordinate increases, the signal gradually switches at the boundary between partial pupil region 1201 and partial pupil region 1202, and the x-direction dependence of the pupil intensity distribution takes on the shape illustrated in Figure 13. Here, the pupil intensity distribution corresponding to PDA201 is called the first pupil intensity distribution 1301, and the pupil intensity distribution corresponding to PDB202 is called the second pupil intensity distribution 1302.

[0070] Next, with reference to Figure 14, the sensor entrance pupil of the image sensor 100 will be described. In the image sensor 100 of this embodiment, the ML421 of each pixel is continuously shifted toward the center of the image sensor 100 according to the image height coordinate on a two-dimensional plane. That is, each ML421 is arranged to be eccentric toward the center as the image height increases. The center of the image sensor 100 and the optical axis of the imaging optical system change due to the mechanism that reduces the effects of blur caused by camera shake, etc., by driving the imaging optical system or the image sensor 100, but they are approximately the same. As a result, the first pupil intensity distribution 1301 and the second pupil intensity distribution 1302 of the horizontally divided pixels 800 arranged at each image height coordinate of the image sensor 100 are configured to be approximately the same at the pupil plane at a distance Ds from the image sensor 100.

[0071] Hereinafter, the first pupil intensity distribution 1301 and the second pupil intensity distribution 1302 will be referred to as the "sensor entrance pupil" of the image sensor 100, and the distance Ds will be referred to as the "entrance pupil distance" of the image sensor 100. It is not necessary for all pixels to have a single entrance pupil distance; for example, the entrance pupil distances of pixels up to 80% of the image height may be made approximately the same, or pixels may be configured to have different entrance pupil distances for each row or detection area.

[0072] Figure 15 shows a schematic diagram of the relationship between the amount of image shift and the amount of defocus between disparity images. The image sensor 100 (not shown) of this embodiment is placed on the imaging surface 1200, and, as in Figure 12, the exit pupil of the imaging optical system is divided into two parts: a partial pupil region 1201 and a partial pupil region 1202.

[0073] The amount of defocus d is defined as the distance from the imaging plane to the subject's imaging position, where |d| is the distance from the subject's imaging position to the imaging plane. A negative value (d<0) indicates a front-focus state where the subject's imaging position is on the subject side of the imaging plane, and a positive value (d>0) indicates a back-focus state where the subject's imaging position is on the opposite side of the imaging plane. The in-focus state where the subject's imaging position is on the imaging plane is d=0. Figure 15 shows an example where the subject on object plane 1501 is in focus (d=0), and the subject on object plane 1502 is front-focused (d<0). The front-focused state (d<0) and the back-focused state (d>0) together constitute a defocused state (|d|>0).

[0074] In the front-focused state (d<0), the light beam from the subject on the object surface 1502 that passes through the partial pupil region 1201 (1202) is focused once, then spreads out with a width Γ1 (Γ2) centered on the centroid position G1 (G2) of the light beam, resulting in a blurred image on the imaging plane 1200. The blurred image is received by the sensitivity region 433 (PDA201) and the sensitivity region 434 (PDB202), and a disparity image is generated. Therefore, the generated disparity image shows the subject on the object surface 1502 blurred with a width Γ1 (Γ2) at the centroid position G1 (G2).

[0075] The blur width Γ1 (Γ2) of the subject image increases roughly proportionally with increasing defocus amount d |d|. Similarly, the amount of image displacement p (=G2-G1) of the subject image between the parallax images |p| also increases roughly proportionally with increasing defocus amount d |d|. The same applies in the back-focused state (d>0), although the direction of image displacement of the subject image between the parallax images is opposite to that of the front-focused state. In the in-focus state (d=0), the centroid positions of the subject images between the parallax images coincide (p=0), and no image displacement occurs.

[0076] Therefore, in the two phase difference signals obtained using the signals in sensitivity region 433 (PDA201) and sensitivity region 434 (PDB202), as the amount of defocus in the disparity image increases, the amount of image shift in the x-direction between the two phase difference signals also increases. Based on this relationship, the amount of image shift calculated by correlation calculation of the x-direction image shift between disparity images is converted into a defocus amount, thereby performing focus detection using the phase difference detection method.

[0077] [Vertical and horizontal pixel divisions in the calculation of defocus amount] In the above calculation of the defocus amount, it is necessary to calculate the amount of image shift, and in order to calculate the amount of image shift, it is necessary to compare two phase difference signals (the signal obtained from PDA201 and the signal obtained from PDB202) at different positions in the pupil division direction. Therefore, in order to calculate the amount of image shift in the vertical direction, it is necessary to compare the phase difference signals between different rows. If the image sensor 100 is driven by a rolling shutter method, for example, instead of a global shutter method, the exposure time will be different for different rows, so the vertical phase difference detection accuracy may be lower than the horizontal phase difference detection accuracy. In contrast, in this embodiment, it is possible to suppress the decrease in vertical phase difference detection accuracy due to the accumulation time difference.

[0078] By configuring the system as described above, it is possible to achieve both high-simultaneity acquisition of vertical / horizontal phase difference signals and high-speed readout in the image sensor.

[0079] <Embodiment 2> Next, Embodiment 2 of the present invention will be described. In Embodiment 1 described above, a configuration in which the image sensor 100 has two frame memories was explained. However, the present invention is not limited to this, and the image sensor may also have a configuration with three frame memories. Embodiment 2 will explain the differences from Embodiment 1, using a configuration of an image sensor having three frame memories as an example.

[0080] [Image sensor configuration] Figure 16 is a schematic diagram showing the overall configuration of the image sensor 1600 according to Embodiment 2 of the present invention. Compared to the image sensor 100, the image sensor 1600 lacks the N signal frame memory 121 and the A / A+B signal frame memory 122, and has a vertical / horizontal (HV) summing circuit 1601, a first live view (LV) frame memory 1602, a second LV frame memory 1603, and an imaging frame memory 1604. The N signal frame memory 121 and A / A+B signal frame memory 122 of the image sensor 100 had memory equal to the number of pixels. The imaging frame memory 1604 of the image sensor 1600 has memory equal to the number of pixels, while the first LV frame memory 1602 and the second LV frame memory 1603 have memory equal to the number of pixels divided by the number added by the HV summing circuit 1601.

[0081] [Live View Operation] Figure 17 schematically shows the LV signal readout flow according to Embodiment 2, and operates in the order of Figures 17(a) to (d). In each figure, the general operation is shown above the overall configuration of the image sensor 1600, the signal flow is indicated by arrows / operating circuit blocks are highlighted and written on the overall configuration of the image sensor 1600, and the types of signals held in the first LV frame memory 1602 and the second LV frame memory 1603 after each operation is completed are shown below the overall configuration of the image sensor 1600. The live view signal readout operation will be explained below using Figure 17. Note that this readout operation may be used not only for live view but also for low-resolution video or still images.

[0082] First, as shown in Figure 17(a), the N signal is converted to AD, and the results are sequentially read from the pixel array, added by the HV adder 1601, and the added result (added noise signal) is output to the first LV frame memory 1602 for storage. Here, the signals for all pixels are read from the pixel array and added by the HV adder 1601, but instead of adding, some may be decimated at a predetermined frequency, or the signals may be read with decimation at a predetermined frequency without using the HV adder 1601, and output to the first LV frame memory 1602 for storage. The first LV frame memory 1602 stores the N added signal obtained by HV adding the N signals.

[0083] Next, as shown in Figure 17(b), the N+A signal is converted to AD, and the results are sequentially read from the pixel array, added by the HV adder 1601, and output to the subtractor 131. At the same time, the corresponding N adder signal is output from the first LV frame memory 1602 to the subtractor 131, and the result of the subtraction is stored in the second LV frame memory 1603. As a result, the second LV frame memory 1603 stores the A adder signal.

[0084] Next, as shown in Figure 17(c), the N+A+B signal is converted to AD, and the results are sequentially read from the pixel array, added by the HV adder 1601, and output to the subtractor 131. At the same time, the corresponding N-add signal is output from the first LV frame memory 1602 to the subtractor 131, and the result of the subtraction is output to the first LV frame memory 1602, sequentially storing the N-add signal in the memory where it was previously held for each add pixel. Through this operation, the signal stored in the first LV frame memory 1602 is rewritten from the N-add signal to the A+B-add signal.

[0085] Next, as shown in Figure 17(d), the A+B sum signal held in the first LV frame memory 1602 and the A sum signal held in the second LV frame memory 1603 are output outside the chip.

[0086] [Imaging Operation] Figure 18 schematically shows the readout process of the imaging signal according to Embodiment 2, and operates in the order of Figures 18(a) to (c). The way each figure is described is the same as in Figure 17. The readout operation of the imaging signal (recording image signal) will be explained below using Figure 18.

[0087] First, as shown in Figure 18(a), the N signal is converted to AD, and the results are sequentially output from the pixel array to the imaging frame memory 1604 for storage.

[0088] Next, as shown in Figure 18(b), the AD conversion of the N+A+B signal is performed, and the results are sequentially read from the pixel array and output to the subtraction circuit 131. At the same time, the corresponding N signal is output from the imaging frame memory 1604 to the subtraction circuit 131, and the result of the subtraction is held in the imaging frame memory 1604. Through this operation, the signal held in the imaging frame memory 1604 is rewritten from the N signal to the A+B signal. Note that here, the AD conversion of the N+A+B signal was performed without performing the AD conversion of the N+A signal. However, by omitting t308~t313 of the detailed operation timings described in Embodiment 1, and turning φTXA203 on and off at the same timing as turning φTXB204 on and off, it is possible to skip the AD conversion of the N+A signal and perform the AD conversion of the N+A+B signal.

[0089] Next, as shown in Figure 18(c), the A+B signal held in the imaging frame memory 1604 is output to the outside of the chip.

[0090] Furthermore, to ensure that the accumulation times for live view operation and imaging operation are separate, the time (t302~t303) from when OFGA205 and OFGB206 are turned on / off until the AD conversion operation begins may be set to be different.

[0091] As described above, by configuring three frame memories in the image sensor, it becomes possible to independently control the accumulation of the LV signal and the accumulation of the still image recording signal during global shutter operation.

[0092] <Embodiment 3> Next, Embodiment 3 of the present invention will be described. In Embodiments 1 and 2 described above, the configurations of an image sensor 100 having two frame memories and an image sensor 1600 having three frame memories were explained. However, the present invention is not limited to these, and the image sensor may also be configured to have four frame memories. Embodiment 3 will explain the configuration of an image sensor having four frame memories, focusing on the differences from Embodiments 1 and 2.

[0093] [Image sensor configuration] Figure 19 is a schematic diagram showing the overall configuration of the image sensor 1900 according to Embodiment 3 of the present invention. The image sensor 1900 has a first frame memory 1901, a second frame memory 1902, a third frame memory 1903, and a fourth frame memory 1904, each having the same number of memories as the number of pixels. [Imaging Operation / Live View Operation A] Figure 20 schematically shows the readout flow of the imaging operation and live view operation A according to Embodiment 3, and the operation proceeds in the order of Figures 20(a) to (d). The notation in each figure is the same as in Figures 17 and 18. The readout operation of the imaging operation and live view operation A will be explained below using Figure 20.

[0094] First, as shown in Figure 20(a), the N signal is converted to AD, and the results are sequentially read from the pixel array and output to the first frame memory 1901 for storage.

[0095] Next, as shown in Figure 20(b), the N+A signal is converted to AD, and the results are sequentially read from the pixel array and output to the subtraction circuit 131. At the same time, the corresponding N signal is output from the first frame memory 1901 to the subtraction circuit 131, and the result of the subtraction is stored in the second frame memory 1902. As a result, the A signal is stored in the second frame memory 1902.

[0096] Next, as shown in Figure 20(c), the N+A+B signal is converted to AD, and the results are sequentially read from the pixel array and output to the subtraction circuit 131. At the same time, the corresponding N signal is output from the first frame memory 1901 to the subtraction circuit 131, and the result of the subtraction is output to the first frame memory 1901, sequentially storing the N signal in the memory where it was previously held for each pixel. Through this operation, the signal stored in the first frame memory 1901 is rewritten from the N signal to the A+B signal.

[0097] Next, as shown in Figure 20(d), the A+B signal held in the first frame memory 1901 and the A signal held in the second frame memory 1902 are output to the outside of the chip.

[0098] The A signal obtained during the imaging operation / live view operation A is called the 1st A signal, and the A+B signal is called the 1st A+B signal.

[0099] [Imaging Operation / Live View Operation B] Image capture operation / live view operation B performs the same operation as image capture operation / live view operation A, but using the third frame memory 1903 instead of the first frame memory 1901, and the fourth frame memory 1904 instead of the second frame memory 1902.

[0100] The A signal obtained during the imaging operation / live view operation B is called the 2nd A signal, and the A+B signal is called the 2nd A+B signal.

[0101] [Dynamic range expansion synthesis] The time elapsed since turning OFGA205 and OFGB206 on and off is set such that the accumulation time of imaging operation / live view operation B is shorter than the accumulation time of imaging operation / live view operation A. 302 ~t 303 The exposure times are made different. The first A signal and second A signal, and the first A+B signal and second A+B signal obtained in this way, with different exposure times, are adjusted and combined in the overall control / calculation unit 2 according to the storage time.

[0102] As described above, by using four frame memory configurations and performing operations with different storage times, it is possible to acquire a phase difference signal and imaging signal with an expanded dynamic range using global shutter operation.

[0103] [Differentiation] Next, we will describe some variations. In Embodiments 1 and 2 described above, each pixel of the image sensor 100, 1600, or 1900 was described as being either a horizontally divided pixel of 800 or a vertically divided pixel of 900. However, the present invention is not limited to this, and the pixels may be composed of pixels with a different number of divisions or division method than those shown in the figures.

[0104] Figure 21 schematically shows the sensitivity region of pixel 2100 and the arrangement of the color filter 501 in a 2x2 pixel area in this modified example.

[0105] In this modified example, pixel 2100R, which has spectral sensitivity for red (R), is located in the upper left; pixel 2100G, which has spectral sensitivity for green (G), is located in the upper right and lower left; and pixel 2100B, which has spectral sensitivity for blue (B), is located in the lower right. Furthermore, each pixel is composed of a first sensitivity region 2101 to a fourth sensitivity region arranged in a 2x2 grid. Therefore, this modified example has four storage regions corresponding to the first sensitivity region 2101 to the fourth sensitivity region.

[0106] The 2x2 arrangement (unit) shown in Figure 21 is applied to the entire pixel array. In this configuration, by adding and reading the signals of the first sensitivity region 2101 and the third sensitivity region 2103, and the second sensitivity region 2102 and the fourth sensitivity region 2104 within each pixel, a signal similar to the horizontally divided pixel 800 in Figure 10 can be obtained. Similarly, by adding and reading the received signals of the first sensitivity region 2101 and the second sensitivity region 2102, and the third sensitivity region 2103 and the fourth sensitivity region 2104 within each pixel, a signal similar to the vertically divided pixel 900 in Figure 10 can be obtained. When using this as an image signal, the signals of the first to fourth sensitivity regions 2101 to 2104 should be added together. Alternatively, a pair of focus detection signals and an image signal can be obtained by reading out a portion of the signals of the first to fourth sensitivity regions 2101 to 2104 and the signal obtained by adding the signals of the first to fourth sensitivity regions 2101 to 2104, and then subtracting them. Other than the above, the embodiment is the same as described above.

[0107] Furthermore, the drive to obtain a signal similar to that of the horizontally divided pixels 800 or a signal similar to that of the vertically divided pixels 900 may be controlled according to the characteristics of the subject. For example, when the subject is wearing vertically striped clothing and there is little change in brightness in the horizontal direction, the drive may be set to obtain a signal similar to that of the vertically divided pixels 900, and when the subject is wearing vertically striped clothing and there is little change in brightness in the vertical direction, the drive may be set to obtain a signal similar to that of the horizontally divided pixels 800.

[0108] <Summary> This embodiment includes the following configuration.

[0109] (Item 1) A pixel array in which multiple pixels are arranged, It has a first memory for holding the signals output from the pixel array, Each of the aforementioned multiple pixels Multiple photoelectric conversion units that convert light passing through different pupil regions of the imaging optical system into photoelectric signals and output analog signals, Analog-to-digital conversion means for converting the analog signals output from the plurality of photoelectric conversion units into digital signals, The system includes latching means for latching the digital signal converted by the analog-to-digital conversion means, The first memory is characterized by holding the digital signal latched by the latching means. (Item 2) The system further includes a driving means for driving the plurality of pixels, The image sensor according to item 1, characterized in that the driving means drives the plurality of pixels to simultaneously output the analog signal to the analog-to-digital conversion means. (Item 3) The image sensor according to item 2, characterized in that the driving means drives the plurality of pixels so as to output the analog signal so that a pair of focus detection signals having a parallax in either a first direction or a second direction orthogonal to the first direction can be obtained from each of the plurality of pixels. (Item 4) Each of the aforementioned multiple pixels is covered by a color filter of one of the following colors: red, green, or blue, in a Bayer array. The image sensor according to item 3, characterized in that the driving means drives a pixel on which one of the four color filters constituting the repeating unit of the Bayer array is arranged, namely red, green, or blue, so as to be able to acquire a pair of focus detection signals having parallax in the first direction, and drives a pixel on which the green color filter is arranged so as to be able to acquire a pair of focus detection signals having parallax in the second direction. (Item 5) A second memory for holding the digital signals latched by the latching means, Subtraction method and An image sensor according to any one of items 2 to 4, further characterized by having the following: (Item 6) The analog signal when each of the plurality of pixels is reset is converted into a digital signal by the analog-to-digital conversion means, and the noise signal latched by the latch means is stored in the first memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the noise signals corresponding to the same pixels held in the first memory are subtracted from the signals latched by the latch means by the subtraction means, and the resulting first signal is held in the second memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting second signal is held in the first memory. The image sensor described in item 5, characterized by the features described herein. (Item 7) An adding means adds the digital signals corresponding to a predetermined number of pixels to the digital signals latched by the latching means, A second memory and a third memory for each holding the added signals obtained by adding them using the aforementioned adding means, Subtraction method and An image sensor according to any one of items 2 to 4, further characterized by having the following: (Item 8) The analog signals generated when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, the noise signals latched by the latch means are added by the adder means, and the added noise signals are stored in the second memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are added together by the add means, and the added noise signal corresponding to the same pixel held in the second memory is subtracted by the subtract means to obtain the first signal, which is then held in the third memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are added together by the add means, and the added noise signal corresponding to the same pixel held in the second memory is subtracted by the subtract means to obtain the second signal, which is then held in the second memory. The image sensor described in item 7, characterized by the features described herein. (Item 9) A second memory and a third memory for each holding digital signals that have been thinned out at a predetermined frequency from the digital signals corresponding to the plurality of pixels, Subtraction method and An image sensor according to any one of items 2 to 4, further characterized by having the following: (Item 10) The analog signals obtained when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the decimated noise signals obtained by decimating the noise signals latched by the latching means are stored in the second memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are thinned out, and the thinned-out noise signals corresponding to the same pixels held in the second memory are subtracted by the subtraction means from the resulting signal, and the resulting first signal is held in the third memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are thinned out, and the thinned-out noise signals corresponding to the same pixels held in the second memory are subtracted by the subtraction means from the resulting signal, and the resulting second signal is held in the second memory. The image sensor described in item 9, characterized by the features described herein. (Item 11) The analog signal when each of the plurality of pixels is reset is converted into a digital signal by the analog-to-digital conversion means, and the noise signal latched by the latch means is stored in the first memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting third signal is held in the first memory. An image sensor according to any one of items 7 to 10, characterized in that it is an image sensor. (Item 12) A second memory, a third memory, and a fourth memory for holding the digital signals latched by the latching means, Subtraction method and An image sensor according to any one of items 2 to 4, further characterized by having the following: (Item 13) The analog signal when each of the plurality of pixels is reset is converted into a digital signal by the analog-to-digital conversion means, and the noise signal latched by the latch means is stored in the first memory. After exposing each of the plurality of photoelectric conversion units of the plurality of pixels for a predetermined first exposure time, the analog signal output from a part of the plurality of photoelectric conversion units is converted into a digital signal by the analog-to-digital conversion means, the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting first signal is held in the second memory. After storing the first signal in the second memory, the analog signals output from each of the multiple photoelectric conversion units of the multiple pixels are converted into digital signals by the analog-to-digital conversion means, the noise signal corresponding to the same pixel stored in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting second signal is stored in the first memory. After storing the second signal in the first memory, the analog signals when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latch means are stored in the third memory. After exposing each of the plurality of pixels to the plurality of photoelectric conversion units for a predetermined second exposure time different from the first exposure time, the analog signals output from some of the plurality of photoelectric conversion units are converted to digital signals by the analog-to-digital conversion means, the noise signals corresponding to the same pixels held in the third memory are subtracted from the signals latched by the latch means by the subtraction means, and the resulting third signal is held in the fourth memory. After storing the third signal in the fourth memory, the analog signals output from each of the multiple photoelectric conversion units of the multiple pixels are converted into digital signals by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel stored in the third memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting fourth signal is stored in the third memory. The image sensor described in item 12, characterized by the features described herein. (Item 14) The plurality of photoelectric conversion units and a portion of the analog-to-digital conversion means are configured on the first substrate. The configuration of the analog-to-digital conversion means, excluding the aforementioned part, the latching means, and the first memory are configured on the second board. The image sensor according to any one of items 1 to 13, characterized in that the first substrate and the second substrate are stacked. (Item 15) The image sensor described in one of items 6, 8, or 10, A focus detection means that performs phase difference type focus detection using a pair of focus detection signals having parallax obtained from the first signal and the second signal. An imaging device characterized by having the following features. (Item 16) The image sensor described in item 13, A dynamic range expansion means that acquires a pair of focus detection signals with parallax and an expanded dynamic range using the first to fourth signals, A focus detection means that performs phase difference type focus detection using a pair of focus detection signals having parallax and an expanded dynamic range. An imaging device characterized by having the following features. (Item 17) The imaging apparatus according to item 16, further characterized in that the dynamic range expansion means generates a single image with an expanded dynamic range using the second signal and the fourth signal. (Item 18) The image sensor described in item 8 or 10, Display means for displaying an image based on the second signal and An imaging device characterized by having the following features.

[0110] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0111] 2...Overall control / calculation unit, 3...Instruction unit, 4...Timing generation unit, 5...Shooting lens unit, 6...Lens drive unit, 7...Signal processing unit, 8...Display unit, 9...Recording unit, 100...Image sensor, 101...PD board, 111...PD board side part, 112...TAD board side part, 113,114...Pixel part, 115...Vertical scanning circuit, 116...Ramp circuit, 121...Frame memory for N signal, 131...Subtraction circuit, 122...Frame memory for A / A+B signal, 141...Output circuit, 230...Positive feedback circuit, 241...Latch circuit, 1601...Vertical / horizontal addition circuit, 1602...Frame memory for 1st live view, 1603...Frame memory for 2nd live view, 1604...Frame memory for imaging, 1901...1st frame memory, 1902...2nd frame memory, 1903...3rd frame memory, 1904...4th frame memory

Claims

1. A pixel array in which multiple pixels are arranged, It has a first memory for storing signals output from the pixel array, Each of the aforementioned multiple pixels Multiple photoelectric conversion units that convert light passing through different pupil regions of the imaging optical system into photoelectric signals and output analog signals, Analog-to-digital conversion means for converting the analog signals output from the plurality of photoelectric conversion units into digital signals, The system includes a latching means for latching the digital signal converted by the analog-to-digital conversion means, The first memory is characterized by holding the digital signal latched by the latching means.

2. The system further includes a driving means for driving the plurality of pixels, The image sensor according to claim 1, characterized in that the driving means drives the plurality of pixels to simultaneously output the analog signal to the analog-to-digital conversion means.

3. The image sensor according to claim 2, characterized in that the driving means drives the plurality of pixels so as to output the analog signal so that a pair of focus detection signals having a parallax in either a first direction or a second direction orthogonal to the first direction can be acquired from each of the plurality of pixels.

4. Each of the aforementioned multiple pixels is covered by a color filter of one of the following colors: red, green, or blue, in a Bayer array. The image sensor according to claim 3, characterized in that the driving means drives a pixel on which one of the four color filters constituting the repeating unit of the Bayer array is arranged, namely red, green, or blue, so as to be able to acquire a pair of focus detection signals having parallax in the first direction, and drives a pixel on which the green color filter is arranged so as to be able to acquire a pair of focus detection signals having parallax in the second direction.

5. A second memory for holding the digital signals latched by the latching means, Subtraction method and The image sensor according to claim 2, further comprising the above.

6. The analog signals generated when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latching means are stored in the first memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the noise signals corresponding to the same pixels held in the first memory are subtracted from the signals latched by the latch means by the subtraction means, and the resulting first signal is held in the second memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting second signal is held in the first memory. The image sensor according to feature 5.

7. An adding means adds the digital signals corresponding to a predetermined number of pixels to the digital signals latched by the latching means, A second memory and a third memory for each holding the added signals obtained by adding them using the aforementioned adding means, Subtraction method and The image sensor according to claim 2, further comprising the above.

8. The analog signals generated when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, the noise signals latched by the latch means are added by the adder means, and the added noise signals are stored in the second memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are added by the add means, and the added noise signal corresponding to the same pixel held in the second memory is subtracted by the subtract means to obtain the first signal, which is then held in the third memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are added by the add means, and the added noise signal corresponding to the same pixel held in the second memory is subtracted by the subtract means from the resulting signal, and the obtained second signal is held in the second memory. The image sensor according to feature 7.

9. A second memory and a third memory for each holding digital signals that have been thinned out at a predetermined frequency from the digital signals corresponding to the plurality of pixels, Subtraction method and The image sensor according to claim 2, further comprising the above.

10. The analog signals obtained when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latching means are thinned out to obtain the thinned-out noise signals which are stored in the second memory. The analog signals output from a portion of the photoelectric conversion units of each of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are thinned out, and the thinned-out noise signals corresponding to the same pixels held in the second memory are subtracted by the subtraction means from the resulting signal, and the resulting first signal is held in the third memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, the signals latched by the latch means are thinned out, and the thinned-out noise signals corresponding to the same pixels held in the second memory are subtracted by the subtraction means from the resulting signal, and the resulting second signal is held in the second memory. The image sensor according to feature 9.

11. The analog signals generated when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latching means are stored in the first memory. The analog signals output from each of the plurality of photoelectric conversion units of the plurality of pixels are converted into digital signals by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting third signal is held in the first memory. The image sensor according to any one of claims 7 to 10.

12. A second memory, a third memory, and a fourth memory for holding the digital signals latched by the latching means, Subtraction method and The image sensor according to claim 2, further comprising the above.

13. The analog signals generated when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latching means are stored in the first memory. After exposing each of the plurality of photoelectric conversion units of the plurality of pixels for a predetermined first exposure time, the analog signal output from a part of the plurality of photoelectric conversion units is converted into a digital signal by the analog-to-digital conversion means, the noise signal corresponding to the same pixel held in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting first signal is held in the second memory. After storing the first signal in the second memory, the analog signals output from each of the multiple photoelectric conversion units of the multiple pixels are converted into digital signals by the analog-to-digital conversion means, the noise signal corresponding to the same pixel stored in the first memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting second signal is stored in the first memory. After storing the second signal in the first memory, the analog signals when each of the plurality of pixels is reset are converted into digital signals by the analog-to-digital conversion means, and the noise signals latched by the latching means are stored in the third memory. After exposing each of the plurality of pixels to the plurality of photoelectric conversion units for a predetermined second exposure time different from the first exposure time, the analog signals output from some of the plurality of photoelectric conversion units are converted to digital signals by the analog-to-digital conversion means, the noise signals corresponding to the same pixels held in the third memory are subtracted from the signals latched by the latch means by the subtraction means, and the resulting third signal is held in the fourth memory. After storing the third signal in the fourth memory, the analog signals output from each of the multiple photoelectric conversion units of the multiple pixels are converted into digital signals by the analog-to-digital conversion means, and the noise signal corresponding to the same pixel stored in the third memory is subtracted from the signal latched by the latch means by the subtraction means, and the resulting fourth signal is stored in the third memory. The image sensor according to feature 12.

14. The plurality of photoelectric conversion units and a portion of the analog-to-digital conversion means are configured on the first substrate. The configuration of the aforementioned analog-to-digital conversion means, excluding the aforementioned part, the latching means, and the first memory are configured on the second circuit board. The image sensor according to claim 1, characterized in that the first substrate and the second substrate are stacked.

15. An image sensor according to any one of claims 6, 8, or 10, A focus detection means that performs phase difference type focus detection using a pair of focus detection signals having parallax obtained from the first signal and the second signal. An imaging device characterized by having the following features.

16. The image sensor according to claim 13, A dynamic range expansion means that acquires a pair of focus detection signals with parallax and an expanded dynamic range using the first to fourth signals, A focus detection means that performs phase difference type focus detection using a pair of focus detection signals having parallax and an expanded dynamic range. An imaging device characterized by having the following features.

17. The imaging apparatus according to claim 16, further characterized in that the dynamic range expansion means generates a single image with an expanded dynamic range using the second signal and the fourth signal.

18. The image sensor according to claim 8 or 10, Display means for displaying an image based on the second signal and An imaging device characterized by having the following features.

Images