3D stacked image sensor and reading method

Abstract

The invention belongs to the technical field of CMOS (complementary metal oxide semiconductor) image sensors, and particularly relates to a global and rolling compatible high-frame-frequency 3D (three-dimensional) stacked image sensor and a readout method.According to the image sensor, a readout circuit and pixel layers are stacked after being designed from the structure, pixel array layers are divided into UP and DN groups, each group is independently subjected to two-way readout, and the readout circuit and the pixel layers are subjected to two-way readout; the reading circuit layer adopts the same group for the column level and adopts a mode of parallel working of a plurality of sub-column level ADCs for the column level ADC to improve the frame frequency; meanwhile, a switched capacitor array is added to an analog front-end circuit (AFE), and the inconsistency of the sequence of integral signals and reset signals output by pixels in a global shutter mode and a rolling shutter mode is adjusted, so that the requirement that a programmable gain amplifier (PGA) and an analog-to-digital converter (ADC) firstly process the reset signals and then process the integral signals is met. Compatibility of a global shutter mode and a rolling shutter mode is achieved, and the pixel structure design is more flexible.

Description

Technical Field

[0001] This invention relates to the field of CMOS image sensors, and more specifically to a high frame rate 3D stacked image sensor and readout method compatible with global and scrolling modes. Background Technology

[0002] Image sensors, as core devices that convert light signals into electrical signals, are widely used in consumer electronics, autonomous driving, aerospace, and machine vision. With the continuous expansion of application scenarios, the market is placing higher demands on image sensors in terms of frame rate, integration, and shutter mode compatibility.

[0003] To improve performance, 3D stacking technology has been introduced into image sensor design. For example, Chinese invention patent CN109587417B discloses a 3D stacked image sensor, which places the pixel array on the upper chip and the analog-to-digital converter array and memory array on the lower chip, achieving high integration through chip stacking. This technology enables parallel signal processing and storage by assigning each pixel unit to an analog-to-digital converter and a memory, thus improving the sensor's frame rate to some extent. However, this existing technology has at least the following shortcomings: First, its readout architecture has not completely escaped the mode of sequential or grouped multi-line readout, and data transmission speed remains a bottleneck restricting further improvements in frame rate; second, this solution does not address the compatibility design between global shutter and rolling shutter, and its memory array is mainly used to temporarily store digital signals to avoid line waiting time, without solving the signal processing problem caused by the different output order of integration signal and reset signal in the two shutter modes.

[0004] Chinese invention patent CN111669526B provides a high-speed, all-digital readout CMOS image sensor to improve frame rate. This technology optimizes timing consistency and latency issues in large-area array sensors by setting up dual-side row drive circuits and column timing buffer circuits, and uses column-level ADC technology to achieve all-digital signal output, effectively improving readout speed. However, this technical solution also has limitations: its core improvement lies in optimizing timing and the digital interface, without changing the readout architecture of the pixel array itself; the data is essentially still read out sequentially in a unidirectional manner, limiting further improvements in parallelism. Furthermore, this solution does not adapt its analog front-end circuitry, making it unable to flexibly handle signal sequence differences under different shutter modes, thus limiting its application in scenarios requiring high dynamic range and distortion-free imaging (such as global shutter mode).

[0005] In summary, existing 3D stacked image sensors, in the pursuit of high frame rates and high integration, generally suffer from the following problems: (1) Insufficient shutter mode compatibility, lack of a mechanism to adaptively adjust the difference in the sequence of global shutter and rolling shutter signals at the analog front end, resulting in the sensor being unable to flexibly adapt to different application scenarios without sacrificing performance.

[0006] (2) Reading architecture efficiency bottleneck: most designs still use unidirectional or semi-parallel reading methods, and there is competition in the data path, which limits the order of magnitude increase in frame rate.

[0007] (3) The system optimization is lacking. It fails to systematically integrate pixel array grouping, parallel readout circuit design and signal order adjustment technology, thus failing to achieve high frame rate, high compatibility and high integration at the same time.

[0008] Therefore, there is an urgent need in this field for an innovative 3D stacked image sensor and readout method to systematically solve the above problems. Summary of the Invention

[0009] In view of this, the present invention aims to overcome the problems of insufficient shutter mode compatibility of existing 3D image sensors, bottleneck of data readout efficiency limited by the number of rows, and optimizable system integration, and provides a high frame rate 3D stacked image sensor compatible with global and rolling shutter and its readout method.

[0010] To achieve the above objectives, the present invention provides the following technical solution: A 3D stacked image sensor is provided, which consists of an upper chip I and a lower chip II stacked one on top of the other, and the upper and lower chips are interconnected through a three-dimensional integration method.

[0011] The upper-layer chip I includes a pixel array configured to operate in either a global shutter mode or a rolling shutter mode. The pixel array consists of multiple pixel units that convert optical signals into analog electrical signals. The pixel array is also divided into independent UP and DN groups in the column direction, and the UP and DN groups are configured to perform bidirectional readout simultaneously.

[0012] The lower-level chip II includes: an analog-to-digital converter array, a memory cell array, a digital logic control circuit, a phase-locked loop (PLL) module, a high-speed serial interface circuit, and an analog front-end circuit.

[0013] The analog-to-digital converter (ADC) array includes multiple sub-column-level ADCs. Each of the UP and DN groups corresponds to M sub-column-level ADCs (M is an integer greater than 1) for the same column of pixels in the upper layer, thus achieving parallel ADC conversion of M rows of pixel signals. Each sub-column-level ADC corresponds to a storage unit located between its counter output and the high-speed serial interface circuit readout interface.

[0014] The storage unit array includes M storage units per column of pixels, used to store digital signals after analog-to-digital conversion, so that M rows of parallel data can be stored in the storage unit array at the same time.

[0015] The PLL module generates the high-frequency clock required by the digital logic control circuit when a reference clock is input.

[0016] The digital logic control circuit operates independently in the UP and DN sections, generating control signals for the row and column address decoding circuit, control signals for the memory cell array, and control signals for other readout circuits.

[0017] The high-speed serial interface circuit is distributed on both sides of the lower-level chip corresponding to the UP group and DN group, and is used to output the processed digital signal in parallel at high speed.

[0018] The analog front-end circuit (AFE) includes a switched capacitor array, which is configured such that when the sensor is in global shutter mode and the pixel outputs an integral signal before a reset signal, the integral signal is sampled and temporarily stored in the capacitor array. After the AFE switches the capacitor array processes the reset signal, it releases and processes the integral signal stored in the capacitor, thereby adjusting the order of the integral signal and the reset signal to meet the requirements of the subsequent programmable gain amplifier (PGA) and analog-to-digital converter (ADC) to process the reset signal before processing the integral signal.

[0019] Preferably, the M pixel column buses of the same column in the upper-layer chip UP group and the M sub-column-level analog-to-digital converter (ADC) input terminals of the lower-layer chip are connected via Cu-Cu bonding or TSV bonding. The control signal lines of each row of pixels in the upper-layer chip UP group and the output signals of the lower-layer row drive circuit are connected via Cu-Cu bonding or TSV bonding. Similarly, the M pixel column buses of the same column in the upper-layer chip DN group and the M sub-column-level ADC input terminals of the lower-layer chip are connected via Cu-Cu bonding or TSV bonding. The control signal lines of each row of pixels in the upper-layer chip DN group and the output signals of the lower-layer row drive circuit are connected via Cu-Cu bonding or TSV bonding.

[0020] Preferably, the size of the pixel array is M times the width of the column-level circuit. This provides sufficient area for the pixel region, allowing for the design of M sub-column-level circuits corresponding to the same column pixel width at the lower layer. Simultaneously, it provides more flexible layout space for the readout circuits of the lower-level chips, facilitating more complex circuit designs and performance optimization.

[0021] Preferably, the pixel array is formed on the upper chip using a back-illuminated process.

[0022] Preferably, the sub-column-level analog-to-digital converter (ADC) is an SS_ADC+TDC structure combining a single-slope ADC and a time-to-digital converter (TDC) to optimize quantization speed and accuracy. The ADC function of the SS_ADC+TDC structure is as follows: the SS_ADC performs coarse quantization (i bits) and the TDC performs fine quantization (j bits), and the TDC fine quantization process does not occupy separate quantization time. The quantization time of the SS_ADC+TDC structure is approximately T = 2. i ×ADC_CLK.

[0023] Preferably, the high-speed serial interface circuit uses a high-speed SerDes interface.

[0024] The present invention also provides a readout method for the above-mentioned 3D stacked image sensor, the readout method comprising the following steps: (1) Synchronous exposure: Control the simultaneous exposure of UP group pixels and DN group pixels; (2) Adaptive signal sampling and sequence adjustment: Determine the sensor's working mode, and sample and adjust the signals output by the UP group and DN group pixels through the AFE switched capacitor array; (3) Signal preprocessing: First process the reset signal, then release and process the integral signal; (4) Parallel analog-to-digital conversion: The analog electrical signals of the two groups of signals are converted in parallel through M sub-column level analog-to-digital conversion units corresponding to the same column of pixels in the UP group and DN group.

[0025] (5) High-speed data output: The digital electrical signals of the UP group and the DN group are output in parallel through high-speed serial interface circuits (such as high-speed Serdes interface) distributed on both sides.

[0026] Preferably, the step of "determining the sensor's operating mode" includes: ① If the sensor is in global shutter mode and the pixels output the integration signal first, then the switched capacitor array is controlled to sample and temporarily store the integration signal, followed by processing the reset signal, and then releasing and processing the temporarily stored integration signal. ② If the sensor is in rolling shutter mode, then the signals output by the pixels in sequence are processed directly.

[0027] Preferably, in the global shutter mode, the step of controlling the switched capacitor array includes: S901: Close the first switch PGA_S1 and sample the integral signal output by the pixel into the storage capacitor for temporary storage; S902: Disconnect the first switch PGA_S1 and store the integral signal in the storage capacitor; S903: Close the second switch PGA_S2 to simulate the front-end circuit processing the reset signal; S904: After the reset signal processing is completed, the first switch PGA_S1 is closed again to release the integral signal temporarily stored in the storage capacitor to the subsequent circuit for processing.

[0028] The beneficial effects of this invention are as follows: (1) By using the signal storage and release mechanism of the AFE switched capacitor array, the problem of mismatch between the signal order and the processing requirements of the subsequent circuit in the global shutter mode is cleverly solved. This allows the same sensor to be compatible with both global and rolling shutter modes without changing the internal structure of the pixels, and the output signal order is not restricted, significantly improving flexibility.

[0029] (2) By dividing the pixel array into two groups, UP and DN, and adopting bidirectional parallel readout, and integrating M sub-column level ADCs for parallel conversion of the same column of pixels in each group, the traditional row-by-row serial readout is transformed into a new mode of group parallel and row-column parallel, which greatly reduces the readout time of single frame data, increases the row time margin, and effectively improves the frame rate of the image sensor without increasing the clock frequency.

[0030] (3) Comprehensive optimization of system performance a. The sub-level analog-to-digital converter adopts a hybrid structure of SS_ADC+TDC, which ensures quantization accuracy and speed without significantly increasing circuit complexity and power consumption.

[0031] b. 3D stacking technology enables vertical interconnection between the pixel layer and the readout circuit layer, improving sensor integration and fill rate, enhancing imaging performance, and improving system integration.

[0032] c. Ensure that the ADC always processes signals in an optimized order of reset followed by integration, effectively implement correlated double sampling, significantly reduce fixed-pattern noise, and improve image quality.

[0033] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a two-dimensional architecture diagram of a 3D stacked CMOS image sensor; Figure 2 This is a schematic diagram of the layout of the upper-layer chip I pixel array structure; Figure 3This is a schematic diagram of the structural layout of the readout circuit of the lower-level chip II; Figure 4 This is a schematic diagram of the circuit structure of an AFE switched capacitor array; Figure 5 This is a diagram illustrating the column-level data path; Figure 6 This is a flowchart of the readout method applied to the 3D stacked image sensor. Detailed Implementation

[0035] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0036] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0037] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0038] This invention provides a high frame rate 3D stacked image sensor compatible with both global and rolling shutter speeds, and its readout method. The following is in conjunction with the appendix... Figure 1-6 The present invention will be described in further detail below.

[0039] The sensor adopts a stacked chip structure and has key technical features such as bidirectional parallel readout, multi-line synchronous processing and shutter mode compatibility.

[0040] Figure 1This is a two-dimensional architecture diagram of the 3D stacked CMOS image sensor described in this invention. Figure 2 and Figure 3 These are schematic diagrams of the pixel array structure layout of upper-layer chip I and the readout circuit structure layout of lower-layer chip II, respectively. The pixel array is divided into two independent parts along the column direction: a UP group and a DN group, corresponding to the column-level circuits of the UP and DN parts in lower-layer chip II, respectively. This grouping structure supports parallel operation of multiple rows of pixels and bidirectional synchronous readout, thereby effectively increasing the available row time and improving the sensor frame rate.

[0041] The upper-layer chip I and the lower-layer chip II are interconnected in three dimensions using Cu-Cu bonding or TSV (Through Silicon Via) technology. The interconnect bonding points are located on the column bus, row drive output line, power line, and ground line, respectively, to ensure the reliability and integrity of signal transmission.

[0042] Figure 4 This is a schematic diagram of the circuit structure of the analog front-end circuit AFE switched capacitor array, including MOS storage capacitors Q1, Q2, and Q3, sampling capacitor Cs, offset storage capacitor Cc, feedback capacitor Cf, switches Test_R and Test_S for controlling the test signal input, and operational amplifier unit A of the programmable gain amplifier (PGA).

[0043] When the sensor is in global shutter mode, the pixel output sequence is integral signal before reset signal. At this time, closing switches PGA_IN_S and PGA_S1 samples the integral signal into the storage capacitor. Vsig_S2 and Vsig_S3 are used to adjust the size of the storage capacitor. Then, PGA_S1 is opened and PGA_S2 is closed, allowing the PGA to process the reset signal. After the reset signal has been processed, PGA_S1 is closed again, releasing and processing the integral signal stored in the capacitor. This adjusts the signal order, satisfying the requirement that the subsequent PGA and ADC process the reset signal before the integral signal.

[0044] When the sensor is operating in rolling shutter mode, the pixel output sequence is that the reset signal precedes the integration signal. At this time, it is only necessary to keep the switch PGA_S1 in the off state to process the signals directly in sequence without any additional adjustment.

[0045] Figure 5This is a schematic diagram of the column-level data path. Taking M=3 as an example, each column-level circuit is configured with 3 sub-column-level analog-to-digital converters (ADCs), allowing UP and DN to process 3 rows of pixel signals in parallel. Each sub-column-level ADC is equipped with an independent storage unit for temporarily storing the quantized digital signal. The high-speed SerDes interface unit is responsible for serially outputting the data from the storage unit to the external chip. Each SerDes interface unit is responsible for outputting data from (3*X) / n memory units, where X is the total number of columns in the pixel array and n is the number of groups in the column direction.

[0046] The following will be combined with the appendix Figure 6 The readout method of this invention is described in detail. This method is based on the aforementioned 3D stacked image sensor hardware structure. Its core lies in achieving high frame rate data readout from the shutter mode compatible image sensor through a series of coordinated steps, including grouped parallel exposure, adaptive signal sequence adjustment, parallel analog-to-digital conversion, and high-speed data output.

[0047] The specific process of the readout method is as follows: Step 1: Simultaneous Exposure The UP and DN groups of pixels in the upper-layer chip I are exposed simultaneously, converting the incident light signal into an analog electrical signal. This synchronization ensures the time consistency of the image data from the UP and DN groups, laying the foundation for subsequent bidirectional parallel readout.

[0048] Step 2: Adaptive signal sampling and sequence adjustment This step is crucial for achieving global compatibility between the sensor and the rolling shutter mode. The digital logic control circuit first determines the sensor's current operating mode and then controls the switched capacitor array in the analog front-end circuit to sample and sequentially adjust the signals output by the UP and DN group pixels. Specifically, this includes two scenarios: If the sensor is in global shutter mode: In this mode, pixels typically output the integration signal first, followed by the reset signal. However, the subsequent programmable gain amplifier (PGA) and analog-to-digital converter (ADC) usually require processing the reset signal first to optimize noise performance.

[0049] Furthermore, the AFE switched capacitor array performs the following operations: (1) Sample the integral signal: Close the first switch PGA_S1 and sample the integral signal output by the pixel into the storage capacitors Q1, Q2, Q3 for temporary storage.

[0050] (2) Disconnect the sampling path: Then disconnect the first switch PGA_S1 and store the integrated signal in the storage capacitor.

[0051] (3) Processing the reset signal: Close the second switch PGA_S2 so that the AFE circuit can process the reset signal subsequently output by the pixel.

[0052] (4) Release and process the integral signal: After the reset signal is processed, close the first switch PGA_S1 again to release the integral signal temporarily stored in the storage capacitor to the subsequent PGA and ADC circuits for processing.

[0053] By using the aforementioned "store first, retrieve later" mechanism, the signal processing order in global shutter mode is adjusted to adapt to the processing flow of subsequent circuits.

[0054] If the sensor is in rolling shutter mode: In this mode, the pixels are output in the order of reset signal first, then integration signal. Therefore, there is no need to enable the signal sequence adjustment function. The AFE switched capacitor array keeps the first switch PGA_S1 in the open state and directly processes the signals that arrive in sequence.

[0055] Step 3: Signal Preprocessing Regardless of which path is taken in step S02, the ultimate goal is to ensure that the subsequent circuitry always processes the reset signal first, followed by the integration signal. This sequence is a crucial prerequisite for performing correlated double sampling to eliminate fixed-mode noise.

[0056] Step 4: Parallel Analog-to-Digital Conversion The analog signals from the UP and DN groups, after being processed by the AFE, are sent to the corresponding analog-to-digital converter (ADC) arrays in the lower-level chip II. Each of the UP and DN groups corresponds to M sub-column-level ADCs. These sub-column-level ADCs operate in parallel under the timing generated by the digital logic control circuit, synchronously converting the analog signals of the M rows of pixels into digital signals. This highly parallel architecture significantly improves the efficiency of data conversion.

[0057] Step 5: High-speed data output The digital signals after analog-to-digital conversion are temporarily stored in the corresponding memory cells of each sub-stage ADC. Subsequently, the high-speed SerDes interface circuits distributed on both sides of the UP and DN groups of the lower-level chip are activated synchronously, outputting the large-scale digital data of the two groups to the off-chip in a serial-to-parallel manner. This bidirectional, parallel data output architecture breaks through the speed bottleneck of traditional unidirectional sequential readout.

[0058] In summary, this invention designs the pixel array and readout circuitry on separate upper and lower chip layers, achieving three-dimensional integration using Cu-Cu bonding or TSV technology. Combined with grouped bidirectional readout, multi-row parallel processing, and a hybrid ADC structure of sub-column SS_ADC+TDC, it significantly increases the row time margin without increasing the system clock frequency, effectively improving the frame rate of the image sensor. Furthermore, by introducing an AFE switched capacitor array in the analog front end, it achieves adaptive adjustment to signal sequence differences in global and rolling shutter modes, enhancing the flexibility of pixel structure design and system compatibility.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A 3D stacked image sensor, characterized by, The upper chip I and the lower chip II are stacked vertically; The upper chip I includes a pixel array configured to work in a global shutter mode or a rolling shutter mode, and is divided into UP groups and DN groups which are independent of each other and configured to simultaneously perform bidirectional readout; The lower chip II includes an analog-to-digital conversion unit array, an analog front-end circuit (AFE), and a high-speed serial interface circuit; The analog-to-digital conversion unit array includes a plurality of sub-column-level analog-to-digital conversion units, and each of the UP groups and the DN groups corresponds to M sub-column-level analog-to-digital conversion units corresponding to the same column of pixels in the upper layer, where M is an integer greater than 1; The analog front-end circuit (AFE) includes a switched capacitor array configured to adjust the order of the integration signal and the reset signal output by the pixel in the global shutter mode so that the reset signal is processed before the integration signal.

2. The 3D stacked image sensor according to claim 1, characterized in that, The upper chip I and the lower chip II are interconnected by a three-dimensional integration method.

3. The 3D stacked image sensor according to claim 2, characterized in that, The three-dimensional integration method is Cu-Cu bonding or TSV silicon through hole.

4. The 3D stacked image sensor according to claim 1 or 2, characterized in that, The pixel array is formed on the upper chip I by a back-illuminated process.

5. The 3D stacked image sensor of claim 1, wherein, The sub-column-level analog-to-digital conversion unit is a SS_ADC+TDC structure combining a single slope analog-to-digital converter and a time-to-digital converter.

6. The 3D stacked image sensor of claim 1, wherein, The high-speed serial interface circuit is a Serdes interface, which is distributed on both sides of the UP groups and the DN groups of the lower chip, and each side is provided with n channels, each of which is responsible for outputting digital data of X / n columns of pixels, where X is the total number of columns of the pixel array.

7. A readout method for a 3D stacked image sensor according to any one of claims 1 to 6, characterized by, The method comprises the following steps: S01: controlling the UP group pixels and the DN group pixels to expose at the same time; S02: judging the sensor working mode, and sampling and adjusting the order of the signals output by the UP group pixels and the DN group pixels through the switched capacitor array of the analog front-end circuit (AFE); S03: processing the reset signal first, and then releasing and processing the integration signal; S04: performing parallel analog-to-digital conversion on the signals of the UP group and the DN group through the M sub-column-level analog-to-digital conversion units; S05: parallel outputting the converted digital signals through the high-speed Serdes interface.

8. The readout method according to claim 7, characterized in that, The judgment of the sensor working mode in step S2 comprises: If the sensor is in the global shutter mode and the pixel outputs the integration signal first, the switched capacitor array is controlled to sample and temporarily store the integration signal, then process the reset signal, and then release and process the temporarily stored integration signal; If the sensor is in the rolling shutter mode, the signals output by the pixel in sequence are directly processed.

9. The readout method according to claim 8, characterized in that, In the global shutter mode, the step of controlling the switched capacitor array comprises: S901: closing a first switch PGA_S1 to sample the integration signal output by the pixel first into a storage capacitor for temporary storage; S902: opening the first switch PGA_S1 to save the integration signal in the storage capacitor; S903: closing a second switch PGA_S2 to process the reset signal by the analog front-end circuit; S904: After the reset signal processing is completed, the first switch PGA_S1 is closed again, and the integral signal temporarily stored in the storage capacitor is released to the subsequent circuit for processing.

Images