Rolling shutter CMOS-TDI image sensor with interval space and method

Abstract

The invention provides a rolling shutter CMOS-TDI image sensor with an interval space and a method, the rolling shutter CMOS-TDI image sensor with the interval space comprises an upper pixel array and a lower pixel array, the pixel arrays are connected with a peripheral control circuit, pixels of the pixel arrays are connected with a reading circuit, an integrator is arranged in the reading circuit, the reading circuit is connected with a memory, and the memory is connected with the storage. The memory is arranged inside or outside the reading circuit; a physical interval space which is accurately calculated according to the pixel width, the line time difference and the frame period is arranged between pixels so as to spatially compensate the time delay. The method has the beneficial effects that the imaging quality is remarkably improved, and motion distortion is eliminated; the signal integration times are multiplied, and the signal-to-noise ratio is greatly enhanced; and the working flexibility and environmental adaptability of the sensor are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of image sensors, and in particular relates to a rolling shutter CMOS-TDI image sensor and method with spaced intervals. Background Technology

[0002] When using TDI technology on CIS in the existing technology, due to the characteristics of RS line-by-line exposure, the exposure time of each pixel line is not synchronized. When the image sensor moves, the line time difference will cause different pixel lines to capture different scene points, resulting in serious distortions such as image blurring, distortion and ghosting, which greatly reduces the imaging effect of TDI.

[0003] The SNR gain of TDI is directly related to the number of signal integrations (i.e., the number of TDI stages), but the number of rows in a single pixel array cannot be increased indefinitely due to the limitations of circuit operating speed.

[0004] The number of TDI stages N and the transit time between each stage are fixed. If the speed of the object being imaged deviates from the preset value, two situations will occur: insufficient image overlap or excessive overlap. This results in a decrease in signal-to-noise ratio due to insufficient signal accumulation, or a blurring effect due to excessive accumulation. Analog circuits are very sensitive to process technology, temperature, and power supply voltage. Fixed hardware parameters can cause gain and bias level drift in different batches of chips or under different operating temperatures, further affecting the linearity and stability of TDI accumulation.

[0005] Area-array CIS is limited by spatial domain resolution, resulting in bottlenecks in readout speed, power consumption, and field-of-view consistency. Linear scan imaging technology, on the other hand, utilizes the linear motion of the object being photographed or the image sensor itself to construct an image in another dimension, improving scanning efficiency and imaging accuracy. It is widely used in remote sensing imaging, industrial monitoring, printing inspection, and various automated inspection fields.

[0006] In low-light and high-speed motion scenarios, the signal-to-noise ratio (SNR) of images often falls short of requirements. To address this issue, TDI (Transient Directional Imaging) technology arranges multiple rows or linear arrays of pixels along a known, uniform direction of target motion and synchronously overlays continuously captured data in the temporal domain. This achieves an effect similar to multiple exposures without extending the exposure time of individual pixels, significantly improving the image's SNR and sensitivity. It is particularly suitable for low-light, high-speed, or long-distance imaging applications. TDI imaging was initially applied in remote sensing and aerospace, and with the expansion of its applications, TDI has gradually gained traction in civilian fields such as industrial inspection, biomedicine, and rail monitoring. Summary of the Invention

[0007] In view of this, the present invention aims to propose a rolling shutter CMOS-TDI image sensor and method with interval space, so as to solve the problems of image distortion, limited integration times of a single sensor, and poor adaptability of fixed hardware to different environments in the prior art.

[0008] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0009] In a first aspect, the present invention provides a rolling shutter CMOS-TDI image sensor with a spaced interval, comprising upper and lower pixel arrays, wherein the pixel arrays are connected to a control circuit on their periphery, the pixels of the pixel arrays are connected to a readout circuit, the readout circuit is provided with an integrator, and the readout circuit is connected to a memory, wherein the memory is arranged inside or outside the readout circuit.

[0010] The circuit structure of the pixel includes a clamping photodiode and a sampling capacitor C. S Transmission transistor M TX Reset transistor M rst Source follower M sf Source follower M sel ;

[0011] The transmission transistor M TX The source of the transistor is connected to the cathode of the clamping photodiode, the anode of the clamping photodiode is grounded, and the transmission transistor M... TX The gate is connected to the control signal line Sgt, and the transmission transistor M TX The drain is connected to the sampling capacitor C. S One end, reset transistor M rst The source and source follower M sf The gate of the sampling capacitor C S The other end is grounded, and the reset transistor M rst The gate is connected to the control signal line Srst, and the reset transistor M rst The drain of the source follower M is connected to the power supply VDD. sf The drain of the source follower M is connected to the power supply VDD. sf The source connection to the source follower M sel The drain of the source follower M sel The gate connection control signal line Srs, the source follower M sel The source is connected to the output signal PXO.

[0012] Furthermore, the two pixel arrays are arranged along the target movement direction, and each column of pixels in both arrays is aligned.

[0013] Furthermore, there is a spacing between adjacent pixels in the pixel column.

[0014] Furthermore, the control signal lines Srst, Sgt, and Srs between pixel columns are all connected in series with buffers.

[0015] Secondly, based on the same concept, the present invention also provides a rolling shutter CMOS-TDI image sensing method with spaced intervals, comprising the following steps:

[0016] If the pixel array captures continuous image data, the width of the interval space is set to the pixel width multiplied by the ratio of the line time difference to the frame period.

[0017] If the pixel array captures non-continuous image data, the width of the interval space is set to the pixel width plus the pixel width multiplied by the ratio of the line time difference to the frame period.

[0018] The pixel array captures continuous image data, including:

[0019] A1. Determine the frame period based on the scene's brightness and the sensitivity of the pixel array;

[0020] A2. The integrator integrates pixel data from the same location in the scene that are related to the pixel array in adjacent image frames;

[0021] A3. The memory integrates the integrated pixel data of the two pixel arrays;

[0022] A4. Enter sleep mode within the time difference between the frame period and the total line exposure time.

[0023] Furthermore, the width of the interval space is set as the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression:

[0024]

[0025] In the formula, W SS,C Where W is the interval space, t is the line time difference, and T is the frame period.

[0026] Furthermore, the pixel array captures non-continuous image data, including:

[0027] The pixel array outputs one frame and then skips to the next.

[0028] Furthermore, the width of the interval space is set to the pixel width plus the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression:

[0029]

[0030] In the formula, W SS,CWhere W is the interval space, t is the line time difference, and T is the frame period.

[0031] Compared with the prior art, the rolling shutter CMOS-TDI image sensor and method with spaced intervals described in this invention have the following advantages:

[0032] (1) Significantly improves image quality and eliminates motion distortion. By setting a physical "interval space" between pixels that is precisely calculated based on pixel width, line time difference and frame period, spatial compensation for time delay is used to ensure that different pixel rows can be precisely aligned with the same scene point at their respective exposure times, thereby eliminating distortion and obtaining a clear image.

[0033] (2) The number of signal integrations is increased several times, which greatly enhances the signal-to-noise ratio. By arranging two or more independently operable pixel arrays along the moving direction, they scan the same scene area one after another and accumulate their respective integration results again, thereby increasing the total number of integrations several times without sacrificing the running speed, and further improving the SNR of the image.

[0034] (3) Enhance the sensor's operational flexibility and environmental adaptability. A more flexible sensor operation mode is provided. By adjusting the line time difference of RS within a fixed physical interval space, the sensor can select the optimal operating mode according to the ambient light or noise level without changing the hardware structure, thus enhancing the sensor's environmental adaptability. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0036] Figure 1 This is a schematic diagram of a conventional CMOS-TDI image sensor system architecture as described in an embodiment of the present invention;

[0037] Figure 2 This is a timing diagram illustrating the working principle of CMOS-TDI as described in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram comparing the imaging of the global shutter and the rolling shutter according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of a CMOS-TDI image sensor with spacing space based on RS, as described in an embodiment of the present invention.

[0040] Figure 5 This is a schematic diagram of the pixel circuit structure of a CMOS-TDI image sensor with spacing space according to an embodiment of the present invention. Detailed Implementation

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0042] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0043] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0044] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] like Figure 1 As shown, the system architecture of the CMOS-TDI image sensor has a spatial resolution of N×M for the pixel array, where N represents the integration stage and M represents the number of columns. During each exposure cycle, each row of pixels outputs its corresponding voltage signal, which is fed into a P×M accumulator array for integration. This accumulation process can be performed in the analog domain as a voltage signal before analog-to-digital conversion, or it can be converted to a digital signal after each exposure stage and then accumulated using a digital domain adder.

[0046] like Figure 2As shown, the CMOS-TDI operating timing diagram illustrates the image sensor imaging along the direction of scene movement. When an object moves through N rows of the pixel array, it is sequentially imaged by each row of pixels. Assuming the object moves at a constant speed, and the exposure time interval between any two adjacent rows is the transit time tL, then at time t-(N-1)tL, the object is exposed by the first row of pixels, resulting in a voltage signal V1(t-(N-1)tL); at time t-(N-2)tL, the object moves to the position of the second row of pixels and is exposed, resulting in a voltage signal V2(t-(N-2)tL); at time t-(N-3)tL, the object moves to the position of the third row of pixels and is exposed, resulting in a voltage signal V3(t-(N-3)tL); and so on until time t, when the object moves to the Nth row and is exposed, resulting in a voltage value VN(t). The integral result of the N consecutive exposures is then output, expressed as follows:

[0047]

[0048] After exposing the same object N times, assuming that the signal voltage obtained from each exposure is the same, with a magnitude of Vsig, and the introduced noise power is V2 noise, the SNR expression is as follows:

[0049]

[0050] As can be seen from the above formula, TDI increases the SNR by 10·lg(N)dB relative to a single exposure. For every 2-fold increase in the number of TDI levels, the signal-to-noise ratio increases by about 3dB.

[0051] like Figure 3 As shown, assuming a CMOS-TDI image sensor has only two rows of pixels, during the uniform rightward movement of object ABC, the determined row time difference directly translates into spatial positional deviation. Without row time delay, the second row of pixels in the second frame is in the same position as the first row of pixels in the first frame. The readout circuit then sums the voltage signals from the two exposed rows, resulting in a clear image of object A. However, with a row time difference, the object continues to move within that time difference. This causes the second row of pixels in the second frame to not perfectly correspond to the position of the first row of pixels in the first frame. The summed readout causes object B to overlap with A, resulting in blurring and ghosting distortions in the image. This significantly limits the application and development of CMOS-TDI technology in high-performance, high-precision imaging.

[0052] In a preferred embodiment of the present invention, such as Figure 4As shown, a rolling shutter CMOS-TDI image sensor with spacing includes two pixel arrays, upper and lower. The pixel arrays are connected to a control circuit, and the pixels of the pixel arrays are connected to a readout circuit. The readout circuit has an integrator inside and is connected to a memory, which is arranged inside or outside the readout circuit. In this embodiment, the integrator integrates pixel data from the same location in the scene that are related to the pixel array in adjacent image frames; the memory integrates the integrated pixel data of the two pixel arrays; control circuit 1 controls the first pixel array to output first pixel data from the lower pixel to the upper pixel in RS mode; after waiting for the compensation time (CT), control circuit 2 controls the second pixel array to output second pixel data from the lower pixel to the upper pixel in RS mode.

[0053] In a preferred embodiment of the present invention, such as Figure 5 As shown, the circuit structure of the pixel includes a clamping photodiode and a sampling capacitor C. S Transmission transistor M TX Reset transistor M rst Source follower M sf Source follower M sel The transmission transistor M TX The source of the transistor is connected to the cathode of the clamping photodiode, the anode of the clamping photodiode is grounded, and the transmission transistor M... TX The gate is connected to the control signal line Sgt, and the transmission transistor M TX The drain is connected to the sampling capacitor C. S One end, reset transistor M rst The source and source follower M sf The gate of the sampling capacitor C S The other end is grounded, and the reset transistor M rst The gate is connected to the control signal line Srst, and the reset transistor M rst The drain of the source follower M is connected to the power supply VDD. sf The drain of the source follower M is connected to the power supply VDD. sf The source connection to the source follower M sel The drain of the source follower M sel The gate connection control signal line Srs, the source follower M sel The source is connected to the output signal PXO; the control signal lines Srst, Sgt, and Srs between pixel columns are all connected in series with buffers. In this embodiment, buffers are connected in series in the control signal lines between every two pixel columns using the spacing space to buffer and amplify the pixel control signals, so that even pixel arrays with a large number of pixel columns can operate accurately.

[0054] In a preferred embodiment of the invention, the two pixel arrays are arranged along the target movement direction, and each column of pixels in both arrays is aligned. In this embodiment, the pixels sequentially pass through the same locations in the scene during imaging to effectively integrate the corresponding pixel data.

[0055] In a preferred embodiment of the present invention, a spacing space is provided between adjacent pixels in a pixel column. In this embodiment, by setting a physical "spacing space" between pixels, which is precisely calculated based on pixel width, row time difference, and frame period, spatial compensation for time delay is provided to ensure that different pixel rows can be precisely aligned with the same scene point at their respective exposure times, thereby eliminating distortion and obtaining a clear image.

[0056] A rolling shutter CMOS-TDI image sensing method with spacing space is proposed. If the pixel array captures continuous image data, the width of the spacing space is set as the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression:

[0057]

[0058] Multiple integrators corresponding to each pixel array are used to integrate pixel data from the same location in the scene that are related to the pixel array in adjacent image frames. Then, the memory is used to integrate the first integrated pixel data related to the first pixel array and the second integrated pixel data related to the second pixel array, thereby doubling the number of integrations. The memory is a frame buffer used to first record the first integrated pixel data output by the first pixel array, and then add the second integrated pixel data output by the second pixel array to the already recorded first integrated pixel data, and vice versa. The memory will not output the integrated pixel data until the pixel data has been integrated a predetermined number of times.

[0059] The frame period T is greater than the sum of the line exposure times of all pixel rows of the first or second pixel array using RS exposure. The frame period is determined based on the scene brightness and the sensitivity of the first or second pixel array, and the moving speed of the CMOS-TDI image sensor or object is equal to the pixel height W divided by the frame period T. During the time difference between the frame period T and the sum of the line exposure times, the CMOS-TDI image sensor enters sleep mode.

[0060] If the pixel array captures non-continuous image data, the width of the interval space is set to the pixel width plus the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression:

[0061]

[0062] Each pixel array captures non-continuous image data, and after each pixel array outputs one frame, it skips one frame. For example, integrator 1 is used to integrate the first pixel data in the first and second image frames, which are related to the first pixel array and correspond to the same position in the scene, where there is an image frame between the first and second image frames. Integrator 2 is used to integrate the second pixel data in the third and fourth image frames, which are related to the second pixel array and correspond to the same position in the scene, where there is an image frame between the third and fourth image frames. There is a CT between the first and third image frames and between the second and fourth image frames. The CT is equal to the distance between corresponding pixels in the first and second pixel arrays divided by the moving speed of the CMOS-TDI image sensor or the object.

[0063] The first pixel in the first image frame used to sense the first pixel data at the same position of the object, and the second pixel in the second image frame used to sense the first pixel data at the same position of the object, are two adjacent pixels in the same pixel column in the first pixel array. Each integrator 1 does not integrate the first pixel data of the first pixel and the second pixel at the same position within one image frame period between the first image frame and the second image frame.

[0064] Similarly, the third pixel in the third image frame used to sense the second pixel data of the same position of the object, and the fourth pixel in the fourth image frame used to sense the second pixel data of the same position of the scene, are two adjacent pixels in the same pixel column in the second pixel array. Each integrator 2 does not integrate the second pixel data of the third pixel and the fourth pixel at the same position within one image frame period between the third image frame and the fourth image frame.

[0065] The advantages and beneficial effects of this invention are as follows:

[0066] (1) Significantly improves image quality and eliminates motion distortion. By setting a physical "interval space" between pixels that is precisely calculated based on pixel width, line time difference and frame period, spatial compensation for time delay is used to ensure that different pixel rows can be precisely aligned with the same scene point at their respective exposure times, thereby eliminating distortion and obtaining a clear image.

[0067] (2) The number of signal integrations is increased several times, which greatly enhances the signal-to-noise ratio. By arranging two or more independently operable pixel arrays along the moving direction, they scan the same scene area one after another and accumulate their respective integration results again, thereby increasing the total number of integrations several times without sacrificing the running speed, and further improving the SNR of the image.

[0068] (3) Enhance the sensor's operational flexibility and environmental adaptability. A more flexible sensor operation mode is provided. By adjusting the line time difference of RS within a fixed physical interval space, the sensor can select the optimal operating mode according to the ambient light or noise level without changing the hardware structure, thus enhancing the sensor's environmental adaptability.

[0069] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A rolling shutter CMOS-TDI image sensor with spaced intervals, characterized in that: It includes two pixel arrays, one above the other, which are connected to a control circuit. The pixels of the pixel array are connected to a readout circuit. The readout circuit has an integrator inside and is connected to a memory. The memory is arranged inside or outside the readout circuit. The circuit structure of the pixel includes a clamping photodiode and a sampling capacitor C. S Transmission transistor M TX Reset transistor M rst Source follower M sf and source follower M sel ; The transmission transistor M TX The source of the transistor is connected to the cathode of the clamping photodiode, the anode of the clamping photodiode is grounded, and the transmission transistor M... TX The gate is connected to the control signal line Sgt, and the transmission transistor M TX The drain is connected to the sampling capacitor C. S One end, reset transistor M rst The source and source follower M sf The gate of the sampling capacitor C S The other end is grounded, and the reset transistor M rst The gate is connected to the control signal line Srst, and the reset transistor M rst The drain of the source follower M is connected to the power supply VDD. sf The drain of the source follower M is connected to the power supply VDD. sf The source connection to the source follower M sel The drain of the source follower M sel The gate connection control signal line Srs, the source follower M sel The source is connected to the output signal PXO.

2. The rolling shutter CMOS-TDI image sensor with spaced intervals according to claim 1, characterized in that: The two pixel arrays are arranged along the target movement direction, and each column of pixels in both arrays is aligned.

3. A rolling shutter CMOS-TDI image sensor with a spaced interval as described in claim 1, characterized in that: There is a space between adjacent pixels in a pixel column.

4. A rolling shutter CMOS-TDI image sensor with a spaced interval as described in claim 1, characterized in that: The control signal lines Srst, Sgt, and Srs between pixel columns are all connected in series with a buffer.

5. A rolling shutter CMOS-TDI image sensing method with spacing, applied to a rolling shutter CMOS-TDI image sensor with spacing as described in any one of claims 1-4, characterized in that: Includes the following steps: If the pixel array captures continuous image data, the width of the interval space is set to the pixel width multiplied by the ratio of the line time difference to the frame period. If the pixel array captures non-continuous image data, the width of the interval space is set to the pixel width plus the pixel width multiplied by the ratio of the line time difference to the frame period. The pixel array captures continuous image data, including: A1. Determine the frame period based on the scene's brightness and the sensitivity of the pixel array; A2. The integrator integrates pixel data from the same location in the scene that are related to the pixel array in adjacent image frames; A3. The memory integrates the integrated pixel data of the two pixel arrays; A4. Enter sleep mode within the time difference between the frame period and the total line exposure time.

6. The rolling shutter CMOS-TDI image sensing method with spaced intervals according to claim 5, characterized in that: The width of the interval space is set as the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression: In the formula, W SS,C Where W is the interval space, t is the line time difference, and T is the frame period.

7. The rolling shutter CMOS-TDI image sensing method with spaced intervals according to claim 5, characterized in that: The pixel array captures non-continuous image data, including: The pixel array outputs one frame and then skips to the next.

8. The rolling shutter CMOS-TDI image sensing method with spaced intervals according to claim 7, characterized in that: The width of the interval space is set to the pixel width plus the pixel width multiplied by the ratio of the line time difference to the frame period, as shown in the following expression: In the formula, W SS,C Where W is the interval space, t is the line time difference, and T is the frame period.

Images