Timing Control Method and Circuit for a TDI Sensor with Adjustable State

Abstract

The present invention discloses a method and a circuit for controlling the timing of a TDI type image sensor with adjustable states, belonging to the field of integrated circuit design. The circuit includes: an on-chip interface circuit, a state control circuit, a row control circuit, a column control circuit, a clock module, and off-chip timing control signals. It can interact with the interface through the off-chip timing control signals, and can achieve adjustable exposure, transfer, and reset state times, as well as dynamically adjustable TDI accumulation levels, effectively adapting to different requirements of TDI image sensors in different working environments. In addition, the present invention provides the working logic for different working modes of the TDI image sensor, such as self-detection, single picture, and continuous pictures, and designs configurable registers on the chip, enabling timely regulation of the chip through an FPGA.

Description

Technical Field

[0001] The present invention relates to a timing control method and circuit for a TDI type image sensor with adjustable state, belonging to the field of integrated circuit design. Background Art

[0002] Time Delay Integration (TDI) is a technology used to improve image quality and is applicable to imaging systems in high-speed motion scenarios. The core idea of TDI technology is to repeatedly expose and accumulate the pixels to improve the signal-to-noise ratio and dynamic range. Traditional image sensors can only capture the instantaneous light intensity information of an object during a single exposure. In a high-speed moving scenario, a longer exposure time is required to capture the complete light signal, but this will result in a blurred image.

[0003] In TDI, the pixel rows of the camera are divided into multiple parallel groups, and each group receives the light signal only at a specific time point. When the object or the camera moves, the next group receives the light signal, and this process is repeated continuously. The outputs of the pixel rows are concatenated to form the final image. The advantage of TDI is that it can accumulate the signals of multiple exposure cycles, thus greatly improving the signal-to-noise ratio and dynamic range of the image. Especially in low-light conditions or high-speed motion scenarios, TDI can effectively reduce noise and improve the details and clarity of the image. TDI technology is widely used in fields such as aerial photography, satellite remote sensing, and medical imaging. It can capture the details of high-speed moving objects and provide a clearer image expression, which is very important for image imaging systems that require high quality and high resolution.

[0004] In the traditional TDI method, the integration time of each pixel row is fixed. When implementing variable integration for a TDI type image sensor in the prior art, a single-stage register bank is used to control the length of the integration time, and it can only be adjusted within one or two orders of magnitude. In actual application scenarios, there may be extremely bright or extremely dark usage environments. Therefore, the photosensitive speed of the pixel matrix may be extremely fast or extremely slow, and the full well charge of the pixel is limited. There is a three-order-of-magnitude difference in exposure from the microsecond level to the millisecond level. Therefore, the adjustable range of the integration time is limited, which results in overexposure of the image in a relatively bright light environment during shooting and underexposure of the image in a relatively weak light environment, and no effective image information can be obtained. The prior art mainly uses a row control circuit to achieve the switching between different working modes of a TDI type image sensor. The existing row control circuit is implemented by a hierarchical decoder, and its advantage is that it has a smaller area and can perform global exposure, progressive rolling shutter exposure, etc. However, since the timing control is implemented outside the chip and only decoding is performed inside the chip, there is a disadvantage that it is easy to cause offsets between row control signals, resulting in multiple rows being enabled simultaneously during rolling shutter exposure, readout disorder, and incorrect image output. Summary of the Invention

[0005] To solve one or more of the above problems, the present application proposes a TDI-type image sensor timing control method and circuit, which support different working modes such as single picture, continuous pictures, and self-detection. It can achieve variable integration time, which means that in areas where more signal accumulation is required (such as low-brightness areas), the integration time of pixel rows can be extended, while in high-brightness areas, the integration time can be shortened, thus improving the dynamic range and detail capture ability of the image, and enhancing the adaptability of the image sensor in different working environments.

[0006] To ensure that the TDI-type image sensor can generate high-quality images in different working environments, the object of the present invention is to provide a TDI-type image sensor timing control method and circuit.

[0007] First, the present invention provides a TDI-type image sensor timing control circuit, including: an on-chip interface circuit, a state control circuit, a row control circuit, a column control circuit, a clock module, and off-chip timing control signals; the part of the pixel device inside the TDI-type image sensor that interacts with the imaging chip includes the gate terminal, drain terminal, and source terminal of the pixel device, and there are also configurable registers inside.

[0008] Among them, the off-chip timing control signals are used to give the enable signal, reset signal, clock signal of the image sensor, and perform data interaction with the image sensor through the interface.

[0009] The on-chip interface circuit is used to configure the required register values, and control the working mode of the image sensor and the time of exposure, reset, and transfer states by changing the register values.

[0010] The state control circuit is used to generate the enable signals required for the pixel array in different working modes of the row control circuit and the column control circuit, and configure the internal configurable registers with the interface circuit. The pixel reset time, pixel exposure time, pixel transfer time, TDI number of stages, working mode, and readout mode signals are configured by the internal registers.

[0011] The row control circuit is implemented by a state machine and is used to generate the row control timing control signals required for the pixel array, which act on the gate terminal of the pixel array.

[0012] The column control circuit is implemented by a state machine and is used to generate the column control timing control signals required for the pixel array, which act on the drain terminal and source terminal of the pixel array, and also provide the enable and timing control signals required for the analog-to-digital converter.

[0013] The clock module is used to provide the system main clock and divided clock, and also synchronize and release the asynchronous signals in the off-chip timing control signals with the internal clock edge.

[0014] In one embodiment, the off-chip timing control signal is generated by a Field-Programmable Gate Array (FPGA), a single-chip microcomputer, a Microcontroller Unit (MCU), or a host computer.

[0015] In one embodiment, the on-chip interface circuit is a slave, which interacts with the off-chip timing control source, and both the data bit width and the address bit width can be customized.

[0016] In one embodiment, the clock module includes a Phase-Locked Loop (PLL) and a clock frequency division circuit.

[0017] In one embodiment, the working modes include a self-check mode, a single-picture mode, and a continuous-picture mode; for the single-picture mode, the image sensor enters the standby mode after capturing one frame of image, for the continuous shooting mode, the image sensor shoots continuously until manually stopped, and the self-check mode is a debug mode that can check whether the on-chip pixel array and the peripheral circuits are working properly.

[0018] The present invention also provides a timing control method for a TDI-type image sensor, which adopts the above-mentioned TDI-type image sensor timing control circuit. The method is as follows:

[0019] Step 1: The clock module stably outputs a lock signal, and a counter is used for counting. A larger count value is taken to eliminate the influence of the clock signal oscillation on the chip. At this time, the output system main clock signal is a valid signal, and before the lock signal is valid, the system main clock signal is an invalid signal;

[0020] Step 2: After the counting ends, a full-chip reset is performed on the chip. All on-chip clock sources are derived from the system main clock or the divided clock. The same-source synchronous clock signal, the asynchronous reset signal, and the system clock are released synchronously;

[0021] Step 3: After the reset ends, the register configuration value is written through the interface module to determine the on-chip working mode and the time of each state such as exposure, reset, and transfer;

[0022] Step 4: The state control circuit controls the row control circuit to be enabled row by row in sequence. The column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signal generated by the pixel array into a digital signal, then converts the parallel digital signal into a serial digital signal through a parallel-to-serial conversion circuit, and then transmits it to the off-chip processing through the IO;

[0023] Step 5: Through the processed output image information, the on-chip exposure, reset, transfer and other state times can be adjusted frame by frame through the off-chip timing control signal, and the working modes of the self-check mode, the single-picture mode, and the continuous-picture mode can be controlled.

[0024] In one embodiment, in the self-detection mode, due to the IO speed limitation, to ensure that the test point data can be transmitted through the IO, the system main clock uses the slow clock of the PLL reference clock; compared with the normal working mode, the main frequency changes, and the internal frequency division relationships remain unchanged. The values of the output test point registers are used to determine whether each internal module is working properly.

[0025] In one embodiment, in the single-picture mode, the interface circuit configures the register for the single-picture mode; the state control circuit operates in the single-picture mode and controls the row control circuit to enable it row by row in sequence. Since the TDI image sensor uses time delay integration and different rows of pixels may be enabled simultaneously, the row control circuit has a one-to-one correspondence with the number of pixel rows; the column control circuit is a global circuit and is controlled by the row control circuit corresponding to the first row of pixels and the global state control circuit; the column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-to-serial conversion circuit, and then transmits them to the outside of the chip through the IO; in the single-picture mode, the end signal generated by the row control circuit of the last row is valid and is fed back to the state control circuit to control the pixel array to stop working after row-by-row exposure and readout; the output of one frame of image automatically stops.

[0026] In one embodiment, in the continuous-picture mode, its working logic is similar to that of the single-picture mode; however, in the continuous-picture mode, the end signal generated by the row control circuit of the last row is invalid. After the state control circuit controls the pixel array to perform row-by-row exposure and readout until the last row, it performs rolling shutter exposure and readout again, and the image sensor outputs pixel information frame by frame to achieve non-stop continuous image output.

[0027] The beneficial effects of the present invention are as follows:

[0028] (1) By means of the built-in configurable register, the present invention can achieve controllable TDI levels, greatly increasing the pixel exposure range, effectively improving the dynamic range of the image sensor, increasing the signal-to-noise ratio, and improving the image quality.

[0029] (2) The present invention uses hardware to implement the control circuit inside the chip and uses a state machine to replace the decoding circuit, reducing the control circuit path from the physical distance, which can effectively reduce the offset between different row control circuits. In addition, using a state machine to implement the row and column control circuits can effectively reduce the use of the chip's IO resources and effectively reduce the chip area in the design of ultra-large-scale chips.

[0030] (3) The time of each state inside the timing control circuit of the present invention, such as exposure, transfer, and readout states, is adjustable. According to different application environments (different light input amounts, extremely dark and extremely bright environments), and different shooting targets (still objects or fast-moving objects), an external control module (such as an FPGA or a single-chip microcomputer) can adjust the time of each state inside the chip according to the output image quality to improve the image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0032] Figure 1 is a schematic structural diagram of the timing control circuit of the TDI image sensor of the present invention;

[0033] Figure 2 is a timing control flowchart of the timing control circuit of the TDI image sensor of the present invention;

[0034] Figure 3 is a schematic diagram of the power-on timing of the timing control circuit of the TDI image sensor of the present invention;

[0035] Figure 4 is a schematic diagram of the row control logic of the timing control circuit of the TDI image sensor of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in detail with reference to the drawings.

[0037] Embodiment 1:

[0038] This embodiment provides a timing control circuit for a TDI image sensor, as Figure 1 shown, including: an SPI interface circuit, a state control circuit, a row control circuit, a column control circuit, a serial-to-parallel conversion circuit, a clock module, and off-chip timing control signals. Among them, all timing control circuits and modules have configurable registers; a pixel array, a readout circuit, and GPIO are not included in the TDI image sensor timing control circuit described herein and are supporting modules within the image sensor; GPIO is an IO unit of the standard cell library and is used for the interaction between on-chip signals and off-chip signals; the pixel array is an optoelectronic device that converts the captured optoelectronic signals into analog signals; the readout circuit is an analog circuit module that converts the analog signals into digital signals;

[0039] Among them, the off-chip timing control signal is connected to the interface module and the state control circuit, and is used to provide the timing control from chip power-on to normal operation and configure the on-chip registers.

[0040] The off-chip timing control signal is connected to the clock module and is used to provide the reference clock required by the PLL.

[0041] The clock module includes a PLL and a clock frequency division circuit, provides on-chip clock signals of various frequencies, and is connected to all on-chip modules.

[0042] The state control circuit is connected to the row control circuit and the column control circuit, and is used to provide the working mode of the image sensor.

[0043] The row control circuit is connected to the pixel array and acts on the pixel gate terminal to switch the pixel array gate terminal level so that the image sensor works in different states.

[0044] The row control circuit is connected to the column control circuit and is used to provide the control signal required by the pixel array drain terminal, and the row and column control circuits work synchronously.

[0045] The parallel-to-serial conversion circuit converts the parallel digital signal converted by the readout circuit into a serial signal and transmits it to the FPGA through the GPIO for acquisition and processing.

[0046] Embodiment 2:

[0047] As Figure 2 shown, this embodiment provides a timing control method for a TDI image sensor, which is implemented based on the timing control circuit of the TDI image sensor described in Embodiment 1, and includes:

[0048] Step 1: First, the PLL is powered on. After the PLL stably outputs the Locked signal, the counter is used for counting, and a larger count value is taken to eliminate the influence of the clock signal oscillation on the chip. The Locked signal is the PLL output signal and is used to indicate that the PLL has locked the output frequency. At this time, the output system main clock signal is a valid signal. Before the Locked signal is valid, the system main clock signal output by the PLL is an invalid signal. The power-on timing of the TDI image sensor is as Figure 3 shown.

[0049] Step 2: After the counting ends, the entire chip is reset. All on-chip clock sources come from the PLL or the PLL output frequency division signal, the same-source synchronous clock signal, and the asynchronous reset signal is asynchronously logically synchronized and released with the 100M clock edge.

[0050] Step 3: After the reset ends, the register configuration value is written through the interface module to determine the on-chip working mode and the time of each state such as exposure, reset, and transfer.

[0051] Step 4: The state control circuit controls the row control circuit to be enabled row by row in sequence. The column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-to-serial conversion circuit, and then transmits them to the off-chip processing through the IO.

[0052] Step 5: Through the processed output image information, the on-chip exposure, reset, transfer and other state times can be adjusted frame by frame through the off-chip timing control signals.

[0053] In the self-detection mode, due to the IO speed limitation, to ensure that the test point data can be transmitted out through the IO, the system main clock uses the slow clock of the PLL reference clock; compared with the normal working mode, the main frequency changes, and the internal frequency division relationships remain unchanged, and the output test point register values are used to determine whether each internal module is working properly.

[0054] In the single-picture mode, the interface circuit configures the register in the single-picture mode; the state control circuit works in the single-picture mode and controls the row control circuit to be enabled row by row in sequence; because the TDI image sensor uses time delay integration and different rows of pixels are enabled simultaneously, the row control circuit has a one-to-one correspondence with the number of pixel rows; the column control circuit is a global circuit and is controlled by the row control circuit corresponding to the first row of pixels and the global state control circuit; the column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-to-serial conversion circuit, and then transmits them to the off-chip through the IO; in the single-picture mode, the end signal generated by the row control circuit of the last row is valid and is fed back to the state control circuit to control the pixel array to stop working after row-by-row exposure and readout; the output of one frame of image automatically stops.

[0055] In the continuous-picture mode, its working logic is similar to that of the single-picture mode; but in the continuous-picture mode, the end signal generated by the row control circuit of the last row is invalid. After the state control circuit controls the pixel array to perform row-by-row exposure and readout to the last row, it performs rolling shutter exposure and readout again, and the image sensor outputs pixel information frame by frame to achieve continuous image output without interruption.

[0056] The timing control method of the present invention can perform information interaction with the interface through the off-chip timing control signals, can realize adjustable exposure, transfer, and reset state times and dynamically adjustable TDI accumulation levels, and can effectively adapt to the different requirements of TDI image sensors in different working environments.

[0057] Embodiment 3:

[0058] According to the timing control method provided in the second embodiment, a row timing control circuit based on a state machine is proposed. Due to the adoption of a column readout structure, the function of the column control circuit is to provide control signals for the pixel array and each analog module, enabling the controlled analog module enable signal to be regulated, enhancing the controllability of the on-chip analog circuit. The column control circuit uses a single multiplexed state machine for the entire chip.

[0059] Each row control circuit corresponds to a row state machine, and its control logic is determined by Figure 4 As shown, the switching between different working states (exposure, reset, transfer, readout) is controlled by the counting of a counter. Idle is the idle state, rst is the reset state, read is the readout state. The reason for dividing into tran1, tran2, tran3, tran4, tran5 is that different control levels are required for charge transfer between pixels. The reason for transfer read1, read2, read3, read4, read5 is that correlated double sampling (CDS) needs to be performed in the readout state, and there needs to be exposure read, reset read, and subtraction of exposure count values. The time required for different state jumps is obtained by counting the counter cnt. And the count values of each state can be configured by the interface circuit. The number of TDI levels can also be configured through the interface circuit. When working with the number of TDI levels being 0, it is a normal camera mode without cumulative integration.

[0060] The column control circuit is multiplexed for the entire chip. It works in cooperation with the row state machine and is designed to perform handshake communication with the row state machine. Its enable signal comes from the first row state machine corresponding to the pixel array. The column state machine provides the control timing required for the analog module. The pulse width of its control signal can also be configured by the interface circuit.

[0061] In the traditional solution, the control signals are all generated outside the chip. The chip only performs row and column decoding inside. The control signals generated outside pass through long traces on the PCB and the chip PAD. After each off-chip control signal enters the chip, there is a phase difference, which easily leads to disorder of the on-chip control signals (in progressive scan readout, multiple rows may be enabled, and in column readout, there may be slight phase disorder of high-parallel output signals). The mode adopted in this article places the control method inside the chip, and only an enable signal is provided outside the chip. Compared with the traditional solution, the control signals do not need to pass through long traces on the PCB and the chip PAD, which can greatly reduce the problem of control signal offset and effectively reduce the phase difference between row and column control signals.

[0062] Some steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk, etc.

[0063] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A TDI type image sensor timing control circuit, characterized in that, The circuit includes: an on-chip interface circuit, a state control circuit, a row control circuit, a column control circuit, a clock module, and off-chip timing control signals; the part where the pixel devices inside the TDI image sensor interact with the imaging chip includes the gate terminal, drain terminal, and source terminal of the pixel devices, and there are also configurable registers inside; Among them, the off-chip timing control signals are used to give the enable signal, reset signal, clock signal of the image sensor, and perform data interaction with the image sensor through the interface; The on-chip interface circuit is used to configure the required register values, and control the working mode of the image sensor and the time of exposure, reset, and transfer states by changing the register values; The state control circuit is used to generate the enable signals required by the pixel array in different working modes of the row control circuit and the column control circuit, and configure the internal configurable registers with the interface circuit. The pixel reset time, pixel exposure time, pixel transfer time, TDI level, working mode, and readout mode signal are configured by the internal registers; The row control circuit is implemented by a state machine and is used to generate the row control timing control signals required by the pixel array, which act on the gate terminal of the pixel array; The column control circuit is implemented by a state machine and is used to generate the column control timing control signals required by the pixel array, which act on the drain terminal and source terminal of the pixel array, and also provide the enable and timing control signals required by the analog-to-digital converter; The clock module is used to provide the system main clock and divided clock, and also synchronously release and process the asynchronous signals in the off-chip timing control signals with the internal clock edge; The working modes include a self-check mode, a single-picture mode, and a continuous-picture mode; the single-picture mode means that the image sensor enters the standby mode after shooting a frame of image; continuous shooting means that the image sensor shoots continuously until manually stopped; the self-check mode can check whether the on-chip pixel devices and the peripheral circuits are working properly.

2. The circuit according to claim 1, wherein The off-chip timing control signals are generated by an FPGA, a single-chip microcomputer, an MCU, and a host computer.

3. The circuit according to claim 1, characterized in that, The on-chip interface circuit is a slave and interacts with the off-chip timing control source for information.

4. The circuit according to claim 1, wherein, The clock module includes a phase-locked loop PLL and a frequency division circuit. The off-chip clock source generates a slow clock, which enters the chip through the IO and is used as the reference clock for the PLL input and the system clock when the image sensor works in the self-detection mode; If working in the normal mode, the PLL multiplies the input reference clock as the system main clock; The frequency division circuit then divides the main clock into clock signals of different frequencies to be used as the clock signals of different internal modules; The interface module is used to perform data interaction with the outside; the row control circuit is used to control the gate of the pixel device to enter different working states; the clocks of the state control circuit and the column control circuit are used to provide various control signals required by the readout circuit module, and their clock signals are all obtained by dividing the main clock.

5. A timing control method for a TDI type image sensor, characterized in that, Adopting the circuit according to any one of claims 1-4, the method includes the following steps: Step 1: The clock module stably outputs a lock signal, counts through a counter, and takes a larger count value to eliminate the influence of the clock signal oscillation on the chip. At this time, the output system main clock signal is a valid signal. Before the lock signal is valid, the system main clock signal is an invalid signal; Step 2: After the counting is completed, perform a full-chip reset on the chip. All on-chip clock sources are derived from the system main clock or the divided clock. The synchronous clock signal of the same source, the asynchronous reset signal, and the system clock are released synchronously. Step 3: After the reset is completed, write the register configuration value through the interface module to determine the on-chip working mode and the state times of exposure, reset, and transfer. Step 4: The state control circuit controls the row control circuit to enable row by row in sequence. The column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-serial conversion circuit, and then transmits them to the off-chip processing through the IO. Step 5: Through the processed output image information, the state times of exposure, reset, and transfer inside the chip can be adjusted frame by frame by the off-chip timing control signal, as well as the control of the working modes of the self-check mode, single-picture mode, and continuous-picture mode.

6. The control method according to claim 5, characterized in that In the self-detection mode, the system main clock uses the slow clock of the PLL reference clock; compared with the normal working mode, the main frequency changes, and the internal division ratios remain unchanged. The values of the test point registers are output to determine whether each internal module is working properly.

7. The control method according to claim 5, wherein In the single-picture mode, the interface circuit configures the register for the single-picture mode; the state control circuit works in the single-picture mode and controls the row control circuit to enable row by row in sequence; the TDI image sensor uses time-delay integration, and different rows of pixels are enabled simultaneously. The row control circuit has a one-to-one correspondence with the number of pixel rows; the column control circuit is a global circuit and is controlled by the row control circuit corresponding to the first row of pixels and the global state control circuit; the column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-serial conversion circuit, and then transmits them to the off-chip through the IO; in the single-picture mode, the end signal generated by the row control circuit of the last row is valid and is fed back to the state control circuit to control the pixel array to stop working after row-by-row exposure and readout; the output of one frame of image automatically stops.

8. The control method according to claim 5, wherein In the continuous-picture mode, the interface circuit configures the register for the continuous-picture mode; the state control circuit works in the continuous-picture mode and controls the row control circuit to enable row by row in sequence; the TDI image sensor uses time-delay integration, and different rows of pixels are enabled simultaneously. The row control circuit has a one-to-one correspondence with the number of pixel rows; the column control circuit is a global circuit and is controlled by the row control circuit corresponding to the first row of pixels and the global state control circuit; the column control circuit generates the timing control signals required by the readout circuit module. The readout circuit module converts the analog signals generated by the pixel array into digital signals, then converts the parallel digital signals into serial digital signals through the parallel-serial conversion circuit, and then transmits them to the off-chip through the IO; in the continuous-picture mode, the end signal generated by the row control circuit of the last row is invalid. After the state control circuit controls the pixel array to perform row-by-row exposure and readout to the last row, it performs rolling shutter exposure and readout again. The image sensor outputs pixel information frame by frame to achieve non-stop continuous image output.

Images