Digital pixel array sensor based on pixel-level ADC architecture and probabilistic image acquisition method thereof

Abstract

The invention discloses a digital pixel array sensor based on a pixel level ADC architecture and a probabilistic image acquisition method thereof, the sensor comprises a digital pixel array, each pixel unit of the digital pixel array comprises an analog-to-digital converter, the analog-to-digital converter is internally provided with a comparator and a latch, and the comparator is connected with the latch. The comparator is used for comparing the voltage on the integrating capacitor of the pixel unit with a preset reference voltage, and the latch is used for latching a comparison result of the comparator; the method comprises the following steps: determining a reference voltage sequence with the length of N based on a probabilistic sampling method; based on the reference voltage sequence, performing image acquisition by using a digital pixel array sensor to obtain N binary images; and reconstructing a grayscale image based on the binarized image. According to the invention, by adjusting the distribution of the reference voltage, finer analog-digital conversion can be selectively carried out on different bright and dark areas in a scene.

Description

Technical Field

[0001] The present invention relates to sensor image acquisition technology, and in particular to a digital pixel array sensor based on a pixel-level ADC architecture and a probabilistic image acquisition method thereof. Background Art

[0002] In CMOS sensors, ADC (analog-to-digital converter) is responsible for converting the analog signals generated by pixels into digital signals. During the image acquisition process, there are various analog-to-digital conversion schemes in CMOS sensors based on the pixel-level ADC architecture, such as multi-channel bit-serial ADC (MCBS ADC) and successive approximation register ADC (SAR ADC) that perform conversions in the voltage domain; Delta-Sigma ADC, pulse frequency modulation ADC (PFM ADC), pulse width modulation ADC (PWM ADC) that perform conversions in the time domain, and triple quantization that performs mixed conversions in the voltage and time domains.

[0003] However, when processing analog signals, these ADC designs cannot select a specific intensity range for conversion, but convert the entire intensity range, resulting in low sensing efficiency and high power consumption. Therefore, when acquiring images, it is necessary to dynamically adjust the sensor's response range according to the light intensity of the scene. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a digital pixel array sensor based on a pixel-level ADC architecture and a probabilistic image acquisition method thereof, which can selectively perform more refined analog-to-digital conversion on different bright and dark areas in a scene by adjusting the distribution of the reference voltage.

[0005] Technical solution: To achieve the above-mentioned purpose, the digital pixel array sensor based on the pixel-level ADC architecture described in the present invention includes a digital pixel array, each pixel unit of the digital pixel array includes an analog-to-digital converter, and the analog-to-digital converter has a built-in comparator and a latch. The comparator is used to compare the voltage on the pixel unit integration capacitor with a preset reference voltage, and the latch is used to latch the comparison result of the comparator.

[0006] Among them, the preset reference voltage is to select a probability distribution type, determine the probability distribution parameters according to the voltage variation range of the integral capacitor of the image acquisition scene, obtain a probability distribution curve, sample the probability distribution curve multiple times, and generate a reference voltage sequence containing multiple random numbers.

[0007] It also includes a row decoder, a row multiplexer array, a column decoder and a readout circuit. The row decoder is used to control the row selection of the pixel array. The row multiplexer array sends three control signals to the pixel unit row selected by the row decoder. The column decoder and the readout circuit are used to control the column selection of the pixel array and read out the digital signal of the selected pixel unit column.

[0008] Each row of pixel units in the digital pixel array shares a row control signal bus, each column of pixel units shares a column readout bus, the row control signal bus of each row is connected to the row multiplexer array, and the column readout bus of each column is connected to the column decoder and the readout circuit.

[0009] Among them, the pixel unit also includes a photosensitive element and a reset transistor connected to an integrating capacitor so that an external power supply can charge the integrating capacitor. The three control signals include a reset signal for controlling the on and off of the reset transistor, a comparison clock signal for triggering the operation of the comparator, and a row selection signal for activating a specific pixel unit row.

[0010] The probabilistic image acquisition method of the digital pixel array sensor described in the present invention comprises the following steps:

[0011] S1: Determine a reference voltage sequence of length N based on a probabilistic sampling method;

[0012] S2: Based on the reference voltage sequence, a digital pixel array sensor is used to acquire images to obtain N binary images;

[0013] S3: Reconstruct the grayscale image based on the binarized image.

[0014] The method for generating the reference voltage sequence of length N in S1 is: selecting a probability distribution type, and determining the probability distribution parameter values, including the mean μ and the variance σ, according to the voltage variation range of the integral capacitor in the image acquisition scene targeted by the digital pixel array sensor. 2 , thereby obtaining a probability distribution curve about the reference voltage, sampling the probability distribution curve N times, and obtaining a sequence containing N random numbers, that is, obtaining a reference voltage sequence with a length of N that satisfies the probability distribution curve.

[0015] The method for obtaining N binary images in S2 is:

[0016] (1) The RG, CLK, and SEL terminals of each row of pixel units in the row multiplexer array input are all low level, and the first reference voltage value of the reference voltage sequence is first applied to the comparator VIN terminal of each pixel unit in the digital pixel array;

[0017] (2) The row decoder selects a specific row, and the row multiplexer array inputs a high-level reset signal to the RG terminal of the pixel unit of the selected row to reset the integration capacitor to the initial state, and then inputs a low level to the RG terminal again;

[0018] (3) Digital pixel array with programmable exposure modes;

[0019] (4) The row multiplexer array inputs a high-level comparison clock signal to the CLK terminal of the pixel unit in the selected row, and the comparator starts to compare the voltage of the integral capacitor on the pixel unit with the first reference voltage value, and transmits the digital signal obtained by comparison to the latch for latching; after the comparison is completed, a low level is input to the CLK terminal again, and the column decoder and readout circuit start to read out the comparison results in parallel, thereby converting the signal collected by the entire pixel array into a binary image;

[0020] (5) Repeat the above processes (1) to (4) for the remaining reference voltages of the reference voltage sequence to complete the probabilistic image acquisition of the image acquisition scene, thereby obtaining N binary images.

[0021] Among them, the process of the column decoder and the readout circuit starting to read out the comparison results in column parallel is as follows: the row decoder first selects the top row of pixel units, and the row multiplexer array inputs a high-level row selection signal to the SEL terminal of the selected row pixel unit to prepare for the readout operation, and the column readout signals of the selected row are grouped by column, and the level of each group of column readout signals is pulled up in turn to perform the column readout operation, so as to realize the parallel reading of multiple columns of the first row, and transmit the compared digital signals to the chip through IO; the operation is repeated for the remaining rows until the comparison results of the entire pixel array are read out.

[0022] The method for reconstructing a grayscale image based on a binary image in S3 is: directly accumulating or bitwise accumulating pixel values ​​at the same position in N binary images to obtain a grayscale image with a grayscale range of 0-N.

[0023] Beneficial effects: The present invention has the following advantages: The present invention combines the advantages of the high dynamic range of analog-to-digital conversion based on the existing pixel-level ADC architecture with probabilistic sampling, and distributes the reference voltage in a selectable range by adjusting parameters such as the mean variance of the reference voltage of the probabilistic sampling, thereby performing selective analog-to-digital conversion on different bright and dark intervals in the scene intensity space. This method realizes an attention mechanism for image intensity at the sensor end, thereby performing high-dynamic range and high-speed perception of important intensity information in the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the structure of a digital pixel array sensor based on a pixel-level ADC architecture;

[0025] Figure 2 is a schematic diagram of a pixel unit structure;

[0026] Figure 3 is the Gaussian distribution curve;

[0027] Figure 4 A schematic diagram for sampling a Gaussian distribution curve;

[0028] Figure 5 It is a timing diagram of single exposure;

[0029] Figure 6 Flowchart of probabilistic image acquisition and image reconstruction. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below in conjunction with the embodiments and drawings.

[0031] like Figure 1 and 2 As shown, a digital pixel array sensor based on a pixel-level ADC architecture includes a digital pixel array, a row decoder, a row multiplexer array, a column decoder, and a readout circuit, wherein each pixel unit in the digital pixel array includes:

[0032] Photosensitive element: generates photogenerated charge under light;

[0033] Integral capacitor: connected to the photosensitive element, used to store the charge generated by the photosensitive element to form a voltage. It is also connected to the external power supply through the reset transistor.

[0034] Reset transistor: The reset transistor is usually a MOSFET, which is turned on when its signal control terminal RG is high, and the external power supply charges the integral capacitor to set the integral capacitor to a known initial state;

[0035] Pixel-level analog-to-digital converter ADC: It has a built-in comparator and latch. The comparator is used to compare the voltage on the pixel unit integration capacitor with a series of preset reference voltages, including the VIN terminal connected to the integration capacitor, the VREF terminal connected to the reference voltage, and the CLK terminal that controls the comparison of the comparator. If the voltage on the integration capacitor is higher than the reference voltage, the comparator outputs a high-level signal; if it is lower than the reference voltage, it outputs a low-level signal, thereby converting the analog voltage signal into a digital signal (high or low), and the comparison result of the comparator will be transmitted to the latch for latching; the output end of the comparator is provided with a SEL switch.

[0036] Each row of pixel units in the digital pixel array shares a row control signal bus, and each column of pixel units shares a column readout bus. The row multiplexer array is connected to a row decoder, and the row decoder is used to control the row selection of the pixel array. The row multiplexer array sends a control signal to the pixel unit row selected by the row decoder through the row control signal bus, including inputting a high and low level reset signal to the RG terminal (which can reset the integration capacitor to an initial state), inputting a high and low level comparison clock signal to the CLK terminal (which can compare the voltage on the integration capacitor with the reference voltage), and inputting a high and low level row selection signal to the SEL terminal (which can read out the output signal of the selected row through the column decoder and the readout circuit); the column readout bus of each column is connected to the column decoder and the readout circuit to read out the output signal of the pixel unit of the selected row.

[0037] Among them, the row decoder can not only select a certain row, but also select all rows, thereby realizing global exposure, and the column decoder and readout circuit perform column selection reading based on the selected row.

[0038] The probabilistic image acquisition method based on the digital pixel array sensor comprises the following steps:

[0039] S1: Select a probability distribution type (such as Gaussian distribution, random distribution, and uniform distribution), and determine the probability distribution parameter values ​​(such as mean μ and variance σ) according to the prior knowledge (voltage variation range of the integral capacitor) of the image acquisition scene (such as dark scenes with different brightness, bright scenes, and light-dark cross scenes) targeted by the digital pixel array sensor. 2 ), thereby obtaining a probability distribution curve about the reference voltage, sampling the probability distribution curve N times, and obtaining a sequence containing N random numbers, that is, obtaining a reference voltage sequence with a length of N that satisfies the probability distribution curve.

[0040] In this embodiment, Gaussian distribution is selected, and the length of the reference voltage sequence is 10. Figure 3 The reference voltage (V ref ), namely V ref ~Ν(μ,σ 2 ), where the horizontal axis represents the voltage value of the reference voltage and the vertical axis represents its corresponding probability of occurrence. At the same time, the digital pixel array sensor acquires images for dark scenes. During the acquisition process, the voltage variation range of the integral capacitor is between 3V and 5V, and the imaging accuracy is 10-bit.

[0041] The parameters of the Gaussian distribution, namely the mean μ and variance σ, are determined according to the prior of the dark scene. 2 , set the initial mean to 4.5 and the variance to 0.5, and obtain the distribution of the reference voltage. Sampling the distribution ten times, obtain a sequence of 10 random numbers that satisfies the above distribution, thereby completing the determination of the reference voltage sequence during sampling, such as Figure 4 shown.

[0042] S2: According to the reference voltage sequence, a digital pixel array sensor is used to acquire images. Taking a digital pixel array sensor with a 128×128 pixel unit array as an example, the digital pixel array performs global exposure and eight columns are read out in parallel. This solution also supports pixel arrays of other sizes, rolling exposure, selective exposure and other readout methods.

[0043] like Figure 5 As shown, the T0-T1 segment is the initial state, the RG terminal, CLK terminal, and SEL terminal of each row of pixel units input into the row multiplexer array are all low level, and the first reference voltage value of the reference voltage sequence is first taken and applied to the comparator VIN terminal of each pixel unit in the digital pixel array;

[0044] In the T1-T2 segment, after the reference voltage is applied, the row decoder selects all rows, and the row multiplexer array inputs a high-level reset signal to the RG end of the pixel units of all rows (the CLK end and the SEL end remain at a low level), resets the integration capacitor to the initial state, and then inputs a low level to the RG end again;

[0045] In the T2-T3 segment, the digital pixel array undergoes programmable global exposure. In this solution, in addition to global exposure, rolling shutter exposure and regional exposure can also be performed;

[0046] After the exposure is completed, the row multiplexer array inputs a high-level comparison clock signal to the CLK terminal of all rows of pixel units (the RG terminal and the SEL terminal remain at a low level), and the comparator starts to compare the voltage of the integral capacitor on the pixel unit with the first reference voltage value, and transmits the digital signal obtained by comparison to the latch for latching;

[0047] After the comparison is completed, a low level is input to the CLK terminal again, and the column decoder and the readout circuit begin to read out the comparison results in columns in parallel. The specific readout process is: the row decoder first selects the first row of pixel units, and the row multiplexer array inputs a high-level row selection signal to the SEL terminal of the first row of pixel units to prepare for the readout operation. The column readout signal of the first row is divided into 128 / 8=16 groups, and the level of each group of column readout signals is pulled high in turn to perform the column readout operation, so as to realize the parallel reading of eight columns of the first row, and transmit the compared digital signal out of the chip through the IO; the operation is repeated for the remaining 127 rows until the comparison results of the entire pixel array are read out, so that the data of the entire pixel array is converted into a binary image;

[0048] (6) Repeat the above process from exposure to readout for the remaining reference voltages in the reference voltage sequence to complete the probabilistic image acquisition of the image acquisition scene, thereby obtaining 10 binary images.

[0049] S3: Figure 6 As shown in the figure, the grayscale image is reconstructed based on the binary image. Taking direct accumulation reconstruction as an example, matrix addition is directly performed on 10 binary images, that is, the pixel values ​​at corresponding positions are directly added, so that a grayscale image with a grayscale range of 0-10 can be obtained, thereby realizing probabilistic and fine detection of the intensity information of dark areas in dark scenes.

Claims

1. A digital pixel array sensor based on a pixel-level ADC architecture, comprising a digital pixel array, characterized in that: Each pixel unit of the digital pixel array includes an analog-to-digital converter, which has a built-in comparator and a latch. The comparator is used to compare the voltage on the pixel unit integration capacitor with a preset reference voltage, and the latch is used to latch the comparison result of the comparator.

2. The digital pixel array sensor based on pixel-level ADC architecture according to claim 1, characterized in that: The preset reference voltage selects a probability distribution type, determines the probability distribution parameters according to the voltage variation range of the integrating capacitor in the image acquisition scene, obtains a probability distribution curve, samples the probability distribution curve multiple times, and generates a reference voltage sequence containing multiple random numbers.

3. The digital pixel array sensor based on pixel-level ADC architecture according to claim 1, characterized in that: It also includes a row decoder, a row multiplexer array, a column decoder and a readout circuit. The row decoder is used to control the row selection of the pixel array. The row multiplexer array sends three control signals to the pixel unit row selected by the row decoder. The column decoder and the readout circuit are used to control the column selection of the pixel array and read out the digital signal of the selected pixel unit column.

4. The digital pixel array sensor based on pixel-level ADC architecture according to claim 3, characterized in that: Each row of pixel units in the digital pixel array shares a row control signal bus, each column of pixel units shares a column readout bus, the row control signal bus of each row is connected to the row multiplexer array, and the column readout bus of each column is connected to the column decoder and the readout circuit.

5. The digital pixel array sensor based on pixel-level ADC architecture according to claim 3, characterized in that: The pixel unit also includes a photosensitive element and a reset transistor connected to an integrating capacitor so that an external power supply can charge the integrating capacitor. The three control signals include a reset signal for controlling the on and off of the reset transistor, a comparison clock signal for triggering the operation of the comparator, and a row selection signal for activating a specific pixel unit row.

6. A probabilistic image acquisition method based on the digital pixel array sensor according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1: Determine a reference voltage sequence of length N based on a probabilistic sampling method; S2: Based on the reference voltage sequence, a digital pixel array sensor is used to acquire images to obtain N binary images; S3: Reconstruct the grayscale image based on the binarized image.

7. The probabilistic image acquisition method according to claim 6, characterized in that: The method for generating the reference voltage sequence of length N in S1 is: selecting a probability distribution type, and determining the probability distribution parameter values, including the mean μ and the variance σ, according to the voltage variation range of the integral capacitor in the image acquisition scene targeted by the digital pixel array sensor. 2 , thereby obtaining a probability distribution curve about the reference voltage, sampling the probability distribution curve N times, and obtaining a sequence containing N random numbers, that is, obtaining a reference voltage sequence with a length of N that satisfies the probability distribution curve.

8. The probabilistic image acquisition method according to claim 6, characterized in that: The method for obtaining N binary images described in S2 is: (1) The RG, CLK, and SEL terminals of each row of pixel units in the row multiplexer array input are all low level, and the first reference voltage value of the reference voltage sequence is first applied to the comparator VIN terminal of each pixel unit in the digital pixel array; (2) The row decoder selects a specific row, and the row multiplexer array inputs a high-level reset signal to the RG terminal of the pixel unit of the selected row to reset the integration capacitor to the initial state, and then inputs a low level to the RG terminal again; (3) Digital pixel array with programmable exposure modes; (4) The row multiplexer array inputs a high-level comparison clock signal to the CLK terminal of the pixel unit in the selected row, and the comparator starts to compare the voltage of the integral capacitor on the pixel unit with the first reference voltage value, and transmits the digital signal obtained by comparison to the latch for latching; after the comparison is completed, a low level is input to the CLK terminal again, and the column decoder and readout circuit start to read out the comparison results in parallel, thereby converting the signal collected by the entire pixel array into a binary image; (5) Repeat the above processes (1) to (4) for the remaining reference voltages of the reference voltage sequence to complete the probabilistic image acquisition of the image acquisition scene, thereby obtaining N binary images.

9. The probabilistic image acquisition method according to claim 8, characterized in that: The process of the column decoder and the readout circuit starting to read out the comparison results in column parallel is as follows: the row decoder first selects the top row of pixel units, the row multiplexer array inputs a high-level row selection signal to the SEL terminal of the selected row pixel unit, and prepares for the readout operation, the column readout signals of the selected row are grouped by column, and the level of each group of column readout signals is raised in turn to perform the column readout operation, realizing multi-column parallel reading of the first row, and the compared digital signal is transmitted out of the chip through IO; the operation is repeated for the remaining rows until the comparison results of the entire pixel array are read out.

10. The probabilistic image acquisition method according to claim 6, characterized in that: The method for reconstructing a grayscale image based on a binary image in S3 is: directly accumulating or bitwise accumulating pixel values ​​at the same position in N binary images to obtain a grayscale image with a grayscale range of 0-N.

Images