Low-light-level image sensor pixel structure with event detection and gray scale quantization functions

Abstract

The invention relates to the technical field of pixel structures, and provides a low-light image sensor pixel structure with event detection and gray scale quantization functions, which comprises an N * N pixel array, the pixel array is arranged in an array according to an RGGB arrangement mode; carrying out charge domain binning on the N * N pixel array to form an M * M macro pixel array; each I < x > J macro pixel arrays form a Tile, and the N Tiles are subjected to voltage domain binning, so that a Tile array is formed. According to the invention, event information can be quickly responded under low light or weak light, and the function of outputting event information and light intensity information at the same time is realized; the pixel structure disclosed by the invention can be completely multiplexed, that is, one pixel structure can be used for CIS gray imaging and can also realize an EVS function to output event information.

Description

Technical Field

[0001] This invention relates to the field of pixel structure technology, and in particular to a pixel structure for a low-light image sensor with event detection and grayscale measurement functions. Background Technology

[0002] Event-based vision sensors (EVS) are bio-inspired sensors. Unlike traditional grayscale cameras (CMOS Image Sensors, CIS), they do not capture images at a fixed rate. Instead, they measure the brightness changes of each pixel and output an event stream containing information about the timing, location, and polarity of these pixel brightness changes. Event cameras offer advantages such as low power consumption, low latency, and high dynamic range, and are widely used in mobile robotics, augmented reality and virtual reality (AR / VR), and video games.

[0003] With the rapid development of CMOS image sensors, many applications require the combination of EVS sensor and CIS information to obtain absolute measurements of light intensity. However, the dual-sensor configuration of event cameras and traditional cameras has some drawbacks, such as parallax errors caused by camera combination, synchronization complexity, and increased costs, such as the need for two pairs of lenses and packaging.

[0004] Current research in this area includes the hybrid EVS / CIS sensor developed by Professor Tobi Delbruck's team at ETH Zurich in 2014. This sensor utilizes photocurrent to support the continuous front-end circuitry of the EVS function, and then integrates this photocurrent onto a sampling capacitor for the CIS function. This method is not easily scalable because the CIS pixel pitch is limited by the EVS pixel pitch; furthermore, while this work reported low latency in the image sensor under strong light (1 k lux), it did not mention low-light performance.

[0005] In 2023, Sony developed an event vision sensor with a 2.97µm pixel pitch using back-illuminated (BSI) stacking technology and front-end circuit sharing. This pixel structure, by using both traditional 4T pixels and EVS pixel structures, simultaneously enables event detection and intensity acquisition, but does not achieve pixel structure reuse. However, the report did not mention the event response sensitivity of the event vision sensor under low light conditions.

[0006] In the same year, OmniVision launched a 3D stacked hybrid image sensor, which alleviated the challenges of synchronization and parallax, requiring only one package and lens, thus reducing space and cost. However, according to reports on the workings of this image sensor, its event camera can only respond to event information quickly under high or ultra-high light conditions, while it takes a long time to respond to changes in light intensity under low or dim light conditions. Therefore, in low light conditions, there are problems with no event response or slow response (milliseconds). Summary of the Invention

[0007] This invention primarily addresses the technical problem of slow response of existing event cameras to changes in light intensity under low light conditions. It proposes a pixel structure for a low-light image sensor with event detection and grayscale quantization functions, which can respond to event information quickly under low or weak light conditions and simultaneously output event information and light intensity information. The pixel structure of this invention can be fully reused, meaning that one pixel structure can be used for both CIS grayscale imaging and EVS functions to output event information.

[0008] This invention provides a pixel structure for a low-light image sensor with event detection and grayscale measurement functions, comprising: an N x N pixel array;

[0009] The pixel array is arranged in an RGGB pattern.

[0010] An N x N pixel array is subjected to charge domain binning to form an M x M macro pixel array;

[0011] A single macropixel includes a first clamping photodiode PPD1, a second clamping photodiode PPD2, a third clamping photodiode PPD3, a fourth clamping photodiode PPD4, a first TG transistor, a second TG transistor, a third TG transistor, a fourth TG transistor, an RST transistor, an SF transistor, and a SEL transistor. The cathode of each clamping photodiode PPD is connected to the source of the corresponding TG transistor, and the drains of each TG transistor are connected together for charge domain binning. The voltage signal after charge domain binning is input to the gate of the SF transistor and output to the drain of the SEL transistor. The source of the SEL transistor is connected to the Col_bus column bus.

[0012] Each I x J macropixel array constitutes a Tile, and voltage domain binning is performed on N Tiles to form a Tile array.

[0013] Preferably, the macro-pixel array operation process includes two stages: a reset stage and a readout stage.

[0014] Macropixel array operation process: At time t0, the RST transistor and multiple TG transistors are turned on simultaneously to reset the macropixel, clearing the charge on multiple PPDs; subsequently, the RST and TG transistors are turned off, and the pixel is exposed for a period of time; at time t1, the RST transistor is turned on, and multiple TG transistors remain off, at which time the pixel is reset again to clear the charge of the storage node FD point, and the signal is amplified by the SF transistor and the pixel reset signal Vrst voltage value is read at t1-t2; at time t3, the RST transistor is turned off, and multiple TG transistors are turned on simultaneously, the charge is transferred through the transmission TG transistor, and the signal is amplified by the SF transistor and the pixel Vsig voltage value is read at t3-t4; during the reading of the Vrst and Vsig signals, the pixel's selection transistor SEL remains on to enable the reading of the pixel signal.

[0015] Correspondingly, the present invention also provides a photosensitive module, including: a macropixel array, an integrator, a storage unit, a comparator, and a logic unit;

[0016] The storage unit includes: a first switch S1, a second switch S2, a third switch S3, a fourth switch S4, a first storage capacitor C1, and a second storage capacitor C2; wherein, the output of the macro-pixel array is connected to an integrator for integration, the result is input into the storage unit for storage, and is connected to a comparator and a logic unit for logical comparison, synchronously outputting whether there is light change information.

[0017] The macro-pixel array can be an N x N pixel array that performs charge domain binning as provided in any embodiment of the present invention, or a pixel array that first performs charge domain binning on N x N pixels and then performs voltage domain binning on multiple macro pixels.

[0018] Preferably, the macro-pixel array is an N x N pixel array that performs charge domain binning;

[0019] The working process of the photosensitive module is as follows: When the pixel is in EVS event detection mode, the control signals of all pixels in the N x N pixel array that is performing charge domain binning are synchronous signals, that is, all pixels are controlled at the same time. After an exposure time T, the gating tubes of all pixels are turned on at the same time, and the output signals of all pixels are on a Col_bus column bus. The output signals are amplified by an integrator and stored in the first storage capacitor C1.

[0020] Repeat the above operation to store the signal obtained after the next exposure time into the second storage capacitor C2. At this time, connect the signals obtained at the two exposure times to the two ends of the comparator respectively, compare the signals obtained from the two exposures, and output the comparison result. In this way, the light intensity change information can be obtained.

[0021] The operating mode for the next exposure time of the N x N pixel array undergoing charge domain binning is determined based on the comparison result; if the output is 1, it switches to CIS operation and performs grayscale quantization operation. At this time, the pixels are sampled in units of the N x N pixel array undergoing charge domain binning; if the output is 0, it continues to operate in EVS event detection mode.

[0022] Preferably, the macro-pixel array is a pixel array that first performs charge domain binning and then voltage domain binning on N x N pixels;

[0023] The working process of the photosensitive module is as follows: When the pixel is in EVS event detection mode, the pixel is sampled in units of Tile. The control signals of all pixels in Tile are synchronous signals, that is, all pixels are controlled at the same time. After an exposure time T, the gating tubes of all pixels are turned on at the same time, and the output signals of all pixels are voltage domain binning on a Col_bus column bus. The output signals are amplified by the integrator and stored in the first storage capacitor C1.

[0024] Repeat the above operation to store the signal obtained after the next exposure time into the second storage capacitor C2. At this time, connect the signals obtained at the two exposure times to the two ends of the comparator respectively, compare the signals obtained from the two exposures, and output the comparison result. In this way, the light intensity change information can be obtained.

[0025] Simultaneously, the operating mode for the next exposure time of the tile is determined based on the comparison result. If the output is 1, it switches to CIS mode to perform grayscale quantization, at which point the pixel samples the signal in micro-pixel units. If the output is 0, it continues to operate in EVS event detection mode. The two storage capacitors enable a pipelined operation for comparing event information, allowing continuous comparison of light intensity information generated within two exposure times.

[0026] Preferably, during the nth frame exposure time, the Integrator high level indicates that the integrator is resetting; then, at time t0, the Integrator low level indicates that the integrator has started integration; after integration for a period of time, the first switch S1 closes and then opens, storing the pixel signal during the nth frame exposure time into the first storage capacitor C1; similarly, during the (n+1)th frame exposure time, the integrator is resetting; at time t2, the integrator starts integration again; after integration for a period of time, the second switch S2 closes and then opens, storing the pixel signal during the (n+1)th frame exposure time into the second storage capacitor C2; subsequently, at time t4, the third switch S3 and the fourth switch S4 close simultaneously, inputting the signals stored in the first storage capacitor C1 and the second storage capacitor C2 respectively into the comparator for comparison, and performing logic operations on the comparison result to output a 1-bit signal.

[0027] Preferably, when the pixel starts working, it first acquires the image information of the nth frame and stores it in the first storage capacitor C1, then acquires the image information of the (n+1)th frame and stores it in the second storage capacitor C2; then it compares the acquired image information of the previous frame with the image information of the next frame and performs logical operations to obtain a 1-bit output signal. If the output is 0, it means that there is no obvious change in light intensity information between the two frames, and the pixel works in event detection mode and continues to acquire the image information of the next frame; if the output is 1, it means that there is a significant change in light intensity information between the two frames, and at this time, the pixel's working mode switches from event detection mode to grayscale quantization mode, and the pixel signal is output; at the same time, it can also be selected whether to output event information. If the light intensity information becomes brighter, the comparator output is 1; if the light intensity information becomes darker, the comparator output is -1.

[0028] This invention provides a pixel structure for a low-light image sensor with event detection and grayscale measurement functions. While outputting light intensity and event information, it also offers a solution to the problem of slow light response in low-light event cameras, filling a gap in this research field. Compared with existing technologies, it has the following advantages:

[0029] 1. Pixel design structures based on charge domain binning can improve the signal-to-noise ratio (SNR) of image sensors under low light conditions. Charge domain binning occurs within the pixel itself, before the charge is converted into a voltage signal by the SF6 (signal generator). Since noise in image sensors mainly occurs in the SF6 and subsequent readout circuitry, charge domain binning can achieve signal superposition without increasing readout noise, thereby improving the SNR of the image sensor. Furthermore, the binning of signal charge is equivalent to expanding the photosensitive area of ​​the image sensor pixel, thus improving its photosensitivity.

[0030] 2. A pixel design structure based on prior charge domain binning followed by voltage domain binning can significantly improve event response sensitivity. In low-light conditions, weak light signals are easily affected by readout noise, and the light signal may even be overwhelmed by noise, leading to image information loss. Charge and voltage domain binning can greatly improve the image signal-to-noise ratio. By accumulating and increasing the signal charge before readout, the risk of being overwhelmed by readout noise is reduced, allowing for the acquisition of more image information and thus improving the response sensitivity of event detection.

[0031] 3. This invention uses a CIS (CMOS Image Sensor) approach to implement EVS (Electronic Display Screen) event detection and pixel structure reuse. By employing a CIS pixel structure to implement the EVS event detection function, it can output both event information and grayscale information, compensating for the EVS's inability to output light intensity information. Simultaneously, in the EVS event detection mode, it can improve event response sensitivity, providing a solution to the problem of slow or no event response speed in low light conditions. Furthermore, the novel pixel structure proposed in this invention can fully realize pixel structure reuse, significantly saving pixel area compared to designing two separate pixel structures with different functions. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of an N x N pixel array provided by the present invention;

[0033] Figure 2 This is a schematic diagram of the macro-pixel array provided by the present invention;

[0034] Figure 3 This is a schematic diagram of the Tile pixel array provided by the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of a single macropixel provided by the present invention;

[0036] Figure 5 This is the working timing diagram of the macropixel provided by the present invention;

[0037] Figure 6 This is a schematic diagram of the structure of the photosensitive unit provided by the present invention. Figure 1 ;

[0038] Figure 7 This is a schematic diagram of the structure of the photosensitive unit provided by the present invention. Figure 2 ;

[0039] Figure 8 This is a timing diagram of the photosensitive unit provided by the present invention;

[0040] Figure 9 This is a flowchart of the operation of the photosensitive unit provided by the present invention. Detailed Implementation

[0041] To make the technical problems solved by this invention, the technical solutions adopted, and the technical effects achieved clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings, not all of them.

[0042] Example 1

[0043] like Figure 1 As shown, the low-light image sensor pixel structure with event detection and grayscale quantization functions provided in this embodiment of the invention includes: an N x N pixel array; the pixel array is arranged in an RGGB pattern.

[0044] Each 2x2 pixel array constitutes a macro-pixel that undergoes charge domain binning. For example... Figure 2 As shown, an N x N pixel array undergoes charge domain binning to form an M x M macropixel array.

[0045] Charge domain binning (a technique for image readout by merging the charges of adjacent pixels) offers several advantages. First, the pixel binning operation occurs within the pixel itself, before the charge is converted into a voltage signal by the SF6 (Source Follower) transistor. Since noise in image sensors primarily occurs in the SF6 transistor and subsequent readout circuitry, charge domain binning can achieve signal superposition without increasing readout noise, thereby improving the signal-to-noise ratio of the image sensor. Furthermore, the binning of signal charges is equivalent to expanding the photosensitive area of ​​the image sensor pixels, thus enhancing their photosensitivity.

[0046] Each I x J macro-pixel array forms a Tile (pixel block), and voltage-domain binning is performed on N Tiles to form a structure like... Figure 3 The tile array shown. Voltage domain binning has the following advantages: since pixel binning in the voltage domain is performed after the signal charge has been amplified, it can reduce the amount of output data through pixel binning, thereby improving data transmission efficiency.

[0047] like Figure 4The diagram shows the structure of a single macropixel after charge domain binning of pixels arranged in a 2x2 RGGB array. A single macropixel includes a first clamping photodiode (PPD) 1, a second clamping photodiode (PPD2), a third clamping photodiode (PPD3), a fourth clamping photodiode (PPD4), a first TG (Transfer Gate) transistor (TG1), a second TG transistor (TG2), a third TG transistor (TG3), a fourth TG transistor (TG4), an RST (Reset) transistor, an SF (Source Follower) transistor, and a SEL (Select) transistor. In this configuration, the cathodes of each clamping photodiode (PPD) are connected to the source terminals of the corresponding TG transistors (the cathode of the first clamping photodiode (PPD1) is connected to the source terminal of the first TG transistor, the cathode of the second clamping photodiode (PPD2) is connected to the source terminal of the second TG transistor, the cathode of the third clamping photodiode (PPD3) is connected to the source terminal of the third TG transistor, and the cathode of the fourth clamping photodiode (PPD4) is connected to the source terminal of the fourth TG transistor). The drain terminals of each TG transistor (first TG transistor, second TG transistor, third TG transistor, and fourth TG transistor) are connected together for charge domain binning. The voltage signal after charge domain binning is input to the gate of the SF transistor and output to the drain terminal of the SEL transistor. The source terminal of the SEL transistor is connected to the col_bus column bus.

[0048] The functions of each MOSFET are as follows: the clamping photodiode (PPD) is used to receive light and convert photons into charges for information acquisition; the TG transistor is the charge transfer gate, which controls charge transfer by providing an enable signal to the gate; the RST transistor is the pixel reset transistor, used to reset the pixel; the SF transistor is the source follower, which converts current into a voltage signal, amplifies it, and transmits it to the next stage; the SEL transistor is the pixel switch transistor, and when the SEL transistor is turned on, the signal is transmitted to the next stage through the Col_bus column bus.

[0049] like Figure 5The diagram shows the timing sequence of the macropixel array. The macropixel array operation consists of two phases: the Reset Phase and the Read Phase. The specific operation is as follows: At time t0, the RST transistor and multiple TG transistors are simultaneously turned on to reset the macropixel, clearing the charge on multiple PPDs. Subsequently, the RST transistor and TG transistors are turned off, and the pixel undergoes exposure for a period of time. At time t1, the RST transistor is turned on, while the multiple TG transistors remain off. At this time, the pixel is reset again, clearing the charge on the storage node FD (Floating Diffusion). The signal is amplified by the SF transistor, and the reset signal Vrst voltage value of the pixel is read at t1-t2 (①). At time t3, the RST transistor is turned off, and multiple TG transistors are simultaneously turned on. The charge is transferred through the transmission TG transistor, and the signal is amplified by the SF transistor, and the Vsig voltage value of the pixel is read at t3-t4 (②). During the reading of Vrst and Vsig signals, the pixel's gate transistor SEL remains on to enable the reading of the pixel signal.

[0050] Example 2

[0051] Based on the image sensor pixel structure of Embodiment 1, this invention provides a photosensitive module, including: a macro-pixel array, an integrator, a storage unit, and a comparator and logic unit;

[0052] The storage unit includes: a first switch S1, a second switch S2, a third switch S3, a fourth switch S4, a first storage capacitor C1, and a second storage capacitor C2. The output of the macro-pixel array (i.e., the output after the SEL transistor is selected) is connected to an integrator for integration. The result is input into the storage unit for storage and then connected to a comparator and logic unit for logical comparison, synchronously outputting information on whether there is a change in light.

[0053] The macro-pixel array can be an N x N pixel array that undergoes charge domain binning or a pixel array that first performs charge domain binning on N x N pixels and then performs voltage domain binning on multiple macro pixels.

[0054] like Figure 6 As shown, the macro-pixel array is an N x N pixel array with charge domain binning. This invention uses an N x N pixel array ( Figure 6 Taking a 2x2 pixel binning as an example, charge domain binning is performed to collect, transmit, and quantize light signals. This pixel structure has two different working modes: event detection mode and grayscale quantization mode. The specific working process is as follows: When the pixel is in EVS event detection mode, the control signals (TG transmission signal, RST reset signal, SEL gating signal) of all pixels in the N x N pixel array performing charge domain binning are synchronized, that is, all pixels are controlled simultaneously. After an exposure time T, the gating transistors (SELs) of all pixels are turned on simultaneously, and the output signals of all pixels are connected to a Col_bus column bus. The output signals are amplified by an integrator and stored in the first storage capacitor C1. The above operation is repeated, and the signal obtained after the next exposure time is stored in the second storage capacitor C2. At this time, the signals obtained from the two exposure times are connected to the two ends of a comparator respectively, and the signals obtained from the two exposures are compared and the comparison result is output. In this way, the light intensity change information can be obtained. Simultaneously, the operating mode for the next exposure time of the N x N pixel array undergoing charge domain binning is determined based on the comparison result. If the output is 1, it switches to CIS mode for grayscale quantization, at which point the pixels sample the signal in units of the N x N charge domain binning pixel array. If the output is 0, it continues to operate in EVS event detection mode. Two storage capacitors enable a pipelined operation for event information comparison, allowing continuous comparison of light intensity information generated within two exposure times. This charge domain pixel binning operation improves the signal-to-noise ratio of the image sensor under low light conditions, thereby enhancing event response sensitivity. Furthermore, the pixel structure proposed in this scheme achieves complete pixel reuse while possessing two different functions, saving pixel area.

[0055] like Figure 7 As shown, the macro-pixel array employs a pixel array that first undergoes charge domain binning followed by voltage domain binning for N x N pixels. This invention first uses an N x N pixel array... Figure 7Taking 2x2 pixel binning as an example, charge domain binning is performed, and the pixel after binning is treated as a macro-pixel unit. Then, voltage domain binning is performed using macro-pixels as units (this diagram uses 4x4 as an example). The macro-pixel units after charge domain binning and voltage domain binning are combined into a Tile. A Tile contains 1 x J macro-pixels. The overall pixel structure has two different working modes: event detection mode and grayscale quantization mode. The specific working process is as follows: When the pixel is in EVS event detection mode, the pixel is sampled in units of Tile. The control signals (TG transmission signal, RST reset signal, SEL gating signal) of all pixels in the Tile are synchronized, that is, all pixels are controlled simultaneously. After an exposure time T, the gating transistors (SEL) of all pixels are turned on simultaneously, and the output signals of all pixels are voltage domain binning on a Col_bus column bus. The output signals are amplified by an integrator and stored in the first storage capacitor C1. Repeat the above operation, storing the signal obtained after the next exposure time in the second storage capacitor C2. Then, connect the signals obtained from the two exposure times to the two ends of a comparator, compare the signals from the two exposures, and output the comparison result. This method allows for the acquisition of light intensity change information. Simultaneously, the operating mode of the Tile for the next exposure time is determined based on the comparison result. If the output is 1, it switches to CIS mode, performing grayscale quantization, and the pixel samples the signal in micro-pixel units. If the output is 0, it continues to operate in EVS event detection mode. The two storage capacitors enable a pipelined operation for event information comparison, allowing continuous comparison of light intensity information generated between two exposure times. This operation, which combines charge domain pixel binning with voltage domain pixel binning, improves event response sensitivity under low light conditions. Furthermore, the pixel structure proposed in this scheme achieves complete pixel reuse while possessing two different functions, saving pixel area.

[0056] The operating timing sequence of the photosensitive module of this invention is as follows: Figure 8As shown, during the nth frame exposure time, the Integrator high level indicates that the integrator is resetting; then, at time t0, the Integrator low level indicates that the integrator has started integration; after integration for a period of time, the first switch S1 closes and then opens, storing the pixel signal during the nth frame exposure time into the first storage capacitor C1; similarly, during the (n+1)th frame exposure time, the integrator is resetting; at time t2, the integrator starts integration again; after integration for a period of time, the second switch S2 closes and then opens, storing the pixel signal during the (n+1)th frame exposure time into the second storage capacitor C2; subsequently, at time t4, the third switch S3 and the fourth switch S4 close simultaneously, inputting the signals stored in the first storage capacitor C1 and the second storage capacitor C2 respectively into the comparator for comparison, and performing logic operations on the comparison result to output a 1-bit signal.

[0057] The working process of the photosensitive module of this invention is as follows: Figure 9 As shown, when a pixel starts working, it first acquires the image information of the nth frame and stores it in the first storage capacitor C1 (see...). Figure 6 Figure 7 Then, the image information of the (n+1)th frame is obtained and stored in the second storage capacitor C2 (see...). Figure 6 Figure 7 Then, the acquired image information from the previous frame is compared with the image information from the next frame, and logical operations are performed to obtain a 1-bit output signal. If the output is 0, it means that there is no obvious change in light intensity information between the two frames, and the pixel works in event detection mode to continue acquiring the image information of the next frame. If the output is 1, it means that there is a significant change in light intensity information between the two frames. At this time, the pixel's working mode is switched from event detection mode to grayscale quantization mode, and the pixel signal will be output. At the same time, it is also possible to choose whether to output event information. If the light intensity information becomes brighter, the comparator output is 1; if the light intensity information becomes darker, the comparator output is -1.

[0058] The following two implementation examples illustrate the invention in detail:

[0059] Implementation Case 1:

[0060] like Figure 6 The diagram shows a pixel design architecture based on charge domain binning, which mainly includes the following functional modules: NXN charge domain binning pixel array, integrator, storage unit, and comparator and logic unit.

[0061] This structure uses NxN pixels (this schematic uses 2x2 pixel binning as an example) for charge domain binning, and performs light signal acquisition, transmission, and quantization. This pixel structure has two different operating modes: event detection mode and grayscale quantization mode. The specific working process is as follows: When the pixel is in EVS event detection mode, the control signals (TG transmission signal, RST reset signal, SEL gating signal) of all pixels within the NxN pixel charge domain binning are synchronized, meaning all pixels are controlled simultaneously. After one exposure time T, the gating transistors (SELs) of all pixels are simultaneously turned on, and the output signals of all pixels are connected to a single Col_bus column bus. The output signals are then amplified by an integrator and stored in the first storage capacitor C1. The above operation is repeated, and the signal obtained after the next exposure time is stored in the second storage capacitor C2. The signals obtained from the two exposure times are then connected to the two ends of a comparator, and the signals from the two exposures are compared. The comparison result is output, and in this way, light intensity change information can be obtained. Simultaneously, the operating mode for the next exposure time is determined based on the comparison result of the NxN pixel charge domain binning. If the output is 1, it switches to CIS mode for grayscale quantization, at which point the pixel samples the signal in units of NxN pixel charge domain binning. If the output is 0, it continues to operate in EVS event detection mode. The two storage capacitors enable a pipelined operation for comparing event information, allowing continuous comparison of light intensity information generated within two exposure times. This charge domain pixel binning operation improves the signal-to-noise ratio of the image sensor under low light conditions, thereby enhancing event response sensitivity. Furthermore, the pixel structure proposed in this scheme achieves complete pixel reuse while possessing two different functions, saving pixel area.

[0062] Implementation Case 2:

[0063] like Figure 7 The diagram shows the pixel design architecture of this invention based on first charge domain binning and then voltage domain binning. It mainly includes the following functional modules: macro-pixel array, integrator, storage unit, and comparator and logic unit.

[0064] This structure first performs charge domain binning using NxN pixels (this schematic diagram uses 2x2 pixel binning as an example), and then treats the binned pixels as a macro-pixel unit. Next, it performs voltage domain binning using macro-pixels as units (this schematic diagram uses 4x4 as an example), and treats the binned macro-pixel units as a tile. A tile contains I x J macro-pixels. The overall pixel structure has two different working modes: event detection mode and grayscale quantization mode.

[0065] The specific working process is as follows:

[0066] When a pixel is in EVS event detection mode, it samples on a tile-by-tile basis. The control signals (TG, RST, SEL) of all pixels within a tile are synchronized, meaning all pixels are controlled simultaneously. After one exposure time T, all pixel gates (SELs) are simultaneously turned on, and the output signals of all pixels are voltage-domain binning on a Col_bus column bus. The output signals are then amplified by an integrator and stored in the first storage capacitor C1. This process is repeated, storing the signal obtained after the next exposure time in the second storage capacitor C2. The signals from the two exposure times are then connected to the two ends of a comparator for comparison, and the comparison result is output. This method provides information on light intensity changes. Simultaneously, the operating mode of the tile for the next exposure time is determined based on the comparison result. If the output is 1, it switches to CIS mode for grayscale quantization, and the pixel samples signals on a micro_pixel basis. If the output is 0, it continues to operate in EVS event detection mode. The two storage capacitors enable a pipelined operation for event information comparison, allowing continuous comparison of light intensity information generated over two exposure times. By employing this method of first binning pixels in the charge domain and then in the voltage domain, the event response sensitivity under low light conditions can be improved. Furthermore, the pixel structure proposed in this scheme achieves complete reuse of the pixel structure while possessing two different functions, thus saving pixel area.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions for some or all of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pixel structure for a low-light image sensor with event detection and grayscale measurement functions, characterized in that, include: An N x N pixel array; The pixel array is arranged in an RGGB pattern. An N x N pixel array is subjected to charge domain binning to form an M x M macro pixel array; A single macropixel includes a first clamping photodiode PPD1, a second clamping photodiode PPD2, a third clamping photodiode PPD3, a fourth clamping photodiode PPD4, a first TG transistor, a second TG transistor, a third TG transistor, a fourth TG transistor, an RST transistor, an SF transistor, and a SEL transistor. The cathode of each clamping photodiode PPD is connected to the source of the corresponding TG transistor, and the drains of each TG transistor are connected together for charge domain binning. The voltage signal after charge domain binning is input to the gate of the SF transistor and output to the drain of the SEL transistor. The source of the SEL transistor is connected to the Col_bus column bus. Each I x J macropixel array constitutes a Tile, and voltage domain binning is performed on N Tiles to form a Tile array.

2. The pixel structure of the low-light image sensor with event detection and grayscale measurement functions according to claim 1, characterized in that, The macropixel array operation process includes two stages: a reset stage and a readout stage. Macropixel array operation process: At time t0, the RST tube and multiple TG tubes are turned on simultaneously to reset the macropixel and clear the charge on multiple PPDs; then, the RST and TG tubes are turned off, and the pixel is exposed for a period of time. At time t1, the RST transistor is turned on, and multiple TG transistors remain off. At this time, the pixel is reset again to clear the charge of the storage node FD. After the signal is amplified by the SF transistor, the reset signal Vrst voltage value of the pixel is read at t1-t2. At time t3, the RST transistor is off, and multiple TG transistors are on simultaneously. Charge is transferred through the TG transistors, and the signal is amplified by the SF transistor. The Vsig voltage value of the pixel is read at time t3-t4. During the reading of the Vrst and Vsig signals, the pixel's gate transistor SEL remains on to enable the reading of the pixel signal.

3. A photosensitive module, characterized in that, include: Macro-pixel array, integrator, storage unit, comparator, logic unit; The storage unit includes: a first switch S1, a second switch S2, a third switch S3, a fourth switch S4, a first storage capacitor C1, and a second storage capacitor C2; wherein, the output of the macro-pixel array is connected to an integrator for integration, the result is input into the storage unit for storage, and is connected to a comparator and a logic unit for logical comparison, synchronously outputting whether there is light change information. The macro-pixel array is an N x N pixel array that undergoes charge domain binning as described in claim 1 or 2, or a pixel array that first performs charge domain binning on N x N pixels and then performs voltage domain binning on multiple macro-pixels.

4. The photosensitive module according to claim 3, characterized in that, When the macro-pixel array is an N x N pixel array that performs charge domain binning; The working process of the photosensitive module is as follows: When the pixel is in EVS event detection mode, the control signals of all pixels in the N x N pixel array that is performing charge domain binning are synchronous signals, that is, all pixels are controlled at the same time. After an exposure time T, the gating tubes of all pixels are turned on at the same time, and the output signals of all pixels are on a Col_bus column bus. The output signals are amplified by an integrator and stored in the first storage capacitor C1. Repeat the above operation to store the signal obtained after the next exposure time into the second storage capacitor C2. At this time, connect the signals obtained at the two exposure times to the two ends of the comparator respectively, compare the signals obtained from the two exposures, and output the comparison result. In this way, the light intensity change information can be obtained. The operating mode for the next exposure time of the N x N pixel array undergoing charge domain binning is determined based on the comparison results; if the output is 1, it switches to CIS operation to perform grayscale quantization, at which point the pixels are switched to sampling the signal in units of the N x N pixel array undergoing charge domain binning. If the output is 0, it will continue to work as an EVS event detection mode.

5. The photosensitive module according to claim 3, characterized in that, When the macro-pixel array adopts a pixel array that first performs charge domain binning and then voltage domain binning on N x N pixels; The working process of the photosensitive module is as follows: When the pixel is in EVS event detection mode, the pixel is sampled in units of Tile. The control signals of all pixels in Tile are synchronous signals, that is, all pixels are controlled at the same time. After an exposure time T, the gating tubes of all pixels are turned on at the same time, and the output signals of all pixels are voltage domain binning on a Col_bus column bus. The output signals are amplified by the integrator and stored in the first storage capacitor C1. Repeat the above operation to store the signal obtained after the next exposure time into the second storage capacitor C2. At this time, connect the signals obtained at the two exposure times to the two ends of the comparator respectively, compare the signals obtained from the two exposures, and output the comparison result. In this way, the light intensity change information can be obtained. Simultaneously, the operating mode for the next exposure time of the tile is determined based on the comparison result. If the output is 1, it switches to CIS mode to perform grayscale quantization, at which point the pixel samples the signal in micro-pixel units. If the output is 0, it continues to operate in EVS event detection mode. The two storage capacitors enable a pipelined operation for comparing event information, allowing continuous comparison of light intensity information generated within two exposure times.

6. The photosensitive module according to claim 4 or 5, characterized in that, The timing sequence of the photosensitive module is as follows: during the nth frame of exposure time, the Integrator high level indicates that the integrator is performing a reset operation; then, at time t0, the Integrator low level indicates that the integrator starts performing an integration operation. After integration for a period of time, the first switch S1 closes and then opens, storing the pixel signal of the nth frame exposure time into the first storage capacitor C1; similarly, during the n+1th frame exposure time, the integrator first performs a reset operation; at time t2, the integrator starts the integration operation again. After a period of integration, the second switch S2 closes and then opens, storing the pixel signal during the exposure time of the (n+1)th frame into the second storage capacitor C2; Subsequently, at time t4, the third switch S3 and the fourth switch S4 are closed simultaneously, inputting the signals stored in the first storage capacitor C1 and the second storage capacitor C2 respectively into the comparator for comparison, and performing Logic operations on the comparison result to output a 1-bit signal.

7. The photosensitive module according to claim 4 or 5, characterized in that, The working process of the photosensitive module is as follows: When the pixel starts working, it first acquires the image information of the nth frame and stores it in the first storage capacitor C1, then acquires the image information of the (n+1)th frame and stores it in the second storage capacitor C2; then it compares the acquired image information of the previous frame with the image information of the next frame and performs logical operations to obtain a 1-bit output signal. If the output is 0, it means that there is no obvious change in light intensity information between the two frames, and the pixel works in event detection mode and continues to acquire the image information of the next frame; if the output is 1, it means that there is a significant change in light intensity information between the two frames, and at this time, the pixel's working mode switches from event detection mode to grayscale quantization mode, and the pixel signal is output; at the same time, it is also possible to choose whether to output event information. If the light intensity information becomes brighter, the comparator output is 1; If the light intensity information becomes darker, the comparator output will be -1.

Images