Pipelined ping-pong buffer readout circuit for event-type vision sensors
By designing a pipelined ping-pong buffer readout circuit, efficient readout of pixel data from event-driven vision sensors was achieved, improving readout rate and chip frame rate, and solving the problems of long readout time and low efficiency in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2023-12-27
- Publication Date
- 2026-05-19
AI Technical Summary
Existing methods for reading out event-based visual sensor data are not efficient enough, and the readout rate of event data is limited.
A pipelined ping-pong buffer readout circuit is adopted, including a pipelined ping-pong buffer system control module, a pipelined column gating module, a column sampling register module, an event encoding module, and an output module. By alternately reading the pixel data of two adjacent columns and processing them at different stages, parallel processing of three columns of pixel data is achieved.
The chip frame rate of the event-driven vision sensor has been increased by up to three times that of the traditional readout circuit, solving the problems of long readout time and low efficiency.
Smart Images
Figure CN117835087B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of readout circuits for event-type vision sensors, and more particularly to a pipelined ping-pong buffer readout circuit for event-type vision sensors. Background Technology
[0002] Event-Based Vision Sensors (EVS) are sensors that detect and output changes in ambient light intensity. Each pixel independently senses changes in light intensity and discretizes them as events of increasing and decreasing light intensity. Compared to traditional integrating CMOS sensors, they output less data, have a faster response time, and consume less power. Therefore, event-based vision sensors are widely used in high-speed, high-real-time, and high-dynamic-range applications.
[0003] The pixels of an event-driven vision sensor mainly consist of a photodiode, a logarithmic amplifier, a source follower, a switched-capacitor amplifier, and a comparator. The photodiode detects changes in light intensity and converts them into photocurrent. The logarithmic amplifier performs a logarithmic conversion between photocurrent and photovoltage. The source follower transmits the voltage signal from the logarithmic amplifier to the switched-capacitor amplifier. The switched-capacitor amplifier outputs the relative value of the voltage change and inputs the result to a signal increase detection comparator (ON comparator) and a signal decrease detection comparator (OFF comparator), respectively. The change in light intensity is compared with a preset threshold, and then the ON and OFF events are output. Subsequently, the switched-capacitor amplifier is reset.
[0004] When reading out the generated event data, event-based vision sensors generally involve the following steps: First, the column selection circuit selects a column that generated the event and transmits the events of that column to the column sampling register circuit; then, the encoding circuit performs AER encoding on these event data, and finally, the encoded data is output through the output interface circuit. After the above process is completed, the column selection circuit selects the next column of pixels to read the events of the next column of pixels, until the data of the entire pixel array is read. However, this data readout method is not efficient enough, and the event data readout rate is limited. Summary of the Invention
[0005] The technical problem to be solved by this invention is that the existing data readout methods for event-type vision sensors are not efficient enough and the readout rate of event data is limited. This invention provides a pipelined ping-pong buffer readout circuit for event-type vision sensors.
[0006] To address the aforementioned technical problems, embodiments of the present invention provide a pipelined ping-pong buffer readout circuit for an event-based visual sensor, comprising: a pipelined ping-pong buffer system control module, a pipelined column gating module, a column sampling register module, an event encoding module, and an output module;
[0007] The pipeline ping-pong buffer system control module is connected to the pipeline column gating module, column sampling register module, event encoding module, and output module respectively, and is used to coordinate the work of each module;
[0008] The pipeline column selection module is used to select a column of pixels in the pixel array of the event-type vision sensor and transmit a reset signal to the event-type vision sensor to reset the pixel array.
[0009] The column sampling register module includes a first column sampling register unit and a second column sampling register unit, which are used to alternately read the pixel column data of two adjacent columns selected by the pipeline column gating module;
[0010] The event encoding module is used to encode the data read by the column sampling register module;
[0011] The output module is used to output the pixel column encoded data;
[0012] The event encoding module encodes one column of pixel data while resetting that column; the column sampling register module reads the next column of pixel data; and the output interface module outputs the encoded pixel data from the previous column. This process simultaneously processes three different columns of pixel data at different stages.
[0013] Preferably, the pipeline ping-pong buffer system control module includes an OR gate, a first AND gate, a second AND gate, a first inverter, a second inverter, a first gating unit, and a second gating unit; the input terminals of the OR gate are respectively connected to the output terminals of the first AND gate and the second AND gate; the input terminals of the first AND gate and the second AND gate are both connected to the event encoding module and the output module; one end of the first inverter is connected to the input terminal of the OR gate, and the other end is connected to the first gating unit; the first gating unit is connected to the first column sampling register unit; one end of the second inverter is connected to the input terminal of the OR gate, and the other end is connected to the second gating unit; the second gating unit is connected to the second column sampling register unit.
[0014] Preferably, the input terminal of the first gate control unit is also connected to the input terminal of the OR gate and the pipeline selection module; the input terminal of the second gate control unit is also connected to the input terminal of the OR gate and the pipeline selection module.
[0015] Preferably, when the first gating unit is turned on, the second gating unit is closed, and the first column sampling register unit reads the pixel column data transmitted by the pipeline column gating module; when the second gating unit is turned on, the first gating unit is closed, and the second column sampling register unit reads the pixel column data transmitted by the pipeline column gating module.
[0016] Preferably, when the event encoding module and the output module complete their work, the first AND gate or the second AND gate is turned on, the OR gate is turned on, and the pipeline column gating module transmits the pixel column data to the first column sampling register unit or the second column sampling register unit through the first gating unit or the second gating unit.
[0017] Preferably, when the output module receives the encoded data of the first column of pixels, the second column sampling register unit completes the reading of the second column of pixel data and transmits it to the event encoding module for encoding; the first column sampling register unit begins to read the data of the third column of pixels; this process is repeated to achieve the output of data for the entire pixel array.
[0018] Preferably, while the event encoding module encodes a column of pixel data, the pipeline column gating module transmits a reset signal for that column of pixels to the event-type vision sensor.
[0019] Preferably, while the pipelined column gating module transmits a column of pixel reset signals to the event-type vision sensor, the pipelined column gating module selects the next column of pixel data in the event-type vision sensor, and the column sampling register unit in the idle state reads the next column of pixel data.
[0020] Preferably, the pipeline column gating module includes a displacement register unit; when the pipeline ping-pong buffer system control module transmits a signal to the pipeline column gating module, the displacement register unit will send a read signal to one column in the pixel array and send a reset signal to the next column in the pixel array.
[0021] Implementing the embodiments of the present invention has the following beneficial effects:
[0022] (1) In this embodiment of the invention, the control module of the pipelined ping-pong buffer system coordinates the operation of the column sampling register module, the event encoding module, and the output module on the pixel column data of different columns simultaneously; then, by using two sets of column sampling register units to alternately read the adjacent pixel column data, the two sets of pixel column data are read without interval, and the three sets of pixel column data are processed synchronously at different stages. Ultimately, this improves the chip frame rate of the event-type vision sensor and solves the problems of long readout time and low efficiency of the readout circuit of the event-type vision sensor. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a schematic diagram of the pipeline ping-pong buffer readout circuit structure for event-type vision sensors provided by the present invention;
[0025] Figure 2 This is a schematic diagram of the pipeline ping-pong buffer system control module in the pipeline ping-pong buffer readout circuit for event-type vision sensors provided by the present invention.
[0026] Figure 3 This is a schematic diagram of the pipeline column gating module structure in a pipeline ping-pong buffer readout circuit for an event-type vision sensor provided by the present invention;
[0027] Figure 4 This is a flowchart of the pipeline ping-pong buffer readout circuit for event-type vision sensors provided by the present invention.
[0028] Figure 5 This is a timing diagram of the pipeline ping-pong buffer readout circuit for event-type vision sensors provided by the present invention.
[0029] 10 - Pipeline ping-pong buffer system control module, 101 - OR gate, 102 - First AND gate, 103 - Second AND gate, 104 - First inverter, 105 - First gating unit, 106 - Second inverter, 107 - Second gating unit, 20 - Pipeline column gating module, 201 - Shift register unit, 30 - Column sampling register module, 301 - First column sampling register unit, 302 - Second column sampling register unit, 40 - Event encoding module, 50 - Output module, 600 - Event-type visual sensor pixel array. Detailed Implementation
[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] Please see Figure 1The first embodiment of the present invention provides a pipelined ping-pong buffer readout circuit for an event-driven vision sensor, comprising: a pipelined ping-pong buffer system control module 10, a pipelined column gating module 20, a column sampling register module 30, an event encoding module 40, and an output module 50. The pipelined ping-pong buffer system control module 10 coordinates the operation of each module. The pipelined column gating module 20 selects a column of pixels from the event-driven vision sensor pixel array 600 and transmits a reset signal to the event-driven vision sensor pixel array 600 to reset it. The column sampling register module 30 reads the pixel column data selected by the pipelined column gating module 20. The event encoding module 40 encodes the data read by the column sampling register module 30; the output module 50 outputs the encoded pixel column data.
[0032] The pipeline ping-pong buffer system control module 10 is connected to the pipeline column gating module 20, column sampling register module 30, event encoding module 40, and output module 50, and is used to coordinate the work of each module. The pipeline column gating module 20 is connected to the event-type visual sensor pixel array 600. The output of the pipeline column gating module 20 is connected to the input of the column sampling register module 30. The input of the event encoding module 40 is connected to the output of the column sampling register module 30. The input of the output module 50 is connected to the event encoding module 40, and its output is connected to the outside world to output the final data.
[0033] The column sampling register module 30 includes a first column sampling register unit 301 and a second column sampling register unit 302. The first column sampling register unit 301 and the second column sampling register unit 302 are used to alternately read pixel column data from two adjacent columns. While the column sampling register module 30 is reading the Nth column of pixel column data, the event encoding module 40 is encoding the (N-1)th column of pixel column data, and the output interface module is outputting the encoded data of the (N-2)th column of pixel column, thus realizing different stages of processing on three different data columns.
[0034] When the event-type visual sensor pixel array 600 is in operation, it generates a series of ON / OFF events based on the relative changes in ambient light intensity. The pipeline column gating module 20 selects the Nth column of pixels in the event-type visual sensor pixel array 600 using the column gating signal ACKX_RD, and sends the selected pixel column data to the column sampling register module 30. After reading the Nth column of pixel data, the column sampling register module 30 sends it to the event encoding module 40 for Address-Event Representation (AER) format encoding. Simultaneously, the column sampling register module 30 sends a reset signal ACKX_RST to the Nth column of pixels, initiating a reset. After encoding, the encoded Nth column of pixel data is sent to the output module 50, which then transmits it to the outside world.
[0035] See Figure 2 The pipeline ping-pong buffer system control module 10 includes: an OR gate 101, a first AND gate 102, a second AND gate 103, a first inverter 104, a second inverter 106, a first gating unit 105, and a second gating unit 107. The input of the OR gate 101 is connected to the outputs of the first AND gate 102 and the second AND gate 103, respectively. The inputs of the first AND gate 102 and the second AND gate 103 are both connected to the event encoding module 40 and the output module 50. One end of the first inverter 104 is connected to the input of the OR gate 101, and the other end is connected to the first gating unit 105. The first gating unit 105 is connected to the first column sampling register unit 301. One end of the second inverter 106 is connected to the input of the OR gate 101, and the other end is connected to the second gating unit 107. The second gating unit 107 is connected to the second column sampling register unit 302. The input terminal of the first gate control unit 105 is also connected to the input terminal of the OR gate 101 and the pipeline selection module 20; the input terminal of the second gate control unit 107 is also connected to the input terminal of the OR gate 101 and the pipeline selection module 20.
[0036] When the event encoding module 40 completes encoding and the output module 50 completes output, either the first AND gate 102 or the second AND gate 103 will be turned on. The OR gate 101 will also be turned on because either the first AND gate 102 or the second AND gate 103 is turned on. The pipeline column selection module 20 will receive the scan signal transmitted by the OR gate and select the next column of pixels. Under the action of the first inverter 104 and the second inverter 106, the first gate unit 105 and the second gate unit 107 will be selectively turned on. When the first AND gate 102 is turned on, the second AND gate 103 is closed, the first gate unit 105 is closed, and the second gate unit 106 is turned on. When the second AND gate 103 is turned on, the first AND gate 102 is closed, the first gate unit 105 is turned on, and the second gate unit 106 is closed. Specifically, when the event encoding module 40 and the output module 50 output completion signals (Finish_e1, Finish_o1) to the first AND gate 102, the first AND gate 102 is turned on. After receiving the transmission signal (Finish_1) from the first AND gate 102, the first inverter 104 outputs a closing signal (N_Finish_1) to the first gating unit 105. The other end of the first gating unit 105 receives the transmission signal (Finish_1) from the first AND gate 102. The first gating unit 105 will close under the combined action of the signals Finish_1 and N_Finish_1. Since the second AND gate 103 has not received the completion signal from the event encoding module 40 and the output signal from the output module 50, the second AND gate 103 is closed. After receiving the non-conducting signal from the second AND gate 103, the second inverter 106 outputs a conduction signal to the second gating unit 106, and the second gating unit 106 is turned on. Then, the second column sampling register unit 302 reads the next column of pixel data. After the second column sampling register unit 302 completes its reading, the event encoding module 40 and the output module 50 finish their work, transmitting the completion signals (Finish_e2, Finish_o2) to the second AND gate 103, which then conducts. Upon receiving the signal (Finish_2) transmitted by the second AND gate 103, the second inverter 106 outputs a closing signal (N_Finish_2) to the second gating unit 107. The other input of the second gating unit 107 receives the signal (Finish_1) transmitted by the second AND gate 103. The first gating unit 105 closes under the combined action of the signals Finish_2 and N_Finish_2. Since the first AND gate 102 does not receive the completion signal from the event encoding module 40 or the output signal from the output module 50, the first AND gate 102 closes.After receiving the non-conducting signal from the first AND gate 102, the first inverter 104 outputs a conduction signal to the first gating unit 105, and the first gating unit 106 is turned on. This enables pipelined cyclic use of the first column sampling register unit 301 and the second column sampling register unit 302, repeating until the last column of pixels is read, thus improving the readout efficiency of the event-type visual sensor pixel array 600.
[0037] See Figure 3 The pipelined column selection module 20 includes a displacement register unit 201. When the pipelined ping-pong buffer system control module 10 transmits a scan signal to the pipelined column selection module 20, the displacement register unit 201 sends a readout signal ACKX_RD to one column of the event-type visual sensor pixel array 600. The pipelined column selection module 20 then selects a column of pixel data from the event-type visual sensor and transmits it to the column sampling register module 30 for reading. Furthermore, the displacement register unit 201 also sends a reset signal ACKX_RST to the previous column of pixels in the event-type visual sensor pixel array 600, resetting that column. The displacement register unit 201 selects the event-type visual sensor pixel array 600 sequentially, starting from the first column and ending at the last column.
[0038] See Figure 4 The pipelined ping-pong buffer readout circuit for event-type vision sensors specifically includes three synchronous steps: reading out the Nth column of pixels, encoding the (N-1)th column of pixels, and outputting the (N-2)th column of pixels.
[0039] Nth column pixel readout: When reading event data, the pipelined column gating module 20 sends a corresponding column gating signal ACKX_RD_N to select the Nth column in the event-type vision sensor pixel array 600. This signal inputs the pixel column data into one of the column sampling register units of the column sampling register module 30 through the first gating unit 105 or the second gating unit 107. After completing the reading of the pixel column data, the pipelined column gating module 20 sends a column reset signal ACKX_RST_N to control the reset of the Nth column pixel column. This process also sends an ACKX_RD_N+1 signal to select the N+1th column. At this time, another register circuit unit starts working, ready to read the N+1th column pixel column data. Specifically, the encoding of the Nth column event data is performed simultaneously with the reading of the N+1th column event data. In this way, when the data of the N+1th column is read out, the Nth column pixel column data has also been encoded and can be output to the outside world through the output module 50. At this point, the Nth column of pixel data stored in the column sampling register circuit can be released and refreshed. Therefore, the pipelined column gating circuit selects the (N+2)th column of pixels and reads the event data of this column into the column sampling register circuit. Simultaneously, the pipelined ping-pong buffer readout circuit for the event-type vision sensor is also outputting the Nth column of pixel data and encoding the (N+1)th column of pixel data. This process continues until all event data from the event-type vision sensor pixel array 600 has been read.
[0040] Pixel Encoding of Column N-1: Event encoding module 40 receives pixel column data output from the first column sampling register unit 301 or the second column sampling register unit 302 and performs AER encoding on them. A Finish_e1 or Finish_e2 pulse signal is generated after encoding. Simultaneously with the pixel column data encoding, the pixels in the corresponding column are reset in preparation for the next readout.
[0041] Pixel output of column N-2: Output module 50 outputs the data processed by the event encoding circuit to the outside world for subsequent processing. After output module 50 has finished outputting all data, it will pull up the Finish_o1 / Finish_o2 signal and maintain it for one clock cycle.
[0042] See Figure 5In the readout process of the pipelined ping-pong buffer readout circuit for the event-driven vision sensor, the ACKX_RD1 signal goes high on a rising edge of the clock signal clk, selecting the first column of pixels. After the event data of the first column of pixels is read, the ACKX_RD1 signal is pulled low, and the first column is reset by pulling the ACKX_RST1 signal high. Simultaneously, the pipelined column gating circuit also sends the ACKX_RD2 signal to read the event data of the second column. Similarly, after the event data of the second column is read, ACKX_RD2 is pulled low, and ACKX_RST2 and ACKX_RD3 are pulled high. At this point, the first column of pixel data is also basically encoded and can be sent to the host computer via the interface circuit for subsequent processing. The valid signal is a valid output data flag. Because the pipelined ping-pong buffer readout circuit for the event-driven vision sensor separates the readout and reset operations of the pixel column data, it avoids the problem that the column gating signal can only select the next column of pixel data after all the work such as data readout, event encoding, and pixel reset is completed. The pipelined ping-pong buffer readout circuit for the event-based vision sensor can process the events of the current column and reset the pixels of the current column while reading out the pixels of the second column in parallel after reading out a column of event data, thereby improving the working efficiency of the readout circuit and increasing the frame rate.
[0043] In summary, the pipelined ping-pong buffer readout circuit for event-driven vision sensors provided by this invention enables a three-stage pipeline for pixel event readout, significantly improving the chip frame rate of event-driven vision sensors. Furthermore, theoretically, it can increase the readout efficiency by up to three times that of traditional readout circuits, effectively avoiding the problems of long readout times and low efficiency in event-driven vision sensors.
[0044] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A pipelined ping-pong buffer readout circuit for an event-driven vision sensor, characterized in that, include: The pipeline ping-pong buffer system includes a control module, a pipeline column gating module, a column sampling register module, an event encoding module, and an output module. The pipeline ping-pong buffer system control module is connected to the pipeline column gating module, column sampling register module, event encoding module, and output module respectively, and is used to coordinate the work of each module; The pipeline column selection module is used to select a column of pixels in the pixel array of the event-type vision sensor and transmit a reset signal to the event-type vision sensor to reset the pixel array. The column sampling register module includes a first column sampling register unit and a second column sampling register unit, which are used to alternately read the pixel column data of two adjacent columns selected by the pipeline column gating module; The event encoding module is used to encode the data read by the column sampling register module; The output module is used to output the pixel column encoded data; The event encoding module encodes one column of pixel data while resetting that column; the column sampling register module reads the next column of pixel data; and the output module outputs the encoded pixel data from the previous column. This process simultaneously processes three different columns of pixel data at different stages. The pipeline ping-pong buffer system control module includes an OR gate, a first AND gate, a second AND gate, a first inverter, a second inverter, a first gating unit, and a second gating unit. The input terminals of the OR gates are connected to the output terminals of the first AND gate and the second AND gate, respectively. The input terminals of the first AND gate and the second AND gate are both connected to the event encoding module and the output module. One end of the first inverter is connected to the input terminal of the OR gate, and the other end is connected to the first gating unit. The first gating unit is connected to the first column sampling register unit. One end of the second inverter is connected to the input terminal of the OR gate, and the other end is connected to the second gating unit. The second gating unit is connected to the second column sampling register unit.
2. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 1, characterized in that, The input terminal of the first gate control unit is also connected to the input terminal of the OR gate and the pipelined column selection module; the input terminal of the second gate control unit is also connected to the input terminal of the OR gate and the pipelined column selection module.
3. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 2, characterized in that, When the first gating unit is turned on, the second gating unit is closed, and the first column sampling register unit reads the pixel column data transmitted by the pipeline column gating module; when the second gating unit is turned on, the first gating unit is closed, and the second column sampling register unit reads the pixel column data transmitted by the pipeline column gating module.
4. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 2, characterized in that, When the event encoding module and the output module complete their work, the first AND gate or the second AND gate is turned on, the OR gate is turned on, and the pipeline column gating module transmits the pixel column data to the first column sampling register unit or the second column sampling register unit through the first gating unit or the second gating unit.
5. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 1, characterized in that, When the output module receives the encoded data of the first column of pixels, the second column sampling register unit completes the reading of the second column of pixel data and transmits it to the event encoding module for encoding; the first column sampling register unit begins to read the data of the third column of pixels; this process is repeated to achieve the output of data for the entire pixel array.
6. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 1, characterized in that, While the event encoding module encodes a column of pixel data, the pipeline column gating module transmits a reset signal for that column of pixels to the event-type vision sensor.
7. The pipelined ping-pong buffer readout circuit for an event-type vision sensor according to claim 1, characterized in that, While the pipelined column gating module transmits a column of pixel reset signals to the event-type vision sensor, the pipelined column gating module also selects the next column of pixel data in the event-type vision sensor, and the column sampling register unit, which is in an idle state, reads the next column of pixel data.
8. The pipelined ping-pong buffer readout circuit for an event-driven vision sensor according to claim 7, characterized in that, The pipeline column selection module includes a displacement register unit; when the pipeline ping-pong buffer system control module transmits a signal to the pipeline column selection module, the displacement register unit will send a read signal to one column in the pixel array and send a reset signal to the next column in the pixel array.