Dynamic visual sensor performance modeling and evaluation method by means of DVS event sequence

Abstract

The invention discloses a dynamic vision sensor performance modeling and evaluating method by means of a DVS event sequence, which comprises the following steps: modeling a DVS pixel circuit based on a pulse sequence in a DVS data set and a Boolean logic technology to obtain a DVS pixel circuit model; based on the structure of the DVS array, modeling the DVS pixel array to obtain a DVS pixel array model; based on the pseudo codes corresponding to the mainstream array readout circuits, modeling the mainstream array readout circuits to obtain a mainstream array readout circuit model; and simulating the DVS pixel circuit model, the DVS pixel array model and the mainstream array readout circuit model so as to evaluate the DVS performance. Scene-specific simulation based on a DVS data set is supported, the event loss rate and the read-out delay can be quantified, and a basis is provided for optimizing a DVS system.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic vision sensors, and in particular, to a method for modeling and evaluating the performance of a dynamic vision sensor by means of a DVS event sequence. Background Art

[0002] A Dynamic Vision Sensor (DVS) is a sensor that mimics the visual system of higher organisms. It generates event data by capturing local brightness changes, and each pixel generates an event when it senses a change in light intensity. These events are characterized by high spatio-temporal correlation, that is, events caused by the movement of the same object are close to each other in space and time. Therefore, compared with traditional imaging sensors, DVS can provide higher temporal resolution in a dynamic environment and is suitable for fields such as high-speed object detection, robot vision, and autonomous driving.

[0003] However, the event reading and processing process of DVS still faces many challenges in the prior art. Existing readout circuits usually use a queuing theory-based modeling method to evaluate the event response process. The queuing theory model assumes that the event generation process of DVS conforms to a Poisson distribution and that the generation of all pixel events is independent of each other. However, the actual situation is that there is significant spatio-temporal correlation in the generation of DVS events, that is, events caused by the movement of the same object will be concentrated on adjacent pixels and time points. Therefore, the assumption based on the Poisson distribution cannot accurately reflect the actual event generation process, resulting in a large deviation between the evaluation results of the queuing model and the real situation.

[0004] In addition, the event readout circuit of DVS usually has a complex arbitration mechanism. For example, arbitration can be performed according to the order of event generation, row scan order, frame-by-frame method, or priority processing based on the region of interest (ROI). These different arbitration strategies will significantly affect the readout delay and overall performance of the events. However, existing queuing theory-based models are difficult to effectively model and analyze these complex arbitration mechanisms, and thus cannot provide effective theoretical guidance for the design and optimization of different arbitration strategies.

[0005] In addition, the queuing theory model assumes that the readout circuit operates in a stable state, but the actual situation is that the event rate of DVS is affected by various factors, such as the movement state, turning, and speed change of objects in the scene. In application scenarios such as autonomous driving, the event generation rate of the DVS array will fluctuate in real time with the speed of the vehicle and environmental changes, which makes the readout circuit unable to always maintain a stable state. Therefore, the evaluation results obtained based on the stable state assumption are not applicable to actual applications.

[0006] Finally, existing queuing theory-based modeling methods usually only focus on the event generation of the DVS array and the performance of the basic readout circuit, but fail to consider the pipeline structure of the entire DVS system, especially the impact of row number arbitration and processing interface circuits. As the number of array rows increases, the calculation time of row number arbitration also increases, resulting in an increase in event readout delay and even affecting the performance of the entire system. Therefore, the existing models do not fully consider these details, leading to inaccurate evaluation results.

[0007] Therefore, there is an urgent need for a method for modeling and evaluating the performance of dynamic vision sensors by means of DVS event sequences. Summary of the Invention

[0008] The present invention provides a method for modeling and evaluating the performance of dynamic vision sensors by means of DVS event sequences to solve the above problems existing in the prior art.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A method for modeling and evaluating the performance of dynamic vision sensors by means of DVS event sequences, including:

[0011] Step 1: Based on the pulse sequence and Boolean logic technology in the DVS dataset, model the DVS pixel circuit to obtain a DVS pixel circuit model;

[0012] Step 2: Based on the structure of the DVS array, model the DVS pixel array to obtain a DVS pixel array model;

[0013] Step 3: Based on the pseudo-code corresponding to each mainstream array readout circuit, model the mainstream array readout circuit to obtain a mainstream array readout circuit model;

[0014] Step 4: Simulate the DVS pixel circuit model, DVS pixel array model, and mainstream array readout circuit model, and then evaluate the DVS performance.

[0015] Among them, the DVS pixel circuit includes: a logarithmic photoreceptor, a switched-capacitor amplifier, a threshold comparator, an event latch, and pixel interface logic;

[0016] Obtaining the DVS pixel circuit model includes:

[0017] The logarithmic photoreceptor converts photocurrent into logarithmic voltage to provide the initial signal conversion of the DVS pixel circuit;

[0018] The switched-capacitor amplifier includes an input capacitor C A , a feedback capacitor C B and an amplifier A 2 , the input capacitor C Aand feedback capacitor C B are connected to form a capacitive amplifier structure. The switched-capacitor amplifier amplifies the AC part of the logarithmic voltage to provide an enhanced signal for subsequent signal processing;

[0019] The threshold comparator is connected to the output of the switched-capacitor amplifier module to detect and output ON events and OFF events. The threshold comparator generates event pulses by comparing the input signal with a preset threshold;

[0020] The event latch receives and latches the ON events and OFF events output by the threshold comparator. The event latch controls the latch state through Boolean logic to achieve stable storage of events;

[0021] The pixel interface logic is connected to the event latch and outputs the latched events through the DVS array interface. The pixel interface logic uses Boolean logic to achieve logical processing and output control of events;

[0022] The logarithmic photoreceptor, switched-capacitor amplifier, threshold comparator, event latch, and pixel interface logic are connected in series in sequence to form a complete DVS pixel circuit model;

[0023] Among them, the output of the threshold comparator is described by a pulse sequence in the DVS dataset and used as the excitation of the event latch, thereby realizing the modeling of the logarithmic photoreceptor, switched-capacitor amplifier, and threshold comparator in the DVS pixel circuit.

[0024] Among them, obtaining the DVS pixel array model includes:

[0025] The pixel array is formed in a two-dimensional manner. The pixel array includes a pixel drive row request signal and a pixel drive column event signal;

[0026] The DVS pixel array model is constructed based on pseudocode descriptions. The pseudocode includes the event stream in the DVS dataset and the output of each pixel event latch in the DVS array;

[0027] Among them, the interface of the pixel array includes interfaces for the row request signal Rreq, row acknowledgement signal Rack, row reset signal Rrst, and column event signals Con and Coff.

[0028] Among them, obtaining the mainstream array readout circuit model includes:

[0029] For the event-driven DVS array, a first readout circuit model based on the first-in-first-out mechanism is constructed through first-in-first-out mechanism pseudocode;

[0030] Construct a second readout circuit model for sequential line selection through sequential line selection pseudocode according to the order of line numbers. Among them, the second readout circuit responds to the lines with event requests line by line in ascending order of line numbers according to the line priority of the event requests.

[0031] Based on the frame-by-frame method pseudocode, construct a third readout circuit model for the frame-by-frame method. Among them, the third readout circuit accumulates events within a fixed period of time through the DVS array and outputs the third readout circuit model as a complete event frame for subsequent processing.

[0032] Based on the priority arbitration pseudocode, construct a fourth readout circuit model for priority arbitration of the region of interest. Among them, the programmable priority of each row of pixels is preset, and events are read according to the predetermined priority order.

[0033] Among them, simulate the DVS pixel circuit model, DVS pixel array model and mainstream array readout circuit model, including:

[0034] Stimulate the DVS array model through the input event sequence. The input event sequence is composed of DVS data sets collected under various scenarios, and each event includes coordinate and timestamp information.

[0035] Run the array readout circuit model in the simulation environment, read the events in the DVS array, generate an output event sequence, and the events in the output event sequence are associated with the events in the input event sequence through an event mapping relationship.

[0036] Calculate the readout delay of each event in the output event sequence relative to the corresponding event in the input event sequence, and mark the events that are lost because they are not answered by the readout circuit in time.

[0037] Based on the event mapping relationship and readout delay, evaluate the performance metrics of the dynamic vision sensor in the specified scenario, including the event loss rate and the average readout delay of the successfully read events.

[0038] Output the performance evaluation results of the dynamic vision sensor. The performance evaluation results are used to optimize the design and configuration of the DVS pixel circuit model and the array readout circuit model to improve the performance of the dynamic vision sensor.

[0039] Among them, it also includes:

[0040] Use the pulse sequence in the DVS data set to describe the output of the threshold comparator and use it as the excitation of the event latch, so as to realize the modeling of the logarithmic photoreceptor, switched-capacitor amplifier and threshold comparator in the DVS pixel circuit.

[0041] Use Boolean logic to model the event latch and pixel interface logic.

[0042] Among them, a pixel array is formed in a two-dimensional manner, including:

[0043] The pixel array includes a plurality of pixels, and each pixel corresponds to at least one pixel drive row request signal and at least one pixel drive column event signal. The pixel array realizes the row-by-row and column-by-column driving and reading of image information through this two-dimensional structure;

[0044] According to the two-dimensional pixel array structure, the timing relationship between the row request signal and the column event signal of each pixel is defined. The row request signal is used to trigger the pixel drive of the corresponding row, and the column event signal is used to generate an event stream related to the column. And this timing relationship ensures that in each cycle, all pixels can synchronously respond to the row request signal and the column event signal to complete the transmission and processing of image data.

[0045] Among them, a DVS pixel array model is constructed based on pseudocode description, including:

[0046] The event stream in the DVS dataset represents the response of each pixel to event changes at different time points;

[0047] In the DVS pixel array model, an event latch is set for each pixel to latch the column event signal corresponding to the pixel and generate a corresponding output signal when an event arrives. This output signal represents the change in the pixel state.

[0048] Among them, the DVS array model is excited by an input event sequence, including:

[0049] The event sequence is composed of DVS datasets in multiple scenarios. The datasets include DVS data collected under different lighting conditions, motion speeds, and object morphologies to ensure the diversity of the event sequence;

[0050] In the DVS dataset, each event includes spatial coordinate information and timestamp information detected by the DVS sensor. The coordinate information is used to represent the position of the event in the sensor's field of view, and the timestamp information is used to represent the time when the event occurs, which is convenient for subsequent timing processing and analysis of the event;

[0051] Based on the DVS dataset, an event sequence including multiple time points is generated. Each event is sorted according to the timestamp to obtain a continuous event sequence arranged in chronological order to ensure that the event sequence reflects the dynamic changes in the actual scene;

[0052] The event sequence is input into the DVS pixel array model for excitation. The DVS pixel array model simulates the response according to the input event sequence and generates a simulated visual output. The visual output is used to further analyze the perception ability and response characteristics of the DVS array in a complex environment.

[0053] Among them, it also includes:

[0054] The DVS pixel array model processes events according to the coordinate information and timestamp information of the event sequence, and takes the temporal and spatial distribution relationship of the events as the input to simulate the visual perception process of the DVS array in a specific scenario, and outputs information including image features and motion trajectories.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] The method for modeling and evaluating the performance of a dynamic vision sensor by means of a DVS event sequence includes: modeling a DVS pixel circuit based on the pulse sequence and Boolean logic technology in a DVS dataset to obtain a DVS pixel circuit model; modeling a DVS pixel array based on the structure of the DVS array to obtain a DVS pixel array model; modeling a mainstream array readout circuit based on the corresponding pseudocode of each mainstream array readout circuit to obtain a mainstream array readout circuit model; simulating the DVS pixel circuit model, the DVS pixel array model and the mainstream array readout circuit model, and then evaluating the DVS performance. It supports scene-specific simulation based on the DVS dataset, can quantify the event loss rate and readout delay, and provides a basis for optimizing the DVS system.

[0057] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by practicing the present invention.

[0058] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0059] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0060] Figure 1 is a flowchart of the method for modeling and evaluating the performance of a dynamic vision sensor in an embodiment of the present invention;

[0061] Figure 2 is a structural diagram of a DVS pixel circuit in an embodiment of the present invention;

[0062] Figure 3 is a model structural diagram of a logarithmic photoreceptor + switched capacitor amplifier + threshold comparator in an embodiment of the present invention;

[0063] Figure 4 is a schematic diagram of pseudocode for DVS pixel array modeling in an embodiment of the present invention;

[0064] Figure 5Schematic diagram of the FIFO mechanism pseudocode in the embodiments of the present invention;

[0065] Figure 6 Schematic diagram of the sequential row selection pseudocode in the embodiments of the present invention;

[0066] Figure 7 Schematic diagram of the frame-by-frame method pseudocode in the embodiments of the present invention;

[0067] Figure 8 Schematic diagram of the priority arbitration pseudocode in the embodiments of the present invention;

[0068] Figure 9 DVS pixel array model in the embodiments of the present invention;

[0069] Figure 10 Three output events of the array readout circuit in the embodiments of the present invention. Detailed implementation manners

[0070] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The embodiments of the present invention provide a method for modeling and evaluating the performance of a dynamic vision sensor by means of a DVS event sequence, including:

[0072] Step 1: Based on the pulse sequence and Boolean logic technology in the DVS dataset, model the DVS pixel circuit to obtain a DVS pixel circuit model;

[0073] Step 2: Based on the structure of the DVS array, model the DVS pixel array to obtain a DVS pixel array model;

[0074] Step 3: Based on the pseudocode corresponding to each mainstream array readout circuit, model the mainstream array readout circuit to obtain a mainstream array readout circuit model;

[0075] Step 4: Simulate the DVS pixel circuit model, the DVS pixel array model, and the mainstream array readout circuit model, and then evaluate the DVS performance.

[0076] The working principle of the above technical solution is as follows: The core of the DVS pixel circuit includes: a logarithmic photoreceptor that converts photocurrent into a logarithmic voltage to simulate the dynamic range of natural light; a switched-capacitor amplifier that amplifies the AC part of the logarithmic voltage to enhance the signal; a threshold comparator that converts signal changes into ON / OFF events; and an event latch and pixel interface logic that stores the event signal and outputs it through the DVS array interface. In the DVS pixel circuit, a pulse sequence in the DVS dataset is used to simulate the behavior of the logarithmic photoreceptor and related modules, and Boolean logic is combined to model the event latch and pixel interface logic. Such a method completely describes the dynamic response characteristics of the pixel circuit.

[0077] Modeling of the DVS pixel array is based on the structure of two-dimensional pixels: Pixels within a row are jointly driven by a row request signal (Rreq), and the event request signal being pulled low indicates that an event has occurred. Pixels within a column are jointly driven by a column event signal (Con / Coff) to control the column event output. The row acknowledge (Rack) and row reset (Rrst) signals coordinate the request and reset of row-level events. Using pseudocode to describe these signal interaction relationships, the dynamic event processing process of the DVS array is modeled to ensure that the spatio-temporal mapping relationship of events is accurately captured.

[0078] Modeling of the mainstream array readout circuit. The modeling of the mainstream readout circuit includes the following four schemes:

[0079] First-in, first-out (FIFO) arbitration: Records the order of event requests and gives priority to responding to the earliest requests, suitable for low event rate scenarios. Sequential row selection: Reads out in the order of row numbers to avoid possible time errors caused by motion artifacts. Frame-by-frame method: Accumulates events for a certain period of time and then reads them out in batches, suitable for specific algorithm processing but sacrificing high time resolution. Region of interest-based: Assigns priorities to rows and quickly responds to events in high-priority regions. Each readout circuit is modeled and described using pseudocode to clarify the specific logic of event requests, responses, and resets.

[0080] Simulation and performance evaluation are carried out through the following simulation method:

[0081] Input event sequence (α): The DVS dataset is used to stimulate the array model in the simulation.

[0082] Output event sequence (β): The events output by the array readout circuit.

[0083] Event mapping relationship (f): Establishes the association between input and output events and marks unread or lost events.

[0084] Evaluation metrics:

[0085] Event loss rate: The ratio of lost events to the total number of events.

[0086] Average readout latency: The time delay of a successful readout event.

[0087] Through this method, the DVS performance in different scenarios is accurately simulated.

[0088] The beneficial effects of the above technical solutions are as follows: The simulation and emulation of the pixel circuit are realized by using pulse sequences and Boolean logic technology, comprehensively covering the behaviors of analog and digital circuits. The array and readout circuit are modeled using pseudocode, simply and efficiently describing complex signal interactions. It supports scenario-specific simulation based on DVS datasets, can quantify the event loss rate and readout latency, and provides a basis for optimizing the DVS system. The comparison of different arbitration readout schemes provides guidance for system designers to select appropriate readout methods in different application scenarios. The model structure is consistent with the actual DVS architecture, facilitating actual hardware implementation. The simulation method is compatible with multiple datasets and supports DVS performance optimization in diverse application scenarios. The detailed modeling of mainstream readout circuits is introduced, providing a reference for the design of future new arbitration mechanisms. The simulation method can adapt to different pixel circuit improvements and array optimization strategies.

[0089] In another embodiment, the DVS pixel circuit includes: a logarithmic photoreceptor, a switched-capacitor amplifier, a threshold comparator, an event latch, and pixel interface logic;

[0090] Obtaining the DVS pixel circuit model includes:

[0091] The logarithmic photoreceptor converts photocurrent into logarithmic voltage, providing the initial signal conversion of the DVS pixel circuit;

[0092] The switched-capacitor amplifier includes an input capacitor C A , a feedback capacitor C B , and an amplifier A 2 . The input capacitor C A and the feedback capacitor C B are connected to form a capacitor amplifier structure. The switched-capacitor amplifier amplifies the AC part of the logarithmic voltage, providing an enhanced signal for subsequent signal processing;

[0093] The threshold comparator is connected to the output of the switched-capacitor amplifier module, detects and outputs ON events and OFF events. The threshold comparator generates event pulses by comparing the input signal with a preset threshold;

[0094] The event latch receives and latches the ON events and OFF events output by the threshold comparator. The event latch controls the latch state through Boolean logic to achieve stable storage of events;

[0095] The pixel interface logic is connected to the event latch and outputs the latched events through the DVS array interface. The pixel interface logic uses Boolean logic to implement logical processing and output control of events;

[0096] The logarithmic photoreceptor, switched-capacitor amplifier, threshold comparator, event latch, and pixel interface logic are connected in series in sequence to form a complete DVS pixel circuit model;

[0097] Among them, the pulse sequence in the DVS dataset is used to describe the output of the threshold comparator and serve as the excitation for the event latch, thereby realizing the modeling of the logarithmic photoreceptor, switched-capacitor amplifier, and threshold comparator in the DVS pixel circuit.

[0098] The working principle of the above technical solution is as follows: The main task of the DVS (Dynamic Vision Sensor) pixel circuit is to convert light intensity changes into pulse events to achieve efficient capture of dynamic scenes. As Figure 2 shown, this pixel circuit consists of multiple modules, which are, in sequence, the logarithmic photoreceptor, switched-capacitor amplifier, threshold comparator, event latch, and pixel interface logic. Each module is interconnected and works together to generate accurate event pulses and output them.

[0099] The logarithmic photoreceptor is responsible for converting light intensity information into a logarithmic voltage. It receives the photocurrent obtained from the photosensor array and uses a logarithmic conversion mechanism to map the amplitude change of the photocurrent into a voltage change. This process helps to handle the non-linear response under different light intensity conditions, greatly expanding the dynamic range of the optical signal, especially suitable for processing rapidly changing light intensity conditions. The output of the logarithmic voltage serves as the input signal for the switched-capacitor amplifier. The switched-capacitor amplifier module consists of an input capacitor (C A ), a feedback capacitor (C B ), and an amplifier A 2 . The input capacitor and the feedback capacitor form a capacitor amplifier structure, and the input signal is stored and amplified by controlling the switch at an appropriate time. In this module, mainly the AC part of the input signal is amplified to increase the signal amplitude for subsequent signal processing. The switched-capacitor amplifier effectively amplifies the voltage signal transmitted from the logarithmic photoreceptor, ensuring that subsequent processing stages can obtain a strong enough signal.

[0100] The threshold comparator is connected to the output of the switched-capacitor amplifier. It is used to detect the input signal and compare it with a preset threshold. If the input signal exceeds the threshold, the threshold comparator outputs an "ON event", and if the input signal is lower than the threshold, it outputs an "OFF event". In this way, the threshold comparator converts the analog signal into digital pulse events, thereby realizing the discretization of light intensity changes. This module is the key to event pulse generation and determines the response characteristics of the DVS pixel.

[0101] The function of the event latch is to receive the ON event and OFF event output by the threshold comparator and latch them. Through latching, the events can be stably stored in time, preventing loss or misjudgment. The event latch usually combines Boolean logic to control the state of the latch to ensure that the latch correctly stores the occurrence of the event when the event is detected. In this way, the event latch can provide reliable event storage, avoiding signal drift or loss. The pixel interface logic module transmits the ON event and OFF event output by the event latch to the interface of the DVS array. As Figure 3 shown, the event latch and the pixel interface circuit are essentially digital circuits, and the output format and timing of the events can be controlled by Boolean logic, and these events are transmitted to an external processor or other systems. The pixel interface logic is the bridge to realize event output, enabling the system to selectively read and process different events according to actual needs.

[0102] The beneficial effects of the above technical solution are as follows: The DVS pixel circuit design realizes an efficient response to dynamic scenes by combining a logarithmic photoreceptor, a switched-capacitor amplifier, a threshold comparator, an event latch, and pixel interface logic. By using a logarithmic photoreceptor, the circuit can adapt to a wide range of light changes and can effectively capture light changes even in very bright or very dark environments. This logarithmic conversion ensures that the DVS pixels can maintain good performance under different light conditions. The DVS pixel circuit converts light changes into event pulses, enabling the system to respond to light changes in dynamic scenes in real time. Compared with traditional frame-mode cameras, DVS provides lower latency and higher temporal resolution, which is especially suitable for capturing high-speed moving objects. The switched-capacitor amplifier can effectively amplify the signal from the logarithmic photoreceptor and improve the detection sensitivity of the subsequent threshold comparator. In this way, the system can accurately capture tiny light changes and avoid missed detections. At the same time, the threshold comparator can effectively extract discrete event pulses from continuous analog signals, further improving the reliability of the signal. The event latch ensures the stable storage of events through Boolean logic. Even in high-speed scenarios, the storage and output of events will not be lost or incorrect. This provides reliable data support for subsequent signal analysis and processing.

[0103] In another embodiment, obtaining a DVS pixel array model includes:

[0104] Constructing a pixel array in a two-dimensional manner, where the pixel array includes a pixel drive row request signal and a pixel drive column event signal;

[0105] Constructing a DVS pixel array model based on pseudocode description, where the pseudocode includes the event stream in the DVS dataset and the output of each pixel event latch in the DVS array;

[0106] Among them, the interface of the pixel array includes the interfaces of the row request signal Rreq, the row response signal Rack, the row reset signal Rrst, and the column event signals Con and Coff.

[0107] The working principle of the above technical solution is as follows: In a DVS (Dynamic Vision Sensor) array, the pixel array is organized in a two-dimensional manner, and each pixel has an independent event detection and output function. Each pixel in the array responds to external changes and generates event signals to represent the dynamic changes at a certain moment. These event signals are propagated and managed through appropriate row and column interfaces, and finally form a complete event stream data set for further processing and analysis.

[0108] The organizational structure of the pixel array: The pixels in the DVS array are arranged in a two-dimensional array form, and each row and each column of the array are controlled through specific signal interfaces. All pixels within each row jointly drive a row request signal Rreq <n>(n represents the line number). These line request signals are open-drain outputs, meaning that as long as any pixel in a line generates an event, the Rreq of that line can be set <n>The signal is pulled low to trigger the event signal corresponding to the column. The row reset signal Rreq is used to reset all pixels to ensure the consistency of the array during operation.

[0109] The pixels within each column jointly drive two column event signals: Con <m>With Coff <m>(m represents the column number). The generation of the column event signal is controlled by the row response signal Rack. Only when a certain row is responded to, the event signal Con of the corresponding column <m>With Coff <m>will be driven by the pixels of this row.

[0110] Event stream and pixel event latch: Each pixel in the DVS pixel array has an event latch for storing the state information of the pixel and then driving the corresponding event output signal. In the model of the DVS array, the event stream is represented by the data set stream<y,x,time,p>dataset_AER, where:

[0111] y and x represent the row and column positions of the pixel respectively. Timet represents the timestamp when the event occurs. p represents the polarity of the event (usually positive or negative).

[0112] For each pixel, the state of the event is saved and updated through the event latch and transmitted through the column event signals Con and Coff. The read and reset operations of the pixel array can be controlled through the row request signal Rreq and the row acknowledgment signal Rack.

[0113] Row-column interface control: Row request signal Rreq <n>: All DVS pixels within each row share the row request signal. When any pixel in that row detects an event, it will cause the Rreq of that row. <n>The signal is pulled low to notify other pixels in the row that an event has occurred.

[0114] Row response signal Rack: Used to control the driving of column event signals Con and Coff. Only when the Rreq signal of a certain row is responded to, the column event signal will be driven from the pixels of that row.

[0115] Row reset signal Rrst: Used to reset the states of all pixels in a row. All pixels share this signal to ensure synchronous reset when needed and avoid inconsistent states.

[0116] Column event signal Con <m>and Coff <m>: These two signals respectively represent the event signals generated by all the pixels in that column. When a pixel event in a certain row is responded to, the event signal Con of the corresponding column <m>or Coff <m>will be activated.

[0117] The model of the DVS array can be Figure 4 described by the pseudo - code shown below. Event latches and output: In the DVS pixel array, each pixel has two event latches - one for storing positive - polarity events (event_on) and the other for storing negative - polarity events (event_off). These two latches are two - dimensional arrays respectively:

[0118] event_on[A height [A width and event_off[A height [A width

[0119] Whenever a pixel detects an event, it updates the states of these two latches and generates corresponding output signals. Controlled by the Rreq and Rack signals, the pixels in a row can synchronously output event signals. The transmission of column signals ensures that the event data of all pixels in the pixel array can be transmitted and responded to in a timely manner.

[0120] The beneficial effects of the above - mentioned technical solution are as follows: The DVS pixel array can effectively capture and transmit the changes of each pixel in a dynamic scene, thereby generating accurate time and space information. This design not only ensures that each pixel event in the pixel array can be quickly responded to, but also effectively avoids signal conflicts and data loss through the reasonable design of the row - column interface. Controlled by the row request signal Rreq and the row acknowledgment signal Rack, the DVS array can efficiently manage the event transmission and reading operations of each pixel, reduce redundant operations, and improve the real - time performance of data. By dynamically capturing change events and converting them into timestamps, DVS pixels can perceive dynamic scenes with extremely high time resolution and are suitable for application scenarios that require quick response, such as motion detection, visual tracking, etc.

[0121] In another embodiment, obtaining the mainstream array read - out circuit model includes:

[0122] For the event - driven DVS array, constructing a first read - out circuit model based on the first - in - first - out mechanism through the first - in - first - out mechanism pseudo - code;

[0123] According to the order of row numbers, constructing a second read - out circuit model of sequential row selection through the sequential row selection pseudo - code, where the second read - out circuit responds to the rows with event requests row by row in ascending order of row numbers according to the row priority of event requests;

[0124] ​Based on the frame-by-frame method pseudocode, construct the third readout circuit model of the frame-by-frame method. Among them, the third readout circuit accumulates events within a fixed period of time through the DVS array and outputs the third readout circuit model as a complete event frame for subsequent processing;

[0125] Based on the priority arbitration pseudocode, construct the fourth readout circuit model for the priority arbitration of the region of interest. Among them, the programmable priority of each row of pixels is preset, and events are read according to the predetermined priority order.

[0126] Among them, construct the first readout circuit model based on the first-in, first-out mechanism through the first-in, first-out mechanism pseudocode, including: comparing the event requests in the event-driven DVS array with the first-in, first-out (FIFO) mechanism;

[0127] When the event requests enter the FIFO queue in chronological order, record the sequence of the event request (Rreq) signals of each row of pixels;

[0128] Based on the order of the FIFO queue, give priority to responding to the row that issues the request earliest;

[0129] After responding to the earliest event request, output the event signal of that row and reset the relevant Rreq signal;

[0130] When there is a new event request in the FIFO queue, allow the new event request to enter the queue and wait for response;

[0131] After each event output, detect the next event request in the FIFO queue and repeat the steps of giving priority to the response;

[0132] Ensure that after each row of event output, the relevant request signal is reset in time so that the next event request can enter the FIFO queue again;

[0133] Among them, the event request includes the dynamic event signals from each pixel row of the DVS array;

[0134] The FIFO mechanism includes a queue structure for storing event requests to ensure that events are processed in chronological order;

[0135] The current event request is scheduled and processed in the first-in, first-out order;

[0136] After each row of event output, the relevant Rreq signal reset operation includes setting the Rreq signal of that row to low level to prepare for the next event request;

[0137] When the FIFO queue is empty, the system enters the idle state and waits for the arrival of new event requests;

[0138] Among them, the sequential response mechanism based on the FIFO queue ensures the fairness and chronological order of event processing;

[0139] The steps of preferentially responding to the earliest event requests include reading the first event request in the FIFO queue and triggering the corresponding readout circuit;

[0140] The steps of resetting the relevant Rreq signal ensure that consecutive event requests in the same row can be processed sequentially without conflicts;

[0141] The steps of allowing new event requests to enter the FIFO queue include detecting the idle positions in the FIFO queue and inserting the newly arrived event requests into the end of the queue in chronological order;

[0142] Based on the above steps, a first readout circuit model based on the first-in-first-out mechanism is constructed, which can effectively manage and schedule event requests in the DVS array, ensuring that events are accurately processed in chronological order.

[0143] The working principle of the above technical solution is as follows: In an event-driven dynamic vision sensor (DVS) array, the design of the readout circuit is crucial because it determines the response speed and processing efficiency of event data. Different readout circuit mechanisms have different arbitration strategies and manage the event outputs of each row of pixels in different ways. This section will explore four main readout circuit models: the first-in-first-out mechanism (FIFO), sequential row selection, frame-by-frame method, and priority-based region of interest arbitration.

[0144] First-in-first-out mechanism (FIFO): FIFO is a readout circuit mechanism based on chronological order. The key to this mechanism is to record the order of event requests (Rreq) signals for each row of pixels and preferentially respond to the row that issues the request earliest. After the event output of each row, the relevant request signal is reset, and the next event request enters the FIFO queue again waiting for response. The FIFO mechanism can provide fast response at low event rates, so it is particularly suitable for scenarios with fewer event generations.

[0145] As Figure 5 shown: The pseudo-code implementation of the first-in-first-out mechanism:

[0146] module FIFO_arbiter(input Rreq[Aheight],fifo<depth=2>row_index);

[0147] fifo pending_queue; / / First-in-first-out queue of row numbers

[0148] int ack_row_number;

[0149] for(i = 0; i < Aheight; i++){

[0150] / / If the Rreq of the i-th row is detected to be pulled high, add it to the first-in, first-out row number queue

[0151] / / #rose() is used to determine whether the signal has a 0→1 transition

[0152] if(#rose(Rreq[i]))

[0153] pending_queue.push(i);

[0154] }

[0155] while(1){

[0156] / / Keep the ping-pong buffer between the waiting row number arbitration circuit and the row response circuit not full

[0157] wait((!row_index.full()) && (!pending_queue.empty()));

[0158] #(Aheight / 16*5)ns;

[0159] ack_row_number = row_idx.pop(); / / Pop the row number to be acknowledged and read from the first-in, first-out queue

[0160] row_index.push(ack_row_number); / / Push the row number into the ping-pong buffer of the row response circuit

[0161] }

[0162] Endmodule

[0163] The sequential row selection mechanism does not consider the time order of event requests for each row, but responds row by row in row number order. That is, regardless of which row issues a request earliest, the array always responds to event requests for each row in ascending row number order. Although this method ensures a uniform response frequency, its drawback is that it cannot guarantee that rows that issue event requests earlier will be responded to first. As Figure 6 shown in the implementation of the sequential row selection pseudocode.

[0164] The core idea of the frame-by-frame method is to accumulate all events within a fixed period of time and output these events as a complete event frame. This approach is suitable for scenarios where all events within a period of time need to be uniformly processed. The frame-by-frame method can reduce the computational burden on the subsequent processing system, but sacrifices the high response speed and time resolution of the array. Therefore, this method is more suitable for occasions that require overall analysis of events within a time period, such as motion analysis or pattern recognition. As Figure 7 shown in the implementation of the frame-by-frame method pseudocode.

[0165] Based on the region of interest (ROI) priority arbitration mechanism, the array presets the programmable priority for each row of pixels. The system reads the events within the row according to these priority orders. The rows within the region of interest have higher priorities, so their events will be responded to first, while the rows outside the region of interest respond more slowly. This mechanism is suitable for applications that need to focus on events within a specific area, such as tracking a target object or changes in a high-concern area. As Figure 8 shown in the implementation of the priority arbitration pseudocode.

[0166] The beneficial effects of the above technical solutions are as follows: When applicable to low-speed scenarios, it can ensure fast response and avoid unnecessary delays. Sequential row selection: By processing events in the order of row numbers, it avoids the queue backlog problem of the FIFO mechanism in high-speed situations and is suitable for high-speed motion monitoring, especially having advantages in large-scale scenarios. Frame-by-frame method: By accumulating events to generate frames, it helps reduce the computational complexity of the subsequent processing system and is applicable to applications that do not require real-time response, especially in image analysis and recognition. Priority arbitration: By prioritizing the region of interest, it improves the response speed of events in key areas and enhances the processing ability of events in specific areas, especially having significant advantages in target tracking and autonomous driving systems. Meeting the requirements of different application scenarios: The implementation of various mechanisms enables different application scenarios to select the most suitable readout circuit according to their needs. For example, the FIFO mechanism is suitable for applications with low event rates, while the frame-by-frame method is suitable for occasions that summarize and analyze events over a period of time. Sequential row selection and priority arbitration provide the ability to balance response speed and processing accuracy, especially being suitable for high-speed scenarios or applications that need to respond to specific areas.

[0167] In another embodiment, simulations are performed on the DVS pixel circuit model, DVS pixel array model, and mainstream array readout circuit model, including:

[0168] The DVS array model is stimulated by inputting an event sequence, and the input event sequence is composed of DVS data sets collected under various scenarios, where each event includes coordinate and timestamp information;

[0169] Run the array readout circuit model in a simulation environment, read out the events in the DVS array, generate an output event sequence, and associate the events in the output event sequence with the events in the input event sequence through an event mapping relationship;

[0170] Calculate the readout delay of each event in the output event sequence relative to the corresponding event in the input event sequence, and mark the events that are lost because they are not answered by the readout circuit in time;

[0171] Based on the event mapping relationship and the readout delay, evaluate the performance metrics of the dynamic vision sensor in a specified scenario, including the event loss rate and the average readout delay of the successfully read events;

[0172] Output the performance evaluation results of the dynamic vision sensor, which are used to optimize the design and configuration of the DVS pixel circuit model and the array readout circuit model to improve the performance of the dynamic vision sensor.

[0173] The working principle of the above technical solution is as follows: The above content models the system architecture, pixels and arrays, and four mainstream array readout circuits of the dynamic vision sensor respectively. Next, the performance evaluation of the dynamic vision sensor and its readout circuit is completed through the simulation of the models.

[0174] If Figure 9 As shown, the DVS data set used as the excitation of the DVS array model is denoted as the input event sequence α, and the events in the array form the output event sequence β after being read out by the array readout circuit. α and β can be represented by expressions (1) and (2), where M and N represent the number of events in the event sequences α and β respectively.

[0175] α={(x 1 ,y 1 ,t 1 ),(x 2 ,y 2 ,t 2 ),(x 3 ,y 3 ,t 3 ),(x 4 ,y 4 ,t 4 )…(x N ,y N ,t N )} (1)

[0176] β={(x′ 1 ,y′ 1 ,t′ 1 ),(x′ 2 ,y′ 2 ,t′ 2 ),(x′ 3 ,y′ 3 , t' 3 ), (x' 4 , y' 4 , t' 4 )…(x' M , y' M , t' M )} (2)

[0177] Since the four readout circuits described above can only possibly lose the events generated by the DVS array, and will not output additional events that did not originally exist, so M ≤ N. At the same time, any "output event" (x' i , y' i , t' i ) in the output event sequence β corresponds to an "input event" (x j , y j , t j ) in the input event sequence α. To determine the correspondence between the input events and the output events, each "input event" is attached with a unique label during the simulation of the array model, and the readout circuit model will read out this label along with the event when reading the events in the DVS array. If the event latched in the DVS pixel is covered by a new event because it is not answered by the readout circuit in time, the label of the event will also be covered by the label of the new event, and the old event is marked as "lost". Through this method, we can obtain the mapping relationship from the output event sequence β to the input event sequence α, and call it the "event mapping relationship f", as shown in (3).

[0178] f((x' i , y' i , t' i ) = (x j , y j , t j ) (3)

[0179] None of the four readout circuits described above will change the coordinates of the events, so any "input-output event pair" satisfies the relationship (4). We call Δt i the "readout delay" of the i-th event output by the readout circuit relative to the event actually generated in the array.

[0180]

[0181] Such as Figure 10 As shown, the relationship between the event mapping function f, the input event sequence α, and the output event queue β is presented. In the figure, the input event sequence α has 4 events, and the output event queue β has 3 events. The expressions for the input event sequence α, the output event queue β, and the mapping function f are as shown in (5), (6), and (7) below. The readout delays of the three output events of the array readout circuit are 30 ns, 20 ns, and 30 ns respectively, and the input event (3, 2, 70 ns) is lost by the readout circuit.

[0182] α = {(2, 6, 50 ns), (3, 2, 70 ns), (5, 3, 100 ns), (3, 2, 120 ns)} (5)

[0183] β = {(2, 6, 80 ns), (5, 3, 120 ns), (3, 2, 150 ns)} (6)

[0184]

[0185] Through the provided method, the dynamic vision sensor can be accurately modeled using DVS datasets collected in multiple scenarios, and the event loss rate (the ratio of lost events to the total events) and the average readout delay of the successfully read events of the dynamic vision sensor in the specified scenario can be simulated.

[0186] The beneficial effects of the above technical solution are as follows: By combining simulation and event mapping relationships, the performance of the dynamic vision sensor in complex scenarios can be evaluated in an accurate manner, providing clear guidance for the optimization of DVS pixels and readout circuits. At the same time, through the input event sequences in multiple scenarios, it helps to improve the robustness and response efficiency of the dynamic vision sensor in practical applications.

[0187] In another embodiment, it further includes:

[0188] Using the pulse sequence in the DVS dataset to describe the output of the threshold comparator and serving as the excitation of the event latch, thereby realizing the modeling of the logarithmic photoreceptor, switched-capacitor amplifier, and threshold comparator in the DVS pixel circuit;

[0189] Using Boolean logic to model the event latch and pixel interface logic.

[0190] The working principle of the above technical solution is as follows: In the modeling of the dynamic vision sensor (DVS), the pulse sequence is an important input signal. By using the pulse sequence in the DVS dataset to describe the output of the threshold comparator and serving as the excitation of the event latch, the modeling of the DVS pixel circuit can be realized, especially the modeling of the logarithmic photoreceptor, switched-capacitor amplifier, and threshold comparator.

[0191] In a DVS, changes in the input image data are typically represented by an event stream (pulse sequence). Each event corresponds to a change in luminance, and the DVS captures dynamic information through asynchronous changes in time and space. By feeding the pulse sequence as an input signal into a threshold comparator, the threshold comparator compares the input signal against a preset threshold. When the input signal exceeds the threshold, the threshold comparator outputs a pulse signal.

[0192] Suppose a DVS pixel generates a luminance change (i.e., a pulse) at time t = 0, and this pulse signal is input to the threshold comparator. If the set threshold of the threshold comparator is a certain fixed value Vth, when the input signal is greater than Vth, the threshold comparator outputs a high-level pulse (e.g., 1). If the input signal is below the threshold, the threshold comparator has no output. This output signal can serve as the excitation signal for the event latch, thereby generating a latched output event.

[0193] The event latch is a key component in a dynamic vision sensor. It is responsible for latching the event state after receiving the output pulse from the threshold comparator. This latch records event information, such as the timestamp and spatial location of the event, based on the triggering of the pulse sequence. The output of the latch can then be processed by the pixel interface logic, which will further control or transmit the event.

[0194] To effectively manage the operation of the latch, Boolean logic can be used to model the event latch and the pixel interface logic. Boolean logic controls the input-output relationship through some simple logic gates (such as AND gates, OR gates, NOT gates, etc.), enabling the judgment of event states and the management of the event stream.

[0195] Suppose the input to the event latch is jointly determined by the output pulse of the threshold comparator and the clock signal. If the output of the threshold comparator is high (an event occurs), and the clock signal is valid (e.g., 1), then the latch outputs the state of the current event (e.g., saves the timestamp and location of the event). Through Boolean logic, conditions can be set such that if the clock signal is not valid, or the output of the threshold comparator has no pulse signal, the latch does not record the event.

[0196] The beneficial effects of the above technical solution are as follows: By accurately modeling modules such as the threshold comparator, event latch, logarithmic sensor, and switched-capacitor amplifier, the role of each module in DVS and their interrelationships can be deeply understood. This simulation provides valuable data support for DVS designers, enabling optimization of circuit design and improvement of system efficiency. By using Boolean logic to control the event latch and pixel interface logic, efficient event capture and status management can be achieved on hardware, contributing to improving the real-time response ability of the DVS system. At the same time, the simplicity of Boolean logic enables low-power hardware implementation, which helps to extend the usage time of the system, especially crucial in embedded systems. By using the method of exciting the event latch with a pulse sequence and the output signal of the threshold comparator, the brightness changes in a dynamic scene can be effectively captured and an accurate event stream can be generated. This method enables the DVS to efficiently capture minute changes in complex scenes, enhancing the performance of the dynamic vision sensor in fast-moving scenes or changing environments. The set threshold of the threshold comparator and the latch control logic can be adjusted according to actual application requirements. For example, in some application scenarios, it is desired to increase sensitivity to capture more minute changes; while in other scenarios, it is desired to reduce noise or unnecessary event outputs. By flexibly adjusting the threshold and logic settings, the performance of the DVS system can be optimized according to requirements.

[0197] In another embodiment, a pixel array is formed in a two-dimensional manner, including:

[0198] The pixel array includes a plurality of pixels, and each pixel corresponds to at least one pixel drive row request signal and at least one pixel drive column event signal. The pixel array realizes the row-by-row and column-by-column drive and reading of image information through this two-dimensional structure;

[0199] According to the two-dimensional pixel array structure, the timing relationship between the row request signal and the column event signal of each pixel is defined. The row request signal is used to trigger the pixel drive of the corresponding row, and the column event signal is used to generate an event stream related to the column. And this timing relationship ensures that within each cycle, all pixels can synchronously respond to the row request signal and the column event signal to complete the transmission and processing of image data.

[0200] The working principle of the above technical solution is as follows: In the design of an image sensor or a dynamic vision sensor (DVS), by constructing a two-dimensional pixel array, the row-by-row and column-by-column drive and reading of image information can be realized. The two-dimensional pixel array is composed of a plurality of pixels, and each pixel drives the information transmission of the row and column where the pixel is located through a row request signal (Row Request Signal) and a column event signal (ColumnEvent Signal) respectively.

[0201] The two-dimensional pixel array is composed of multiple pixel units, and each pixel unit can interact with its corresponding row and column. Each pixel unit has two main signals: Row request signal: used to trigger the driving of the row where the pixel is located. This signal is usually sent by the row driving circuit (row selector) of the pixel array. Column event signal: used to generate an event stream related to the column where the pixel is located. This signal is usually sent by the column selector and event processing unit of the pixel array, indicating that there are changes or events in the image data on the current column.

[0202] In such a structure, the row request signal is used to drive the pixel array row by row to read out each row of image data; while the column event signal is associated with the pixel activities in that column and is usually used to represent the dynamic changes or trigger event streams of the pixels in that column.

[0203] Within each cycle, all pixels need to synchronously respond to the row request signal and the column event signal. To ensure synchronization, the timing of the row request signal and the column event signal needs to be precisely designed to ensure that within each clock cycle, the image data can be transmitted stably and accurately.

[0204] Triggering of the row request signal: The row request signal triggers the pixels of the pixel array row by row. For example, at a certain moment in the cycle, the row request signal first triggers the pixels in the first row, and then triggers the pixels in the second row at the next moment, and so on. This ensures that the image data is read out row by row in sequence.

[0205] Synchronization of the column event signal: The column event signal is related to the pixel activities in the column, and the triggering of the column signal is usually based on a specific change pattern (such as the change of pixel values in the image). Within each cycle, when an event occurs to the pixels in a certain column, the column event signal will be activated, thereby generating an event stream for that column. Through precise timing control, it is ensured that within each cycle, all pixels can synchronously respond to the row request and column event signals, thus completing the effective transmission and processing of the image data.

[0206] Under the two-dimensional pixel array structure, the row request signal and the column event signal work together, enabling the entire pixel array to efficiently drive and read the image data row by row and column by column. Through this timing relationship, the transmission of the image data can not only ensure synchronization but also handle the local area changes of the image at each moment. Through the activation of the column event signal, the system can capture and process the dynamic changes in the image in real time, thereby generating accurate image output or event streams.

[0207] The beneficial effects of the above technical solution are as follows: Through the precise timing control of the row request signal and the column event signal, it is ensured that each pixel can accurately respond to the corresponding drive signal. Whether it is the reading of a static image or the capture of dynamically changing events, the pixel array can efficiently transmit data in a predetermined order and timing, ensuring the synchronization of the entire system.

[0208] In another embodiment, a DVS pixel array model is constructed based on pseudocode description, including:

[0209] The event stream in the DVS dataset represents the response of each pixel to event changes at different time points;

[0210] In the DVS pixel array model, an event latch is set for each pixel to latch the column event signal corresponding to the pixel and generate a corresponding output signal when an event arrives, and this output signal represents the change in the pixel state.

[0211] The working principle of the above technical solution is as follows: DVS is an event-driven vision sensor. Different from traditional frame image cameras, DVS generates a data stream by monitoring the brightness changes of each pixel point in the scene. Each pixel emits an event when the brightness of the scene changes. These events occur continuously, rather than based on a fixed time frame. This way enables DVS to capture fast movements and changes in high dynamic range, especially suitable for capturing high-speed moving objects, rapidly changing scenes, and details in low-light environments.

[0212] In the pixel array of DVS, each pixel is equipped with an event latch. The function of this latch is to record and hold the event signal of each pixel. When the brightness of a certain pixel in the scene changes (for example, from light to dark or from dark to light), an event will be generated and transmitted to the corresponding latch. This latch generates an output signal by "latching" the event, and this signal reflects the change in the state of the pixel. For example, when a certain part of the scene suddenly brightens, the DVS sensor detects this change and generates an event. This event will be captured and output by the latch of the corresponding pixel. When the brightness change returns, the state of the latch will be updated to a new output, reflecting the real-time change of the scene.

[0213] The beneficial effects of the above technical solution are as follows: By adopting the event stream and event latch mechanism, the DVS sensor can accurately capture every tiny change in the scene and output corresponding event signals in real time. Compared with the traditional frame image processing method, this method has obvious advantages in dynamic response, low latency, high dynamic range, and low power consumption. Whether it is high-speed motion capture, real-time object detection, dynamic scene analysis, or applications in low-light environments, DVS has demonstrated its unique advantages. For example, in autonomous driving, robot navigation, and fast motion analysis, DVS provides more reliable data support for real-time decision-making.

[0214] In another embodiment, the DVS array model is excited by inputting an event sequence, including:

[0215] The event sequence is composed of DVS data sets in multiple scenarios. The data sets include DVS data collected under different lighting conditions, motion speeds, and object shapes, ensuring the diversity of the event sequence;

[0216] In the DVS data set, each event includes spatial coordinate information and timestamp information detected by the DVS sensor. The coordinate information is used to represent the position of the event in the sensor's field of view, and the timestamp information is used to represent the time when the event occurs, facilitating subsequent chronological processing and analysis of the events;

[0217] Based on the DVS data set, an event sequence containing multiple time points is generated. Each event is sorted according to the timestamp to obtain a continuous event sequence arranged in chronological order, ensuring that the event sequence reflects the dynamic changes in the actual scene;

[0218] The event sequence is input into the DVS pixel array model for excitation. The DVS pixel array model simulates the response according to the input event sequence and generates a simulated visual output, which is used to further analyze the perception ability and response characteristics of the DVS array in complex environments.

[0219] The working principle of the above technical solution is as follows: The DVS data set is composed of event data in multiple scenarios, which cover different lighting conditions, object shapes, and motion speeds to ensure the diversity of the event sequence. This diversity enables the generated event sequence to reflect visual changes in various dynamic environments. For example, the data set can include scenes from low-light environments to strong-light environments, and changes from stationary objects to high-speed moving objects. These data help analyze the performance of DVS under different conditions.

[0220] In the DVS dataset, each event contains two main pieces of information: spatial coordinates and timestamp. The spatial coordinates represent the location where the event occurs, that is, the position of the pixel corresponding to the event in the sensor's field of view; the timestamp identifies the exact time when the event occurs, usually in microseconds. With these two pieces of information, the temporal characteristics of the events can be accurately reconstructed, facilitating subsequent dynamic analysis and processing. For example, in a scene with a fast-moving object, DVS records the brightness changes of each pixel at different time points and outputs corresponding events. By using the timestamps, the events can be sorted to restore the object's motion trajectory.

[0221] By sorting each event according to the timestamp, a continuous event sequence is generated. This event sequence reflects the dynamic changes in the scene. The time interval between events can be very short (e.g., a few microseconds), which enables DVS to accurately capture rapid light changes and object movements. For example, in the case of high-speed motion, traditional frame images cannot capture details due to frame rate limitations, while DVS can capture each dynamic change with a continuous event stream.

[0222] By inputting the event sequence into the DVS pixel array model, the model simulates the response of the pixels based on the input events, thereby generating a visual output. The excitation response of the DVS pixel array model can reflect how the sensor processes light changes and object movements in the actual scene. The generated visual output can be used to further analyze the perception ability and response characteristics of the DVS array in complex environments, such as its performance under high dynamic range, fast-moving objects, and changing light conditions.

[0223] The beneficial effects of the above technical solutions are as follows: By constructing a diverse DVS dataset, generating an accurate event sequence, and using the DVS pixel array model for excitation and response, the system can accurately perceive and reconstruct the changes in the real world under different environments and dynamic scenarios. This event-driven perception mode not only improves the response speed and accuracy but also effectively reduces the data storage and processing burden, especially having important application values in fields such as autonomous driving, robot control, and fast dynamic monitoring.

[0224] In another embodiment, it further includes:

[0225] The DVS pixel array model processes the events based on the coordinate information and timestamp information of the event sequence, and uses the temporal and spatial distribution relationship of the events as input to simulate the visual perception process of the DVS array in a specific scenario, and outputs information including image features and motion trajectories.

[0226] The working principle of the above technical solution is as follows: The DVS pixel array model processes according to the input event sequence. Each event includes spatial coordinate information and timestamp information, where the spatial coordinates represent the position in the sensor's field of view when the event occurs, and the timestamp represents the exact moment when the event occurs. By processing this information, the DVS model can accurately simulate the visual perception process. Coordinate information: Through the spatial coordinates, the DVS model can determine the pixel position where each event occurs, thus assigning a specific spatial position to each pixel. Timestamp information: Through the timestamp, the DVS model can accurately arrange the events in chronological order and simulate the temporal evolution of the events, reflecting the dynamic changes in the scene. For example, in a scene with a fast-moving object, as the object moves, pixels at different positions will generate events according to the light changes. The DVS pixel array model can accurately restore the motion trajectory and image features of the object using the coordinate and time information of these events.

[0227] Simulating the visual perception process: In the model, the temporal and spatial distribution relationship of the event sequence is the key input. The model will simulate the response process of each pixel according to the temporal information of the events, reflect the dynamic changes of the image, and then simulate the motion trajectory of the object, the light change, and its impact on visual perception. By continuously updating and processing these events, the DVS pixel array model can output information including image features (such as edges, textures, etc.) and the object's motion trajectory in a specific scene. This information not only reveals the basic characteristics of visual perception but can also be used for subsequent image understanding and analysis tasks, such as object detection, behavior recognition, trajectory prediction, etc.

[0228] Outputting image features and motion trajectories: Through the input of the event sequence and spatio-temporal analysis, the output of the model includes: Image features: including the object contour, texture changes, edge information, etc. in the scene. These features reflect the shape of the object and the lighting conditions. Motion trajectory: Based on the chronological sorting of timestamps, the DVS model can calculate the moving trajectory of the object in the visual scene, showing information such as the object's motion path and speed. The output of this model is a continuous dynamic data stream, rather than a traditional static image. Therefore, it can provide continuous dynamic visual information in an extremely short time and is suitable for real-time monitoring and perception in dynamic environments.

[0229] The beneficial effects of the above technical solution are as follows: By taking the coordinate information and timestamp information of the event sequence as inputs, the DVS pixel array model can accurately simulate the visual perception process and output image features and motion trajectory information. The beneficial effect of this model lies in its ability to provide higher-precision perception and analysis capabilities in complex environments such as dynamic, high-speed, and low-light conditions. By optimizing image feature extraction, motion tracking, and perception capabilities, the DVS pixel array model provides significant technical advantages for fields such as real-time monitoring, autonomous driving, and robot vision, promoting the development of various high-performance applications.

[0230] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.< / m> < / m> < / m> < / m> < / n> < / n> < / m> < / m> < / m> < / m> < / n> < / n>

Claims

1. A method for modeling and evaluating the performance of a dynamic visual sensor using a DVS event sequence, characterized in that: include: Step 1: Based on the pulse sequence and Boolean logic technology in the DVS data set, the DVS pixel circuit is modeled to obtain the DVS pixel circuit model; Step 2: Based on the structure of the DVS array, the DVS pixel array is modeled to obtain a DVS pixel array model; Step 3: Modeling the mainstream array readout circuit based on the pseudo code corresponding to each mainstream array readout circuit to obtain the mainstream array readout circuit model; Step 4: Simulate the DVS pixel circuit model, DVS pixel array model, and mainstream array readout circuit model to evaluate the DVS performance.

2. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 1, characterized in that: The DVS pixel circuit includes: a logarithmic sensor, a switched capacitor amplifier, a threshold comparator, an event latch, and a pixel interface logic; Get the DVS pixel circuit model, including: The logarithmic receptor converts the photocurrent into a logarithmic voltage, providing the initial signal conversion for the DVS pixel circuit; The switched capacitor amplifier includes an input capacitor C A , feedback capacitor C B and amplifier A2, input capacitor C A and feedback capacitor C B The connection forms a capacitor amplifier structure, and the switched capacitor amplifier amplifies the AC part of the logarithmic voltage to provide an enhanced signal for subsequent signal processing; The threshold comparator is connected to the output of the switched capacitor amplifier module, detects and outputs ON events and OFF events, and the threshold comparator generates event pulses by comparing the input signal with a preset threshold. The event latch receives and latches the ON event and OFF event output by the threshold comparator. The event latch controls the latch state through Boolean logic to achieve stable storage of events. The pixel interface logic is connected to the event latch and outputs the latched event through the DVS array interface. The pixel interface logic uses Boolean logic to implement logical processing and output control of the event; The logarithmic sensor, switched capacitor amplifier, threshold comparator, event latch and pixel interface logic are connected in series to form a complete DVS pixel circuit model; The pulse sequence in the DVS data set is used to describe the output of the threshold comparator and serve as the excitation of the event latch, thereby realizing the modeling of the logarithmic sensor, switched capacitor amplifier and threshold comparator in the DVS pixel circuit.

3. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 1, characterized in that: Get the DVS pixel array model, including: A pixel array is formed in a two-dimensional manner, wherein the pixel array includes a pixel driving row request signal and a pixel driving column event signal; Construct a DVS pixel array model based on a pseudocode description, which includes the event flow in the DVS data set and the output of each pixel event latch in the DVS array; The interface of the pixel array includes interfaces of a row request signal Rreq, a row response signal Rack, a row reset signal Rrst, and column event signals Con and Coff.

4. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 1, characterized in that: Obtain mainstream array readout circuit models, including: For the event-driven DVS array, the first readout circuit model based on the first-in-first-out mechanism is constructed through the pseudo code of the first-in-first-out mechanism; According to the order of row numbers, a second readout circuit model of sequential row selection is constructed through a sequential row selection pseudo code, wherein the second readout circuit responds to rows with event requests row by row according to the row priority of the event request and in the order of row numbers from small to large; Based on the pseudo code of the frame-by-frame method, a third readout circuit model of the frame-by-frame method is constructed, wherein the third readout circuit accumulates events within a fixed period of time through the DVS array, and outputs the third readout circuit model as a complete event frame for subsequent processing; Based on the priority arbitration pseudocode, a fourth readout circuit model for priority arbitration of the region of interest is constructed, wherein the programmable priority of each row of pixels is preset and events are read according to a predetermined priority order.

5. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 1, characterized in that: Simulate the DVS pixel circuit model, DVS pixel array model, and mainstream array readout circuit model, including: The DVS array model is stimulated by an input event sequence, which consists of DVS data sets collected in various scenarios, where each event includes coordinate and timestamp information; Run the array readout circuit model in the simulation environment, read out the events in the DVS array, generate an output event sequence, and associate the events in the output event sequence with the events in the input event sequence through an event mapping relationship; Calculate the readout delay of each event in the output event sequence relative to the corresponding event in the input event sequence, and mark the events that are lost because they are not responded to in time by the readout circuit; Based on the event mapping relationship and readout delay, the performance indicators of dynamic vision sensors in specified scenarios are evaluated, including event loss rate and average readout delay of successfully read events; The performance evaluation results of the dynamic vision sensor are output. The performance evaluation results are used to optimize the design and configuration of the DVS pixel circuit model and the array readout circuit model to improve the performance of the dynamic vision sensor.

6. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 2, characterized in that: Also includes: The pulse sequence in the DVS dataset is used to describe the output of the threshold comparator and as the stimulus of the event latch, so as to realize the modeling of the logarithmic sensor, switched capacitor amplifier and threshold comparator in the DVS pixel circuit. The event latch and pixel interface logic are modeled using Boolean logic.

7. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 3 is characterized in that: The pixel array is formed in a two-dimensional manner, including: The pixel array includes a plurality of pixels, each pixel corresponds to at least one pixel driving row request signal and at least one pixel driving column event signal, and the pixel array realizes row-by-row and column-by-column driving and reading of image information through the two-dimensional structure; According to the two-dimensional pixel array structure, the timing relationship between the row request signal and the column event signal of each pixel is defined. The row request signal is used to trigger the pixel drive of the corresponding row, and the column event signal is used to generate an event stream related to the column. This timing relationship ensures that within each cycle, all pixels can synchronously respond to the row request signal and the column event signal to complete the transmission and processing of image data.

8. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 3 is characterized in that: The DVS pixel array model is constructed based on the pseudocode description, including: The event stream in the DVS dataset represents the response of each pixel to event changes at different time points; In the DVS pixel array model, an event latch is set for each pixel to latch the column event signal corresponding to the pixel and generate a corresponding output signal when an event arrives. The output signal indicates the change of the pixel state.

9. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 5, characterized in that: The DVS array model is stimulated by a sequence of input events, including: The event sequence is composed of DVS data sets in multiple scenarios. The data sets include DVS data collected under different lighting conditions, motion speeds, and object shapes to ensure the diversity of the event sequence; In the DVS dataset, each event includes spatial coordinate information and timestamp information detected by the DVS sensor. The coordinate information is used to indicate the location of the event in the sensor field of view, and the timestamp information is used to indicate the time when the event occurred, which facilitates subsequent time series processing and analysis of the event. Based on the DVS dataset, an event sequence containing multiple time points is generated, and each event is sorted by timestamp to obtain a continuous, chronologically arranged event sequence, ensuring that the event sequence reflects the dynamic changes in the actual scene; The event sequence is input to the DVS pixel array model for stimulation. The DVS pixel array model simulates the response according to the input event sequence and generates simulated visual output. The visual output is used to further analyze the perception ability and response characteristics of the DVS array in complex environments.

10. The method for dynamic visual sensor performance modeling and evaluation based on DVS event sequence according to claim 9, characterized in that: Also includes: The DVS pixel array model processes events according to the coordinate information and timestamp information of the event sequence, and takes the temporal and spatial distribution relationship of events as input to simulate the visual perception process of the DVS array in a specific scene, and outputs information including image features and motion trajectories.

Images