Intra-pixel event data generation method and event visual sensor

Abstract

This disclosure provides a method for generating intra-pixel event data and an event vision sensor. The method is applied to an event vision sensor comprising multiple pixel units. The method includes: determining multiple periodic discrete time points to form a time reference sequence; synchronously sampling the brightness signal of the photosensitive unit of each pixel unit at each discrete time point, and determining the brightness change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points; comparing the brightness change information of each pixel unit with a preset threshold, and generating event data for each pixel unit based on the comparison result and the timestamp of each discrete time point, wherein the comparison of the brightness change information with the preset threshold occurs within a time window corresponding to each discrete time point and located between two discrete time points. This disclosure can constrain the time of event data generation, generating ordered event data.

Description

Technical Field

[0001] This disclosure relates to the field of sensor technology, and in particular to a method for generating intra-pixel event data and an event vision sensor. Background Technology

[0002] An event-based vision sensor (EVS), also known as an event camera, only generates data when a pixel detects a change in brightness. Therefore, it can record only pixels in a scene whose brightness changes. Each pixel in an event-based vision sensor can operate independently. When the brightness of the light detected by a pixel changes and the change exceeds a threshold, the corresponding pixel triggers data generation; this data is called event data. By outputting brightness change events, rather than complete image frames, event-based vision sensors achieve low latency, high dynamic range, and low power consumption in visual perception.

[0003] The existing event vision sensors generate event data based solely on changes in brightness. The timing of event data generation is unrestricted; the generation time can be concentrated or sparse, making it difficult to utilize the event data more effectively in later stages. Summary of the Invention

[0004] This disclosure provides a method for generating intra-pixel event data and an event sensor, which can constrain the timing of event data generation to generate ordered event data, enabling the event data to be used more effectively in later stages.

[0005] According to a first aspect, embodiments of this disclosure provide a method for generating intra-pixel event data, applied to an event visual sensor, the event visual sensor including multiple pixel units, each pixel unit including a photosensitive unit, the method comprising: determining multiple periodic discrete time points for triggering sampling of the brightness signal of the photosensitive unit, the discrete time points constituting a time reference sequence for each pixel unit to generate the event data; at each discrete time point in the time reference sequence, synchronously sampling the brightness signal of the photosensitive unit of each pixel unit, and determining brightness change information of each pixel unit between adjacent discrete time points based on a time structure defined by the discrete time points; comparing the brightness change information of each pixel unit with a preset threshold, and generating event data of each pixel unit based on the comparison result and the timestamp of each discrete time point, wherein the comparison of the brightness change information with the preset threshold occurs within an event evaluation time window corresponding to each discrete time point and located between two discrete time points.

[0006] According to a second aspect, embodiments of this disclosure provide an event vision sensor, comprising: a plurality of pixel units, each pixel unit including a photosensitive unit; a reference time determination module configured to determine a plurality of periodic discrete time points for triggering sampling of the brightness signal of the photosensitive unit, the discrete time points constituting a time reference sequence for each pixel unit to generate the event data; a change information determination module configured to synchronously sample the brightness signal of the photosensitive unit of each pixel unit at each discrete time point in the time reference sequence, and determine the brightness change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points; and an event data generation module configured to compare the brightness change information of each pixel unit with a preset threshold, and generate event data of each pixel unit based on the comparison result and the timestamp of each discrete time point, wherein the comparison of the brightness change information with the preset threshold occurs within an event evaluation time window corresponding to each discrete time point and located between two discrete time points.

[0007] The pixel-based event data generation method and event vision sensor of this disclosure use a time structure defined by periodic discrete time points as a time reference sequence to trigger the sampling of the brightness signal of the photosensitive unit of the pixel unit. This can constrain the time of event data generation by the pixel unit, limiting the generation of event data to discrete time points. This allows the generation of event data to be driven by both the brightness change of the photosensitive unit and the discrete time points, making the generation of event data orderly and constrained. This is beneficial for the event data to be used more effectively in the later stages.

[0008] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] Figure 1 This is a flowchart of a method for generating intra-pixel event data according to an embodiment of the present disclosure; Figure 2 This is a schematic diagram of discrete time points and corresponding event signals according to embodiments of the present disclosure; Figure 3A This is a schematic diagram illustrating the random occurrence of events using existing visual sensors. Figure 3B This is a schematic diagram showing the asynchronous events of a single pixel in an existing event vision sensor arranged on a timeline; Figure 4 This is a flowchart for determining the brightness change information of a pixel unit according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of a discrete-time sampling circuit according to some embodiments of the present disclosure; Figure 6 This is a flowchart illustrating how event data for pixel units is generated by judging brightness change information according to an embodiment of this disclosure. Figure 7 This is a timing diagram of multi-phase time-division multiplexing according to some embodiments of this disclosure; Figure 8 This is a schematic diagram of devices multiplexing in different phases according to some embodiments of this disclosure; Figure 9 This is a block diagram of an event vision sensor according to an embodiment of the present disclosure. Detailed Implementation

[0010] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0011] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0012] Figure 1 A flowchart 100 of an intra-pixel event data generation method according to an embodiment of the present disclosure is shown. The intra-pixel event data generation method of the present disclosure is applied to an event vision sensor, which includes multiple pixel units, each pixel unit including a photosensitive unit, such as... Figure 1 As shown, process 100 may include the following steps: Step 101: Determine multiple periodic discrete time points for triggering the sampling of the brightness signal of the photosensitive unit. These multiple discrete time points constitute a time reference sequence for the generation of event data by each pixel unit.

[0013] In this embodiment, the execution entity can be an event vision sensor, which can determine multiple periodic discrete time points for triggering the sampling of the brightness signal of the photosensitive unit. These multiple discrete time points can constitute a time reference sequence for each pixel unit to generate event data.

[0014] In this embodiment, discrete time points can refer to specific moments that are independent and discrete, selected according to certain rules on a continuous time axis. The time reference sequence can contain at least two discrete time points, providing a basis for each photosensitive unit to determine the brightness change information of adjacent discrete time points. The embodiments of this disclosure do not limit the number of discrete time points contained in the time reference sequence.

[0015] In this embodiment, as Figure 2 As shown, discrete time points t0, t1, t2, t3... can be selected on a continuous time axis at the same time interval. The fixed time interval is the period of recurrence of the discrete time points. The embodiments of this disclosure do not limit the time interval for selecting discrete time points.

[0016] In this embodiment, discrete time points can be selected according to a preset time interval. Each discrete time point can have a definite time coordinate, and all photosensitive units are sampled at each discrete time point. The time of each sample is known. For example, a discrete time point can be selected every 100μs, and the relative time coordinates of the discrete time points can be obtained as 0μs, 100μs, 200μs, 300μs, etc.

[0017] In one embodiment, a synchronous clock can be used to provide periodic discrete time points. By providing a unified clock reference for all pixel units, the temporal consistency of event data generated by all pixel units can be ensured. For example, a high-precision clock signal can be generated by a master clock, and sampling control signals can be distributed to each photosensitive unit through a clock distribution network, thereby enabling periodic sampling of the brightness state values ​​of each photosensitive unit. The time reference sequence composed of discrete time points can be expressed as Equation 1: (Formula 1) in, It is the period of the time reference sequence; It is an integer and ,For example .

[0018] Step 102: At each discrete time point in the time reference sequence, the brightness signal of the photosensitive unit of each pixel unit is synchronously sampled, and the brightness change information of each pixel unit between adjacent discrete time points is determined based on the time structure defined by the discrete time points.

[0019] In this embodiment, the execution entity can synchronously sample the brightness signal of the photosensitive unit of each pixel unit at each discrete time point in the time reference sequence, and based on the time structure defined by the discrete time points, determine the brightness change information of each pixel unit between adjacent discrete time points according to the sampled brightness state value and the brightness state value obtained in the previous sampling.

[0020] In this embodiment, the photosensitive unit is a device capable of converting light signals into electrical signals. For example, the photosensitive unit can be a photosensitive element of a pixel unit in an event vision sensor (EVS), a hybrid vision sensor (HVS), a high-speed CMOS image sensor, or a neuromorphic vision sensor; the embodiments disclosed herein are not limited to this. Multiple pixel units in an event vision sensor can be arranged in an array. For example, multiple pixel units can form an m×n sensor array consisting of m rows and n columns.

[0021] In this embodiment, sampling the luminance signal of the photosensitive unit can refer to acquiring the continuous luminance electrical signal output by the photosensitive unit at discrete time points to obtain a luminance state value that reflects the luminance state of the light received by the photosensitive unit. The luminance state value is usually represented by the current value or voltage value of the photosensitive unit at discrete time points.

[0022] In this embodiment, at each discrete time point determined by the time reference sequence, the sampling operation of the brightness state values ​​of all photosensitive units in the event visual sensor is performed synchronously. For each photosensitive unit, a brightness state value is collected at different discrete time points in the time reference sequence, thereby obtaining the discrete brightness state values ​​of that photosensitive unit.

[0023] In one embodiment, an event vision sensor containing m×n pixels also has m×n photosensitive units. Discrete time points are set as t0=0μs, t1=100μs, t2=200μs, t3=300μs, and so on. At time t0, all m×n photosensitive units are sampled simultaneously to obtain the brightness state value of each unit. At time t1, all m×n photosensitive units are sampled simultaneously again to obtain their respective brightness state values, and so on. This method ensures that each photosensitive unit acquires its brightness state value at the same time point.

[0024] In this embodiment, the brightness state value can represent the perception state of the photosensitive unit of external light, i.e., ambient brightness, at each discrete time point. If the external light, i.e., ambient brightness, does not change, the brightness state of the photosensitive unit will remain consistent, and the discrete brightness state values ​​will be basically the same (ignoring the influence of noise); if the external light, i.e., ambient brightness, changes, the brightness state of the photosensitive unit will change accordingly, and the discrete brightness state values ​​will fluctuate accordingly.

[0025] Therefore, in this embodiment, for each photosensitive unit, the amount of change in the brightness state of the photosensitive unit can be determined by the brightness state values ​​of the photosensitive unit collected at two adjacent discrete time points, such as the brightness state value obtained by the current sampling and the brightness state value obtained by the previous sampling, thereby obtaining the brightness change information of the photosensitive unit.

[0026] Step 103: Compare the brightness change information of each pixel unit with the preset threshold, and generate event data for each pixel unit based on the comparison results and the timestamp of each discrete time point. The comparison of brightness change information with the preset threshold occurs within the event evaluation time window corresponding to each discrete time point and located between two discrete time points.

[0027] In this embodiment, the execution entity can compare the brightness change information of each pixel unit with a preset threshold, and generate event data for each pixel unit based on the comparison result and the timestamp of each discrete time point. The comparison of brightness change information with the preset threshold occurs within the event evaluation time window corresponding to each discrete time point and located between two discrete time points. The generation of event data is constrained by the time structure defined by the discrete time points, rather than based on a continuous time triggering mechanism.

[0028] In this embodiment, the event evaluation time window refers to a time interval after each discrete time point arrives, during which the brightness change information at that discrete time point is compared and evaluated to generate event data. The length of the event evaluation time window needs to satisfy the following condition: the comparison and evaluation must be completed between the current discrete time point and the next discrete time point to avoid affecting the sampling and processing of the next discrete time point. The event evaluation of all photosensitive units of the event vision sensor only occurs within the event evaluation time window corresponding to each discrete time point in the time reference sequence.

[0029] In this embodiment, setting an event evaluation time window allows sufficient preparation time for the sampling operation at the next discrete time point, ensuring that the processes of brightness state value sampling, brightness change information determination, and event evaluation can proceed continuously and stably without process conflicts. This guarantees the real-time generation of event data and avoids overlap between event evaluation and the sampling operation at the next discrete time point.。

[0030] In this embodiment, the preset threshold can be a pre-set judgment condition used to determine what type of event data the brightness change information generates. For example, the preset threshold may include a positive event threshold and a negative event threshold. When the brightness change information is greater than the positive event threshold, it is determined that the photosensitive unit has generated a positive event, and positive event data is generated. When the brightness change information is less than the negative event threshold, it is determined that the photosensitive unit has generated a negative event, and negative event data is generated. When the brightness change information is between the positive event threshold and the negative event threshold, it is determined that the photosensitive unit has generated a zero event, and zero event data is generated.

[0031] In one embodiment, the threshold decision can be made according to Formula 2: (Formula 2) in, It is an event signal used to indicate the type of event data; It is the positive event threshold, and is a positive value; It is the threshold for negative events, and is a negative value; It is the change in brightness state of a pixel unit with pixel coordinates (x, y) at the kth discrete time point, i.e., brightness change information.

[0032] In this embodiment, after comparing the brightness change information with the preset threshold and generating the event signal corresponding to each photosensitive unit, the timestamp of the current discrete time point can be obtained. The timestamp is then associated with all event signals generated at the current discrete time point to generate corresponding event data. That is, all event signals generated at the current time point are assigned the timestamp of the current time point, so that each event signal carries a timestamp of the current time point.

[0033] In this embodiment, a timestamp is a time value corresponding to a discrete time point, reflecting the specific moment when the event data was generated. A timestamp can be an integer on the order of microseconds (μs), with an accuracy reaching the microsecond level. Typically, a timestamp can be represented by the number of microseconds elapsed from when the sensor starts operating (or at a certain reference time) to that discrete time point.

[0034] In this embodiment, by assigning a timestamp to all event signals generated at the current time, all event signals can be made to monotonically increase in chronological order of their generation, thus satisfying the monotonicity constraint of the timing sequence. The relationship is shown in Equation 3: (Formula 3) Wherein, for any pixel unit, if the occurrence time of event A is... Earlier than the time of event B Then the event signal of event A The timestamp must be less than the event signal of event B. Timestamp.

[0035] In this embodiment, the timing order of event signals can be ensured by the monotonicity constraint of the timing, avoiding the situation where timestamps are reversed. By defining discrete time points as globally unique time series, the event signals generated according to the time series can meet strict timing monotonicity and match the actual timing of the events, thus eliminating the influence of event arbitration or transmission on the order of event data.

[0036] The pixel-based event data generation method provided in this disclosure uses a time structure defined by periodic discrete time points as a time reference sequence to trigger sampling of the brightness signal of the photosensitive unit of the pixel unit. This constrains the generation time of event data by limiting the event data generation to discrete time points. This ensures that the generation of event data is driven by both the brightness change of the photosensitive unit and the discrete time points, making the event data generation orderly and constrained. This facilitates more effective utilization of the event data later. For example, event signals with the same timestamp can obtain synchronized event data, and all event data with the same timestamp can form an event frame.

[0037] The core of the pixel-based event data generation method provided by the embodiments of this disclosure lies in reconstructing the definition of event data through a periodic discrete-time structure, changing the generation of event data from a continuous-time triggering mechanism to a discrete-time driven mechanism. That is, event data is generated at various discrete time points defined under a unified time base, which is different from the existing continuous-time triggering method.

[0038] In existing event vision sensors, the brightness of the photosensitive unit is the sole determining factor in generating event data. Event data is generated whenever the brightness of the photosensitive unit changes, and the timing of event data generation is unrestricted; therefore, this is a continuously triggered mechanism. Figure 3A As shown, in arrive Events that cause changes in the brightness of the photosensitive unit may continue to occur within a certain time period, such as... Figure 3B As shown, an event in which the brightness of the photosensitive unit changes at a certain moment is represented by a vertical line. In real-world scenarios, the triggering time of events may be concentrated or sparse, and the triggering time is completely unrestricted.

[0039] Figure 4 The flowchart illustrating an embodiment of this disclosure describes a process for determining brightness variation information of a pixel unit. For example... Figure 4As shown, step 102, at each discrete time point in the time reference sequence, synchronously samples the brightness signal of the photosensitive unit of each pixel unit, and determines the brightness change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points. This may include the following steps: Step 401: In response to the target discrete time point of the arrival time reference sequence, the brightness signal of the photosensitive unit of each pixel unit is synchronously sampled to obtain the current brightness state value of each pixel unit.

[0040] In this embodiment, the execution entity can be an event vision sensor. In response to the arrival of a target discrete time point in the time reference sequence, the event vision sensor synchronously samples the brightness signal of the photosensitive unit of each pixel unit to obtain the current brightness state value of each pixel unit. The target discrete time point can be any discrete time point in the time reference sequence other than the first discrete time point. Multiple periodic discrete time points can be generated based on a unified clock reference. The determination of brightness change information, the comparison of brightness change information with a preset threshold, and the generation of event data are all completed within a uniformly scheduled synchronous timing sequence.

[0041] In one embodiment, such as Figure 5 As shown, the event vision sensor may include a photosensitive unit consisting of a photodiode (PD) and an analog front end (AFE), a controllable switch (S1), and a storage capacitor (Cstore). At the sampling time, the controllable switch (S1) is closed, and the voltage of the photosensitive unit at the sampling time is input into the storage capacitor (Cstore). Then, the controllable switch (S1) is opened, and the voltage on the storage capacitor (Cstore) is maintained until the next sampling.

[0042] When discrete time points arrive, a synchronous trigger sampling control signal is sent to the control terminals of all photosensitive units in the event vision sensor, for example... Figure 5 The controllable switch S1 can synchronously sample and save the brightness state values ​​of all or part of the photosensitive units at the sampling time. It should be noted that the time interval between discrete time points needs to match the speed of the sampling process to ensure that all sampling operations can be completed between two adjacent discrete time points, avoiding incomplete sampling. For example... Figure 5 The response time of the storage capacitor Cstore is less than the time interval between discrete time points.

[0043] Step 402: Determine the current brightness change information of each pixel unit based on the current brightness state value and the reference brightness state value of each pixel unit.

[0044] In this embodiment, after obtaining the current brightness state value of each pixel unit by synchronously sampling the brightness signal of the photosensitive unit, the aforementioned execution entity can determine the current brightness change information of each pixel unit based on the current brightness state value and the reference brightness state value. The reference brightness state value is the brightness state value of each pixel unit at the previous discrete time point adjacent to the target discrete time point.

[0045] In this embodiment, determining brightness change information based on brightness state values ​​between adjacent discrete time points can be achieved using at least one of the following methods: difference operation, ratio operation, time-domain filtering, and prediction error based on historical states. For example, a combination of at least two of the above methods can be used. The embodiments of this disclosure do not limit the implementation method for determining brightness change information based on brightness state values.

[0046] In one embodiment, differential operations can be used to calculate the change in brightness state between adjacent discrete time points based on the brightness state value of each photosensitive unit at the current discrete time point and the brightness state value at the previous discrete time point, thus obtaining the brightness change information of each photosensitive unit between adjacent discrete time points. The current discrete time point can be denoted as... (k > 0); the previous discrete time point refers to the discrete time point in the time reference sequence that is adjacent to the current discrete time point and precedes the current discrete time point, and can be denoted as... . It can represent the first discrete time point within a certain processing time period, which is the first discrete time point relative to the local time domain. The local time domain can be selected and defined as needed.

[0047] In this embodiment, the current discrete time point Compared with the previous discrete time point These are two adjacent discrete time points, and the time interval between them is the period of the time reference sequence. If we skip a discrete time point during the difference operation, for example, for a discrete time point... and The brightness state value is differentially calculated, which is equivalent to calculating according to twice the period. Performing differential operations is essentially equivalent to taking time reference sequences with different periods. It is important to clarify that the time interval between two adjacent discrete time points used to determine brightness change information should be sufficient to reflect the actual brightness changes of the photosensitive unit. If the period is too long, brightness changes occurring within the period may not be captured and reflected.

[0048] Step 403: Update the reference brightness state value of each pixel unit according to the current brightness state value of each pixel unit.

[0049] In this embodiment, after determining the current brightness change information of each pixel unit based on the current brightness state value and the reference brightness state value, the execution entity can update the reference brightness state value of each pixel unit according to the current brightness state value of each pixel unit. The updated reference brightness state value, after acquiring the brightness state value at the next discrete time point, can be compared with the brightness state value at the next discrete time point to determine the brightness change information. Steps 401, 402, and 403 are iteratively executed until the last discrete time point in the time reference sequence, completing the sampling of the brightness state values ​​of all photosensitive units and the determination of the brightness change information.

[0050] Optionally, step 102, which involves synchronously sampling the luminance signal of the photosensitive unit of each pixel unit at each discrete time point in the time reference sequence, and determining the luminance change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points, may further include the following step: in response to the arrival of the first discrete time point in the time reference sequence, synchronously sampling the luminance signal of the photosensitive unit of each pixel unit to obtain a reference luminance state value for each pixel unit. This step is used at the first discrete time point in the time reference sequence. At this point, the brightness status values ​​of all photosensitive units are sampled. After obtaining the brightness status values, it is only necessary to store the brightness status values, without needing to determine the brightness change information or generate event data.

[0051] Figure 6 The present disclosure illustrates a flow of generating event data for pixel units by judging brightness change information. For example... Figure 6 As shown, step 103, which compares the brightness change information of each pixel unit with a preset threshold and generates event data for each pixel unit based on the comparison results and the timestamp of each discrete time point, may include the following steps: Step 601: Within the event evaluation time window corresponding to the current discrete time point, compare the current brightness change information of each pixel unit with the preset threshold to determine the current brightness change direction of each pixel unit.

[0052] In this embodiment, the execution entity can be an event vision sensor. After determining the current brightness change information at a target discrete time point, the event vision sensor compares the current brightness change information of each pixel unit with a preset threshold within the event evaluation time window corresponding to the current discrete time point to determine the brightness change direction of each pixel unit. The target discrete time point can be any discrete time point in the time reference sequence other than the first discrete time point. Within the event evaluation time window corresponding to the same discrete time point, the comparison operation of the current brightness change information of all photosensitive units of the event vision sensor with the preset threshold is performed synchronously, thereby generating a synchronous event signal for all photosensitive units. This event signal can be used to represent the current brightness change direction of the photosensitive units.

[0053] In one embodiment, for the event evaluation time window corresponding to the k-th discrete time point of the time reference sequence, the start time of the event evaluation time window is no earlier than the k-th discrete time point, the end time of the event evaluation time window is earlier than the (k+1)-th discrete time point, and the duration of the event evaluation time window is less than the duration of one period of the time reference sequence. Where k is zero or a positive integer, The duration of the event evaluation time window, the event signal Within the time interval corresponding to the event assessment time window [ , + Generate within.

[0054] For example, the period of the time reference sequence, i.e., the sampling period. The length of the event assessment time window Current discrete time point At 100 μs, the next discrete time point At 200μs. The event evaluation time window ranges from 102μs to 127μs. The event signal is generated at 127μs, followed by a 73μs preparation time for circuit reset, signal conditioning, and other tasks to prepare for the sampling operation at 200μs, ensuring that the sampling operation can start on time and avoiding process conflicts.

[0055] In this embodiment, the duration or length of the event evaluation time window It can be adjusted according to actual application needs. Optionally, The range of values ​​can be ~ For example, when hour, It can be set to 10μs~100μs. The smaller the value, the faster the event signal is generated and the stronger the real-time performance, but it is necessary to ensure that the duration is within a certain range. It can compare the change in brightness state with a preset threshold. The larger the value, the more time is available for event evaluation, but the real-time performance will be reduced.

[0056] In this embodiment, determining the preset threshold requires comprehensive consideration of factors such as the noise level of the photosensitive unit and the brightness variation requirements of the application scenario. For example, the preset threshold can be determined by the following method: First, under stable ambient light conditions, collect the brightness state changes at multiple discrete time points and statistically analyze the fluctuation range of these brightness state changes, i.e., the noise range; then, set the preset threshold to 1.5 to 2 times the maximum value of the noise range. This effectively eliminates minor changes caused by noise while ensuring that genuine brightness changes are not missed.

[0057] In one embodiment, under stable lighting conditions, the fluctuation range of the brightness state change of the photosensitive unit is -5 to 5, which is the noise range. Therefore, the preset threshold is set to... , When the brightness state change of a pixel unit is 20, it is greater than the positive threshold of 10 and is judged as a positive event; when the brightness state change of a pixel unit is -15, it is less than the negative threshold of -10 and is judged as a negative event; when the brightness state change of a pixel unit is 8, it is between -10 and 10, and is judged as a small change caused by noise and is judged as a zero event.

[0058] In some embodiments, the preset threshold can be dynamically adjusted according to the actual application scenario. For example, in low-light environments, the noise level of the photosensitive unit will increase, and the preset threshold can be appropriately increased to avoid the generation of false event signals; in high-light environments, the noise level of the photosensitive unit is low, and the preset threshold can be appropriately decreased to improve the sensitivity of the event signal and capture more subtle brightness changes. Therefore, the intra-pixel event data generation method may further include: adjusting the preset threshold according to the usage scenario of the event visual sensor, wherein the usage scenario may include at least one of ambient brightness, light source flicker, and motion state within the scene.

[0059] In this embodiment, the direction of brightness change, i.e., the event signal, obtained by comparing the brightness change information of the photosensitive unit with a preset threshold, can be set according to actual application requirements. For example, a 2-bit binary signal can be used to represent the direction of brightness change of the photosensitive unit, with "10" representing a positive event, "01" representing a negative event, and "00" representing a zero event. A positive event can indicate an increase in the brightness of the photosensitive unit, a zero event can indicate no change in the brightness of the photosensitive unit, and a negative event can indicate a decrease in the brightness of the photosensitive unit.

[0060] In this embodiment, the event signal is a synchronous event signal, meaning that the event signals of all photosensitive units are generated synchronously within the event evaluation time window, ensuring that the event signals of different photosensitive units correspond to the same time reference. Furthermore, since the event signal is generated within the event evaluation time window corresponding to the current discrete time point, it can promptly reflect changes in ambient light brightness, meeting the requirements of real-time signal processing.

[0061] Step 602: Generate current event data for each pixel unit based on the current brightness change direction, pixel coordinates, and timestamp of the current discrete time point.

[0062] In this embodiment, after comparing the current brightness change information with a preset threshold to determine the current brightness change direction of each pixel unit, the execution entity can generate the current event data of each pixel unit according to the current brightness change direction of each pixel unit, the pixel coordinates of the corresponding pixel unit, and the timestamp of the current discrete time point.

[0063] In one embodiment, event data can be represented as a quadruple of (x, y, p, t). Here, (x, y) represents the position of the pixel unit or the pixel coordinates; p represents the polarity of the event or the direction of brightness change, i.e., the direction of light intensity change; and t represents the time the event occurred or the time the event data was generated, stored as a timestamp. That is, the event signal, pixel coordinates, and timestamp are packaged into a set of data to characterize the brightness change event of the pixel unit.

[0064] For example, to accommodate sufficiently large count values, a 64-bit unsigned integer is typically used to store the timestamp. Event data (123, 456, 1, 1235624) could represent that at 1235624 microseconds after sensor activation, the pixel at (123, 456) detected an increase in brightness. Alternatively, storing a complete 64-bit timestamp directly in the data stream for each event would be very bandwidth-intensive; using a very compact encoding method for timestamps at the hardware level can save storage space.

[0065] In some optional embodiments of this disclosure, the intra-pixel event data generation method may further include the following steps: within each period of the time reference sequence, after the event data is generated, the event data of each generated pixel unit is stored, and the photosensitive unit of each pixel unit is reset. One period of the time reference sequence may include at least four stages: brightness sampling, change information determination, threshold decision, and storage reset, which are not overlapping in time. Four non-overlapping phase control signals may be generated within the period of the time reference sequence based on the master clock, and the four stages are executed using a multi-phase time-division strategy. In the first period of the time reference sequence, since only brightness state values ​​are sampled and no event data is generated, only the photosensitive unit of each pixel unit is reset, without storing the event data.

[0066] In this embodiment, a multi-phase time-division multiplexing mechanism is introduced, which enables the brightness sampling, change information determination, threshold decision and storage reset operations to achieve global synchronization within the same sampling period by using the same set of hardware circuits.

[0067] like Figure 7 As shown, the embodiments of this disclosure can organize the event data generation process using a multi-phase time-division operation mechanism. Within a sampling period, it can sequentially include four phases: sampling phase Φ1 (brightness sampling stage), differential phase Φ2 (change information determination stage), threshold evaluation phase Φ3 (threshold decision stage), and state hold / reset phase Φ4 (storage reset stage), each corresponding to one of the four stages of operation. It should be noted that the differential phase Φ2 is named for obtaining brightness change information in a differential manner. In other embodiments, obtaining brightness change information can be achieved through differential operation, ratio operation, temporal filtering, and prediction errors based on historical states, or one or more of these methods. Therefore, the naming of the differential phase is not intended to limit the method used for the phase function.

[0068] In this implementation example, each phase can be driven by a global synchronous clock and satisfy strict non-overlapping timing constraints. Specifically, the clock for each phase can be generated from the master clock of the event vision sensor via an on-chip phase-locked loop (PLL), time-locked loop (DLL), or delay chain to form non-overlapping phase control signals. The four phases constitute a complete operating cycle, and its total duration can satisfy Equation 4: (Formula 4) in These are the time lengths corresponding to phases Φ1, Φ2, Φ3, and Φ4, respectively.

[0069] In sampling phase Φ1 (corresponding to step 401), when the global synchronization signal is at the discrete time node Upon activation, each photosensitive unit in the pixel array simultaneously enters the sampling phase, sampling its current photoelectric response to obtain the brightness state value corresponding to that time point. In this way, all pixels acquire their brightness status under a unified time reference, achieving synchronized sampling across the entire array.

[0070] In differential phase Φ2 (corresponding to step 402), the event vision sensor can determine the current discrete time point. Sample values , and the previous discrete time point The brightness status value is latched or stored. The input differential calculation unit calculates the change in brightness state, as shown in Formula 5: = (Formula 5) Through differential operations, the event vision sensor can obtain information on brightness changes between two adjacent discrete time points.

[0071] In the threshold evaluation phase Φ3 (corresponding to step 601), the event visual sensor can evaluate the results of the differential calculation. Threshold decision is performed. If the result of the decision difference calculation exceeds a positive threshold, a positive event is generated; if the result of the decision difference calculation is below a negative threshold, a negative event is generated; if the result of the decision difference calculation is between the positive and negative thresholds, it is considered that the pixel unit has not generated event data in the current sampling period. This results in three event states: positive event, negative event, and no event. No event can also be called zero event.

[0072] In the state hold / reset phase Φ4 (corresponding to step 602), there are two operating modes:

[0073] Operating mode 1: Regardless of whether a pixel unit triggers a positive or negative event, the event vision sensor writes the event data generated in the current cycle into the event register.

[0074] Operating Mode 2: The event vision sensor only writes the event data generated in the current cycle to the event register when a pixel unit triggers a positive or negative event. If no pixel unit triggers a positive or negative event, the event vision sensor does not write the event data generated in the current cycle to the event register; that is, it does not write event data with zero events to the event register.

[0075] The event data can then be accessed by the readout module, where the timestamps of the event data are provided by a global synchronization counter, ensuring that the timestamps of the event data monotonically increase strictly according to discrete time points. Subsequently, a reset operation is performed on the photosensitive units in the pixel unit to prepare for the next sampling cycle.

[0076] The intra-pixel event data generation method provided by the embodiments of this disclosure is applicable to a variety of photoelectric detection devices that generate event data, such as event vision sensors (EVS), hybrid vision sensors (HVS), high-speed CMOS image sensors, or neuromorphic vision sensors, etc., and the embodiments of this disclosure are not limited thereto.

[0077] In some optional embodiments of this disclosure, in the four-stage operation performed within one period of the time reference sequence, the device performing the operation in the corresponding stage is at least partially multiplexed in at least two stages. To improve the area utilization of the sensor, the intra-pixel event data generation method provided by the embodiments of this disclosure can introduce a device time-division multiplexing mechanism. The same circuit device can perform different functions in different operating phases, and the relationship can be abstractly expressed as Equation 6: (Formula 6) Where Function(t) represents the function executed at time t, and Device represents the hardware device or functional module used in the circuit. This indicates the phase corresponding to the current operation. Through the mapping relationship in Formula 6, the same hardware device can perform different processing functions at different phases, thereby enabling circuit multiplexing in the time dimension.

[0078] like Figure 8 As shown, in the sampling phase Φ1, device 1 and device 2 work together to achieve the sampling function; in the differential phase Φ2, device 1 and device 3 jointly participate in the brightness differential calculation; in the threshold evaluation phase Φ3, device 1, device 2 and device 3 are combined to form the threshold decision circuit; in the state hold / reset phase Φ4, device 2 and device 3 jointly complete the state hold and pixel reset functions.

[0079] In the pixel-level signal processing structure of existing event vision sensors, each pixel unit typically requires an independent differential calculation circuit and threshold comparator. This leads to an increase in the number of transistors inside the pixel unit, thereby limiting the fill factor of the photodiode region and making it difficult to further reduce the size of the pixel unit.

[0080] This disclosure can utilize the strictly non-overlapping timing relationship of four phases, Φ1, Φ2, Φ3 and Φ4, to move the core analog processing unit from the pixel level to the region group level. For example, 4x4 pixel units can share a set of differential calculation circuits and threshold comparators, and shared operation can be achieved through time-division multiplexing, thereby significantly reducing the number of transistors while ensuring functional integrity.

[0081] Specifically, in the differential phase Φ2, the amount of change in brightness state is used to calculate the change. The differential calculation circuit does not need to be set up independently within each pixel unit. Instead, a regional pixel group shared structure can be adopted, in which the differential calculation circuit is arranged within a regional pixel group consisting of 4x4 pixel units. In the Φ2 phase, through pixel-by-pixel gating control, the brightness state values ​​output by the photosensitive units of the same regional pixel group are sequentially input into the shared differential calculation circuit of that regional pixel group, thereby completing the differential calculation of the entire group of pixel units.

[0082] Similarly, in the threshold evaluation phase Φ3, the comparator that performs event decision based on the differential calculation result can also adopt a region pixel group shared structure. Each region pixel group is configured with one comparator, and the input of the comparator is connected to the differential calculation circuit of that region pixel group. The threshold reference voltage provided by the global digital-to-analog converter (DAC) can be used as the preset threshold of the comparator. Through pixel-by-pixel processing, the comparator of that region pixel group can sequentially complete the event decision for the pixel units within the entire region pixel group. The threshold reference voltage is uniformly generated by the global DAC, which avoids repeatedly arranging the threshold generation circuit in each pixel unit or each region pixel group.

[0083] During the state hold / reset phase Φ4, event data generated by the pixel unit can be written to a shared or separate digital register, while pixel reset can be uniformly controlled by a global reset signal.

[0084] Through the aforementioned hierarchical sharing mechanism, this disclosure can form a layered event vision sensor architecture: the first layer is the pixel unit, and within the pixel unit, only basic photoelectric detection structures, sample-and-hold capacitors, and a minimal number of control switches can be retained, such as... Figure 5 As shown; the second level is the regional pixel group, in which differential calculation circuits and comparators can be centrally arranged; the third level is the global circuit, which can be configured on the periphery of the chip, and can include global timing control circuits, threshold DACs, and digital readout interfaces.

[0085] This architecture allows for the relocation of area-intensive analog processing units from within pixel cells to regional pixel groups and global circuitry, significantly reducing pixel cell size or achieving higher resolution within the same chip area. Furthermore, because the four phase operations are strictly separated in the time dimension, circuit sharing avoids conflicts, enabling low-power, high-area-efficiency event vision sensors.

[0086] It should be noted that in the pixel-based event data generation method provided in the embodiments of this disclosure, the device reuse method is not limited to the method provided in the above embodiments, and can be determined according to specific needs based on the basic technical concept provided in the above embodiments.

[0087] Since the number of components is directly proportional to the functionality, the pixel area is strongly correlated with the number of components. Without reusing components, the number of components is large. By reusing components, the number of components is reduced, thereby enabling the reduction of pixel area.

[0088] In the pixel-based event data generation method provided in the embodiments of this disclosure, for a determined sampling period The maximum bandwidth can be defined and can be determined according to Formula 7: (Formula 7) in, This can be calculated based on the sensor's resolution and the number of bits occupied by the event signal. For example, if the resolution of an event vision sensor is 1280×720, the maximum data size of the event data can be 1280×720×2, approximately 1.8Mb. If we take... If the ms interval is 1ms, then the maximum bandwidth is approximately 1.8Gbps.

[0089] Similarly, since the maximum number of events is fixed, congestion control can be achieved through a buffer queue. The capacity of the buffer queue does not need to exceed the maximum number of events in each sampling period.

[0090] Existing event vision sensors mainly adopt the following three architectures.

[0091] 1. Asynchronous event visual sensor

[0092] A typical technology in this field is the Dynamic Vision Sensor (DVS). In this type of technology, each pixel continuously monitors changes in brightness, generating event data when the brightness change exceeds a threshold. Its mathematical model is typically Equation 8: (Formula 8) Where: I is pixel brightness, θ is threshold, and event data is generated when the conditions of the above formula are met.

[0093] This type of architecture has the following characteristics: 1) Event triggering is asynchronous and continuous; 2) Each pixel works independently; 3) Events are output through an arbitration network. This architecture also has the following problems: 1) The event time structure is uncertain, as the event time depends on pixel triggering and the arbitration network, and there is no unified time reference; 2) Timestamps may be in reverse order, which may occur in high-event-density scenarios. but The problems include: 1) timestamp reversal; 2) unpredictable bandwidth, with the number of events depending on the dynamic changes in the scene; 3) complex pixel circuitry, where each pixel unit needs to continuously monitor brightness changes and perform comparison operations.

[0094] 2. Frame Differential Event Generation Method

[0095] Another type of architecture generates image frames and calculates the difference between adjacent frames, as shown in Equation 9: (Formula 9) in, and These are the nth frame and the (n+1)th frame, respectively. This represents the difference between adjacent frames. Event data is generated when the difference between adjacent frames exceeds a threshold.

[0096] This type of architecture typically requires the configuration of a frame buffer, frame differential circuitry, and frame-level readout structure. It has the following drawbacks: 1) Complete frame data must be generated before differential operations and event data generation can be performed; 2) Frame buffering increases power consumption; 3) Resolution is limited by the frame rate.

[0097] 3. Sampling and Holding Time Differential Method

[0098] Some architectures employ sample-and-hold circuitry, as shown in Equation 10: (Formula 10) While this architecture can achieve temporal difference, its core still relies on integrating photocurrent (i.e., exposure) to obtain brightness, followed by digital-to-analog conversion for measurement in the digital domain. Because integration is inherently a time-span process and requires adjustments to the integration time based on light intensity, this architecture cannot capture information at a specific point in time and lacks a unified event-time structure. Furthermore, it still performs full-frame difference calculations outside the pixel array.

[0099] In summary, existing event vision sensors generally suffer from the following problems: (1) The event time structure is missing, and there is no unified time node in the sensor. (2) Event timestamps may not be monotonous. Arbitration, delay, etc. may cause the event order to be disordered. Due to the delay in event signal propagation and the influence of the arbitration mechanism, the timestamp order may be reversed. That is, the event data that should be earlier has a later timestamp than the event data that should be later due to the delay. (3) The bandwidth is unpredictable, and the number of events depends on the dynamic changes of the scene. Events are sudden, and the amount of data required is related to the number of pixels, the frequency of brightness change, the intensity of brightness change, and the number of data bits. The number of event pixels that occur ranges from [0, m×n] (m×n is the resolution of the event vision sensor). The frequency of brightness changes in the real world may be very random, possibly reaching as high as 10000Hz. The data bit representing the event is generally 2 bits. The resolution of mainstream event vision sensors can usually reach 1280×720. Therefore, the burst full data volume of event data can reach: 1280×720×10000×2=17.2Gbps. However, it is rare for a pixel to have such a large number of events at the same time. Designing it to the maximum bandwidth for data reading and transmission would be wasteful. Designing it to a smaller compromise bandwidth would cause data loss in the event of a burst. Moreover, the compromise bandwidth is difficult to define. (4) The pixel circuit is complex. The pixel needs to continuously compare brightness changes and arbitrate events to generate event data.

[0100] The pixel-based event data generation method disclosed herein, based on a unified time base and a discrete-time point-based event definition and synchronous event generation mechanism, can transform event generation from a continuous-time triggering mechanism to a discrete-time driven mechanism. By establishing a unified time base, defining discrete-time events, and introducing a synchronous event evaluation time window, event time structuring can be achieved, the timestamp reversal problem can be eliminated, event bandwidth can be predicted, and the complexity of pixel circuitry can be reduced.

[0101] This can be achieved by introducing a multi-phase time-division multiplexing mechanism, allowing devices within a pixel unit to be reused. Since the number of devices is directly proportional to their functionality, and pixel area is strongly correlated with the number of devices, the number of devices is large without reuse. Reusing devices allows for compression of the device count, thus enabling pixel area compression. For a given sampling period, the maximum bandwidth is defined, and therefore the bandwidth requirement is definable. Similarly, because the maximum number of events is fixed, congestion control can be achieved through a buffer queue. The capacity of the buffer queue does not exceed the maximum number of events in each sampling period.

[0102] In summary, the pixel-based event data generation method provided in this disclosure can eliminate the problem of timestamp reversal by synchronously generating event signals at discrete time points, thereby making bandwidth requirements definable, data congestion predictable, and pixel area scalable.

[0103] It should be noted that the pixel-based event data generation method provided in this disclosure determines the brightness change information and compares the thresholds within the sensor pixel array, and does not output intermediate data to the subsequent readout circuit before generating event data.

[0104] Regarding the second architecture mentioned in the prior art, when using a high-speed CMOS image sensor to generate event data in a frame differential manner, the image signal acquisition is completed by pixel acquisition and then the frame data is buffered. After being read out by the readout circuit as frame image data during the readout stage, the frame image data is differentially processed to obtain event data.

[0105] The method provided in this disclosure completes the acquisition of brightness state values, the generation of brightness change information, and the generation of event data through threshold comparison within the sensor pixel array. The event data is then output to the subsequent readout circuit. No intermediate data is output to the subsequent readout circuit, and a frame buffer structure is not required.

[0106] Please refer to further information. Figure 9 As an implementation of the methods shown in the above figures, this disclosure also provides an embodiment of an event visual sensor, which is similar to... Figure 1 Corresponding to the method embodiments shown, this event vision sensor can be specifically applied to various electronic devices.

[0107] like Figure 9 As shown, the event vision sensor in this embodiment may include: multiple pixel units 901, a reference time determination module 902, a change information determination module 903, and an event data generation module 904.

[0108] Pixel unit 901 includes a photosensitive unit. Reference time determination module 902 is configured to determine multiple periodic discrete time points for triggering sampling of the brightness signal of the photosensitive unit. These discrete time points constitute a time reference sequence for generating event data for each pixel unit. Change information determination module 903 is configured to synchronously sample the brightness signal of the photosensitive unit of each pixel unit at each discrete time point in the time reference sequence, and determine the brightness change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points. Event data generation module 904 is configured to compare the brightness change information of each pixel unit with a preset threshold, and generate event data for each pixel unit based on the comparison result and the timestamp of each discrete time point. The comparison of the brightness change information with the preset threshold occurs within the event evaluation time window corresponding to each discrete time point and located between two discrete time points.

[0109] In some optional implementations of this disclosure, the plurality of pixel units are divided into a plurality of region pixel groups, and the number of region pixel groups is less than the number of pixel units; The pixel unit includes a photoelectric conversion circuit, a storage capacitor, and a control switch; wherein, the photoelectric conversion circuit forms the photosensitive unit; the control switch is used to sample the brightness signal of the photosensitive unit at the discrete time points; and the storage capacitor is used to store the brightness state values ​​sampled at the discrete time points. The region pixel group includes the pixel unit, the differential operation circuit, and the threshold comparator; wherein, all pixel units of the region pixel group are controlled by pixel-by-pixel gating, and the differential operation circuit and threshold comparator of the region pixel group are shared by time-division multiplexing; the differential operation circuit is used to perform differential operation on the brightness state values ​​sampled at two adjacent discrete time points; the threshold comparator is used to compare the brightness change information obtained by the differential operation with a preset threshold. The event vision sensor further includes: a global timing control circuit, a threshold generation circuit, and a data transmission interface; wherein, all pixel groups in all regions of the event vision sensor share the global timing control circuit, the threshold generation circuit, and the data transmission interface; the global timing control circuit is used to determine periodic discrete time points and control the control switch to sample at the discrete time points; the threshold generation circuit is used to generate a preset threshold for comparison with the brightness change information; the data transmission interface is used to output the event data generated by the pixel unit to the subsequent circuit.

[0110] In some optional implementations of this disclosure, the change information determination module 903 includes: A luminance sampling unit is configured to synchronously sample the luminance signal of the photosensitive unit of each pixel unit in response to the arrival of a target discrete time point in the time reference sequence, so as to obtain the current luminance state value of each pixel unit, wherein the target discrete time point is any discrete time point in the time reference sequence other than the first discrete time point. The brightness comparison unit is configured to determine the current brightness change information of each pixel unit based on the current brightness state value and the reference brightness state value of each pixel unit, wherein the reference brightness state value is the brightness state value of each pixel unit at the previous discrete time point adjacent to the target discrete time point. The brightness update unit is configured to update the reference brightness state value of each pixel unit according to the current brightness state value of each pixel unit.

[0111] In some optional implementations of this disclosure, the plurality of periodic discrete time points are generated based on a unified clock reference, and the determination of the brightness change information, the comparison of the brightness change information with the preset threshold, and the generation of the event data are all completed in a unified scheduling synchronous timing sequence.

[0112] In some alternative implementations of this disclosure, the brightness comparison unit is further configured to determine the brightness change information based on the brightness state values ​​between the adjacent discrete time points using at least one of the following methods: difference operation, ratio operation, time-domain filtering, and prediction error based on historical state.

[0113] In some alternative implementations of this disclosure, the event data generation module 904 includes: The threshold comparison unit is configured to compare the current brightness change information of each pixel unit with a preset threshold within the event evaluation time window corresponding to the current discrete time point, and determine the current brightness change direction of each pixel unit. The event generation unit is configured to generate current event data for each pixel unit based on the current brightness change direction, pixel coordinates, and timestamp of the current discrete time point.

[0114] In some alternative implementations of this disclosure, the event vision sensor also includes: The threshold adjustment module is configured to adjust the preset threshold according to the usage scenario of the event visual sensor, wherein the usage scenario includes at least one of ambient brightness, light source flicker, and motion state within the scene.

[0115] In some optional implementations of this disclosure, within each period of the time reference sequence, after the event data is generated, the event data of each generated pixel unit is stored, and the photosensitive unit of each pixel unit is reset. The time reference sequence includes at least four stages: brightness sampling, change information determination, threshold decision and storage reset, which are not overlapping in time. Based on the master clock, four phase control signals that are not overlapping are generated within the period of the time reference sequence, and the operation of the four stages is performed using a multi-phase time division strategy.

[0116] In some alternative implementations of this disclosure, in the four-stage operation performed within one period of the time reference sequence, the device performing the operation of the corresponding stage is at least partially multiplexed in at least two stages.

[0117] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0118] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating intra-pixel event data, characterized in that, Applied to an event vision sensor, the event vision sensor comprising a plurality of pixel units, each pixel unit comprising a photosensitive unit, the method comprising: Multiple periodic discrete time points are determined to trigger sampling of the brightness signal of the photosensitive unit, and the discrete time points constitute a time reference sequence for each pixel unit to generate the event data; At each discrete time point in the time reference sequence, the brightness signal of the photosensitive unit of each pixel unit is synchronously sampled, and the brightness change information of each pixel unit between adjacent discrete time points is determined based on the time structure defined by the discrete time points. The brightness change information of each pixel unit is compared with a preset threshold, and event data of each pixel unit is generated based on the comparison result and the timestamp of each discrete time point. The comparison of the brightness change information with the preset threshold occurs within the event evaluation time window corresponding to each discrete time point and located between two discrete time points.

2. The method according to claim 1, characterized in that, At each discrete time point in the time reference sequence, the brightness signal of the photosensitive unit of each pixel unit is synchronously sampled, and based on the time structure defined by the discrete time points, the brightness change information of each pixel unit between adjacent discrete time points is determined, including: In response to the arrival of the target discrete time point of the time reference sequence, the brightness signal of the photosensitive unit of each pixel unit is synchronously sampled to obtain the current brightness state value of each pixel unit, wherein the target discrete time point is the other discrete time point in the time reference sequence except for the first discrete time point. Based on the current brightness state value and reference brightness state value of each pixel unit, the current brightness change information of each pixel unit is determined respectively, wherein the reference brightness state value is the brightness state value of each pixel unit at the previous discrete time point adjacent to the target discrete time point. The reference brightness state value of each pixel unit is updated based on the current brightness state value of each pixel unit.

3. The method according to claim 2, characterized in that, The multiple periodic discrete time points are generated based on a unified clock reference. The determination of the brightness change information, the comparison of the brightness change information with the preset threshold, and the generation of the event data are all completed in a unified scheduling synchronous timing sequence.

4. The method according to claim 2, characterized in that, The brightness change information is determined based on the brightness state values ​​between adjacent discrete time points using at least one of the following methods: difference operation, ratio operation, time-domain filtering, and prediction error based on historical state.

5. The method according to claim 1, characterized in that, The step of comparing the brightness change information of each pixel unit with a preset threshold, and generating event data for each pixel unit based on the comparison result and the timestamp of each discrete time point, includes: Within the event evaluation time window corresponding to the current discrete time point, the current brightness change information of each pixel unit is compared with a preset threshold to determine the current brightness change direction of each pixel unit. Based on the current brightness change direction, pixel coordinates, and timestamp of the current discrete time point of each pixel unit, current event data for each pixel unit is generated.

6. The method according to claim 5, characterized in that, The method further includes: The preset threshold is adjusted according to the usage scenario of the event visual sensor, wherein the usage scenario includes at least one of ambient brightness, light source flicker, and motion state within the scene.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Within each period of the time reference sequence, after the event data is generated, the event data of each generated pixel unit is stored, and the photosensitive unit of each pixel unit is reset. The time reference sequence includes at least four stages: brightness sampling, change information determination, threshold decision and storage reset, which are not overlapping in time. Based on the master clock, four phase control signals that are not overlapping are generated within the period of the time reference sequence, and the operation of the four stages is performed using a multi-phase time division strategy.

8. The method according to claim 7, characterized in that, In the four-stage operation performed within one period of the time reference sequence, for the device performing the operation of the corresponding stage, it is at least partially multiplexed in at least two stages.

9. An event vision sensor, characterized in that, include: Multiple pixel units, wherein each pixel unit includes a photosensitive unit; The reference time determination module is configured to determine a plurality of periodic discrete time points for triggering sampling of the brightness signal of the photosensitive unit, the discrete time points constituting a time reference sequence for generating event data for each of the pixel units; The change information determination module is configured to synchronously sample the brightness signal of the photosensitive unit of each pixel unit at each discrete time point in the time reference sequence, and determine the brightness change information of each pixel unit between adjacent discrete time points based on the time structure defined by the discrete time points. The event data generation module is configured to compare the brightness change information of each pixel unit with a preset threshold, and generate event data for each pixel unit based on the comparison result and the timestamp of each discrete time point. The comparison of the brightness change information with the preset threshold occurs within the event evaluation time window corresponding to each discrete time point and located between two discrete time points.

10. The sensor according to claim 9, characterized in that, The plurality of pixel units are divided into a plurality of region pixel groups, and the number of region pixel groups is less than the number of pixel units; The pixel unit includes a photoelectric conversion circuit, a storage capacitor, and a control switch; wherein, the photoelectric conversion circuit forms the photosensitive unit; the control switch is used to sample the brightness signal of the photosensitive unit at the discrete time points; and the storage capacitor is used to store the brightness state values ​​sampled at the discrete time points. The region pixel group includes the pixel unit, the differential operation circuit, and the threshold comparator; wherein, all pixel units of the region pixel group are controlled by pixel-by-pixel gating, and the differential operation circuit and threshold comparator of the region pixel group are shared by time-division multiplexing; the differential operation circuit is used to perform differential operation on the brightness state values ​​sampled at two adjacent discrete time points; the threshold comparator is used to compare the brightness change information obtained by the differential operation with a preset threshold. The event vision sensor further includes: a global timing control circuit, a threshold generation circuit, and a data transmission interface; wherein, all pixel groups in all regions of the event vision sensor share the global timing control circuit, the threshold generation circuit, and the data transmission interface; the global timing control circuit is used to determine periodic discrete time points and control the control switch to sample at the discrete time points; the threshold generation circuit is used to generate a preset threshold for comparison with the brightness change information; the data transmission interface is used to output the event data generated by the pixel unit to the subsequent circuit.

Images