A low-cost method and apparatus for dynamic visual sensor event frame conversion

Abstract

This invention relates to a low-cost method and apparatus for frame conversion of dynamic visual sensor events. The method includes: determining the number of sampling regions based on the DVS image size and downsampling factor, and mapping each sampling region to a counter; whenever a new DVS event arrives, calculating the sampling region corresponding to the DVS event and updating the counter corresponding to the sampling region of the DVS event; when the framesync signal indicating the start of a new frame is valid, starting from the first address of the storage space, sequentially reading and clearing each address to obtain the event frame. Compared with the prior art, this invention performs downsampling and pixel counting simultaneously, reducing the number of counters, thereby reducing the storage space size and lowering the cost; abandoning the ping-pong buffer method, the storage space only stores the counter of one event frame, further reducing the storage space size and lowering the cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a low-cost method and apparatus for converting dynamic visual sensor events into frames. Background Technology

[0002] Traditional visual image acquisition methods are based on "frames" acquired at a fixed frequency. When performing image processing and feature extraction based on these continuous image frames, the image acquisition sensor needs to operate at a high frame rate. Moreover, the output image stream contains a large amount of redundant data that is irrelevant to the target, resulting in drawbacks such as high redundancy, high latency, high noise, low dynamic range, and high data volume.

[0003] In existing technologies, address- and event-based visual image sensors (DVS) mimic the working mechanism of biological vision. Pixels operate asynchronously; for a single pixel, an event (pulse) signal is only output when the light intensity it receives changes. For example, if the brightness change exceeds a threshold, an event indicating a brightness change at that pixel is added, outputting the event's address (corresponding pixel location) and the event's nature (brighter or darker, etc.). Dynamic vision sensors output only the addresses of pixels with changing light intensity and the sign of the change, rather than passively reading information from every pixel within a "frame," eliminating redundant data at the source. This results in features such as real-time dynamic response to scene changes, ultra-sparse image representation, and asynchronous event output. When pixel light intensity changes infrequently, the number of events output by the DVS is small, reducing redundant data and lowering system power consumption. Each pixel operates independently; when pixel light intensity changes rapidly, the DVS outputs the corresponding event within a microsecond delay. Combined with the pulse input characteristics of spiking neural networks (SNNs), image classification can be achieved within a short delay, making it applicable to scenarios such as high-speed target tracking.

[0004] Deep neural network (DNN) research has made rapid progress and has seen initial applications in recent years. Implementing such deep learning algorithms typically requires a large amount of computing power. In image processing methods based on deep neural networks, training DVS data is similar to training ordinary image data based on active pixel sensors (APS). Currently, the mainstream algorithms still operate on a frame-by-frame basis. This requires converting DVS events (carrying pixel address information and the sign of the pulse) into frames. Of course, the frame rate can be much higher than the APS frame rate, such as 1000 frames per second.

[0005] Chinese patent CN113012200A discloses a method, device, electronic device, and storage medium for locating a moving object. It acquires event stream information and samples the event stream information according to a preset sampling period to obtain sampled event frames, which are then used for subsequent processing. In the prior art, the general implementation method for DVS event-to-frame conversion is as follows: a time interval is set, during which the DVS must output an event at least once. During the time interval, for each pixel that receives an event, its count is incremented. After the time interval ends, the counts of each pixel are counted, and the counts corresponding to all pixels are output to obtain the event frame. Specifically: when the framesync signal indicating the start of a new frame is valid, the count value of each pixel is output as the event frame, and the count values ​​of these pixels are cleared to zero, starting pixel counting for the next frame.

[0006] Assuming the DVS pixel count is 1024*768, when converting events to frames by counting each pixel, and assuming each pixel's counter bit width is 8 bits, the required storage space for this conversion is 1024*768*8 bits = 1024*768 bytes = 768KB. After framesync is enabled, the storage space is read and cleared, which takes a considerable amount of time. If the storage space is an SRAM with one read port and one write port, a depth of 192KB, and a width of 32 bits (width refers to the length of data that can be stored, and depth refers to the number of such data sets that can be stored), this read process will last for 192KB clock cycles. During this time, new DVS events are likely to arrive. To prevent interference between the next frame and the currently read frame, the storage space is generally divided into two parts (ping-pong buffer): the previous frame is in the first buffer, and events arriving during the reading process from the first buffer are included in the statistics of the next frame, while the counter for the next frame is in the second buffer. When the next framesync is active, the second part of the cache is read out, and the new event count is entered into the first part of the cache. Therefore, under the ping-pong caching condition, the required storage space is 768K * 2 = 1536K bytes.

[0007] Because the pixel count of DVS (e.g., 1024*768) is relatively large, often exceeding the maximum pixel count supported by neural networks (e.g., 224*224), and neural networks do not require high resolution for classification, it is necessary to downsample (or pool) the converted frames. A typical implementation of DVS data event-to-frame conversion is as follows... Figure 2As shown, the conversion from event to frame is achieved by counting each pixel in the DVS (Dynamic Visual Sensor). Then, the data in a frame is downsampled (e.g., 4:1 downsampling) to obtain an event frame suitable for a neural network. During downsampling, the bit width of the downsampled data may exceed the preset bit width, which can be truncated (with minimal reduction in accuracy). In existing technologies, considering the downsampling process, two additional storage spaces are required: one for storing the frame obtained from pixel counting and another for storing the event frame obtained from downsampling the data within the frame, further increasing the required storage space. This large storage space leads to higher hardware costs, affects read / write speed, and increases time costs. Therefore, a low-cost method for converting dynamic visual sensor events to frames is needed. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art by providing a low-cost method and apparatus for dynamic visual sensor event frame conversion.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] A low-cost method for implementing dynamic visual sensor event frame conversion includes the following steps:

[0011] S1. Obtain the DVS image size and downsampling factor. Determine the number of sampling regions based on the DVS image size and downsampling factor. Map the sampling regions to counters in the storage space and record the address of the counter corresponding to each sampling region in the storage space.

[0012] S2. Whenever a new DVS event arrives, obtain the pixel position of the DVS event, calculate the sampling area corresponding to the DVS event, and update the counter corresponding to the sampling area of ​​the DVS event.

[0013] S3. When the framesync signal indicating the start of a new frame is valid, starting from the first address of the storage space, read and clear operations are performed on each address in sequence to obtain an event frame. The read and clear operation specifically involves reading the reading of the counter in the current address and clearing the counter in the current address to zero.

[0014] Furthermore, in step S3, the reading of the counter in the storage space is read into the external memory to obtain an event frame. When performing the read-clear operation, the reading of the counter in the current address is first read into the on-chip cache, and the counter in the current address is cleared. Then, the reading of the counter is written from the on-chip cache into the external memory.

[0015] Further, during the process of reading and clearing the counter in address i, if a new DVS event arrives, calculate the address j in the storage space corresponding to the counter of the new DVS event:

[0016] If j > i, continue to read and clear the counter in address i. After completing the reading and clearing operation of the counter in address i, update the counter corresponding to the new DVS event, and continue with the reading and clearing operation of the next address; when reading the counter in address j, this new DVS event will be read out and reflected in the currently read event frame;

[0017] If j < i, continue to read and clear the counter in address i. After completing the reading and clearing operation of the counter in address i, update the counter corresponding to the new DVS event, and continue with the reading and clearing operation of the next address; this new DVS event will be reflected in the event frame read next time;

[0018] If j = i and the reading of the counter has not been written to the on-chip cache, read out the reading of the counter in address i, update the read counter reading based on the new DVS event, write the updated counter reading to the on-chip cache, clear the counter in address i, and continue with the reading and clearing operation of the next address; this new DVS event will be reflected in the currently read event frame;

[0019] If j = i and the reading in the counter has already been written to the on-chip cache, directly write the reading of the counter based on the new DVS event, and continue with the reading and clearing operation of the next address; this DVS event will be reflected in the event frame read next time. Directly writing the reading of the counter based on the new DVS event can be understood as first clearing the counter and then updating the counter based on the new DVS event.

[0020] Further, updating the read counter reading based on the new DVS event is specifically as follows: if the counter has not counted up to the maximum value of the counter, add 1 to the counter reading, otherwise, the counter reading remains unchanged; directly writing the reading of the counter based on the new DVS event is specifically as follows: perform a write 1 operation on the counter.

[0021] Further, in step S2, updating the counter corresponding to the sampling area of the DVS event is specifically as follows: if the counter has not counted up to the maximum value of the counter, add 1 to the counter, otherwise, the counter remains unchanged; the specific operation of adding 1 to the counter is: read out the reading of the counter, add 1, and then rewrite it to the counter.

[0022] For a single pixel, when the light intensity it receives changes, an event (pulse) signal will be output. For example, if the brightness change exceeds a threshold, an event for the brightness change of that pixel, namely a DVS event, will be added. The address of the event (corresponding pixel position) and the nature of the event (brighter or darker, etc.) will be output. This application does not focus on the nature of the event. For each DVS event received, the sampling area to which it belongs is determined according to the pixel position corresponding to the DVS event, and then the counter corresponding to the sampling area is updated. If the counter has not yet counted to the maximum value, the counter is incremented by 1; otherwise, the counter remains unchanged.

[0023] Furthermore, the DVS image size is ROW_MAX * COL_MAX, where ROW_MAX represents the number of pixel rows in the DVS image and COL_MAX represents the number of pixel columns in the DVS image. The downsampling factor includes a row downsampling factor and a column downsampling factor, where the row downsampling factor is ROW_POOL: 1 and the column downsampling factor is COL_POOL: 1. The size of the downsampling region is ROW_POOL * COL_POOL. Wherein, ROW_MAX, COL_MAX, ROW_POOL, and COL_POOL are all positive integers.

[0024] Furthermore, PIXEL_WIDTH represents the preset bit width of the counter corresponding to one pixel, so the bit width of the counter corresponding to one sampling region is:

[0025] PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL))

[0026] The size of the storage space is then:

[0027] Num*(PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL)))

[0028] The function ceil(x) returns the smallest integer value not less than x, and Num represents the number of counters.

[0029] Furthermore, the number of sampling regions is equal to the number of counters, and the number of counters, Num, is:

[0030] Num=(ceil(ROW_MAX / ROW_POOL))*(ceil(COL_MAX / COL_POOL))

[0031] The function ceil(x) returns the smallest integer value that is not less than x.

[0032] Furthermore, in step S3, the framesync signal, which indicates the start of a new frame, is valid once every preset time interval.

[0033] A low-cost implementation device for dynamic visual sensor event frame conversion, based on the low-cost implementation method for dynamic visual sensor event frame conversion as described above, includes a storage space, an external memory, and a controller; a counter in the storage space is used to count DVS events, the external memory is used to store event frames, and the controller is used to receive DVS events and perform read, write, and update operations on the storage space based on the received DVS events, and perform write operations on the external memory.

[0034] The shortcoming of this application lies in the unclear boundaries between frames. Because new DVS events also affect the current event frame during the reading of the current frame, the time interval between adjacent frames is not fixed. However, DVS events themselves are event-based; a DVS event is generated when the brightness change exceeds a threshold. They do not inherently have clear frame boundaries. The frames described in existing technologies are merely statistical representations of DVS events over a period of time obtained through artificially defined time intervals, intended only to facilitate frame division for subsequent data processing, AI training, etc. The frames in this application do not have clearly fixed time intervals; they simply use a different frame division method than existing technologies. However, they do not lose DVS events, ensuring that all information is available for subsequent data processing and AI training without affecting these processes.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) Downsampling and pixel counting are performed simultaneously, which greatly reduces the number of counters, thereby reducing the storage space and lowering the cost.

[0037] (2) The ping-pong caching method is abandoned. Only the counter of one event frame is stored in the storage space. If a new DVS event arrives during the counter reading process, the new DVS event is merged into the current frame or the next frame, which further reduces the size of the storage space and lowers the cost.

[0038] (3) Every DVS event can be reflected in the event frame without losing any DVS events, ensuring that all information is obtained during subsequent data processing and artificial intelligence training after the DVS event is converted to a frame. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating the simultaneous downsampling and pixel counting during DVS event frame conversion in this invention;

[0040] Figure 2This is a schematic diagram of DVS event frame conversion in the prior art;

[0041] Figure 3 This is a schematic diagram illustrating the simultaneous downsampling and pixel counting in this invention. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0043] Example 1:

[0044] A low-cost implementation method for dynamic visual sensor event frame conversion, such as... Figure 1 As shown, it includes the following steps:

[0045] S1. Obtain the DVS image size and downsampling factor. Determine the number of sampling regions based on the DVS image size and downsampling factor. Map the sampling regions to counters in the storage space and record the address of the counter corresponding to each sampling region in the storage space.

[0046] S2. Whenever a new DVS event arrives, obtain the pixel position of the DVS event, calculate the sampling area corresponding to the DVS event, and update the counter corresponding to the sampling area of ​​the DVS event.

[0047] In step S2, updating the counter corresponding to the sampling area of ​​the DVS event is specifically done as follows: if the counter has not yet counted to its maximum value, then the counter is incremented by 1; otherwise, the counter remains unchanged. The specific operation of incrementing the counter by 1 is to read the counter reading, increment it by 1, and then rewrite it to the counter.

[0048] For a single pixel, when the light intensity it receives changes, an event (pulse) signal will be output. For example, if the brightness change exceeds a threshold, an event for the brightness change of that pixel, namely a DVS event, will be added. The address of the event (corresponding pixel position) and the nature of the event (brighter or darker, etc.) will be output. This application does not focus on the nature of the event. For each DVS event received, the sampling area to which it belongs is determined according to the pixel position corresponding to the DVS event, and then the counter corresponding to the sampling area is updated. If the counter has not yet counted to the maximum value, the counter is incremented by 1; otherwise, the counter remains unchanged.

[0049] S3. When the framesync signal indicating the start of a new frame is valid, starting from the first address of the memory space (i.e., address 0), read and clear operations are performed on each address in sequence to obtain the event frame. The read and clear operation is as follows: read the reading of the counter in the current address and clear the counter in the current address to zero.

[0050] In step S3, every time a preset time interval elapses, the framesync signal indicating the start of a new frame is valid once.

[0051] In step S3, the reading of the counter in the storage space is read into the external storage device to obtain an event frame. When performing the read and clear operation, first read the reading of the counter at the current address into the on-chip cache, and at the same time clear the counter at the current address, and then write the reading of the counter from the on-chip cache into the external memory. The read and write operations, parallel execution, etc. of the memory are well-known to those skilled in the art and will not be elaborated here.

[0052] During the process of performing the read and clear operation on the counter at address i, if a new DVS event arrives, calculate the address j in the storage space corresponding to the counter of the new DVS event:

[0053] If j > i, continue to perform the read and clear operation on the counter at address i. After completing the read and clear operation on the counter at address i, update the counter corresponding to the new DVS event, and continue to perform the read and clear operation on the next address; when reading the counter at address j, this new DVS event will be read out and reflected in the currently read event frame;

[0054] [[ID=1�]]If j < i, continue to perform the read and clear operation on the counter at address i. After completing the read and clear operation on the counter at address i, update the counter corresponding to the new DVS event, and continue to perform the read and clear operation on the next address; this new DVS event will be reflected in the event frame read next time;

[0055] If j = i and the reading of the counter has not been written into the on-chip cache, read the reading of the counter at address i, update the read counter reading based on the new DVS event, write the updated counter reading into the on-chip cache, clear the counter at address i, and continue to perform the read and clear operation on the next address; this new DVS event will be reflected in the currently read event frame; specifically, updating the read counter reading based on the new DVS event means: if the counter has not counted up to the maximum value of the counter, increment the counter reading by 1, otherwise, the counter reading remains unchanged;

[0056] If j = i, and the reading in the counter has already been written to the on-chip cache, then the reading of the counter is directly written based on the new DVS event, and the read and clear operation of the next address continues. This DVS event will be reflected in the event frame of the next read. The reading of the counter directly written based on the new DVS event can be understood as first clearing the counter to zero, and then updating the counter based on the new DVS event. Specifically, the reading of the counter directly written based on the new DVS event is: performing a write 1 operation on the counter.

[0057] A low-cost implementation device for dynamic visual sensor event frame conversion, based on the above-mentioned low-cost implementation method for dynamic visual sensor event frame conversion, includes a storage space, an external memory, and a controller; a counter in the storage space is used to count DVS events, the external memory is used to store event frames, the controller is used to receive DVS events, and perform read, write, and update operations on the storage space based on the received DVS events, and perform write operations on the external memory.

[0058] The DVS image size is ROW_MAX * COL_MAX, where ROW_MAX represents the number of pixel rows in the DVS image, and COL_MAX represents the number of pixel columns in the DVS image. The downsampling factor includes a row downsampling factor and a column downsampling factor, where the row downsampling factor is ROW_POOL: 1, the column downsampling factor is COL_POOL: 1, and the size of the downsampling region is ROW_POOL * COL_POOL. Here, ROW_MAX, COL_MAX, ROW_POOL, and COL_POOL are all positive integers.

[0059] PIXEL_WIDTH represents the preset bit width of the counter corresponding to one pixel. Therefore, the bit width of the counter corresponding to one sampling region is:

[0060] PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL))

[0061] The size of the storage space is then:

[0062] Num*(PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL)))

[0063] The function ceil(x) returns the smallest integer value not less than x, and Num represents the number of counters.

[0064] The number of sampling regions is equal to the number of counters, and the number of counters, Num, is:

[0065] Num=(ceil(ROW_MAX / ROW_POOL))*(ceil(COL_MAX / COL_POOL))

[0066] The function ceil(x) returns the smallest integer value that is not less than x.

[0067] In this embodiment, as Figure 3 As shown, the DVS image has a resolution of 8*8, meaning the DVS image size is 8*8 pixels. The row downsampling factor is 4:1, the column downsampling factor is 4:1, and there are four sampling regions. All 16 pixels in each sampling region are mapped to the same counter. For example... Figure 3 The counter for the middle pixel (0,0) is pool(0,0), the counter for the pixel (2,4) is pool(0,1), and the counters for the pixels (6,0) and (7,2) are both pool(1,0).

[0068] Upon receiving a DVS event, the downsampled pixel position of the DVS event is determined based on the position information (pixel_row, pixel_col) carried in the DVS event. Then, the sampling region to which the event belongs is calculated, and the counter corresponding to the sampling region is incremented by 1. For example, if the position information of the first DVS event is pixel_row = 1 and pixel_col = 2, its corresponding counter is pool(0,0). If the count of this counter has not reached the maximum value, the counter is incremented by 1; if it reaches the maximum value, it remains unchanged. If the position information of the second DVS event is pixel_row = 6 and pixel_col = 5, its corresponding counter is pool(1,1). If the count of this counter has not reached the maximum value, the counter is incremented by 1; if it reaches the maximum value, it remains unchanged.

[0069] (1) Traditional DVS event frame conversion methods, such as Figure 2 As shown:

[0070] For a 768*1024 DVS image, the traditional DVS event-to-frame method allocates 768*1024 counters in storage space, each corresponding to a pixel. When the brightness change of a pixel exceeds a threshold, the dynamic vision sensor generates a DVS event. When a DVS event arrives, the counter corresponding to that event is incremented by 1. The bit width of the counter can be set to 8 bits. When the framesync signal indicating the start of a new frame is valid, the readings of each counter in storage space are read out to obtain a 768*1024 frame. The magnitude of each value in the frame represents the number of times the brightness change of a pixel exceeds the threshold.

[0071] During the process of reading the counter readings in the storage space, if the storage space is an SRAM with a depth of 192K and a width of 32 bits (one read port and one write port, where width refers to the length of data that can be stored and depth refers to the number of such data entries that can be stored), then this read process lasts for 192K clock cycles, which is a relatively long read time. Considering that new DVS events may occur during this time, a ping-pong buffering method is adopted: the entire storage space is divided into two parts. The size of the first part is 768*1024*8=768K bytes, and the size of the second part is also 768*1024*8=768K bytes. Assuming that pixel counting is currently being performed in the first part, when the framesync signal is valid, the values ​​of each counter are read from the first part, and at the same time, pixel counting begins in the second part, until the framesync signal becomes valid again, at which point the values ​​of each counter are read from the second part, and pixel counting begins in the first part, and so on.

[0072] The pixel count of a DVS (e.g., 1024*768) may exceed the maximum pixel count supported by the neural network (e.g., 224*224). Since the neural network does not require high resolution for classification, the 768*1024 frame obtained from the read counter needs to be downsampled. If downsampling is performed, and the read counter obtains a 768*1024 frame, then downsampling both rows and columns of this frame by a 4:1 ratio results in a 192*256 event frame. This requires additional storage space to store this downsampled event frame. Afterward, the bit width is truncated and sent to external DDR memory.

[0073] If the space occupied by downsampling is not included in the storage space, then the storage space size is at least 768K*2=1536KByte.

[0074] (2) This application performs downsampling and DVS frame counting simultaneously, such as Figure 1 As shown:

[0075] For a 768*1024 DVS image, both the rows and columns are downsampled by a factor of 4:1. The number of counters is 768*1024 / 16, which is reduced to 1 / 16 of the original. Each counter corresponds to a sampling area. When the brightness change of a pixel exceeds the threshold, the dynamic vision sensor generates a DVS event. When a DVS event arrives, calculate the sampling area to which the DVS event belongs, and increment the counter corresponding to that sampling area by 1. At the same time, since the counter needs to count 16 pixels within the sampling area, its bit width is increased to 8+ceil(log2(4*4)) = 12. When the framesync signal indicating the start of a new frame is valid, read the readings of each counter in the storage space to obtain a 192*256 frame. The magnitude of each value in the frame represents the number of times the brightness change of all pixels in a sampling area exceeds the threshold. Then perform bit width truncation and send it to the external memory DDR.

[0076] This application does not adopt the ping-pong buffer method. During the process of reading the readings of the counters in the storage space, if a new DVS event arrives, based on the address of the counter corresponding to the new DVS event and the currently being read address, decide whether to incorporate the new DVS event into the current event frame or the next event frame.

[0077] During the process of reading and clearing the counter at address i, if a new DVS event arrives, calculate the address j of the counter corresponding to the new DVS event in the storage space:

[0078] If j>i, continue to perform the read and clear operation on the counter at address i. After completing the read and clear operation on the counter at address i, update the counter corresponding to the new DVS event and continue with the read and clear operation for the next address; when reading the counter at address j, this new DVS event will be read out and reflected in the currently read event frame;

[0079] If j<i, continue to perform the read and clear operation on the counter at address i. After completing the read and clear operation on the counter at address i, update the counter corresponding to the new DVS event and continue with the read and clear operation for the next address; this new DVS event will be reflected in the event frame read next time;

[0080] If j = i, and the counter reading has not yet been written to the on-chip cache, then read the counter reading at address i, update the read counter reading based on the new DVS event, write the updated counter reading to the on-chip cache, clear the counter at address i, and continue to the next address for read and clear operation; this new DVS event will be reflected in the current read event frame; the update of the read counter reading based on the new DVS event is as follows: if the counter has not yet counted to its maximum value, then increment the counter reading by 1; otherwise, the counter reading remains unchanged;

[0081] If j = i, and the reading in the counter has already been written to the on-chip cache, then the reading of the counter is directly written based on the new DVS event, and the read and clear operation of the next address continues. This DVS event will be reflected in the event frame of the next read. The reading of the counter directly written based on the new DVS event can be understood as first clearing the counter to zero, and then updating the counter based on the new DVS event. Specifically, the reading of the counter directly written based on the new DVS event is: performing a write 1 operation on the counter.

[0082] Therefore, the storage space in this application is 192*256*12=73.728K Byte, while the traditional DVS event frame conversion method requires 1536K Byte. Obviously, this application greatly reduces the storage space.

[0083] On the one hand, this application performs downsampling and pixel counting simultaneously, which greatly reduces storage space (73.728K Byte in this application, compared to 768K Byte in conventional technology). On the other hand, instead of using a ping-pong buffer, it stores only one frame counter, further reducing storage space (73.728K Byte in this application, compared to 1536K Byte in conventional technology).

[0084] The shortcoming of this application lies in the unclear boundaries between frames. Because new DVS events also affect the current event frame during the reading of the current frame, the time interval between adjacent frames is not fixed. However, DVS events themselves are event-based; a DVS event is generated when the brightness change exceeds a threshold. They do not inherently have clear frame boundaries. The frames described in existing technologies are merely statistical representations of DVS events over a period of time obtained through artificially defined time intervals, intended only to facilitate frame division for subsequent data processing, AI training, etc. The frames in this application do not have clearly fixed time intervals; they simply use a different frame division method than existing technologies. However, they do not lose DVS events, ensuring that all information is available for subsequent data processing and AI training without affecting these processes.

[0085] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for low-cost implementation of dynamic vision sensor event frame transfer, characterized in that, The method comprises the following steps: S1, acquiring a DVS image size and a down-sampling factor, determining a number of sampling regions based on the DVS image size and the down-sampling factor, and one-to-one corresponding the sampling regions to counters in a storage space; S2, whenever a new DVS event arrives, acquiring a pixel position of the DVS event, calculating a sampling region corresponding to the DVS event, and updating the counter corresponding to the sampling region of the DVS event; S3, when a framesync signal indicating the start of a new frame is valid, starting from a first address of the storage space, sequentially performing a read and clear operation on each address to obtain an event frame, wherein the read and clear operation specifically comprises: reading a reading of the counter in the current address, and clearing the counter in the current address.

2. The method of claim 1, wherein, In step S3, the reading of the counter in the storage space is read to an external memory to obtain an event frame. When the read and clear operation is performed, the reading of the counter in the current address is first read to an on-chip cache, and the counter in the current address is cleared, and then the reading of the counter is written to the external memory from the on-chip cache.

3. The method of claim 2, wherein, In the process of performing the read and clear operation on the counter in the address i, if a new DVS event arrives, an address j of the counter corresponding to the new DVS event in the storage space is calculated: If j>i, the read and clear operation on the counter in the address i is continued, after the read and clear operation on the counter in the address i is completed, the counter corresponding to the new DVS event is updated, and the read and clear operation on the next address is continued; If j<i, the read and clear operation on the counter in the address i is continued, after the read and clear operation on the counter in the address i is completed, the counter corresponding to the new DVS event is updated, and the read and clear operation on the next address is continued; If j=i, and the reading of the counter has not been written to the on-chip cache, the reading of the counter in the address i is read out, the read counter reading is updated based on the new DVS event, the updated counter reading is written to the on-chip cache, the counter in the address i is cleared, and the read and clear operation on the next address is continued; If j=i, and the reading of the counter has been written to the on-chip cache, the reading of the counter is directly written based on the new DVS event, and the read and clear operation on the next address is continued.

4. The method of claim 3, wherein, The updating of the read counter reading based on the new DVS event specifically comprises: if the counter has not counted to the maximum value of the counter, the counter reading is increased by 1, otherwise, the counter reading remains unchanged; the direct writing of the counter reading based on the new DVS event specifically comprises: performing a write-1 operation on the counter.

5. The method of claim 1, wherein, In step S2, the updating of the counter corresponding to the sampling region of the DVS event specifically comprises: if the counter has not counted to the maximum value of the counter, the counter is increased by 1, otherwise, the counter remains unchanged.

6. The method of claim 1, wherein, The DVS image size is ROW_MAX*COL_MAX, wherein ROW_MAX represents the number of pixel rows of the DVS image, and COL_MAX represents the number of pixel columns of the DVS image, the downsampling factor includes a row downsampling factor and a column downsampling factor, wherein the row downsampling factor is ROW_POOL:1, the column downsampling factor is COL_POOL:1, and the size of the downsampling region is ROW_POOL*COL_POOL; wherein ROW_MAX, COL_MAX, ROW_POOL, and COL_POOL are all positive integers.

7. The method of claim 6, wherein, PIXEL_WIDTH represents the bit width of a counter corresponding to one pixel, and the bit width of a counter corresponding to one sampling region is: PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL)) The size of the storage space is: Num*(PIXEL_WIDTH+ceil(log2(ROW_POOL*COL_POOL))) Wherein, the function ceil(x) returns the minimum integer value not less than x, and Num represents the number of counters.

8. The method of claim 7, wherein, The number of sampling regions is equal to the number of counters, and the number of counters Num is: Num=(ceil(ROW_MAX / ROW_POOL))*(ceil(COL_MAX / COL_POOL)) Wherein, the function ceil(x) returns the minimum integer value not less than x.

9. The method of claim 1, wherein, In step S3, the framesync signal indicating the start of a new frame is valid once every preset time interval.

10. A low cost implementation of dynamic vision sensor event frame transfer, characterized by, The low-cost implementation method of the dynamic vision sensor event frame conversion based on any one of claims 1-9 comprises a storage space, an external memory, and a controller; the counters in the storage space are used for counting DVS events, the external memory is used for storing event frames, and the controller is used for receiving DVS events and performing read operation, write operation, and update operation on the storage space and write operation on the external memory based on the received DVS events.

Images