A method and system for visualizing asynchronous signals of event cameras

Abstract

The present invention provides a method and system for visualizing an asynchronous signal of an event camera. The method includes: obtaining an event set within a preset time interval; performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; stacking the temporary image frame onto a pre-initialized event frame to form a current time frame; performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame, and the number of pixel overlaps between the current time frame and the temporary image frame, to obtain a first pixel overlap number and a second pixel overlap number; determining whether event extraction is complete based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the asynchronous signal of the event camera based on whether event extraction is complete and the current time frame.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for visualizing asynchronous signals of an event camera. Background Art

[0002] In recent years, researchers have proposed a series of new imaging devices based on biomimetic principles, exemplified by the Dynamic Vision Sensor (DVS). These devices utilize an "address event representation" approach, where each pixel in their internal sensing array operates independently, responding only to relative changes in logarithmic light intensity. Sensing results are asynchronously output when the relative change exceeds a specific threshold. Each sensing result is called an event, e = (x, y, p, t), where x and y are the pixel coordinates, p is the polarity of the light intensity increase or decrease, and t is time. The continuous output of these events constitutes an event stream. Event cameras revolutionize traditional photoelectric conversion imaging, offering advantages such as high temporal resolution, wide dynamic range, low power consumption, minimal latency, and minimal redundancy. Event cameras eliminate the concept of frames and are unaffected by absolute light intensity. Driven by data at a macro level, they only image objects that are in relative motion with the event camera, significantly reducing the bandwidth and throughput required for data transmission.

[0003] For visual sensors, visualizing imaging results is a primary task. However, event cameras output asynchronous, sparse event streams that need to be stacked into image frames for visualization.

[0004] Currently, there are two main ways to stack event streams into image frames: stacking by time interval and stacking by number of events. Stacking by time interval means initializing an image frame with width W, height H, and number of channels C (different channels store different polarities), taking the event set e within an event interval Δt Δt ={e i},i=1~N,e i =(x i ,y i ,t i ,p i ). Whenever an event is received, the polarity p i Find the image frame corresponding to channel c i (e.g. negative polarity corresponds to the first channel, positive polarity corresponds to the second channel) the corresponding pixel position (x i ,y i ), assign it to v, and finally get an image frame. The process of stacking by the number of events is similar to stacking by time interval, the difference is that the events taken are a fixed number of event sets e N .

[0005] Stacking by time interval allows for real-time, synchronized display, but when the subject moves rapidly, excessive events can accumulate, resulting in severe motion blur. Stacking by number of events into frames avoids blurring caused by excessive event accumulation, but it lacks temporal synchronization. Furthermore, the number of events required varies significantly for different scenes, objects, and event cameras, requiring manual adjustment of the number of stacked events. Therefore, the existing method of stacking event streams into image frames suffers from weak generalization and motion blur. Summary of the Invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a method and system for visualizing asynchronous signals of an event camera.

[0007] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0008] In a first aspect, the present invention provides a method for visualizing an asynchronous signal of an event camera, comprising:

[0009] S101, obtaining an event set within a preset time interval;

[0010] S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame;

[0011] S103, stacking the temporary image frame onto the pre-initialized event frame to form a current time frame;

[0012] S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame;

[0013] S105, calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number;

[0014] S106. Determine whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal based on whether the event extraction is completed and the current time frame.

[0015] Optionally, after stacking the temporary image frame onto the pre-initialized event frame to form the current time frame, the method for visualizing an asynchronous signal of an event camera further includes:

[0016] Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result;

[0017] Perform non-maximum suppression on the current time frame to obtain a non-maximum current time frame, including:

[0018] A non-maximum suppression process is performed on the first down-sampling result to obtain a non-maximum current time frame.

[0019] Optionally, calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes:

[0020] Determine whether the pixel values ​​of all event frames in the first down-sampling result and the second down-sampling result are consistent by traversing, and obtain the number of first pixel overlaps;

[0021] Whether the pixel values ​​of the non-maximum current time frame and all event frames in the second downsampling result are consistent is determined by traversal, and the second pixel overlap number is obtained.

[0022] Optionally, judging whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal based on whether the event extraction is completed and the current time frame, including:

[0023] Calculating a ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio;

[0024] Determining whether the value of the third pixel overlap ratio is greater than a preset ratio value;

[0025] If the value of the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0026] Optionally, if the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including:

[0027] Linearly stretch or pixel map the current time frame to obtain the visualization result of the event camera.

[0028] Optionally, if the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0029] Optionally, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes:

[0030] Determine whether each pixel of the current time frame in the preset source frame is the maximum value in the preset direction within the neighborhood of the current pixel;

[0031] If it is the maximum value, the current pixel is retained; otherwise, the value of the current pixel is set to 0 to obtain a non-maximum current time frame.

[0032] Optionally, performing event extraction to extract multiple events from the event set and stacking the multiple events to form a temporary image frame includes:

[0033] Extract multiple events from an event set;

[0034] Determine the channel of the event based on the polarity information in each event;

[0035] According to the event channel and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned to the time information in the event to form a temporary image frame.

[0036] In a second aspect, the present invention provides a visualization system for asynchronous signals of an event camera, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When a visualization system for asynchronous signals of an event camera is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the method of the first aspect mentioned above.

[0037] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in the first aspect are executed.

[0038] The present invention provides a method and system for visualizing asynchronous signals of an event camera. The method includes: S101, obtaining an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; S103, stacking the temporary image frame onto a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame, and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106, judging whether event extraction is complete based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the asynchronous signal of the event camera based on whether event extraction is complete and the current time frame. In the present invention, by sampling small batches of events from an event set within a preset time interval, it is possible to achieve time synchronization while ensuring the output of clear image frames even in fast motion conditions. In addition, the use of the first pixel overlap number and the second pixel overlap number to determine event extraction can meet the needs of a variety of scenes, multiple objects, and multiple event cameras, with strong adaptability, thereby improving the robustness of the event camera's asynchronous signal visualization and the image visualization quality.

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a method for visualizing asynchronous signals of an event camera provided by an embodiment of the present invention;

[0041] Figure 2 The results of the method of the present invention running on the DAVIS 346 event camera;

[0042] Figure 3 The results of the method of the present invention running on the CeleX V event camera;

[0043] Figure 4 A schematic diagram of a visualization system for event camera asynchronous signals provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0045] In order to improve the robustness and visualization quality of event camera asynchronous signal visualization, an embodiment of the present invention provides a method for visualizing event camera asynchronous signals. Figure 1The flowchart of the method for visualizing asynchronous signals of an event camera provided by an embodiment of the present invention is as follows. Figure 1 Shown, including:

[0046] S101: Acquire an event set within a preset time interval.

[0047] S102: Execute event extraction to extract multiple events from the event set, and stack the multiple events to form a temporary image frame.

[0048] Optionally, performing event extraction to extract multiple events from the event set and stacking the multiple events to form a temporary image frame includes:

[0049] Extract multiple events from an event set;

[0050] Determine the channel of the event based on the polarity information in each event;

[0051] According to the event channel and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned to the time information in the event to form a temporary image frame.

[0052] It should be noted that the event extraction in the embodiment of the present invention is sequential extraction, and the extraction method is small batch event sampling.

[0053] In the embodiment of the present invention, the stacking process of multiple events may illustratively be:

[0054] For the event set e N ={e i Each event e in}(i=1~N) i =(x i ,y i ,t i ,p i ), N represents the number of extracted events, e i Represents the event set e N Any event in x i ,y i ,t i ,p i Respectively represent e i The horizontal coordinate, vertical coordinate, event time and event polarity of i Find the corresponding channel c of the preset initial temporary image frame i (If it is negative polarity, it corresponds to the first channel, if it is positive polarity, it corresponds to the second channel) the corresponding pixel position (x i ,y i ) and assign t to the pixel position i .

[0055] S103: stacking the temporary image frame onto the pre-initialized event frame to form a current time frame.

[0056] It should be noted that the stacking method of the image frames in step S103 may refer to step S102, and the specific implementation process will not be repeated in this embodiment.

[0057] S104: Perform non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame.

[0058] Optionally, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes:

[0059] Determine whether each pixel of the current time frame in the preset source frame is the maximum value in the preset direction within the neighborhood of the current pixel;

[0060] If it is the maximum value, the current pixel is retained; otherwise, the value of the current pixel is set to 0 to obtain a non-maximum current time frame.

[0061] Optionally, after stacking the temporary image frame onto the pre-initialized event frame to form the current time frame, the method for visualizing an asynchronous signal of an event camera further includes:

[0062] Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result;

[0063] Perform non-maximum suppression on the current time frame to obtain a non-maximum current time frame, including:

[0064] A non-maximum suppression process is performed on the first down-sampling result to obtain a non-maximum current time frame.

[0065] It should be noted that when the current time frame is I and the temporary image frame is I temp . to I and I temp Perform "down sampling" respectively to obtain the first down sampling result d(I) and the second down sampling result d(I temp ). The "downsampling" operation can be exemplarily as follows: given a source frame with width W, height H, and number of channels C, set the downsampling factor B. For each non-overlapping B×B neighborhood in the temporary image frame or the current time frame, each neighborhood represents a corresponding pixel of the non-maximal current time frame, then the corresponding pixel of the non-maximal current time frame takes the maximum value of the neighborhood pixels t max The final output non-maximum current time frame width is W / B, height is H / B, and the number of channels is C.

[0066] Perform non-maximum suppression on d(I) to obtain n(d(I)). Specifically, the non-maximum suppression operation is performed on a source frame with width W1, height H1, and number of channels C1. For each pixel in the source frame, if the pixel is the maximum value in a preset neighborhood and a preset direction, the pixel is retained; otherwise, the pixel value is set to 0.

[0067] For example, in the embodiment of the present invention, the preset neighborhood may be specifically 8 neighborhoods, and the preset directions may be specifically 0°, 45°, 90°, and 135°. It should be noted that the preset directions in the embodiment of the present invention are only exemplary, and the specific direction settings can be flexibly adjusted according to specific needs.

[0068] S105 , calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number.

[0069] Optionally, calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes:

[0070] Determine whether the pixel values ​​of all event frames in the first down-sampling result and the second down-sampling result are consistent by traversing, and obtain the number of first pixel overlaps;

[0071] Whether the pixel values ​​of the non-maximum current time frame and all event frames in the second downsampling result are consistent is determined by traversal, and the second pixel overlap number is obtained.

[0072] S106. Determine whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal based on whether the event extraction is completed and the current time frame.

[0073] An embodiment of the present invention provides a method for visualizing an asynchronous signal of an event camera, including: S101, obtaining an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; S103, stacking the temporary image frame into a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame, and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106, judging whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the asynchronous signal of the event camera based on whether the event extraction is completed and the current time frame. In an embodiment of the present invention, by sampling small batches of events from an event set within a preset time interval, it is possible to achieve time synchronization while ensuring the output of clear image frames even in fast motion conditions. In addition, event extraction is judged based on the number of first pixel overlaps and the number of second pixel overlaps, which can meet the needs of various scenes, various objects, and various event cameras, has strong adaptability, and improves the robustness of the event camera asynchronous signal visualization and the image visualization quality.

[0074] Optionally, judging whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal based on whether the event extraction is completed and the current time frame, including:

[0075] Calculating a ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio;

[0076] Determining whether the value of the third pixel overlap ratio is greater than a preset ratio value;

[0077] If the value of the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0078] In the embodiment of the present invention, when the first pixel overlap number is rate1 and the second pixel overlap number is rate2, the third pixel overlap ratio rate=rate1 / rate2, and the preset ratio value is Thre.

[0079] When rate>Thre or the event set is empty, it indicates that event extraction has been completed. At this time, event stacking is stopped, the value of the current time frame I is stretched to a linear range of [0,255] or the value greater than 0 is mapped to 255, and the visualization result of the event camera asynchronous signal is output.

[0080] It should be noted that the embodiment of the present invention determines whether event extraction is completed based on the third pixel overlap ratio and the preset ratio value Thre, and the judgment method has a strong generalization capability, mainly based on:

[0081] Suppose that a sequence of events captured over a period of time contains a subset of events that clearly depict the outline of an object. Since the object's outline is fixed, the number of events in this subset does not change with the value of Δt. When visualizing, clarity is achieved when the selected events approximate these subsets. rate2 is the mathematical description of the events in this subset, and rate1 is the mathematical description of the currently selected events. rate represents the ratio of the number of events currently selected relative to the number of events in this subset. When the ratio is too large, too many events are selected, resulting in blurring; when the ratio is too small, too few events are selected to adequately depict the object's outline. Therefore, an intermediate value, Thre, must exist between these two thresholds. Since this intermediate value, Thre, is a ratio, it does not change with changes in the size of the object's outline. Therefore, if the event camera is functioning properly, once Thre is determined through preliminary experiments, there is no need to change the threshold, resulting in strong generalization capabilities.

[0082] The embodiment of the present invention defines an indicator Thre that indicates whether the frame stacking is clear by combining the characteristics of stacking by time interval and stacking by time quantity. For an event sequence within a certain time interval, frames are continuously stacked in small batches and cumulatively. If the indicator is reached, the stacking is terminated early. Otherwise, the stacking is continued until the end of the sequence is reached, and finally a time-synchronized and clear image frame is obtained.

[0083] Optionally, if the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including:

[0084] Linearly stretch or pixel map the current time frame to obtain the visualization result of the event camera.

[0085] Optionally, if the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0086] In order to illustrate the effectiveness of the method for visualizing asynchronous signals of an event camera provided by an embodiment of the present invention, a simulation test was also performed in the embodiment of the present invention.

[0087] Experimental setup

[0088] 1. Hyperparameter Setting

[0089] In the actual testing process, for different scenarios and different devices, the algorithm's hyperparameter settings are: preset time interval Δt = 30ms, preset ratio value Thre = 1.25, and the number of small batch events extracted N = 1000.

[0090] 2. Test results presentation and analysis

[0091] Figure 2 and Figure 3 These are all test comparison charts provided by embodiments of the present invention. Figure 2 and Figure 3 These are the results of the method of the present invention running on different event cameras.

[0092] Figure 2 is the result of running the method of the present invention on the DAVIS 346 event camera, Figure 2 Figure (a) is a test result diagram using the method of the present invention. Figure 2 Figure (b) shows the test results using the time interval stacking method. Figure 2 The first to fourth rows of are respectively a fast-moving car, a chessboard shot by a slow-moving camera, an indoor scene shot by a fast-moving camera, and an outdoor scene shot by a fast-moving camera. Figure 2 It can be seen from the figure that, under the premise of achieving timing synchronization, the present invention can robustly output relatively clear image frames for different scenes, different motion speeds, different object sizes, etc.

[0093] Figure 3 is the result of running the method of the present invention on the CeleX V event camera, Figure 3 Figure (a) is a test result diagram using the method of the present invention. Figure 3 Figure (b) shows the test results using the time interval stacking method. Figure 3 It can be seen that in Figure 3 The method provided by the present invention can still obtain good experimental results on the CeleX V camera, which shows that the method provided by the present invention can be applied to different event cameras and has strong versatility.

[0094] The method provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc. This is not limited here; any electronic device that can implement the present invention falls within the scope of protection of the present invention.

[0095] Based on the same inventive concept, an embodiment of the present invention further provides a visualization system for asynchronous signals of an event camera. Figure 4A schematic diagram of a system for visualizing asynchronous signals from an event camera provided in an embodiment of the present invention includes a processor 710, a storage medium 720, and a bus 730. The storage medium 720 stores machine-readable instructions executable by the processor 710. When the system for visualizing asynchronous signals from an event camera is in operation, the processor 710 communicates with the storage medium 720 via the bus 730, and the processor 710 executes the machine-readable instructions to perform the steps of the aforementioned method embodiment. The specific implementation methods and technical effects are similar and will not be further described here.

[0096] The storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the storage medium may be at least one storage device located away from the processor.

[0097] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0098] The present invention also provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, any of the above-mentioned method steps for visualizing an asynchronous signal of an event camera is implemented.

[0099] It should be noted that the terms "first," "second," and the like are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0100] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0101] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the word "comprising" does not exclude other components or steps, "one" or "a" does not exclude multiple situations, and "multiple" means two or more, unless otherwise clearly and specifically defined. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0102] As for the device / electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for visualizing asynchronous signals of an event camera, characterized in that: include: S101, obtaining an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; S103, stacking the temporary image frame onto a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106. Determine whether the event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal based on whether the event extraction is completed and the current time frame.

2. The method for visualizing an asynchronous signal of an event camera according to claim 1, characterized in that: After stacking the temporary image frame onto the pre-initialized event frame to form the current time frame, the method for visualizing an event camera asynchronous signal further includes: Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result; The performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes: Non-maximum suppression processing is performed on the first downsampling result to obtain a non-maximum current time frame.

3. The method for visualizing an event camera asynchronous signal according to claim 2, characterized in that: The calculating the number of pixel overlaps between the non-maximal current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes: Determine whether the pixel values ​​of all event frames in the first downsampling result and the second downsampling result are consistent by traversing, and obtain the number of first pixel overlaps; Whether the pixel values ​​of the non-maximum current time frame and all event frames in the second down-sampling result are consistent is determined by traversal, and the second pixel overlap number is obtained.

4. The method for visualizing an event camera asynchronous signal according to claim 3, characterized in that: The determining whether the event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame, includes: Calculating a ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio; Determining whether the value of the third pixel overlap ratio is greater than a preset ratio value; If the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

5. The method for visualizing an event camera asynchronous signal according to claim 4, characterized in that: If the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including: The current time frame is linearly stretched or pixel mapped to obtain the visualization result of the event camera.

6. The method for visualizing an event camera asynchronous signal according to claim 4, characterized in that: If the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that the event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

7. The method for visualizing an event camera asynchronous signal according to claim 1, characterized in that: The performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes: Determine whether each pixel of the current time frame in the preset source frame is a maximum value in a preset direction within the neighborhood of the current pixel; If it is a maximum value, the current pixel is retained; otherwise, the value of the current pixel is set to 0 to obtain the non-maximum current time frame.

8. The method for visualizing an event camera asynchronous signal according to claim 1, characterized in that: The performing event extraction to extract a plurality of events from the event set and stacking the plurality of events to form a temporary image frame includes: Extracting a plurality of events from the event set; determining a channel of the event according to polarity information in each of the events; According to the channel of the event and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned to the time information in the event to form the temporary image frame.

9. A visualization system for event camera asynchronous signals, characterized in that include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the visualization system for asynchronous signals of an event camera is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the method according to any one of claims 1 to 8.

Images