Event camera asynchronous signal visualization method and system

Abstract

The invention provides an event camera asynchronous signal visualization method and system. The method comprises the following steps: acquiring an event set in a preset time interval; executing event extraction to extract a plurality of events from the event set, and stacking the plurality of events to form a temporary image frame; stacking the temporary image frame to a pre-initialized event frame to form a current time frame; performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; calculating the pixel overlapping number of the non-maximum current time frame and the temporary image frame and the pixel overlapping number of the current time frame and the temporary image frame, and correspondingly obtaining the first pixel overlapping number and the second pixel overlapping number; and judging whether event extraction is completed or not according to the first pixel overlapping number, the second pixel overlapping number or whether the event set is empty or not, and generating a visualization result of the event camera asynchronous signal according to whether the event extraction is completed or not and the current time frame.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and system for visualizing asynchronous signals of an event camera. Background Art

[0002] In recent years, researchers have proposed a series of new imaging devices based on bionic principles, represented by network video servers (Dynamic Vision Sensor, DVS for short) - event cameras. This type of imaging device adopts the "address event representation method". Its internal sensing array works independently with each individual pixel, and only responds to the relative change of logarithmic light intensity. When the relative change exceeds a specific threshold, it will asynchronously output the sensing result. Each sensing result is called an event e = (x, y, p, t), where x and y are pixel space coordinates, p is the polarity of light intensity increase or decrease, t is time, and the continuously output events constitute an event stream. Event cameras have subverted the traditional photoelectric conversion imaging method and have the advantages of high temporal resolution, large dynamic range, low power consumption, low delay, and less redundancy. Event cameras no longer have the concept of frames, are no longer affected by absolute light intensity, are driven by data on a macro scale, and only image objects that move relative to the event camera, which can greatly reduce the bandwidth and data throughput required for data transmission.

[0003] For visual sensors, visualizing imaging results is a primary task. However, event cameras output asynchronous, sparse event streams that need to be stacked into image frames for visualization.

[0004] Currently, there are two main ways to stack event streams into image frames: stacking by time interval and stacking by number of events. Stacking by time interval means initializing an image frame with width W, height H, and number of channels C (different channels store different polarities), taking an event set e within an event interval Δt Δt ={e i},i=1~N,e i =(x i ,y i ,t i ,p i ). Whenever an event is received, the polarity p i Find the image frame corresponding to channel c i (e.g. negative polarity corresponds to the first channel, positive polarity corresponds to the second channel) the corresponding pixel position (x i ,y i ), assign it to v, and finally get an image frame. The process of stacking by number of events is similar to stacking by time interval, except that the events taken are a fixed number of event sets e N .

[0005] The stacking method based on time intervals can achieve real-time synchronous display, but when the target moves quickly, too many events will be accumulated, resulting in severe motion blur; while the stacking method based on the number of events can avoid blur caused by too many events, but it cannot achieve time sequence synchronization, and the number of events required for different scenes, different objects, different event cameras, etc. varies greatly, and the number of stacked events needs to be manually adjusted. Therefore, the existing method of stacking event streams into image frames has the problems of weak generalization ability and motion blur. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a method and system for visualizing asynchronous signals of an event camera.

[0007] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0008] In a first aspect, the present invention provides a method for visualizing an asynchronous signal of an event camera, comprising:

[0009] S101, obtaining an event set within a preset time interval;

[0010] S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame;

[0011] S103, stacking the temporary image frame onto the pre-initialized event frame to form a current time frame;

[0012] S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame;

[0013] S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number;

[0014] S106. Determine whether event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame.

[0015] Optionally, after stacking the temporary image frame to the pre-initialized event frame to form the current time frame, a method for visualizing an asynchronous signal of an event camera further includes:

[0016] Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result;

[0017] Perform non-maximum suppression on the current time frame to obtain a non-maximum current time frame, including:

[0018] The first down-sampling result is subjected to non-maximum suppression processing to obtain a non-maximum current time frame.

[0019] Optionally, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes:

[0020] Determine whether the pixel values ​​of all event frames in the first down-sampling result and the second down-sampling result are consistent by traversing, and obtain the number of first pixel overlaps;

[0021] It is determined by traversing whether the pixel values ​​of the non-maximum current time frame and all event frames in the second down-sampling result are consistent, and the second pixel overlap number is obtained.

[0022] Optionally, judging whether event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame, including:

[0023] Calculating the ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio;

[0024] Determine whether the value of the third pixel overlap ratio is greater than a preset ratio value;

[0025] If the value of the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and the visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0026] Optionally, if the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including:

[0027] Linearly stretch or pixel map the current time frame to obtain the visualization result of the event camera.

[0028] Optionally, if the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0029] Optionally, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes:

[0030] Determine whether each pixel of the current time frame in the preset source frame is a maximum value in a preset direction within the neighborhood of the current pixel;

[0031] If it is the maximum value, the current pixel is retained, otherwise the value of the current pixel is set to 0 to obtain a non-maximum current time frame.

[0032] Optionally, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame, including:

[0033] Extract multiple events from an event set;

[0034] According to the polarity information in each event, the channel of the event is determined;

[0035] According to the channel of the event and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned to the time information in the event to form a temporary image frame.

[0036] In a second aspect, the present invention provides a visualization system for asynchronous signals of an event camera, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the visualization system for asynchronous signals of an event camera is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the method of the first aspect.

[0037] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method in the first aspect are executed.

[0038] The present invention provides a method and system for visualizing asynchronous signals of an event camera. The method includes: S101, obtaining an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking multiple events to form a temporary image frame; S103, stacking the temporary image frame to a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106, judging whether event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the asynchronous signal of the event camera according to whether event extraction is completed and the current time frame. In the present invention, by sampling small batches of events from an event set within a preset time interval, it is possible to achieve time synchronization and at the same time ensure that clear image frames can be output even in fast motion conditions; in addition, event extraction is judged using the first pixel overlap number and the second pixel overlap number, which can meet the needs of a variety of scenes, a variety of objects, and a variety of event cameras, has strong adaptability, and improves the robustness of the event camera asynchronous signal visualization and the image visualization quality.

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a method for visualizing an asynchronous signal of an event camera provided by an embodiment of the present invention;

[0041] Figure 2 The results of the method of the present invention running on the DAVIS 346 event camera;

[0042] Figure 3 The results of the method of the present invention running on the CeleX V event camera;

[0043] Figure 4 A schematic diagram of a visualization system for event camera asynchronous signals provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0045] In order to improve the robustness and visualization quality of event camera asynchronous signal visualization, an embodiment of the present invention provides a method for visualizing event camera asynchronous signals. Figure 1The present invention provides a flow chart of a method for visualizing an asynchronous signal of an event camera. Figure 1 As shown, including:

[0046] S101. Acquire an event set within a preset time interval.

[0047] S102: Execute event extraction to extract multiple events from the event set, and stack the multiple events to form a temporary image frame.

[0048] Optionally, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame, including:

[0049] Extract multiple events from an event set;

[0050] According to the polarity information in each event, the channel of the event is determined;

[0051] According to the channel of the event and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned to the time information in the event to form a temporary image frame.

[0052] It should be noted that the event extraction in the embodiment of the present invention is sequential extraction, and the extraction method is small batch event sampling.

[0053] In the embodiment of the present invention, the stacking process of multiple events may exemplarily be:

[0054] For the event set e N ={e i Each event e in}(i=1~N) i =(x i ,y i ,t i ,p i ), N represents the number of extracted events, e i Represents the event set e N Any event in x i ,y i ,t i ,p i Respectively represent e i The horizontal coordinate, vertical coordinate, event time and event polarity of i Find the corresponding channel c of the preset initial temporary image frame i (If it is negative polarity, it corresponds to the first channel, if it is positive polarity, it corresponds to the second channel) i ,y i ) and assign t to the pixel position i .

[0055] S103: stacking the temporary image frame to the pre-initialized event frame to form a current time frame.

[0056] It should be noted that the stacking method of the image frames in step S103 may refer to step S102, and the specific implementation process will not be repeated in this embodiment.

[0057] S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame.

[0058] Optionally, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes:

[0059] Determine whether each pixel of the current time frame in the preset source frame is a maximum value in a preset direction within the neighborhood of the current pixel;

[0060] If it is the maximum value, the current pixel is retained, otherwise the value of the current pixel is set to 0 to obtain a non-maximum current time frame.

[0061] Optionally, after stacking the temporary image frame to the pre-initialized event frame to form the current time frame, a method for visualizing an asynchronous signal of an event camera further includes:

[0062] Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result;

[0063] Perform non-maximum suppression on the current time frame to obtain a non-maximum current time frame, including:

[0064] The first down-sampling result is subjected to non-maximum suppression processing to obtain a non-maximum current time frame.

[0065] It should be noted that when the current time frame is I and the temporary image frame is I temp . to I and I temp The first downsampling result d(I) and the second downsampling result d(I) are obtained by performing "downsampling" respectively. temp ). The "downsampling" operation can be exemplarily: given a source frame with a width of W, a height of H, and a number of channels of C, set the downsampling ratio B. For each non-overlapping B×B neighborhood in the temporary image frame or the current time frame, each neighborhood represents a corresponding pixel of the non-maximum current time frame, then the corresponding pixel of the non-maximum current time frame takes the maximum value of the neighborhood pixels t max The final output non-maximum current time frame width is W / B, height is H / B, and the number of channels is C.

[0066] Perform "non-maximum suppression" on d(I) to obtain n(d(I)). The "non-maximum suppression" operation can be specifically a source frame with a given width of W1, a height of H1, and a number of channels of C1. For each pixel in the source frame, if the pixel is the maximum value of a preset direction in a preset neighborhood, the pixel is retained, otherwise the pixel value is set to 0.

[0067] Exemplarily, in the embodiment of the present invention, the preset neighborhood may be specifically 8 neighborhoods, and the preset directions may be specifically 0°, 45°, 90°, and 135°. It should be noted that the preset directions in the embodiment of the present invention are only exemplary, and the specific direction settings may be flexibly adjusted according to specific needs.

[0068] S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number.

[0069] Optionally, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes:

[0070] Determine whether the pixel values ​​of all event frames in the first down-sampling result and the second down-sampling result are consistent by traversing, and obtain the number of first pixel overlaps;

[0071] It is determined by traversing whether the pixel values ​​of the non-maximum current time frame and all event frames in the second down-sampling result are consistent, and the second pixel overlap number is obtained.

[0072] S106. Determine whether event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame.

[0073] An embodiment of the present invention provides a method for visualizing an asynchronous signal of an event camera, comprising: S101, acquiring an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; S103, stacking the temporary image frame to a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106, judging whether event extraction is completed based on the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the asynchronous signal of the event camera based on whether event extraction is completed and the current time frame. In an embodiment of the present invention, by sampling small batches of events from an event set within a preset time interval, it is possible to achieve time synchronization and at the same time ensure that clear image frames can be output even in fast motion conditions; in addition, event extraction is judged using the first pixel overlap number and the second pixel overlap number, which can meet the needs of a variety of scenes, a variety of objects, and a variety of event cameras, has strong adaptability, and improves the robustness of the event camera asynchronous signal visualization and the image visualization quality.

[0074] Optionally, judging whether event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame, including:

[0075] Calculating the ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio;

[0076] Determine whether the value of the third pixel overlap ratio is greater than a preset ratio value;

[0077] If the value of the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and the visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0078] In the embodiment of the present invention, when the first pixel overlap number is rate1 and the second pixel overlap number is rate2, the third pixel overlap ratio rate=rate1 / rate2, and the preset ratio value is Thre.

[0079] When rate>Thre or the event set is empty, it indicates that event extraction has been completed. At this time, event stacking is stopped, the value of the current time frame I is stretched to linear [0,255] or the value greater than 0 is mapped to 255, and the visualization result of the event camera asynchronous signal is output.

[0080] It should be noted that the embodiment of the present invention determines whether event extraction is completed based on the third pixel overlap ratio and the preset ratio value Thre, and the generalization ability of this judgment method is relatively strong, mainly based on:

[0081] Assume that, in a sequence of events captured over a period of time, some of the events describe the clear outline of an object. Since the outline of the object is fixed, the number of events in this part will not change with the value of Δt. When visualizing, a clear effect can be achieved when the events taken are close to these events. rate2 is the mathematical description of the events of this part of the outline, rate1 is the mathematical description of the events currently taken, and rate represents the ratio of the number of events currently taken to the number of events in this part of the outline. When the ratio is too large, too many events are taken, which will cause blurring; when the ratio is too small, too few events are taken, which is not enough to describe the outline of the object. Therefore, there must be an intermediate value Thre in the middle, and this intermediate value Thre, because it is a ratio, does not change with the change of the size of the object outline. Therefore, when the event camera works normally, after determining Thre through preliminary experiments, there is no need to change the threshold, so the generalization ability is strong.

[0082] The embodiment of the present invention defines an indicator Thre indicating whether frame stacking is clear by combining the characteristics of stacking by time interval and stacking by time quantity. For an event sequence within a certain time interval, frames are continuously stacked in small batches and cumulatively. If the indicator is reached, the stacking is terminated in advance, otherwise the stacking is continued until the end of the sequence is reached, so as to finally obtain a time-synchronized and clear image frame.

[0083] Optionally, if the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including:

[0084] Linearly stretch or pixel map the current time frame to obtain the visualization result of the event camera.

[0085] Optionally, if the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

[0086] In order to illustrate the effectiveness of the method for visualizing an asynchronous signal of an event camera provided by an embodiment of the present invention, a simulation test is also performed in the embodiment of the present invention.

[0087] Experimental setup

[0088] 1. Hyperparameter settings

[0089] In the actual testing process, for different scenarios and different devices, the algorithm's hyperparameter settings are: preset time interval Δt=30ms, preset ratio value Thre=1.25, and the number of small batch events extracted N=1000.

[0090] 2. Test results display and analysis

[0091] Figure 2 and Figure 3 All of them are test comparison diagrams provided by embodiments of the present invention. Figure 2 and Figure 3 They are respectively the results of the method of the present invention running on different event cameras.

[0092] Figure 2 The results of the method of the present invention running on the DAVIS 346 event camera are shown in Figure 2. Figure 2 Figure (a) is a test result diagram using the method of the present invention. Figure 2 Figure (b) shows the test results using the time interval stacking method. Figure 2 The first to fourth rows of are respectively a fast-moving car, a chessboard photographed by a slow-moving camera, an indoor scene photographed by a fast-moving camera, and an outdoor scene photographed by a fast-moving camera. Figure 2 It can be seen from the figure that, under the premise of achieving timing synchronization, the present invention can robustly output relatively clear image frames for different scenes, different motion speeds, different object sizes, etc.

[0093] Figure 3 is the result of the method of the present invention running on the CeleX V event camera, Figure 3 Figure (a) is a test result diagram using the method of the present invention. Figure 3 Figure (b) shows the test results using the time interval stacking method. Figure 3 It can be seen that in Figure 3 The method provided by the present invention can still obtain good experimental results on the CeleX V camera, which shows that the method of the present invention can be applied to different event cameras and has strong versatility.

[0094] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0095] Based on the same inventive concept, an embodiment of the present invention further provides a visualization system for asynchronous signals of an event camera. Figure 4A schematic diagram of a visualization system for asynchronous signals of an event camera provided by an embodiment of the present invention includes: a processor 710, a storage medium 720 and a bus 730, wherein the storage medium 720 stores machine-readable instructions executable by the processor 710, and when the visualization system for asynchronous signals of an event camera is running, the processor 710 communicates with the storage medium 720 via the bus 730, and the processor 710 executes the machine-readable instructions to perform the steps of the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0096] The storage medium may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk storage. Optionally, the storage medium may also be at least one storage device located away from the aforementioned processor.

[0097] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0098] The present invention also provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, any of the above-mentioned method steps for visualizing an asynchronous signal of an event camera is implemented.

[0099] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0100] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0101] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other changes to the disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the term "comprising" does not exclude other components or steps, "one" or "an" does not exclude multiple situations, and "multiple" means two or more, unless otherwise clearly and specifically limited. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0102] As for the device / electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for visualizing asynchronous signals of an event camera, characterized in that: include: S101, obtaining an event set within a preset time interval; S102, performing event extraction to extract multiple events from the event set, and stacking the multiple events to form a temporary image frame; S103, stacking the temporary image frame onto a pre-initialized event frame to form a current time frame; S104, performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame; S105, calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number; S106. Determine whether the event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generate a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame.

2. The method for visualizing an asynchronous signal of an event camera according to claim 1, characterized in that: After stacking the temporary image frame to the pre-initialized event frame to form the current time frame, the method for visualizing an event camera asynchronous signal further includes: Downsampling the current time frame and the temporary image frame respectively to obtain a first downsampling result and a second downsampling result; The performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes: The first down-sampling result is subjected to non-maximum suppression processing to obtain a non-maximum current time frame.

3. The method for visualizing an asynchronous signal of an event camera according to claim 2, characterized in that: The calculating the number of pixel overlaps between the non-maximum current time frame and the temporary image frame and the number of pixel overlaps between the current time frame and the temporary image frame, and correspondingly obtaining a first pixel overlap number and a second pixel overlap number, includes: Determine whether the pixel values ​​of all event frames in the first down-sampling result and the second down-sampling result are consistent by traversing, and obtain the number of overlaps of the first pixel; Whether the pixel values ​​of the non-maximum current time frame and all event frames in the second down-sampling result are consistent is determined by traversal, so as to obtain the second pixel overlap number.

4. The method for visualizing an event camera asynchronous signal according to claim 3, characterized in that: The determining whether the event extraction is completed according to the first pixel overlap number and the second pixel overlap number or whether the event set is empty, and generating a visualization result of the event camera asynchronous signal according to whether the event extraction is completed and the current time frame, includes: Calculating a ratio of the first pixel overlap number to the second pixel overlap number to obtain a third pixel overlap ratio; Determine whether the value of the third pixel overlap ratio is greater than a preset ratio value; If the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

5. The method for visualizing an event camera asynchronous signal according to claim 4, characterized in that: If the value of the third pixel overlap ratio is greater than a preset ratio value or the event set is empty, it indicates that the event extraction has been completed, and a visualization result of the event camera asynchronous signal is generated according to the current time frame, including: The current time frame is linearly stretched or pixel mapped to obtain the visualization result of the event camera.

6. The method for visualizing an event camera asynchronous signal according to claim 4, characterized in that: If the value of the third pixel overlap ratio is less than the preset ratio value or the event set is not empty, it indicates that the event extraction is not completed, and steps S102-S105 are repeated until the third pixel overlap ratio is greater than the preset ratio value or the event set is empty, and a visualization result of the event camera asynchronous signal is generated according to the current time frame.

7. The method for visualizing an event camera asynchronous signal according to claim 1, characterized in that: The performing non-maximum suppression processing on the current time frame to obtain a non-maximum current time frame includes: Determine whether each pixel of the current time frame in the preset source frame is a maximum value in a preset direction within the neighborhood of the current pixel; If it is a maximum value, the current pixel is retained; otherwise, the value of the current pixel is set to 0 to obtain the non-maximum current time frame.

8. The method for visualizing an event camera asynchronous signal according to claim 1, characterized in that: The performing event extraction to extract a plurality of events from the event set and stacking the plurality of events to form a temporary image frame comprises: Extracting a plurality of events from the event set; Determining a channel of the event according to polarity information in each of the events; According to the channel of the event and the coordinate information in the event, the event is set at the temporary coordinate position of the preset initial temporary image frame, and the temporary coordinate position is assigned as the time information in the event to form the temporary image frame.

9. A visualization system for event camera asynchronous signals, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the visualization system for event camera asynchronous signals is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are executed.

Images