A method, system and storage medium for suppressing tailing effect of event camera

Abstract

The present invention discloses a method, system and storage medium for suppressing the tailing effect of an event camera, which belongs to the field of image processing technology. The tailing suppression method includes: inputting a continuous event stream, and traversing and judging whether each event is a tailing event according to the principle that the tailing events are distributed from dense to sparse with the same polarity, and modifying the timestamp of the tailing event to the sum of the timestamp of the previous adjacent event of the same pixel and the minimum time resolution of the event camera, and finally obtaining a set of events without tailing. The method of the present invention suppresses the event tailing effect without deleting any event under any lighting conditions, thereby improving the signal-to-noise ratio of the event data. The present invention also provides a corresponding adaptive representation network construction method and an adaptive representation method, which uses an event adaptive representation network to learn the mapping of tailing events to a new representation method without tailing; the adaptive representation network can be embedded in any downstream task network for fine-tuning or joint training, which can improve the performance of downstream tasks.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and more specifically, relates to a method, system and storage medium for suppressing the tailing effect of an event camera. Background Art

[0002] The event camera is a new type of neuromorphic sensor. Each pixel is triggered asynchronously, and it has the characteristics of high dynamics, low data redundancy and low latency. Although the event camera has the above advantages, as the external light decreases, the noise level of the sensor increases, and the sensor response speed slows down as the cutoff frequency decreases, forming an obvious event tailing phenomenon. The reduction of signal-to-noise ratio in low-light environments seriously affects the application of event cameras in downstream tasks (such as imaging, motion estimation, target detection, semantic segmentation, etc. in high-speed and high-dynamic range scenes).

[0003] In the prior art, the tail of an event is generally regarded as noise, and an event camera noise suppression method is used to suppress the tail. However, the existing event camera noise suppression methods are mostly oriented to background event noise (such as fixed pattern noise, photon shot noise, dark current noise, etc.), which has the characteristics of global stray distribution. The tail event is only generated at the edge of the moving object, which is different from the distribution characteristics of this type of noise. Therefore, using the noise suppression method to process the tail event will cause the valid signal event to be mistakenly deleted, which is not conducive to the improvement of the signal-to-noise ratio. Summary of the invention

[0004] In view of the defects of the prior art and the need for improvement, the present invention provides a method, system and storage medium for suppressing the tailing effect of an event camera, which aims to suppress the event tailing effect while not deleting any events under any lighting conditions, so as to improve the signal-to-noise ratio of event data.

[0005] To achieve the above object, according to a first aspect of the present invention, a method for suppressing the tailing effect of an event camera is provided, comprising:

[0006] S1. Get the event set E of any scene collected by the event camera i (x i ,y i ,t i ,p i ), where i is the sequence number of the event, (x i ,y i ) is the pixel position where the event is triggered, t i is the timestamp of event triggering, p i for the polarity of the event;

[0007] S2, for the current pixel position (x i ,y i), get all the corresponding events; perform tailing judgment on each event in all events in order: if the nth event satisfies both:

[0008] Polarity of the nth event Polarity with the n-1th event Same, n>2;

[0009] The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, Indicates the timestamp when the nth event is triggered;

[0010] The time interval Δt between the nth event and the n-1th event n Less than the set threshold;

[0011] If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponding to the set of no-tailing events;

[0012] S3, taking the next pixel position as the current pixel position, and jumping to S2, until the events corresponding to all pixel positions complete the tailing judgment, and obtaining the event set without tailing corresponding to all pixel positions.

[0013] According to a second aspect of the present invention, a method for constructing an adaptive representation network of an event camera is provided, comprising:

[0014] The event set E i (x i ,y i ,t i ,p i ) After being processed by the tailing suppression method described in the first aspect, a set of events E without tailing is obtained. i ′ (x i ,y i ,t i ,p i );

[0015] The event set E without tail i ′ (x i ,y i ,t i ,p i) to a tail-free voxel V ′ (x,y,t);

[0016] Construct an adaptive representation network for representing event data voxels, and transform the event set E i (x i ,y i ,t i ,p i ) as the input of the adaptive representation network, the tail-free voxel V ′ (x, y, t) is used as a label, and the adaptive representation network is trained end-to-end to obtain a trained adaptive representation network.

[0017] Further, the event set E without tailing i ′ (x i ,y i ,t i ,p i ) to a tail-free voxel V ′ (x,y,t), including:

[0018]

[0019]

[0020] Among them, t i is the timestamp of event triggering, p i is the polarity of the event, N is the total number of events, B is the number of units quantized in the time dimension, k b (a) is the interpolation function.

[0021] Furthermore, the interpolation function k b (a) is: k b (a) = max(0,1-|a|).

[0022] According to a third aspect of the present invention, there is provided an adaptive characterization method for an event camera, comprising:

[0023] The actual event data to be represented is input into the trained adaptive representation network, and the tail-free voxels are output; wherein the adaptive representation network is constructed by the adaptive representation network construction method described in any one of the second aspects.

[0024] According to a fourth aspect of the present invention, there is provided an application network of an event camera, comprising an adaptive representation network and a task network connected in sequence;

[0025] Wherein, the adaptive representation network is constructed by the adaptive representation network construction method described in any one of the second aspects; the task network includes an image reconstruction network, a target detection network, a target recognition network, a semantic segmentation network, a motion estimation network, a motion segmentation network or a simultaneous positioning and mapping network.

[0026] According to a fifth aspect of the present invention, there is provided a system for suppressing the tailing effect of an event camera, comprising:

[0027] The event data acquisition module is used to obtain the event set E of any scene collected by the event camera. i (x i ,y i ,t i ,p i ), where i is the sequence number of the event, (x i ,y i ) is the pixel position where the event is triggered, t i is the timestamp of event triggering, p i for the polarity of the event;

[0028] The first tailing suppression module is used to suppress the current pixel position (x i ,y i ), get all the corresponding events; perform tailing judgment on each event in all events in order: if the nth event satisfies both:

[0029] Polarity of the nth event Polarity with the n-1th event Same, n>2;

[0030] The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, Indicates the timestamp when the nth event is triggered;

[0031] The time interval Δt between the nth event and the n-1th event n Less than the set threshold;

[0032] If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponding to the set of no-tailing events;

[0033] The second tailing suppression module is used to take the next pixel position as the current pixel position and repeatedly execute the first tailing suppression module until the events corresponding to all pixel positions complete the tailing judgment, thereby obtaining a set of events without tailing corresponding to all pixel positions.

[0034] According to a sixth aspect of the present invention, there is provided an electronic device comprising a computer-readable storage medium and a processor;

[0035] The computer-readable storage medium is used to store executable instructions;

[0036] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the tailing suppression method described in the first aspect, or / and the adaptive characterization network construction method described in the second aspect, or / and the adaptive characterization method described in the third aspect.

[0037] According to the seventh aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for suppressing tailing as described in the first aspect, or / and the method for constructing an adaptive characterization network as described in the second aspect, or / and the adaptive characterization method as described in the third aspect is implemented.

[0038] According to an eighth aspect of the present invention, a computer product is provided. When the computer product runs on a computer, the computer executes the tailing suppression method described in the first aspect, or / and the adaptive characterization network construction method described in the second aspect, or / and the adaptive characterization method described in the third aspect.

[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0040] (1) The method for suppressing the tailing effect of the event camera of the present invention does not regard the tailing event as noise, but starts from the essence of the event tailing effect. Event tailing is essentially the delayed triggering of signal events. By traversing the event stream, according to the principle that tailing events are distributed from dense to sparse with the same polarity, the polarity relationship between adjacent events and the relationship between the time intervals of events are used to judge whether the event at the current pixel position is a tailing event. After the tailing event is determined, the timestamp of the tailing event is modified to the sum of the timestamp of the previous adjacent event of the same pixel and the minimum time resolution of the event camera, that is, the timestamp of the tailing event is adjusted forward, thereby realizing tailing suppression instead of directly deleting the tailing event. In this way, it is achieved that the event tailing effect is suppressed without deleting any event under any lighting conditions, thereby improving the signal-to-noise ratio of the event data.

[0041] (2) Furthermore, based on the tail suppression method of the present invention, the time set containing the tail is subjected to tail suppression to obtain a tail-free event set. After the tail-free event set is converted into a voxel grid, the voxel grid at this time also does not have a tail. Therefore, it is used as a label of the adaptive representation network and is formed into a paired data set with the original tail event. The event adaptive representation network is used to learn the mapping from the tail event to the new tail-free representation method, which solves the problem in the prior art that the tail event cannot be handled well and thus the representation that does not contain a tail cannot be learned.

[0042] (3) Furthermore, the application network of the event camera of the present invention embeds the adaptive representation network with event tailing suppression function into the downstream task network for fine-tuning or joint training, which can improve the performance of downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the process of suppressing the tailing effect of an event camera in an embodiment of the present invention.

[0044] FIG. 2( a ) is a schematic diagram of an original event sequence before tailing event correction according to an embodiment of the present invention.

[0045] FIG. 2( b ) is a schematic diagram of an event sequence after tailing event correction in an embodiment of the present invention.

[0046] FIG. 3( a ) is a schematic diagram of real event data before tailing events are suppressed in an embodiment of the present invention.

[0047] FIG3( b ) is a schematic diagram of the suppression of tailing events in real event data according to an embodiment of the present invention.

[0048] Figure 4 Schematic diagram of an adaptive representation network for tailing event suppression in an embodiment of the present invention.

[0049] Figure 5 It is a flowchart of the tailing effect suppression, adaptive characterization and application of the event camera in the embodiment of the present invention.

[0050] Figure 6 This is a schematic diagram of the use of an adaptive representation network in combination with any downstream task in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0052] Example 1

[0053] like Figure 1 As shown, an embodiment of the present invention provides a method for suppressing the tailing effect of an event camera, which mainly includes:

[0054] S1. Input event stream: Get the event set E of any scene collected by the event camera i (x i ,y i ,t i ,p i ), where i represents the sequence number of the event, (x i ,y i ) represents the pixel position where the event is triggered, t i Indicates the timestamp of event triggering, p i is the polarity of the event. In the embodiment of the present invention, p i ∈[1,-1], when the light intensity at the pixel changes from dark to bright, p i =1, otherwise p i =-1.

[0055] S2, for the current pixel position (x i ,y i ), get all corresponding events; if the current pixel position (x i ,y i ) If the number of all events corresponding to is less than or equal to 2, there is no need to perform tailing judgment. Otherwise, perform tailing judgment on each event in all events in sequence: If the nth event satisfies the following three conditions at the same time:

[0056] (1) Polarity of the nth event Polarity with the n-1th event Same, n>2;

[0057] (2) The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, The timestamps corresponding to the triggering of the nth event, the n-1th event, and the n-2th event respectively;

[0058] (3) The time interval Δt between the nth event and the n-1th event n is less than a set threshold value thr; wherein the threshold value thr is selected based on experience;

[0059] If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponds to a set of non-tailing events; in the embodiment of the present invention, the minimum time resolution of the event camera is T=10 -6 , in seconds.

[0060] S3, taking the next pixel position as the current pixel position, and jumping to S2, until the events corresponding to all pixel positions complete the tailing judgment, and obtaining the event set without tailing corresponding to all pixel positions.

[0061] Specifically, in S2, the embodiment of the present invention generates m empty arrays according to the number of pixels m in program implementation; wherein the number of pixels m=M*N, M is x i The maximum value of , that is, the maximum number of pixels of the event camera sensor width, N is y i The maximum value, i.e. the maximum number of pixels of the event camera sensor;

[0062] Traverse the event set E i , each pixel position (x i ,y i ) is stored in the pixel position (x i ,y i ) in the empty array corresponding to ;

[0063] For the current pixel position (x i ,y i ), if the number of events in the array is less than or equal to 2, no tailing judgment is performed; otherwise, a tailing judgment is performed on each event in the array according to the above three conditions. If the current nth event is a tailing event, the timestamp of the current nth event is modified to the sum of the timestamp of the previous adjacent event of the same pixel and the minimum time resolution of the event camera; if the current nth event is not a tailing event, a judgment is performed on whether the next event is a tailing event until the current pixel position (x i ,y i ) is completed, and the current pixel position (x i ,y i ) The event set without tailing corresponding to the next pixel position is then judged, until all pixel positions are judged, and the event set without tailing corresponding to all pixel positions is obtained, thus achieving the correction of tailing events at all pixel positions.

[0064] As shown in Figure 2(a), it is a schematic diagram of the original event sequence before the tailing event correction. After judging by the tailing judgment condition in S2, the schematic diagram of the event sequence after the tailing event correction is shown in Figure 2(b). It can be seen that the delayed triggering effect of the event is correctly corrected. Taking the tailing suppression in the real event data as an example, as shown in Figure 3(a) and Figure 3(b), a, b, c, and d in Figure 3(a) and Figure 3(b) are four areas with tailing effects, 3(a) is the effect before correction, and 3(b) is the effect after correction. The lower area corresponding to Figure 3(a) and Figure 3(b) is a local enlarged view corresponding to the four areas a, b, c, and d. It can be seen that the edges of the objects in the corrected tailing areas are sharper.

[0065] The method for suppressing the tailing effect of the event camera of the present invention takes into account that the signal model of the event camera can be equivalent to a first-order low-pass filter, and the characteristic is that after the event camera receives external light stimulation, the photovoltage in the pixel circuit changes from fast to slow, so the number of events triggered by the event camera changes from dense to sparse, and this effect becomes more obvious as the cutoff frequency decreases. According to the cutoff frequency formula (Among them, C D is the gate-drain capacitance of the transistor in the pixel circuit, U t is the ambient temperature, I ph is the photocurrent), cut-off frequency f 3db and photocurrent I ph is proportional to the light intensity, and the photocurrent is proportional to the light intensity. Therefore, when the external light intensity decreases, the cutoff frequency decreases accordingly, thereby producing an obvious event tailing effect. Based on the above considerations, the present invention does not regard the tailing event as noise, but starts from the essence of the event tailing effect, that is, the event tailing is essentially the delayed triggering of the signal event. By traversing the event stream, according to the principle that the tailing event is distributed from dense to sparse with the same polarity, the polarity relationship between adjacent events and the relationship between the time intervals of the events are used to determine whether the event at the current pixel position is a tailing event. After determining the tailing event, the timestamp of the tailing event is modified to the sum of the timestamp of the previous adjacent event of the same pixel and the minimum time resolution of the event camera, that is, the timestamp of the tailing event is adjusted forward, thereby achieving tailing suppression, rather than deleting the tailing event. In this way, it is achieved that the event tailing effect is suppressed without deleting any event under any lighting conditions, thereby improving the signal-to-noise ratio of the event data.

[0066] Example 2

[0067] Event data is often represented in voxel form before being applied to downstream tasks. Given an event sequence {(x i ,y i ,t i ,p i )} i∈[1,N], the corresponding conversion formula is:

[0068]

[0069]

[0070] Among them, i represents the sequence number of the event, N represents the total number of events, B is the number of units quantized in the time dimension, and k is b (a) = max(0,1-|a|) is the interpolation function, and V(x,y,t) is the voxel grid generated by the event sequence. If the event itself has tailing, there will also be tailing in the converted voxels.

[0071] Based on this, Figure 4 As shown, an embodiment of the present invention provides an adaptive network construction method for an event camera, comprising:

[0072] The event set E i (x i ,y i ,t i ,p i ) After being processed by the tailing suppression method in Example 1, an event set E without tailing is obtained. i ′ (x i ,y i ,t i ,p i );

[0073] The event set E without tail i ′ (x i ,y i ,t i ,p i ) to a tail-free voxel V ′ (x,y,t);

[0074] Construct an adaptive representation network U for representing event data voxels, and transform the event set E i (x i ,y i ,t i ,p i ) as network input, no tailing voxel V ′ (x, y, t) is used as a label, and the adaptive representation network U is trained end-to-end to obtain the trained adaptive representation network U.

[0075] Example 3

[0076] An embodiment of the present invention provides an adaptive characterization method for an event camera, comprising:

[0077] The actual event data to be represented is input into the adaptive representation network U trained in Example 2, and the tail-free voxels are output, thereby realizing the adaptive representation and tail suppression of the event stream.

[0078] The adaptive representation network with event tailing suppression function can be embedded in any downstream task network for fine-tuning.

[0079] The adaptive characterization method of the event camera of the present invention is based on the tail suppression method of the present invention, and suppresses the tail of the time set containing the tail to obtain an event set without tail. After the event set without tail is converted into a voxel grid, the voxel grid at this time also does not have tail. Therefore, it is used as a label of the adaptive characterization network and is composed of a paired data set with the original tail event. The event adaptive characterization network is used to learn the mapping from the tail event to the new characterization method without tail, which solves the problem in the prior art that the tail event cannot be well handled and the characterization without tail cannot be learned.

[0080] Example 4

[0081] like Figure 5 As shown, an embodiment of the present invention provides an application network of an event camera, including a pre-trained adaptive representation network U with tailing suppression capability and a task network, wherein the adaptive representation network U is constructed by the adaptive representation network construction method in the above-mentioned embodiment 2; the task network can be any one of tasks such as image reconstruction, target detection, target recognition, semantic segmentation, motion estimation, motion segmentation, simultaneous positioning and mapping, etc.

[0082] like Figure 6 As shown, in an embodiment of the present invention, taking the reconstruction of an image as an example, in the task of event reconstruction of an image, the event representation part of the front end of the reconstruction network is embedded in the adaptive representation network U for joint training, and finally an image without tailing can be reconstructed.

[0083] The present invention can significantly improve the signal-to-noise ratio of event signals by suppressing the tailing effect of the event camera. The tailing suppression method proposed in the present invention can be applied to tasks such as event-based image reconstruction, target detection, recognition, semantic segmentation, motion estimation, motion segmentation, simultaneous localization and mapping (SLAM), etc. to improve algorithm performance, and has wide reference value.

[0084] The application network of the event camera of the present invention embeds the adaptive representation network with event tailing suppression function into the downstream task network for fine-tuning or joint training, which can improve the performance of downstream tasks.

[0085] Example 5

[0086] The embodiment of the present invention provides a tailing effect suppression system for an event camera, comprising: an event data acquisition module, a first tailing suppression module and a second tailing suppression module;

[0087] The event data acquisition module is used to obtain the event set E of any scene collected by the event camera. i (x i ,y i ,t i ,p i ), where i represents the sequence number of the event, (x i ,y i ) represents the pixel position where the event is triggered, t i Indicates the timestamp of event triggering, p i for the polarity of the event;

[0088] The first tailing suppression module is used to suppress the current pixel position (x i ,y i ), get all the corresponding events; perform tailing judgment on each event in all events in order: if the nth event satisfies both:

[0089] Polarity of the nth event Polarity with the n-1th event Same, n>2;

[0090] The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, Indicates the timestamp when the nth event is triggered;

[0091] The time interval Δt between the nth event and the n-1th event n Less than the set threshold;

[0092] If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponding to the set of no-tailing events;

[0093] The second tailing suppression module is used to take the next pixel position as the current pixel position and repeatedly execute the first tailing suppression module until the events corresponding to all pixel positions complete the tailing judgment, thereby obtaining a set of events without tailing corresponding to all pixel positions.

[0094] Among them, the specific implementation of each of the above modules refers to the specific description of the corresponding steps in Example 1, which will not be repeated here.

[0095] Example 6

[0096] An embodiment of the present invention provides an electronic device, including a computer-readable storage medium and a processor;

[0097] Computer-readable storage media for storing executable instructions;

[0098] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the tailing suppression method of the event camera in the above-mentioned embodiment 1, or / and, the adaptive characterization network construction method of the event camera in embodiment 2, or / and, the adaptive characterization method of the event camera in embodiment 3.

[0099] Example 7

[0100] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for suppressing tailing of an event camera in the above-mentioned embodiment 1, or / and the method for constructing an adaptive representation network of an event camera in embodiment 2, or / and the method for constructing an adaptive representation network of an event camera in embodiment 3 is implemented.

[0101] Example 8

[0102] An embodiment of the present invention provides a computer product. When the computer product runs on a computer, the computer executes the tailing suppression method for the event camera in the above-mentioned embodiment 1, or / and the adaptive characterization network construction method for the event camera in embodiment 2, or / and the adaptive characterization method for the event camera in embodiment 3.

[0103] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for suppressing the tailing effect of an event camera, characterized in that: include: S1. Get the event set E of any scene collected by the event camera i (x i ,y i ,t i ,p i ), where i is the sequence number of the event, (x i ,y i ) is the pixel position where the event is triggered, t i is the timestamp of event triggering, p i for the polarity of the event; S2, for the current pixel position (x i ,y i ), get all the corresponding events; perform tailing judgment on each event in order: if the nth event satisfies both: Polarity of the nth event Polarity with the n-1th event Same, n>2; The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, Indicates the timestamp when the nth event is triggered; The time interval Δt between the nth event and the n-1th event n Less than the set threshold; If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponding to the set of no-tailing events; S3, taking the next pixel position as the current pixel position, and jumping to S2, until the events corresponding to all pixel positions complete the tailing judgment, and obtaining the event set without tailing corresponding to all pixel positions.

2. A method for constructing an adaptive representation network for an event camera, characterized in that: include: The event set E i (x i ,y i ,t i ,p i ) is processed by the tailing effect suppression method according to claim 1 to obtain an event set E without tailing i ′ (x i ,y i ,t i ,p i ); The event set E without tail i ′(x i ,y i , t i , p i ) is converted into a tail-free voxel V′(x, y, t); Construct an adaptive representation network for representing event data voxels, and transform the event set E i (x i ,y i , t i , p i ) is used as the input of the adaptive representation network, the tail-free voxel V′(x, y, t) is used as a label, and the adaptive representation network is trained end-to-end to obtain a trained adaptive representation network.

3. The method for constructing an adaptive representation network according to claim 2, characterized in that: The event set E without tail i ′(x i ,y i , t i , p i ) into a tail-free voxel V′(x, y, t), including: Among them, t i is the timestamp of event triggering, p i is the polarity of the event, N is the total number of events, B is the number of units quantized in the time dimension, k b (a) is the interpolation function.

4. The method for constructing an adaptive representation network according to claim 3, characterized in that: The interpolation function k b (a) is: k b (a) = max(0, 1 - |a|).

5. An adaptive characterization method for an event camera, characterized in that: include: The actual event data to be represented is input into the trained adaptive representation network, and the tail-free voxels are output; wherein the adaptive representation network is constructed by the adaptive representation network construction method according to any one of claims 2 to 4.

6. An application network of event cameras, characterized in that: It includes an adaptive representation network and a task network connected in sequence; Wherein, the adaptive representation network is constructed by the adaptive representation network construction method described in any one of claims 2-4; the task network includes an image reconstruction network, a target detection network, a target recognition network, a semantic segmentation network, a motion estimation network, a motion segmentation network or a simultaneous positioning and mapping network.

7. A tailing effect suppression system for an event camera, characterized in that: include: The event data acquisition module is used to obtain the event set E of any scene collected by the event camera. i (x i ,y i ,t i ,p i ), where i is the sequence number of the event, (x i ,y i ) is the pixel position where the event is triggered, t i is the timestamp of event triggering, p i for the polarity of the event; The first tailing suppression module is used to suppress the current pixel position (x i ,y i ), get all the corresponding events; perform tailing judgment on each event in order: if the nth event satisfies both: Polarity of the nth event Polarity with the n-1th event Same, n>2; The time interval Δt between the nth event and the n-1th event n Greater than the time interval Δt between the n-1th event and the n-2th event n-1 ;in, Indicates the timestamp when the nth event is triggered; The time interval Δt between the nth event and the n-1th event n Less than the set threshold; If the nth event is a tailing event, the timestamp of the nth event is modified to the sum of the timestamp of the n-1th event and the minimum time resolution of the event camera; otherwise, the next event is judged until all events have completed the tailing judgment, and the current pixel position (x i ,y i ) corresponding to the set of no-tailing events; The second tailing suppression module is used to take the next pixel position as the current pixel position and repeatedly execute the first tailing suppression module until the events corresponding to all pixel positions complete the tailing judgment, thereby obtaining a set of events without tailing corresponding to all pixel positions.

8. An electronic device, characterized in that: comprising a computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the tailing effect suppression method described in claim 1, or / and, the adaptive characterization network construction method described in claim 2, or / and, the adaptive characterization method described in claim 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the tailing effect suppression method according to claim 1, or / and, the adaptive characterization network construction method according to claim 2, or / and, the adaptive characterization method according to claim 5.

10. A computer program product, characterized in that It comprises a computer program, which, when running on a computer, enables the computer to execute the tailing effect suppression method according to claim 1, or / and the adaptive characterization network construction method according to claim 2, or / and the adaptive characterization method according to claim 5.

Images