An event data denoising method based on polar triplet attenuation

Abstract

This invention discloses a method for denoising event data based on triple polarity attenuation, belonging to the field of image sensor technology. The method targets background activity noise and flicker noise in event camera output data by constructing and dynamically maintaining three two-dimensional weight maps: positive polarity, negative polarity, and polarity change, to simulate the spatiotemporal distribution characteristics of event data. The steps include: acquiring event data; performing global attenuation on the three weight maps; updating the corresponding polarity weight map locally based on the current event polarity, enhancing the center and attenuating the neighborhood; updating the polarity change weight map by enhancing or attenuating it depending on whether the polarity changes; and finally, performing dual threshold judgment based on the updated weight maps to accurately distinguish between real signals and noise and remove noise. This invention achieves low-latency denoising through event-by-event processing, effectively preserving scene details, significantly improving the quality of event data, and is suitable for high dynamic range visual tasks.

Description

Technical Field

[0001] This invention relates to the field of image sensor technology, specifically to an event data denoising method based on polarity triple attenuation, and more particularly to an event data denoising method that dynamically maintains a neighborhood weight graph by event polarity and achieves noise suppression based on a weight threshold. Background Technology

[0002] The human retina possesses the ability to simultaneously perceive dynamic changes and static information such as color, detail, and texture. To mimic the working mode of the human retina, neuromorphic vision sensors (also known as event cameras, brain-like vision sensors, or silicon retina) have been developed. Neuromorphic vision sensors capture logarithmic relative changes in light intensity. When the light intensity change exceeds a positive or negative change threshold, they output event data with positive or negative polarity (1 or -1, representing the direction of the light intensity change). The event data is presented in tuples. Expressed in the form of, where, Indicates the event is in The coordinates of the axis, Indicates the polarity of the event. This indicates the event trigger time. This approach achieves high temporal resolution, high dynamic range, and low power consumption for sensing dynamic changes.

[0003] However, event cameras are susceptible to various noise sources, leading to performance degradation. BA noise (background activity noise) refers to random events spontaneously generated by event sensor pixels in the absence of actual lighting changes or motion. These events are irregularly distributed in time and space, with random positive and negative polarities. Besides inherent noise from circuit manufacturing processes, event cameras are also susceptible to optical interference, such as flicker noise from artificial light sources. This type of flicker, caused by rapid lighting changes, generates a large number of events, severely contaminating the output signal. Therefore, effectively suppressing noise is crucial for improving the reliability of event-based vision systems. Summary of the Invention In view of the above problems, the present invention provides an event data denoising method based on polarity triple attenuation, which improves the denoising accuracy of event data and solves the problem of severe stroboscopic noise contaminating event data.

[0004] This invention provides a method for denoising event data based on polarity triple attenuation, comprising the following steps: Acquire multiple event data points, each of which must include at least coordinates, polarity, and event trigger time; Construct two-dimensional positive polarity weight maps, two-dimensional negative polarity weight maps, and two-dimensional polarity change weight maps that match the resolution of the event data; In response to the input of event data, the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map, and the two-dimensional polarity change weight map are updated with global weight decay. Based on the polarity of the current event data, the corresponding polarity weight graph is updated with local time decay. Based on the comparison between the polarity of the current event data and the polarity of the event data input at the current event coordinate, the polarity change weight map is updated by polarity change decay. Based on the updated positive polarity weight map, negative polarity weight map, and polarity change weight map, the current event data is determined to be either a real signal or noise, and event data identified as noise is removed.

[0005] Optionally, the global weight decay update specifically involves: if the event trigger time of the current event data is greater than the last global decay time than a preset global decay interval, then the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map, and the two-dimensional polarity change weight map are multiplied by their respective global decay coefficients, and the last global decay time is updated to the event trigger time of the current event data.

[0006] Optionally, the local time decay update specifically includes: for the center coordinate of the current event data, if its corresponding polarity weight value is zero, it is initialized to a fixed value; if it is not zero, the polarity weight of the center coordinate is enhanced; and the same polarity weight in the preset eccentric neighborhood of the center coordinate is multiplied by the local time decay eccentric neighborhood decay coefficient.

[0007] Optionally, the polarity change decay update specifically includes: if the polarity of the current event data is the same as the polarity of the previously input event data, then the polarity change weight of the center coordinate is multiplied by the polarity change decay center coordinate decay coefficient; if the polarities are different, then the polarity change weight of the center coordinate is exponentially amplified.

[0008] Optionally, the enhancement coefficient used in the exponential amplification is dynamically determined based on the average polarity change weight and the number of active neighbors within the preset decentered neighborhood of the center coordinates.

[0009] Optionally, determining whether the current event data is a real signal or noise specifically includes: Calculate the sum of polarity weights within the preset dedentate neighborhood of the center coordinates of the current event data, and use it as the first statistical value; The sum of the polarity change weights within the centered neighborhood is calculated as the second statistical value; If the first statistical value is greater than or equal to the preset polarity weight threshold, and the second statistical value is less than the preset polarity change weight threshold, then the current event data is determined to be a real signal; otherwise, it is determined to be noise.

[0010] Optionally, the method further includes: after each weight update, if the updated weight value is less than a preset weight reset threshold, then the weight value is set to zero.

[0011] Optionally, the preset dedentation neighborhood involved in the local time decay update and polarity change decay update is a 3×3 dedentation neighborhood.

[0012] Optionally, the global decay interval is positively correlated with the global decay coefficient to control the duration of the influence of historical event data on the weight graph.

[0013] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The weight graph of the present invention is dynamically updated based on event information and multiple spatiotemporal update mechanisms. At the time level, the weight graph changes with the global decay of each input event data; at the spatial level, the weights are updated based on event coordinates, local time decay and polarity change decay; thus realizing the modeling and analysis of the spatiotemporal distribution of event polarity.

[0014] (2) The weighted graph and dual-threshold denoising judgment of the present invention flexibly meet the usage requirements of different denoising scenarios. The polarity change weighted graph and the corresponding threshold scheme are suitable for removing flicker noise caused by rapid changes in illumination. The polarity weighted graph and its corresponding threshold scheme are suitable for removing BA noise, etc., where the polarity spatiotemporal distribution differs significantly from the neighborhood.

[0015] (3) The event data denoising method of the present invention breaks through the existing denoising method based on event accumulation frame image or time window segmentation processing. It adopts the method of inputting events one by one and dynamically updating the weight map, which can perform denoising while the processor receives events, further reducing the amount of data and speeding up the processing speed of event data. Attached Figure Description

[0016] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0017] Figure 1 This is the initialization method for local time decay in the event data denoising method based on polarity triple decay of the present invention.

[0018] Figure 2 This invention relates to a polarity weighted global attenuation method for event data denoising based on polarity triple attenuation.

[0019] Figure 3 This invention relates to a method for enhancing the center coordinates of local time decay in an event data denoising method based on polarity triple decay.

[0020] Figure 4This invention relates to a local temporal attenuation method for denoising event data based on polarity triple attenuation, specifically a de-centered neighborhood attenuation method.

[0021] Figure 5 This is the initialization method for polarity change attenuation in the event data denoising method based on polarity triple attenuation of the present invention.

[0022] Figure 6 This invention relates to a polarity change weight global attenuation method for event data denoising based on polarity triple attenuation.

[0023] Figure 7 This invention relates to a method for enhancing the center coordinate of polarity change attenuation in an event data denoising method based on polarity triple attenuation.

[0024] Figure 8 This invention relates to a polarity change attenuation center coordinate attenuation method in the event data denoising method based on polarity triple attenuation.

[0025] Figure 9 This is an event frame image before denoising, based on the event data denoising method of polarity triple attenuation of the present invention.

[0026] Figure 10 This is a segment of an event frame image after data denoising based on the event data denoising method of the present invention, which is based on polarity triple attenuation.

[0027] Figure 11 This is a segment of event frame image after denoising using the EDformer: Transformer-Based Event Denoising Across Varied NoiseLevels denoising method.

[0028] Figure 12 This is a segment of event frame image after denoising, based on the Polarity-Focused Denoising for Event Cameras method. Detailed Implementation

[0029] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0030] A specific embodiment of the present invention, such as Figure 1-12 As shown, a method for denoising event data based on polarity triple attenuation is disclosed, and the specific steps are as follows: Step 1. Obtain multiple event data; Step 2. The weighted graph network constructs two-dimensional positive polarity weight graphs, two-dimensional negative polarity weight graphs, and two-dimensional polarity change weight graphs of the same size based on the resolution of each event data.

[0031] Each pixel coordinate of an event corresponds to a weight value on the positive polarity weight map, the negative polarity weight map, and the polarity change weight map.

[0032] Step 3. See Figure 2 , Figure 6 The weights of the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map, and the two-dimensional polarity change weight map for each event are updated globally by weight decay.

[0033] Specifically, if the time between the current event's trigger time and the last decay time is greater than a specified interval, the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map, and the two-dimensional polarity change weight map are multiplied by the corresponding decay coefficients to suppress the long-term impact of event data on the weight maps. Subsequently, the recorded decay time is updated to the current event trigger time. The expression for global weight decay is:

[0034] in, Indicates the event trigger time of the current event; This indicates the time of the last global decay event; Indicates the global decay interval; This represents the positive polarity weight map before the update. This represents the negative polarity weight map before the update. This represents the weighted graph showing the polarity changes before the update. This represents the updated positive polarity weighting graph; This represents the updated negative polarity weight map; This represents the updated polarity change weighting graph; This represents the global decay coefficient of the polarity weighting graph; This represents the global attenuation coefficient of the polarity change weighting graph.

[0035] Furthermore, the longer the decay time interval, the higher the decay coefficient, and the longer the time span of the event data's influence on the weight graph.

[0036] Understandably, if the time between the current event's trigger time and the last decay time is less than or equal to the specified interval, the step will not be triggered, and subsequent steps will proceed normally.

[0037] Step 4. See Figure 1 , Figure 3-4 The corresponding polarity weight graph is updated locally with time decay based on the polarity of the current event.

[0038] Specifically, for the center coordinates of the input event If the polarity weight value corresponding to the center coordinate of the event is 0, it is initialized to a fixed value; if the polarity weight value is not 0, the polarity weight of the center coordinate of the input event is enhanced; then a 3×3 centered neighborhood decay is performed to obtain the updated polarity weight map.

[0039] Furthermore, for the same polarity weights corresponding to the 3×3 eccentric neighborhood coordinates of this event, multiplied by the local time decay eccentric neighborhood decay coefficient, the expression for local time decay is:

[0040]

[0041]

[0042] in, Represents the center coordinates of the input event The 3×3 centered neighborhood; This represents the i-th coordinate within the 3×3 denoted neighborhood of the input event; Indicates the polarity of the input event; This represents the negative polarity weight map before the update. This represents the updated negative polarity weight map; This represents the positive polarity weight map before the update. This represents the updated positive polarity weighting graph; This represents the decay coefficient of the local time decayed center-neighbor area; This indicates taking the minimum value; This represents the enhancement factor of the center coordinates of the local time decay input event. This indicates that the weights are given initial values.

[0043] Step 5. See Figure 5 , Figure 7-8 The current coordinate of the polarity of the input event data is compared with the polarity of the previous input event at the same coordinate. Based on the polarity judgment result, the polarity change weight map is attenuated by polarity change.

[0044] Specifically, first, the mean of polarity change weights within the 3×3 centered neighborhood of the input event is calculated as the normalized polarity change activity within the centered neighborhood. The number of neighbors whose polarity change weights are greater than the neighborhood activity judgment threshold is calculated to improve the detection sensitivity of legitimate polarity changes in motion scenes.

[0045] If there is no previous event input for the event coordinates, the polarity of the event will be recorded as the polarity to be used for the next comparison of the current coordinates, and the weight of polarity change will not be updated.

[0046] If the polarity remains unchanged (based on the comparison between the polarity of the input event data and the polarity of the previous input event at the current coordinate, as mentioned in step five), then center coordinate attenuation is performed, multiplying the polarity change weight of the center coordinate of the event by the polarity change weight center coordinate attenuation coefficient; if the polarity changes, the polarity change attenuation center coordinate enhancement coefficient is obtained based on the normalized polarity change activity and the number of active neighbors within the 3×3 dedentate neighborhood, and the polarity change weight is exponentially amplified to quickly accumulate evidence of high-frequency oscillations. The polarity change attenuation expression is:

[0047]

[0048]

[0049]

[0050] in, This indicates the activity of normalized polarity changes within the decentered neighborhood; This indicates addition and summation; This indicates the number of active neighbors within the de-centric neighborhood; This indicates the threshold for determining neighborhood activity. Indicates the attenuation coefficient at the coordinates of the attenuation center due to polarity change; Indicates the enhancement coefficient of the coordinates of the attenuation center due to polarity change; This represents the weighted graph showing the polarity changes before the update. This represents the updated polarity change weighting graph; Preferably, after each update of a weight in the positive / negative polarity weight map or the polarity change weight map, if the weight is less than the weight reset threshold, it is set to zero, and the expression is as follows:

[0051] in, This represents the positive and negative polarity weights or polarity change weights after any operation update. The threshold for resetting the representation right is zero.

[0052] Step 6. Dual-threshold noise determination: After a single event is input and updated, the event is determined to be a real signal or noise based on the updated positive and negative polarity weight maps and polarity change weight maps. The noise is removed and the real signal is retained.

[0053] Finally, two threshold judgments are performed on the corresponding polarity weight map and polarity change weight map for this event. For the polarity weight map, the weights of its 3×3 centered neighborhood are summed; for the polarity change weight map, the weights of its 3×3 centered neighborhood are summed. Events whose summation statistical value of the polarity weight neighborhood is greater than the polarity weight threshold but less than the polarity change weight threshold are considered true signals, expressed as:

[0054]

[0055]

[0056] in, This represents the summation statistics of the polarity weight neighborhood; This represents the summation statistics of the neighborhood of the polarity change weights; This indicates addition and summation; This indicates that the event has been determined. This is a real signal; Indicates the polarity weight threshold; This represents the threshold for polarity change weighting.

[0057] The denoised results are compared with existing methods EDformer and Polarity-Focused Denoising for Event Camera, for example. Figure 9-12 The image shows a segment of event data recorded from a stadium stands. Event frames are generated based on the presence or absence of an event at each pixel coordinate; a black pixel indicates an event, while a white pixel indicates no event. All three denoising methods effectively remove minor noise from the image. However, compared to the other two methods, this invention better preserves image details. The EDformer method removes a significant amount of the actual event signal and fails to retain the mesh structure of the stadium stands well. The Polarity-Focused Denoising for Event Camera method also removes some of the mesh structure and fails to preserve the two layers of railings on the left side of the stands well.

[0058] While the specific embodiments of the present invention depict actions or steps in a particular order, this should be understood as requiring such actions or steps to be performed in the shown specific order or sequential order, or requiring all illustrated actions or steps to be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations. The above descriptions are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0059] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for denoising event data based on polarity triple attenuation, characterized in that, Includes the following steps: Acquire multiple event data points, each of which must include at least coordinates, polarity, and event trigger time; Construct two-dimensional positive polarity weight maps, two-dimensional negative polarity weight maps, and two-dimensional polarity change weight maps that match the resolution of the event data; In response to the input of event data, the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map, and the two-dimensional polarity change weight map are updated with global weight decay. Based on the polarity of the current event data, the corresponding polarity weight graph is updated with local time decay. Based on the comparison between the polarity of the current event data and the polarity of the event data input at the current event coordinate, the polarity change weight map is updated by polarity change decay. Based on the updated positive polarity weight map, negative polarity weight map, and polarity change weight map, the current event data is determined to be either a real signal or noise, and event data identified as noise is removed.

2. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The specific global weight decay update is as follows: if the event trigger time of the current event data is greater than the previous global decay time than the preset global decay interval, then the two-dimensional positive polarity weight map, the two-dimensional negative polarity weight map and the two-dimensional polarity change weight map are multiplied by the corresponding global decay coefficients, and the previous global decay time is updated to the event trigger time of the current event data.

3. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The local time decay update specifically includes: for the center coordinate of the current event data, if its corresponding polarity weight value is zero, it is initialized to a fixed value; if it is not zero, the polarity weight of the center coordinate is enhanced; and the same polarity weight in the preset eccentric neighborhood of the center coordinate is multiplied by the local time decay eccentric neighborhood decay coefficient.

4. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The polarity change decay update specifically includes: if the polarity of the current event data is the same as the polarity of the previously input event data, then the polarity change weight of the center coordinate is multiplied by the polarity change decay center coordinate decay coefficient; if the polarities are different, then the polarity change weight of the center coordinate is exponentially amplified.

5. The event data denoising method based on polarity triple attenuation according to claim 4, characterized in that, The enhancement coefficient used in the exponential amplification is dynamically determined based on the average polarity change weight and the number of active neighbors within the preset decentered neighborhood of the center coordinates.

6. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The determination of whether the current event data is a real signal or noise specifically includes: Calculate the sum of polarity weights within the preset dedentate neighborhood of the center coordinates of the current event data, and use it as the first statistical value; The sum of the polarity change weights within the centered neighborhood is calculated as the second statistical value; If the first statistical value is greater than or equal to the preset polarity weight threshold, and the second statistical value is less than the preset polarity change weight threshold, then the current event data is determined to be a real signal; otherwise, it is determined to be noise.

7. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The method further includes: after each weight update, if the updated weight value is less than a preset weight reset threshold, then the weight value is set to zero.

8. The event data denoising method based on polarity triple attenuation according to claim 1, characterized in that, The preset dedentation neighborhood involved in the local time decay update and polarity change decay update is a 3×3 dedentation neighborhood.

9. The event data denoising method based on polarity triple attenuation according to claim 2, characterized in that, The global decay interval is positively correlated with the global decay coefficient to control the duration of the influence of historical event data on the weight graph.

Images