Label-independent and non-monotonic event camera denoising evaluation method

Abstract

The invention relates to a label-independent and non-monotonous event camera denoising evaluation method, which belongs to the technical field of brain-like perception and computing, and comprises the following steps of: acquiring a continuous comparison curve area, taking the comparison curve area as an event camera denoising evaluation index, and performing event camera denoising evaluation to obtain an optimal event camera denoising method. According to the evaluation index, the evaluation process of the event camera denoising task is remarkably enhanced, the optimal parameter of the denoising method can be selected, and the denoising solution is more efficient and effective due to the development of the evaluation index; according to the method, the performance of the event camera denoising method can be objectively evaluated, and the highest evaluation is provided for a method which retains a large number of effective events and filters out the most noise at the same time; according to the method, evaluation does not need to be carried out on a data set with labels added manually, evaluation of the noise removal effect can be carried out on any processed event sequence, and the applicability of noise evaluation is greatly enhanced; moreover, the method is simple in calculation and convenient and direct to use, and improves the working efficiency.

Description

Technical Field

[0001] The present invention relates to the field of brain-like perception and computing technology, and in particular to a label-independent, non-monotonic event camera denoising evaluation method. Background Art

[0002] Event cameras are a new type of biomimetic visual sensor. Unlike traditional cameras that capture absolute light intensity values ​​through global exposure, event cameras encode relative changes in light intensity and produce sparse, asynchronous event streams. Therefore, event cameras have excellent performance in various challenging application scenarios, such as target detection, pose estimation, motion analysis, and SLAM.

[0003] The event camera simulates the imaging mechanism of the human eye by implementing the three-level abstraction of the photoreceptor layer, bipolar cell layer, and ganglion cell layer in the retinal structure. They independently encode positive relative increases and negative relative decreases in light intensity and output events representing these changes.

[0004] However, previous event cameras were merely simplified hardware abstractions of the retina, which, by contrast, can capture richly detailed textures, perceive color, and acutely detect motion.

[0005] To more closely mimic biological visual imaging mechanisms, many solutions embed RGB pixels into event cameras. This has led to sensors such as the Asynchronous Time-Based Image Sensor (ATIS) and the Dynamic and Active Pixel Vision Sensor (DAVIS), which can output both events and frames. The latest technology trends involve the use of back-illuminated (BI) and stacking techniques. Distributed switch imaging chips now include both high-resolution RGB pixels and event-generating pixels with event pixel resolutions up to 1 million. In addition, the latest solutions provide four event-generating pixel channels for red, green, blue, and white light events.

[0006] However, the increase in the number of pixels and the complexity of technology make the pixel circuit more susceptible to noise.

[0007] The noise in event cameras is mainly BA noise. BA noise refers to the noise caused by the current generated by the pixel when the illumination remains unchanged. This noise is related to the ambient temperature and the manufacturing process of the pixel circuit. Typical influencing factors include leakage current that causes charge accumulation (exceeding the threshold and falsely triggering the event), threshold drift of the triggering event, process defects that generate noise, random photon fluctuations, etc.

[0008] Although many new techniques have been adopted to reduce the generation of BA noise, it still exists in large quantities, especially in low-light conditions. Therefore, denoising of event cameras has become a fundamental and critical problem that needs to be solved.

[0009] Current denoising methods for event cameras usually rely on the spatiotemporal correlation of events, event density, motion consistency of events, and learning-based methods. Most denoising methods are based on spatiotemporal-temporal correlation. The double window filter (DWF) cleverly establishes the spatiotemporal correlation of events using two first-in, first-out queues to effectively denoise while retaining valid events. The quantized multi-layer perceptron filter (QMLPF) is a learning-based denoising method that treats denoising as a binary classification task. The QMLPF method calculates a predicted value for each event and classifies it as noise or valid signal. Some denoising methods no longer treat denoising as a binary classification task, such as the learning-based EventZoom and NeuroZoom. Instead, they perform a projection operation on the events and combine denoising with downstream tasks, treating it as a subtask of the overall process. This denoising method prevents the direct distinction between noise and valid events, even if the event stream is pre-labeled. Summary of the invention

[0010] In view of the above problems, the present invention provides a label-independent, non-monotonic event camera denoising evaluation method, obtains the area of ​​continuous contrast curve (AOCC), takes the contrast curve area as the event camera denoising evaluation index, evaluates the event camera denoising, and obtains the optimal event camera denoising method. The evaluation index significantly enhances the evaluation process of the event camera denoising task and helps to select the optimal parameters of the denoising method. The development of the evaluation index makes the denoising solution more efficient and effective.

[0011] The present invention provides a label-independent, non-monotonic event camera denoising evaluation method, comprising:

[0012] S1, defining a brightness signal of an event, and establishing an event generation model based on the brightness signal;

[0013] S2, obtaining an event sequence; setting a time interval of the event sequence; and obtaining multiple event streams of the event sequence based on the time interval;

[0014] Based on the event generation model, the events triggered in each event stream are acquired and accumulated, and a corresponding event framework is established to obtain multiple event frameworks;

[0015] S3, obtaining pixels corresponding to each event in each event frame; calculating pixel gradients corresponding to each event; and obtaining corresponding event frame contrast based on the pixel gradients corresponding to each event;

[0016] S4, obtaining the contrast of each event frame and taking the average value to obtain the average event frame contrast; the event frame is a binary event frame;

[0017] S5, defining the average event frame contrast as an original continuous time contrast curve; obtaining the area of ​​the original continuous time contrast curve;

[0018] S6. defining a time interval array; the time interval array includes multiple groups of time intervals;

[0019] The event stream is segmented repeatedly for multiple times according to the time interval array, the event frames corresponding to the event streams of each segment are obtained, and the contrast of the events in each event frame is calculated to obtain the event contrast of the event streams of each segment;

[0020] Rewriting the original continuous time contrast curve area using the event contrast of each segmented event stream to obtain a continuous time contrast curve area;

[0021] S7, using the continuous time contrast curve area as a label-independent, non-monotonic event camera denoising evaluation indicator;

[0022] The continuous time contrast curve area is used to evaluate event camera denoising.

[0023] Preferably, the method further comprises step S8, evaluating event camera denoising by using the continuous time contrast curve area to obtain an optimal event camera denoising method.

[0024] Preferably, the expression of the brightness signal of the event in step S1 is:

[0025]

[0026] Among them, L(·) is the brightness signal function, I(·) is the light intensity function, and u i is the coordinate of the i-th pixel, u i =(x i ,y i ), x i is the x-axis coordinate of the ith pixel, y i is the y-axis coordinate of the i-th pixel, t i is the timestamp corresponding to the i-th pixel.

[0027] Preferably, the event generation model is expressed as:

[0028]

[0029] Among them, ΔL(·) is the brightness change function, C represents the threshold value of the event being triggered, and p i is the event polarity corresponding to the i-th pixel, and Δt′ represents the time interval between pixels.

[0030] It can be understood that the event camera simulates the imaging mechanism of the human eye by implementing the three-level abstraction of the photosensitive layer, bipolar cell layer, and ganglion cell layer in the retinal structure. They independently encode the positive relative increase and negative relative decrease of light intensity and output events representing these changes.

[0031] Exemplarily, the event flow of the event sequence in step S2 is: for an event sequence E with a duration of T, ΔT is selected as the time interval, then the event flow is T / ΔT segments, and each segment of the event flow includes a plurality of pixel points.

[0032] Preferably, the expression of the event framework in step S2 is:

[0033]

[0034] I t =∑I(u i ) t =∑I(x k ,y k ) t , among which, I(u i ) t =1 (4)

[0035] Among them, 1(·) is an indicator function, which takes the value 1 when the condition is met, otherwise it takes the value 0. i ) t is the coordinate of the i-th pixel in the event stream at the t-th interval. i ) t =1, it means that the pixel triggers an event, which is marked as an event. The pixel value 1 of the event stream at the tth interval is mapped to the corresponding image channel intensity 255, which is characterized as an event. The event includes positive (ON) events and negative (OFF) events. The positive (ON) event indicates that the light intensity increases beyond the threshold, and the negative (OFF) event indicates that the light intensity decreases. When the brightness change ΔL reaches the threshold, an event will be triggered at the corresponding pixel. k is the x-axis coordinate of the k-th event, y k is the y-axis coordinate of the k-th event; I t is the event frame of the tth interval time, e is the event stream, E is the event sequence, t0 represents the initial time in the tth interval time, t1 represents the end time in the tth interval time, t=0,1,…T / ΔT, T / ΔT represents the number of interval times within the duration T, ΔT represents the time interval for dividing the event stream into event sequences, within this time interval, the event frame corresponding to each event stream is accumulating events, and T represents the duration of the event sequence.

[0036] Preferably, the contrast expression of the event frame in step S3 is:

[0037]

[0038] Where C(t) is the contrast of the event frame at the tth interval, (G k ) t is the gradient of the pixel corresponding to the kth event at the tth interval, k = 1, 2, 3…K, K is the total number of events.

[0039] Furthermore, the pixel gradient expression is:

[0040]

[0041] in, is the Sobel operator in the vertical direction of the event frame at the t-th interval time, is the Sobel operator in the horizontal direction of the event frame at the tth interval time; (G k ) t is the gradient of the pixel corresponding to the kth event at the tth interval.

[0042] In the present invention, events are more likely to be triggered at edge contours where brightness changes significantly, rather than within a scene or object, resulting in a lack of fine texture information in the generated data. In addition, due to the sparseness of event triggering, events representing edges may not form continuous lines. Suppose that an ultra-high-speed camera and an event camera with the same field of view work simultaneously in the same short time window. In this case, the RGB camera will clearly capture the outline of the scene and more texture details, while the event camera will mainly record high brightness changes along the edges of the scene.

[0043] Preferably, the event frame is a binary event frame; it can be understood that the event frame is represented as a binary image. If each pixel in the event frame triggers an event, the pixel value of the pixel that triggers the event is set to 255, and the pixel value of the pixel that does not contain the event is set to 0, thereby obtaining a binary event frame.

[0044] Preferably, the contrast of the event frame is a function related to the time interval (t, t+ΔT);

[0045] Furthermore, the expression of the time interval correlation function is:

[0046]

[0047] Wherein, ΔT is the time interval for dividing the event sequence into event streams. Within this time interval, events are accumulated in each event stream to form an event frame; C(t) is the contrast of the event frame at the t-th interval; if the selected time interval ΔT is large enough, then the time interval contains multiple events and multiple pixels, and the events and pixels will be evenly present. The contrast C(t) of the event frame at the t-th interval tends to 0, resulting in a lack of contrast; if the selected time interval ΔT is too small, since the events are sparsely distributed in the entire event sequence, the event will not exist in the time interval. The contrast C(t) of the event frame at the t-th interval also tends to 0, resulting in the minimum contrast; optionalΔT is the optimal time interval, and the corresponding contrast C(t) of the event frame at the t-th interval is the largest.

[0048] Preferably, the average event frame contrast expression in step S5 is:

[0049]

[0050] Among them, C(t k ) is the contrast of the kth event in the event frame of the tth interval, C avg (t) is the average event contrast of the event frame at the tth interval.

[0051] Preferably, the expression of the original continuous time contrast curve area (AOCC) in step S5 is:

[0052]

[0053] Among them, τ is the duration of the event stream, and the duration of the event stream is the sum of the intervals of multiple pixels in the event stream, 0≤t≤τ, A′ C is the original continuous time contrast curve area, C avg (t) is the average event contrast of the event frame at the tth time interval.

[0054] Preferably, the event contrast expression of each segmented event stream in step S6 is:

[0055]

[0056] Among them, C k,p is the contrast of the kth event in the event frame corresponding to the pth segmented event stream, that is, the event stream includes multiple pixels, each of which either triggers an event or does not trigger an event; Δt p is the interval time between events in the event frame corresponding to the event stream of the pth segment; τ p The duration of the event stream of the pth segment; kp is the kth event in the event frame corresponding to the event stream of the pth segment, k p =1,2,3…K p , K p The total number of events in the event frame corresponding to the event stream of the p-th segment.

[0057] Preferably, the expression of the continuous time comparison curve area in step S6 is:

[0058]

[0059] Among them, A C is the area of ​​the continuous time versus curve.

[0060] Specifically, the evaluation of the event camera denoising method in step S8 specifically includes:

[0061] The continuous time contrast curve area is used to evaluate the best overall performance provided by various event camera denoising methods in multiple time intervals, and the optimal event camera denoising method is obtained.

[0062] The optimal event camera denoising method is a method that retains a large number of valid events while filtering out the most noise.

[0063] The present invention designs the continuous time contrast curve area A C The metric retains all events that outline the edges of a scene or object, ensuring that the highest contrast value is obtained within any accumulation time interval of event frames.

[0064] Compared with the prior art, the present invention has at least the following beneficial effects:

[0065] (1) The event camera denoising evaluation method of the present invention is non-monotonic, so that the present invention can objectively evaluate the performance of the event camera denoising method, and give the highest evaluation to the method that retains a large number of valid events and filters out the most noise; and give the lowest evaluation to the method that removes a large number of valid events and noise at the same time, and the method that retains a large number of valid events and retains a large amount of noise;

[0066] (2) The event camera denoising evaluation method of the present invention has the characteristic of being independent of labels and does not need to be evaluated on a dataset with artificial labels. The noise removal effect can be evaluated for any processed event sequence, which greatly enhances the applicability of noise evaluation.

[0067] (3) The event camera denoising evaluation method of the present invention is simple in calculation, convenient and direct to use, and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0069] Figure 1 (a)-(d) are schematic diagrams of images captured by an active pixel sensor (APS) of an event camera in an embodiment of the present invention and corresponding grayscale images, binary frames, and event frames;

[0070] Figure 2 Schematic diagram of an example of a continuous time comparison curve CCC using the QMLPF method in an embodiment of the present invention;

[0071] FIG3( a ) is a schematic diagram of a curve when the noise level is 3 Hz / pixel in an embodiment of the present invention;

[0072] FIG3( b ) is a schematic diagram of a curve when the noise level is 1 Hz / pixel in an embodiment of the present invention;

[0073] FIG3( c ) is a schematic diagram of a curve when the noise level is 5 Hz / pixel in an embodiment of the present invention;

[0074] FIG4( a ) is a schematic diagram of a continuous time area contrast curve AOCC at a noise level of 1 Hz / pixel in an embodiment of the present invention;

[0075] FIG4( b ) is a schematic diagram of a continuous time area contrast curve AOCC at a noise level of 3 Hz / pixel in an embodiment of the present invention;

[0076] FIG4( c ) is a schematic diagram of a continuous time area contrast curve AOCC at a noise level of 5 Hz / pixel in an embodiment of the present invention;

[0077] FIG4( d ) is a schematic diagram of a baseline measurement with a noise level of 1 Hz / pixel according to an embodiment of the present invention;

[0078] FIG4( e ) is a schematic diagram of a baseline measurement with a noise level of 3 Hz / pixel according to an embodiment of the present invention;

[0079] FIG4( f ) is a schematic diagram of a baseline measurement with a noise level of 5 Hz / pixel according to an embodiment of the present invention;

[0080] Figure 5 Schematic diagram of the receiver operating characteristic (ROC) and area under the curve (AUC) of the QMLPF method under different noise levels in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0082] A specific embodiment of the present invention, as Figure 1-5 , discloses a label-independent, non-monotonic event camera denoising evaluation method. In order to illustrate the effectiveness of the method proposed in the present invention, the above technical solution of the present invention is described in detail through a specific embodiment below. The specific implementation steps are as follows:

[0083] The present invention provides a label-independent, non-monotonic event camera denoising evaluation method, comprising:

[0084] S1, defining a brightness signal of an event, and establishing an event generation model based on the brightness signal;

[0085] S2, obtaining an event sequence; setting a time interval of the event sequence; and obtaining multiple event streams of the event sequence based on the time interval;

[0086] Based on the event generation model, the events triggered in each event stream are acquired and accumulated, and a corresponding event framework is established to obtain multiple event frameworks;

[0087] S3, obtaining pixels corresponding to each event in each event frame; calculating pixel gradients corresponding to each event; and obtaining corresponding event frame contrast based on the pixel gradients corresponding to each event;

[0088] S4, obtaining the contrast of each event frame and taking the average value to obtain the average event frame contrast; the event frame is a binary event frame;

[0089] S5, defining the average event frame contrast as an original continuous time contrast curve; obtaining the area of ​​the original continuous time contrast curve;

[0090] S6. defining a time interval array; the time interval array includes multiple groups of time intervals;

[0091] The event stream is segmented repeatedly for multiple times according to the time interval array, the event frames corresponding to the event streams of each segment are obtained, and the contrast of the events in each event frame is calculated to obtain the event contrast of the event streams of each segment;

[0092] Rewriting the original continuous time contrast curve area using the event contrast of each segmented event stream to obtain a continuous time contrast curve area;

[0093] S7, using the continuous time contrast curve area as a label-independent, non-monotonic event camera denoising evaluation indicator;

[0094] The continuous time contrast curve area is used to evaluate event camera denoising.

[0095] Preferably, the method further comprises step S8, evaluating event camera denoising by using the continuous time contrast curve area to obtain an optimal event camera denoising method.

[0096] Preferably, the expression of the brightness signal of the event in step S1 is:

[0097]

[0098] Among them, L(·) is the brightness signal function, I(·) is the light intensity function, and u i is the coordinate of the i-th pixel, u i =(x i ,y i ), x i is the x-axis coordinate of the ith pixel, y i is the y-axis coordinate of the i-th pixel, t i is the timestamp corresponding to the i-th pixel.

[0099] Preferably, the event generation model is expressed as:

[0100]

[0101] Among them, ΔL(·) is the brightness change function, C represents the threshold value of the event being triggered, and p i is the event polarity corresponding to the i-th pixel, and Δt′ represents the time interval between pixels.

[0102] It can be understood that the event camera simulates the imaging mechanism of the human eye by implementing the three-level abstraction of the photosensitive layer, bipolar cell layer, and ganglion cell layer in the retinal structure. They independently encode the positive relative increase and negative relative decrease of light intensity and output events representing these changes.

[0103] Exemplarily, the event flow of the event sequence in step S2 is: for an event sequence E with a duration of T, ΔT is selected as the time interval, then the event flow is T / ΔT segments, and each segment of the event flow includes a plurality of pixel points.

[0104] Preferably, the expression of the event framework in step S2 is:

[0105]

[0106] It =∑I(u i ) t =∑I(x k ,y k ) t , among which, I(u i ) t =1 (4)

[0107] Among them, 1(·) is an indicator function, which takes the value 1 when the condition is met, otherwise it takes the value 0. i ) t is the coordinate of the i-th pixel in the event stream at the t-th interval. i ) t =1, it means that the pixel triggers an event, which is marked as an event. The pixel value 1 of the event stream at the tth interval is mapped to the corresponding image channel intensity 255, which is characterized as an event. The event includes positive (ON) events and negative (OFF) events. The positive (ON) event indicates that the light intensity increases beyond the threshold, and the negative (OFF) event indicates that the light intensity decreases. When the brightness change ΔL reaches the threshold, an event will be triggered at the corresponding pixel. k is the x-axis coordinate of the k-th event, y k is the y-axis coordinate of the k-th event; I t is the event frame of the tth interval time, e is the event stream, E is the event sequence, t0 represents the initial time in the tth interval time, t1 represents the end time in the tth interval time, t=0,1,…T / ΔT, T / ΔT represents the number of interval times within the duration T, ΔT represents the time interval for dividing the event stream into event sequences, within this time interval, the event frame corresponding to each event stream is accumulating events, and T represents the duration of the event sequence.

[0108] Preferably, the contrast expression of the event frame in step S3 is:

[0109]

[0110] Where C(t) is the contrast of the event frame at the tth interval, (G k ) t is the gradient of the pixel corresponding to the kth event at the tth interval, k = 1, 2, 3…K, K is the total number of events.

[0111] Furthermore, the pixel gradient expression is:

[0112]

[0113] in, is the Sobel operator in the vertical direction of the event frame at the t-th interval time, is the Sobel operator in the horizontal direction of the event frame at the tth interval time; (G k ) t is the gradient of the pixel corresponding to the kth event at the tth interval.

[0114] In the present invention, events are more likely to be triggered at edge contours where brightness changes significantly, rather than within a scene or object, resulting in a lack of fine texture information in the generated data. In addition, due to the sparseness of event triggering, events representing edges may not form continuous lines. Suppose that an ultra-high-speed camera and an event camera with the same field of view work simultaneously in the same short time window. In this case, the RGB camera will clearly capture the outline of the scene and more texture details, while the event camera will mainly record high brightness changes along the edges of the scene.

[0115] Preferably, the event frame is a binary event frame; it can be understood that the event frame is represented as a binary image. If each pixel in the event frame triggers an event, the pixel value of the pixel that triggers the event is set to 255, and the pixel value of the pixel that does not contain the event is set to 0, thereby obtaining a binary event frame.

[0116] Preferably, the contrast of the event frame is a function related to the time interval (t, t+ΔT);

[0117] Furthermore, the expression of the time interval correlation function is:

[0118]

[0119] Wherein, ΔT is the time interval for dividing the event sequence into event streams. Within this time interval, events are accumulated in each event stream to form an event frame; C(t) is the contrast of the event frame at the t-th interval; if the selected time interval ΔT is large enough, then the time interval contains multiple events and multiple pixels, and the events and pixels will be evenly present. The contrast C(t) of the event frame at the t-th interval tends to 0, resulting in a lack of contrast; if the selected time interval ΔT is too small, since the events are sparsely distributed in the entire event sequence, the event will not exist in the time interval. The contrast C(t) of the event frame at the t-th interval also tends to 0, resulting in the minimum contrast; optionalΔT is the optimal time interval, and the corresponding contrast C(t) of the event frame at the t-th interval is the largest.

[0120] Preferably, the average event frame contrast expression in step S5 is:

[0121]

[0122] Among them, C(t k) is the contrast of the kth event in the event frame of the tth interval, C avg (t) is the average event contrast of the event frame at the tth interval.

[0123] Preferably, the expression of the original continuous time contrast curve area (AOCC) in step S5 is:

[0124]

[0125] Among them, τ is the duration of the event stream, and the duration of the event stream is the sum of the intervals of multiple pixels in the event stream, 0≤t≤τ, A′ C is the original continuous time contrast curve area, C avg (t) is the average event contrast of the event frame at the tth time interval.

[0126] Preferably, the event contrast expression of each segmented event stream in step S6 is:

[0127]

[0128] Among them, C k,p is the contrast of the kth event in the event frame corresponding to the pth segmented event stream, that is, the event stream includes multiple pixels, each of which either triggers an event or does not trigger an event; Δt p is the interval time between events in the event frame corresponding to the event stream of the pth segment; τ p The duration of the event stream of the pth segment; k p The kth event in the event frame corresponding to the event stream of the pth segment.

[0129] Preferably, the expression of the continuous time comparison curve area in step S6 is:

[0130]

[0131] Among them, A C is the area of ​​the continuous time versus curve.

[0132] Specifically, the evaluation of the event camera denoising method in step S8 specifically includes:

[0133] The continuous time contrast curve area is used to evaluate the best overall performance provided by various event camera denoising methods in multiple time intervals, and the optimal event camera denoising method is obtained.

[0134] The optimal event camera denoising method is a method that retains a large number of valid events while filtering out the most noise.

[0135] The present invention designs the continuous time contrast curve area AC The metric retains all events that outline the edges of a scene or object, ensuring that the highest contrast value is obtained within any accumulation time interval of event frames.

[0136] Embodiment 1

[0137] A. Experimental Setup

[0138] Using existing parameter-adjustable denoising methods,

[0139] First, the CCC under different parameters was plotted;

[0140] Then, AOCC is calculated based on CCC and other label-related denoising evaluation metrics, and they are compared;

[0141] Finally, the advantages of using AOCC curves under different parameters to select the optimal denoising method are demonstrated.

[0142] like Figure 1 (a) Schematic diagram of an image captured by the active pixel sensor (APS) of the DAVIS346 event camera; Figure 1 (b) Schematic diagram of a grayscale image captured by the active pixel sensor (APS) of the DAVIS346 event camera; Figure 1 (c) Schematic diagram of a binary frame corresponding to a contrast ratio of 211.5 for an image captured by the active pixel sensor (APS) of the DAVIS346 event camera; Figure 1 (d) Schematic diagram of the event frame corresponding to the image contrast of 72.9 captured by the active pixel sensor (APS) of the DAVIS346 event camera;

[0143] B. Dataset

[0144] The DND21 dataset is used for evaluation;

[0145] The DND21 dataset is generated by simulated events and is inherently noise-free;

[0146] Different levels of labeling noise were introduced in the DND21 sequence, especially at 1 Hz / pixel, 3 Hz / pixel and 5 Hz / pixel; the introduced noise followed a Poisson distribution.

[0147] C. Benchmark Evaluation Metrics

[0148] 1) Label-related metrics: Experiments were conducted using the most common label-related denoising evaluation metrics, noise event removal rate (NeRr), valid event removal rate (VeRr), signal-to-noise ratio (SNR), true positive rate (TPR), false positive rate (FPR), accuracy (ACC), receiver operating characteristic (ROC) curve, and area under the curve (AUC).

[0149] The noise event removal rate NeRr, effective event removal rate VeRr, signal-to-noise ratio SNR, true positive rate TPR, false positive rate FPR, and accuracy ACC are expressed as follows:

[0150]

[0151] Among them, true positive TP classification means classifying signal events as valid events, true negative TN classification means classifying noise events as noise events, false positive FP classification means classifying noise events as valid events, and false negative FN classification means classifying signal events as noise events; the ROC curve is obtained by plotting TPR on the y-axis and FPR on the x-axis to obtain different thresholds.

[0152] TPR=1-VeRr (17)

[0153] FPR=1-NeRr (18)

[0154]

[0155] D. Comparison with Label-Dependent Denoising Evaluation Indicators

[0156] For the evaluation, the QMLPF method was used; e.g. Figure 2 A schematic diagram of an example of a continuous time comparison curve CCC using the QMLPF method;

[0157] The closer the value predicted by the QMLPF method is to 1, the higher the probability that the event is a valid signal.

[0158] The output of the QMLPF method at different thresholds was then used to plot continuous contrast curves.

[0159] The threshold adjustment range of the QMLPF method is from 0.02 to 0.98, and the increment is set to 0.02; when the threshold exceeds 0.94, all events are removed, so the results above this threshold are omitted.

[0160] 1) CCC of QMLPF method: Most current denoising evaluations of event cameras rely on labeled event data. We first evaluate the performance of the proposed denoising metric AOCC by comparing it with existing label-dependent denoising evaluation metrics. The time interval for plotting CCC ranges from 2ms to 400ms with an increment of 2ms. When the time interval is 0, AOCC is also equal to 0.

[0161] Continuous comparison curves of noise-free driving sequences and noisy driving sequences in the DND21 dataset are plotted to compare their performances; as shown in Figure 3;

[0162] exist Figure 3a In , the QMLPF method produces different curves at different thresholds, but they all follow the same trend. At the same time, it can be observed that the curves of some parameters exceed the curves of the noise-free data. This is because even pure event data contains glitches; Figure 3b and Figure 3c In the figure, corresponding to very high noise levels, the curves derived from the noise-free data and the unprocessed noise data surround the data plotted under various parameters. In addition, as the noise level increases, it becomes increasingly difficult for the curve of the denoised sequence to approach the curve of the noise-free event sequence. That is, as the difficulty of denoising increases, the performance of the denoising method using the same parameters also decreases, which confirms the effectiveness of the method proposed in the present invention.

[0163] 2) AOCC and label-related indicators: Subsequently, the present invention shows the results of AOCC, NeRr, VeRr, ACC and SNR in FIG4 ;

[0164] As can be seen from the first row of Figure 4, the AOCC curve is non-monotonic. However, the monotonicity of the NeRr, VeRr, and SNR curves shows that these label-dependent metrics cannot be relied on alone for evaluation because they cannot identify the best solution. In addition, ACC is a non-monotonic metric that is more effective in evaluating classification results. However, classification accuracy and the quality of denoising effect are not synonymous. For example, in 1Hz / pixel sequences, low NeRr and VeRr also obtain high ACC values.

[0165] Next, the present invention shows the ROC curve of the QMLPF method under different noise levels and calculates the AUC, such as Figure 5 As shown in Figure 2, ROC is plotted with the TPR and FPR of the denoising method under different parameters, providing a more objective evaluation. However, it still relies on labeled data and is not convenient for selecting the optimal parameters.

[0166] 3) AOCC for Optimal Parameter Selection: In addition, the AOCC proposed in the present invention can easily identify the optimal parameters of the denoising method, or the interval where these optimal parameters lie, which is very conducive to fair comparison.

[0167] The NeRr, VeRr, SNR and ACC values ​​calculated at the threshold parameters that produced the best AOCC are summarized in Table 1.

[0168] Table 1 Best AOCC metric and other label-based metrics

[0169]

[0170] In Table 1, obtaining the best AOCC is associated with a balance of NeRr, VeRr, ACC, and SNR. When the threshold is maximized, the achieved AOCC is significantly reduced, characterized by significantly high VeRr and NeRr, significantly reduced ACC, and maximum SNR. This situation shows that although a considerable amount of noise is removed, the residual effective signal is minimal, indicating that the denoising result is insufficient despite the increase in SNR. Conversely, lowering the threshold results in a decrease in AOCC, a decrease in both VeRr and NeRr, a decrease in SNR, and a significant increase in ACC. Although this configuration retains a large amount of effective signal, it only removes minimal noise, indicating that the denoising effect is not ideal. Therefore, AOCC, as a non-monotonic indicator, provides an objective measure of the denoising effect.

[0171] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A label-independent, non-monotonic event camera denoising evaluation method, characterized in that include: S1, defining a brightness signal of an event, and establishing an event generation model based on the brightness signal; S2, obtain event sequence; Set the time interval between event sequences; Based on the time interval, obtaining multiple event streams of the event sequence; Based on the event generation model, the events triggered in each event stream are acquired and accumulated, and a corresponding event framework is established to obtain multiple event frameworks; S3, obtaining pixels corresponding to each event in each event frame; Calculating the pixel gradient corresponding to each event; obtaining the corresponding event frame contrast based on the pixel gradient corresponding to each event; S4, obtaining the contrast of each event frame and taking the average value to obtain the average event frame contrast; the event frame is a binary event frame; S5, defining the average event frame contrast as an original continuous time contrast curve; obtaining the area of ​​the original continuous time contrast curve; S6. defining a time interval array; the time interval array includes multiple groups of time intervals; The event stream is segmented repeatedly for multiple times according to the time interval array, the event frames corresponding to the event streams of each segment are obtained, and the contrast of the events in each event frame is calculated to obtain the event contrast of the event streams of each segment; Rewriting the original continuous time contrast curve area using the event contrast of each segmented event stream to obtain a continuous time contrast curve area; S7, using the continuous time contrast curve area as a label-independent, non-monotonic event camera denoising evaluation indicator; The continuous time contrast curve area is used to evaluate event camera denoising.

2. The label-independent, non-monotonic event camera denoising evaluation method according to claim 1, characterized in that: The event generation model in step S1 is expressed as: Among them, ΔL(·) is the brightness change function, C represents the brightness threshold at which the event is triggered, and p i is the event polarity corresponding to the i-th pixel, Δt′ represents the time interval between pixels, and u i is the coordinate of the i-th pixel, t i is the timestamp corresponding to the i-th pixel, and L(·) is the brightness signal function.

3. The label-independent, non-monotonic event camera denoising evaluation method according to claim 2, characterized in that: The expression of the event framework in step S2 is: I t =∑I(u i ) t =∑I(x k ,y k ) t , among which, I(u i ) t =1 (4) where 1(·) is an indicator function, which takes the value 1 when the condition is met and takes the value 0 otherwise. i ) t is the coordinate of the i-th pixel in the event stream at the t-th interval. i ) t =1, it means that the pixel triggers an event and is marked as an event; k is the x-axis coordinate of the k-th event, y k is the y-axis coordinate of the k-th event; I t is the event frame of the tth interval, e is the event flow, E is the event sequence, t0 represents the initial time in the tth interval, t1 represents the end time in the tth interval, t=0,1,…T / ΔT, T / ΔT represents the number of intervals in duration T, ΔT represents the time interval of the event sequence, and T represents the duration of the event sequence.

4. The label-independent, non-monotonic event camera denoising evaluation method according to claim 3, characterized in that: The contrast expression of the event frame in step S3 is: Where C(t) is the contrast of the event frame at the tth interval, (G k ) t is the gradient of the pixel corresponding to the kth event at the tth interval, k = 1, 2, 3…K, K is the total number of events.

5. The label-independent, non-monotonic event camera denoising evaluation method according to claim 4, characterized in that: The step S3 uses the Sobel operator to calculate the pixel gradient corresponding to each event.

6. The label-independent, non-monotonic event camera denoising evaluation method according to claim 4, characterized in that: The average event frame contrast expression in step S4 is: Among them, C(t k ) is the contrast of the kth event in the event frame of the tth interval, C avg (t) is the average event contrast of the event frame at the tth interval.

7. The label-independent, non-monotonic event camera denoising evaluation method according to claim 4, characterized in that: The event contrast expression of each segmented event stream in step S6 is: Among them, C k,p is the contrast of the kth event in the event frame corresponding to the pth segmented event stream; Δt p is the interval time between events in the event frame corresponding to the event stream of the pth segment; τ p The duration of the event stream of the pth segment; k p is the kth event in the event frame corresponding to the event stream of the pth segment, K p The total number of events in the event frame corresponding to the event stream of the p-th segment.

8. The label-independent, non-monotonic event camera denoising evaluation method according to claim 7, characterized in that: The expression of the continuous time comparison curve area in step S6 is: Among them, A C is the area of ​​the continuous time contrast curve, A′ C is the original continuous time versus curve area.

9. The label-independent, non-monotonic event camera denoising evaluation method according to claim 8, characterized in that: The method further includes step S8 of evaluating event camera denoising using the continuous time contrast curve area to obtain an optimal event camera denoising method.

10. The label-independent, non-monotonic event camera denoising evaluation method according to claim 9, characterized in that: The evaluation of the event camera denoising method in step S8 specifically includes: The continuous time contrast curve area is used to evaluate the best overall performance provided by various event camera denoising methods in multiple time intervals, and the optimal event camera denoising method is obtained. The optimal event camera denoising method is a method that retains a large number of valid events while filtering out the most noise.

Images