Real-time noise filtering method based on event camera

Abstract

The invention discloses a real-time noise filtering method based on an event camera. The objective of the invention is to improve the real-time performance and precision of data processing. According to the method, original event data are collected through an event camera, an event stream is generated, a fixed window filter is used for carrying out noise filtering on the event stream, and a Manhattan distance algorithm is used for judging and filtering noise events. The processed event stream data is displayed through a real-time visual interface, and it is ensured that the event stream data are synchronized with the output of an event camera. The method optimizes the algorithm performance of an embedded system, has an efficient noise filtering effect, and is suitable for dynamic monitoring and real-time processing in an intelligent industrial environment.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision, event camera technology and data processing, and in particular to a real-time noise filtering method based on an event camera. Background Art

[0002] With the rapid development of computer vision and automation technology, traditional frame image camera technology is gradually facing challenges, especially in scenarios with high real-time and low power requirements. As an emerging sensor technology, event cameras have the advantages of high dynamic range, low latency, and low power consumption, and can effectively capture high-speed dynamic events in complex industrial environments. However, the data generated by event cameras is asynchronous and sparse, and usually contains a large number of noise events. These noises seriously affect the accuracy of the data, and thus affect the effectiveness of subsequent processing tasks such as target detection and tracking.

[0003] Existing noise filtering methods, such as fixed window-based denoising methods, have certain filtering effects, but their real-time performance is often challenged when dealing with complex dynamic scenes. Especially in smart industrial environments, the system is required to quickly process a large number of asynchronous event streams to ensure the real-time and accuracy of decision-making. However, existing embedded systems and traditional denoising methods often find it difficult to achieve efficient real-time processing on hardware platforms with limited resources.

[0004] The present invention focuses on improving the real-time processing capability based on event cameras. Through hardware optimization of the fixed window denoising method and improvement of the processing architecture, the present invention can greatly improve the response speed and real-time performance of the system in complex scenarios, meeting the demand for efficient and real-time data processing in industrial environments. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time noise filtering method based on an event camera to solve the problem that the prior art is insufficient in real time in complex scenes.

[0006] The technical solution to achieve the purpose of the present invention is: a real-time noise filtering method based on an event camera, comprising the following steps:

[0007] (1): Use an event camera to collect raw event data and output it in the form of an event stream to generate an initial event stream.

[0008] (2): Apply a fixed window filter to the event stream data obtained in step (1) to perform preliminary noise filtering to obtain filtered event stream data.

[0009] (3): Display the event stream data processed in step (2) in real time in a new window.

[0010] Furthermore, the step (1) specifically includes the following sub-steps:

[0011] Step (11), during the data collection process of the event camera, the pixels in the sensor of the event camera sense the change in the logarithm of the light intensity, and determine whether the difference in the change reaches the condition for exciting the event. If so, a single pixel is excited, and the process proceeds to step (12);

[0012] ΔL(x,y,t)=log(I(x,y,t))-log(I(x,y,t-Δt))>C

[0013] Where ΔL(x,y,t) represents the brightness change at pixel (x,y) at time t; I(x,y,t) represents the brightness value at pixel (x,y) at time t, Δt is the interval between two time points, and C is the brightness change threshold of the event camera;

[0014] Step (12): After a single pixel is excited, it outputs an event e, which includes the pixel position, time and polarity of the event, that is, e = (x, y, t, p) is a four-tuple; as a series of events occur, the event camera outputs an event stream, which is a sequence of four-tuples of all events.

[0015] Furthermore, the step (2) specifically includes the following sub-steps:

[0016] Step (21): Window initialization: Set a fixed-size window, the size of which is L. Slide the window over the event stream data, and the window will contain the most recent L events.

[0017] Step (22) Event stream input: The event stream data is a series of four-tuple events e = (x, y, t, p) collected by the event camera, where x and y are the event pixel positions, t is the timestamp, and p is the polarity of the event.

[0018] Step (23) Sliding window operation: For each newly arrived event e i , calculate the distance between the event and all events in the window. The distance uses the Manhattan distance formula:

[0019] D(e i ,e j )=|x i -x j |+|y i -y j |

[0020] Among them, e i is the current event, e j For events within the window.

[0021] Step (24) Minimum distance judgment: Calculate the current event e i The Manhattan distances of all events in the window are calculated, and the minimum value Dmin of these distances is found. If Dmin is less than the preset noise filtering threshold s, then e is considered i If Dmin is greater than or equal to s, then e i It's noise, filter it out.

[0022] Step (25) Window update: Update the current event e i If the window is full, the earliest event that enters the window is removed and the current event is replaced into the window to keep the number of events in the window at a fixed size L.

[0023] Step (26) outputs the filtering result: the event stream data processed by the fixed window filter is the event stream after noise filtering, that is, the valid event stream after noise filtering. filtered After each event passes through this process, it is considered as a valid event or a noise event.

[0024] Compared with the prior art, the present invention has the following significant advantages: (1) Significantly improve real-time performance: The present invention greatly improves the real-time performance of event data processing by optimizing the combination of fixed window filtering algorithm and event camera. Compared with traditional denoising methods, the present invention can quickly process a large number of asynchronous event streams in complex and highly dynamic environments, ensuring effective operation in scenarios with high real-time requirements in intelligent industrial environments. (2) Efficient noise filtering effect: The present invention adopts a fixed window filter and quickly and accurately filters noise events based on the Manhattan distance calculation of events. When the distance between an event in the event stream and a historical event in the window is small and lower than a set threshold, the event is retained, otherwise it is considered as noise and filtered out. This method can effectively reduce invalid events, thereby improving the efficiency and accuracy of subsequent data processing. (3) Adaptation to embedded systems and efficient resource utilization: In view of the limited computing resources and low power consumption requirements of embedded systems, the denoising method of the present invention has been optimized by algorithm so that it can be efficiently executed on embedded hardware platforms. Compared with the denoising methods with large computational complexity in the prior art, the computational complexity of the present invention is significantly reduced, making it more suitable for resource-constrained application scenarios, especially in industrial automation equipment. (4) Real-time visualization of event processing results: The present invention not only processes the event stream efficiently in the background, but also displays the processed data in real time in the window, using different colors to distinguish the polarity of the event, so that users can intuitively observe the dynamic changes of the event stream. This kind of real-time event visualization provides strong support for system debugging, data analysis and real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of a real-time noise filtering method based on an event camera. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] The embodiment provides a real-time noise filtering method based on an event camera, and the method flow is as follows: Figure 1 As shown, the method comprises the following steps:

[0028] Step (1): In the present invention, an event camera is first used to collect event data. The pixel sensor in the event camera responds to the change in light intensity in the scene and determines whether an event is triggered based on the logarithmic change in light intensity. Specifically, at the pixel (x, y), the brightness change ΔL(x, y, t) at time t is calculated by the following formula:

[0029] ΔL(x,y,t)=log(I(x,y,t))-log(I(x,y,t-Δt))>C

[0030] Where I(x, y, t) represents the light intensity value at the pixel (x, y) at time t, Δt is the time interval, and C is the threshold value of brightness change.

[0031] When the absolute value of the brightness change ΔL(x,y,t) exceeds the threshold C, the pixel triggers an event. The triggered event includes the pixel position (x,y) where the event occurs, the time t, and the event polarity p (i.e., whether the event increases or decreases in brightness). Therefore, each event can be represented as a four-tuple e = (x,y,t,p), and all events form an event stream, which is continuously output.

[0032] Step (2): Noise filtering of event data

[0033] The present invention adopts a fixed window filter to filter the collected event stream in real time. The filter adopts a sliding window of a fixed size, which slides on the event stream to retain valid events and filter out noise events.

[0034] Set a fixed-size window, the size of the window is L. The window is used to accommodate the most recent L events. The event stream is the four-tuple sequence e = (x, y, t, p) output by the event camera. When a new event e i When it arrives, calculate the event and all events e in the window j The Manhattan distance between them is:

[0035] D(e i ,e j )=|x i -xj |+|y i -y j |

[0036] Among them, (x i ,y i ) and (x j ,y j ) are respectively events e i and e j The spatial coordinates of .

[0037] Find the event i The minimum Manhattan distance D to all events in the window min If D min If the noise level is less than the preset noise threshold s, the event e is considered i is a valid event, keep the event; if D min If it is greater than or equal to the threshold s, then the event e is considered i is noise, filter it out. i Add to the window. If the window is full, remove the earliest event that enters the window, keeping the number of events in the window unchanged.

[0038] Step (3) Visualization of event data

[0039] The event stream data processed by the fixed window filter is a valid event after noise filtering. The present invention provides a function of real-time visualization of event processing results so that users can observe the dynamics of the processed event stream. The specific implementation includes: spatial coordinate and polarity visualization: the filtered events are drawn in the display window according to their spatial coordinates (x, y). In order to distinguish the polarity of the event (i.e., increase or decrease in brightness), different colors are used for marking, such as red for events with increased brightness and blue for events with decreased brightness. Real-time refresh: In order to ensure synchronization with the output of the event camera, the display window will refresh in real time to show the latest event stream data, so that users can intuitively see the changes in the event data.

[0040] The present invention significantly improves the real-time performance of event stream data processing by optimizing the fixed window denoising method and combining it with the hardware characteristics of the embedded system, so that it can achieve efficient and stable noise filtering and event processing in a resource-constrained environment, which is suitable for dynamic monitoring and real-time decision-making applications in smart industrial scenarios.

Claims

1. A method for real-time noise filtering based on event camera, characterized in that: The steps include: Step (1): Use an event camera to collect raw event data and output it in the form of an event stream to generate an initial event stream; Step (2): Using a fixed window filter to filter noise from the event stream data obtained in step (1) to obtain filtered event stream data; Step (3): Display the event stream data processed in step (2) in real time in a new window.

2. The method for real-time noise filtering based on event camera according to claim 1, characterized in that: In the step (1): Step (11), during the data collection process of the event camera, the pixels in the sensor of the event camera sense the change in the logarithm of the light intensity, and determine whether the difference in the change reaches the condition for exciting the event. If so, a single pixel is excited, and the process proceeds to step (12); ΔL(x,y,t)=log(I(x,y,t))-log(I(x,y,t-Δt))>C Wherein, ΔL(x, y, t) represents the brightness change at pixel (x, y) at time t; I(x, y, t) represents the brightness value at pixel (x, y) at time t, Δt is the interval between two time points, and C is the brightness change threshold of the event camera; Step (12), after a single pixel is excited, an event e is output, and the event e includes the pixel position, time and polarity of the event, that is, e = (x, y, t, p) is a four-tuple; as a series of events occur, the event camera outputs an event stream, which is a sequence of four-tuples of all events.

3. The method for real-time noise filtering based on event camera according to claim 1, characterized in that: The step (2) "using a fixed window filter" specifically includes the following: Step (21), window initialization: set a fixed-size window, the size of the window is L; slide the window on the event stream data, and the window will accommodate the most recent L events; Step (22), event stream input: the event stream data is a series of four-tuple events e = (x, y, t, p) collected by the event camera, where x and y are event pixel positions, t is the timestamp, and p is the polarity of the event; Step (23), sliding window operation: for each newly arrived event e i , calculate the distance between the event and all events in the window; the distance uses the Manhattan distance formula: D(e i ,e j )=|x i -x j |+|y i -y j | Among them, e i is the current event, e j For events within the window; Step (24), minimum distance judgment: calculate the current event e i Manhattan distances to all events in the window, and find the minimum of these distances D min ; If D min If the noise filtering threshold s is smaller than the preset value, then e i If it is not noise, keep the event; if D min If it is greater than or equal to s, then e is considered i It is noise, filter it out; Step (25), window update: update the current event e i Add to the window; if the window is full, remove the earliest event that enters the window and replace the current event into the window to keep the number of events in the window at a fixed size L; Step (26), output filtering result: the event stream data processed by the fixed window filter is the event stream after noise filtering, that is, the valid event stream after noise filtering. filtered After each event passes through this process, it is considered as a valid event or a noise event.

Images