An event camera adaptive target extraction method under back-light disturbance conditions

Abstract

The application is a kind of event camera adaptive target extraction method under the condition of reverse light disturbance. It includes: defining a fixed time window, reading the event stream in a loop, and mapping the event stream to the image plane grid; statistics of the grid event stream, calculation of the number of positive and negative events in each grid and the total number of events, construction of event density and polarity imbalance degree; according to the event density and polarity imbalance degree, a grid comprehensive score model is constructed, and a candidate grid area is generated according to the quantile threshold of the comprehensive score; the candidate grid area is analyzed by connected domain analysis to determine the target connected area; according to the target connected area, the reliability is judged, if it meets, the current time window target extraction result is output, if it does not meet, the last time window target extraction result is inherited. The application can stably extract the target related area under the condition of reverse light dynamic change, reduce the interference of strong light background events on the target extraction result, and provide stable input for target state estimation and event camera parameter adjustment.

Description

Technical Field

[0001] This invention belongs to the fields of event vision, backlight disturbance target detection, and event camera data processing, and specifically relates to an adaptive target extraction method for event cameras under backlight disturbance conditions. Background Technology

[0002] In space target detection, target recognition against strong light backgrounds, and high-dynamic visual perception tasks, the observed targets typically possess complex reflective properties and unstable motion. Especially under strong light sources such as sunlight and Earth's reflected light, the observation scene is prone to backlighting, significantly increasing the difficulty of distinguishing between the effective target signal and disturbances. Traditional frame-based visual sensors are prone to problems such as localized overexposure, blurred target edges, and loss of detail under these conditions, making it difficult to meet the requirements for stable target detection in complex lighting environments.

[0003] An event camera is a novel visual sensor based on a brightness change triggering mechanism. Unlike traditional cameras, it only outputs event information when the pixel brightness change exceeds a threshold. Event cameras offer advantages such as high dynamic range, high temporal resolution, low latency, and low redundancy output, providing a new technical approach for target detection under backlighting and perturbation conditions. However, under strong backlighting or dynamic stray light conditions, strong background light, reflected light spots, and areas of abrupt brightness changes can also trigger a large number of non-target events, causing a rapid increase in background perturbation events in the event stream and forming high-density spatial regions on the image plane.

[0004] Existing target extraction methods for event cameras mostly rely on the number of events, event clustering, optical flow estimation, deep learning detection boxes, or fixed thresholds to filter target regions. For example, the patent "Adaptive Target Detection Method, System, and Device Based on Event Camera" applied for by the University of Science and Technology of China in 2022 converts the event stream into an adaptive channel-number hyperhistogram based on the density of the event stream, preserving polarity and spatiotemporal information, and then performs weighted dimensionality reduction before inputting it into the target detection network. The adaptivity of this method is reflected in the number of event representation channels, but the problem is that the computational cost is too high and the computational efficiency is low.

[0005] Furthermore, under backlighting disturbance conditions, both high-density background disturbance events and real target events can generate significant event responses within a short time window. Simply relying on the number of events or a general detection box can easily misidentify reflected light spots and stray light abrupt changes as target areas, leading to target extraction result drift, false background detection, or short-term loss of the target area. Therefore, the target extraction problem under backlighting disturbance is not a generalized target detection problem, but rather a problem that requires suppressing disturbances directly at the front end of the event stream and stably outputting the target-related region. Summary of the Invention

[0006] The purpose of this invention is to provide an adaptive target extraction method based on event camera grid statistics under backlight perturbation conditions. This method primarily addresses the problems of insufficient adaptability of fixed thresholds, susceptibility to false detection of high-density background events, easy submersion of target events, and easy drift or short-term loss of target extraction results when extracting target regions under backlight perturbation conditions caused by event cameras. This method adaptively determines the target-related region based on the grid statistical characteristics of the event flow within the current time window, and inherits the target extraction results from the previous time window when detection is unreliable, thereby improving the stability of the target extraction output under backlight perturbation conditions.

[0007] The technical solution to achieve the purpose of this invention is: an adaptive target extraction method for an event camera under backlight disturbance conditions, comprising the following steps:

[0008] Step (1): Define a fixed time window, read in the event stream output by the event camera under backlight disturbance conditions in a loop, and map the event stream to the image plane grid;

[0009] Step (2): Statistically analyze the grid event flow obtained in step (1), calculate the number of positive and negative events and the total number of events in each grid, and construct the event density and polarity imbalance.

[0010] Step (3): Construct a grid comprehensive scoring model based on the event density and polarity imbalance obtained in step (2), generate candidate grid regions based on the quantile threshold of the comprehensive score, and perform connectivity analysis on the candidate grid regions. Combine the comprehensive score, area, shape compactness and spatial continuity with the target region of the connectivity region to determine the target connected region.

[0011] Step (4): Based on the target connected region obtained in step (3), a reliability judgment is made. The reliability judgment includes at least the area constraint of the connected region, the average comprehensive score constraint, and the displacement continuity constraint with the target extraction result of the previous time window. If the reliability conditions are met, the target extraction result of the current time window is output. If the reliability conditions are not met, the target extraction result of the previous time window is inherited.

[0012] Furthermore, step (1) specifically includes the following steps:

[0013] Step (11): Use the event camera to detect the target under backlight disturbance conditions, read the event stream cyclically, and decode the event stream; the decoded event stream is represented as:

[0014] ,

[0015] in,( () represents the pixel coordinates of the event. For event timestamps, Where N represents the event polarity, and N is the number of event data points.

[0016] Step (12): Define a fixed time window:

[0017] ,

[0018] in, For the k-th time window, The length of the time window;

[0019] Step (13): Divide the image plane into a set of grids:

[0020] ,

[0021] in, For the (u,v)th grid cell, , These represent the number of horizontal and vertical grid lines in the image plane, respectively.

[0022] Furthermore, step (2) specifically includes the following steps:

[0023] Step (21): For the (u,v)th grid cell within the kth time window, count the number of positive events. negative event count Total number of events ;

[0024] Step (22): Calculate the grid event density :

[0025]

[0026] in, This represents the number of pixels contained in a grid cell.

[0027] Step (23): Calculate the degree of grid polarity imbalance :

[0028]

[0029] in, These are non-zero stable parameters.

[0030] Furthermore, step (3) specifically includes the following steps:

[0031] Step (31): Normalize the event density:

[0032] ,

[0033] Normalize the degree of polarity imbalance:

[0034] ,

[0035] Step (32): Construct a comprehensive scoring model using normalized event density and normalized polarity imbalance:

[0036] ,

[0037] in, , These are weight parameters;

[0038] Step (33): Determine the candidate region screening threshold based on the comprehensive scoring model:

[0039] ,

[0040] in, (·) is the first Quantile function Set the threshold for filtering candidate regions;

[0041] Step (34): Binarize the grid comprehensive score according to the candidate region screening threshold:

[0042] ,

[0043] in, For binary candidate grid masks;

[0044] Step (35): Perform connected component analysis on the binary candidate mesh mask to obtain the set of connected components:

[0045] ,

[0046] in, Let m be the m-th connected region;

[0047] Step (36): Select the target connected component from the set of connected components:

[0048] ,

[0049] in, For the evaluation function of connected components;

[0050] Furthermore, the connected component evaluation function Represented as:

[0051] .

[0052] Furthermore, step (4) specifically includes the following steps:

[0053] Step (41): Map the target connected region to the pixel space to obtain the target extraction result for the current time window:

[0054] ,

[0055] Step (42): Construct the reliability function of the target connected component And set a reliability threshold. ;

[0056] Step (43): Output the final target extraction result based on the reliability judgment result:

[0057] ,

[0058] in, This is the final target extraction result output for the k-th time window. This is the final target extraction result output from the previous time window.

[0059] Furthermore, the reliability function of the target connected component is expressed as:

[0060] ,

[0061] in, The area of ​​the target connected region.

[0062] Compared with the prior art, the significant advantages of this invention are:

[0063] (1) This invention treats backlight disturbance as a special processing object, uses event-based sensor output as input, and takes advantage of the high dynamic range, high temporal resolution and low redundancy output of the event camera to obtain target and background brightness change information under complex backlight conditions such as sunlight, albedo or reflected light spots.

[0064] (2) The present invention adopts a gridded event statistics method to map the event flow to the image plane grid, and constructs the event density and polarity imbalance by combining the number of positive and negative events, and then generates target-related candidate regions through grid comprehensive scoring; compared with the method that simply relies on the number of events, fixed threshold or general target detection box, it can more fully characterize the difference in event structure between target-related regions and disturbed regions in backlight scenes;

[0065] (3) The present invention adopts a closed-loop strategy that combines candidate mesh generation, connected component selection, reliability judgment and history inheritance, which can maintain the stability of target extraction results under strong light disturbance, short-term event sparsity or local false detection conditions, and reduce the risk of target extraction results drift caused by backlight background events.

[0066] (4) The method of the present invention does not rely on target category training samples, optical flow velocity estimation or target geometry prior, and can be used as a front-end target related region extraction module for subsequent target detection, state estimation and event camera parameter adjustment. It has the advantages of clear calculation process, low implementation complexity and strong engineering adaptability. Attached Figure Description

[0067] Figure 1 This is a flowchart of the adaptive target extraction method based on backlight perturbation using an event camera according to the present invention.

[0068] Figure 2 Comparison of imaging effects under backlight disturbance; (a) is a regular industrial camera, (b) is an event camera.

[0069] Figure 3 Comparison of the effects of target extraction: (a) direct extraction from the original data, (b) extraction by this method. Detailed Implementation

[0070] The present invention will now be described in further detail with reference to the accompanying drawings.

[0071] This application provides an adaptive target extraction method for an event camera under backlight disturbance conditions, the method flow is as follows: Figure 1 As shown, the method includes the following steps:

[0072] Step (1): Define a fixed time window, read in the event stream output by the event camera under backlight disturbance conditions in a loop, and map the event stream to the image plane grid;

[0073] Step (2): Statistically analyze the grid event flow obtained in step (1), calculate the number of positive and negative events and the total number of events in each grid, and construct the event density and polarity imbalance.

[0074] Step (3): Construct a comprehensive scoring model based on the event density and polarity imbalance obtained in step (2), generate candidate grid regions based on the comprehensive scoring model, and determine the target connected region through connected component analysis;

[0075] Step (4): Based on the target connected region obtained in step (3), perform reliability judgment. If the reliability conditions are met, output the target extraction result of the current time window; if the reliability conditions are not met, inherit the target extraction result of the previous time window.

[0076] In this embodiment, the target extraction process does not require a trained target detection network, optical flow clustering results, or a fixed event number threshold as necessary inputs. Instead, the target extraction results are constrained by event density, polarity imbalance, connected component structure, and temporal continuity, thereby enhancing the ability to distinguish from disturbed regions.

[0077] Step (1) specifically includes the following steps:

[0078] Step (11): Use the event camera to detect the target under backlight disturbance conditions, read the event stream cyclically, and decode the address event stream. The decoded event stream is represented as:

[0079]

[0080] in,( ) represents the pixel coordinates of the event. For event timestamps, Let N represent the event polarity, and N be the number of event data points.

[0081] Step (12): Define a fixed time window:

[0082]

[0083] in, For the k-th time window, The length of the time window;

[0084] Step (13): Divide the image plane into a set of grids:

[0085]

[0086] in, For the (u,v)th grid cell, , These represent the number of horizontal and vertical grid lines in the image plane, respectively.

[0087] Step (2) specifically includes the following steps:

[0088] Step (21): For the (u,v)th grid cell within the kth time window, count the number of positive events. negative event count Total number of events ;

[0089] Step (22): For a grid event stream with a fixed time window, calculate the number of events per unit time and per unit grid area, i.e., the grid event density. :

[0090]

[0091] in, This represents the number of pixels contained in a grid cell.

[0092] Step (23): Calculate the degree of grid polarity imbalance :

[0093]

[0094] in, These are non-zero stable parameters.

[0095] Step (3) specifically includes the following steps:

[0096] Step (31): Normalize the event density:

[0097]

[0098] Normalize the degree of polarity imbalance:

[0099]

[0100] Step (32): Construct a comprehensive scoring model using normalized event density and normalized polarity imbalance:

[0101]

[0102] in, , These are weight parameters;

[0103] Step (33): Determine the candidate region screening threshold based on the comprehensive scoring model:

[0104]

[0105] in, (·) is the first Quantile function Set the threshold for filtering candidate regions;

[0106] Step (34): Binarize the grid comprehensive score according to the candidate region screening threshold:

[0107]

[0108] in, For binary candidate grid masks;

[0109] Step (35): Perform connected component analysis on the binary candidate mesh mask to obtain the set of connected components:

[0110]

[0111] in, Let m be the m-th connected region.

[0112] Step (36): Select the target connected component from the set of connected components:

[0113]

[0114] in, For the evaluation function of connected components;

[0115] The connected component evaluation function can be expressed as:

[0116]

[0117] Step (4) specifically includes the following steps:

[0118] Step (41): Map the target connected region to the pixel space to obtain the target extraction result for the current time window:

[0119]

[0120] Step (42): Construct the reliability function of the target connected component And set a reliability threshold. .

[0121] Step (43): Output the final target extraction result based on the reliability judgment result:

[0122]

[0123] in, This is the final target extraction result output for the k-th time window. This is the final target extraction result output from the previous time window.

[0124] The reliability function of the target connected component can be expressed as:

[0125]

[0126] in, The area of ​​the target connected region.

[0127] Through the above steps, this invention can adaptively determine the target-related region based on the spatial distribution of the current event flow, the event polarity structure, and the reliability of the connected regions under backlight disturbance conditions. When the target connected regions become unreliable due to event sparsity, sudden changes in strong light, or local false detections within the current time window, the output remains stable by inheriting the target extraction results from the previous time window, thus avoiding frequent jumps in the target extraction results.

[0128] Example:

[0129] Further explanation Figure 2 and Figure 3Under the same backlighting disturbance scenario, a conventional industrial camera and an event camera were used to simultaneously observe the target. The conventional industrial camera output color frame images with a fixed exposure time, while the event camera output an event stream. The two sensors were installed at positions close to the line of sight, and the target was placed within a region of strong light illumination or reflected light interference. During the acquisition process, the target's motion state, illumination direction, and observation distance were kept consistent to compare the target visibility under backlighting disturbance. Figure 2 (a) is an image obtained by a regular industrial camera under backlight conditions. Due to overexposure and strong background light, the texture and edges of the target are not obvious. Figure 2 (b) is the event imaging result obtained by the event camera based on the change in brightness. The target edge and local structure can still form a relatively obvious event response.

[0130] Figure 3 This is achieved by directly extracting the raw data from the same event stream and then processing it using the method described above. Direct extraction of the raw data projects events onto the image plane based solely on the number of events or a simple threshold, which easily mixes backlighting disturbances and abrupt changes in reflected light spots with the target events, resulting in… Figure 3 (a) shows the adhered, broken, or offset target regions. In this method, the event flow is first statistically analyzed in a gridded manner according to a fixed time window to calculate the event density and polarity imbalance. Then, candidate grid masks are generated based on a comprehensive scoring model, and target connected regions are selected through a connected component evaluation function. For windows with insufficient reliability, the target extraction results from the previous time window are inherited. This yields... Figure 3 (b) shows that the target extraction results have a more concentrated target area, less background disturbance residue, and better contour continuity.

[0131] In practice, the time window length can be set to 20 ms, and the grid size to 16×16 pixels. The candidate region selection threshold is adaptively determined using the high quantile of the comprehensive score, here set to the 90th percentile. The weights in the connected component evaluation function are set according to the application scenario. By setting the above parameters, the target region output can be kept stable under conditions of changes in backlight intensity, sudden increases in event density, or short-term weak response of the target.

[0132] The above describes preferred embodiments of the present invention. The descriptions of the embodiments are only for the purpose of helping to understand the method and core ideas of the present invention, but the embodiments of the present invention are not limited to the above content. For those skilled in the art, based on the ideas of the present invention, there will be changes in specific implementation methods and application scope. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. An adaptive target extraction method for an event camera under backlight perturbation conditions, characterized in that, Includes the following steps: Step (1): Define a fixed time window, read in the event stream output by the event camera under backlight disturbance conditions in a loop, and map the event stream to the image plane grid; Step (2): Statistically analyze the grid event flow obtained in step (1), calculate the number of positive and negative events and the total number of events in each grid, and construct the event density and polarity imbalance. Step (3): Construct a grid comprehensive scoring model based on the event density and polarity imbalance obtained in step (2), generate candidate grid regions based on the quantile threshold of the comprehensive score, and perform connectivity analysis on the candidate grid regions. Combine the comprehensive score, area, shape compactness and spatial continuity with the target region of the connectivity region to determine the target connected region. Step (4): Based on the target connected region obtained in step (3), a reliability judgment is made. The reliability judgment includes at least the area constraint of the connected region, the average comprehensive score constraint, and the displacement continuity constraint with the target extraction result of the previous time window. If the reliability conditions are met, the target extraction result of the current time window is output. If the reliability conditions are not met, the target extraction result of the previous time window is inherited.

2. The method according to claim 1, characterized in that, Step (1) specifically includes the following steps: Step (11): Use the event camera to detect the target under backlight disturbance conditions, read the event stream cyclically, and decode the event stream; the decoded event stream is represented as: ， in,( () represents the pixel coordinates of the event. For event timestamps, Where N represents the event polarity, and N is the number of event data points. Step (12): Define a fixed time window: ， in, For the k-th time window, The length of the time window; Step (13): Divide the image plane into a set of grids: ， in, For the (u,v)th grid cell, , These represent the number of horizontal and vertical grid lines in the image plane, respectively.

3. The method according to claim 2, characterized in that, Step (2) specifically includes the following steps: Step (21): For the (u,v)th grid cell within the kth time window, count the number of positive events. negative event count Total number of events ; Step (22): Calculate the grid event density : ， in, This represents the number of pixels contained in a grid cell. Step (23): Calculate the degree of grid polarity imbalance : ， in, These are non-zero stable parameters.

4. The method according to claim 1, characterized in that, Step (3) specifically includes the following steps: Step (31): Normalize the event density: ， Normalize the degree of polarity imbalance: ， Step (32): Construct a comprehensive scoring model using normalized event density and normalized polarity imbalance: ， in, , These are weight parameters; Step (33): Determine the candidate region screening threshold based on the comprehensive scoring model: ， in, (·) is the first Quantile function Set the threshold for filtering candidate regions; Step (34): Binarize the grid comprehensive score according to the candidate region screening threshold: ， in, For binary candidate grid masks; Step (35): Perform connected component analysis on the binary candidate mesh mask to obtain the set of connected components: ， in, Let m be the m-th connected region; Step (36): Select the target connected component from the set of connected components: ， in, This is the evaluation function for connected components.

5. The method according to claim 4, characterized in that, Connectivity evaluation function Represented as: 。 6. The method according to claim 5, characterized in that, Step (4) specifically includes the following steps: Step (41): Map the target connected region to the pixel space to obtain the target extraction result for the current time window: ， Step (42): Construct the reliability function of the target connected component And set a reliability threshold. ; Step (43): Output the final target extraction result based on the reliability judgment result: ， in, This is the final target extraction result output for the k-th time window. This is the final target extraction result output from the previous time window.

7. The method according to claim 6, characterized in that, The reliability function of the target connected component is expressed as: ， in, The area of ​​the target connected region.

Images