Adaptive differential event detection method for infrared moving target

Abstract

The invention discloses a self-adaptive differential event detection method for an infrared moving target, which relates to the technical field of infrared detection and infrared space, and comprises the following steps: S101, initializing a current frame and a threshold value; s102, infrared differential event detection of a self-adaptive threshold value is carried out; s103, preprocessing an infrared event frame; s104, target event feature extraction is carried out; and S105, abnormal events are removed. In order to balance a detection rate and an error triggering rate for a space-based detection background with complex change and a dark moving target observation task, the invention provides a method for adjusting an event triggering threshold value based on triggering rate feedback, self-adapting to a strong radiation background with high dynamic change, flexibly and efficiently obtaining an infrared differential event frame, reducing static background redundant information, and improving the detection efficiency. And the detection efficiency and the moving target discovery capability are improved. And furthermore, the time-space characteristics of the target event are utilized to extract the centroid coordinates of the target area, so that high detection rate and low false alarm rate are realized, and the method has real-time performance and stability and relatively high engineering application value.

Description

Technical Field

[0001] The invention belongs to the field of infrared detection and infrared space technology, in particular to an adaptive differential event detection method for infrared moving targets. Background Art

[0002] The event-triggered sensor is a new type of bionic visual sensor. It is triggered asynchronously by sensing changes in ambient light. It has the advantages of high temporal resolution, low latency, and high sensitivity. It is particularly suitable for observing scenes with moving objects and changes in light intensity. It can be applied to autonomous driving, robotics, space situational awareness, and other fields. The principle of event triggering is that if the light intensity change of the scene exceeds the contrast threshold, the pixel triggers the event independently. The event consists of the coordinates, timestamp, and polarity of the response pixel. Therefore, the trigger threshold plays a vital role in the ability to detect moving targets. Usually, the threshold is preset or manually adjusted.

[0003] The downward-looking scenes of space-based detection include cities, clouds, land, ocean, etc., with strong background infrared radiation and large fluctuations. Aerial targets appear as point targets on the space-based imaging plane, and they move fast and over a wide range. In order to quickly perceive dim and weak moving targets and adapt to the complex and changing high-background infrared radiation environment, taking the space-based detection scene as an example, an infrared differential event detection method with adaptive threshold is urgently needed to generate sparsely distributed infrared event frames to solve the problems of severe noise, low spatial resolution, and difficulty in detecting weak and small targets at long distances. Summary of the invention

[0004] The purpose of the present invention is to provide an adaptive differential event detection method for infrared moving targets, so as to balance the detection rate and the error trigger rate, and realize the rapid perception and high-precision detection of long-distance dim moving targets.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] An adaptive differential event detection method for infrared moving targets comprises the following steps:

[0007] S101, initializing the current frame and threshold;

[0008] S102, infrared differential event detection with adaptive threshold;

[0009] S103, infrared event frame preprocessing;

[0010] S104, target event feature extraction;

[0011] S105. Remove abnormal events.

[0012] In S101, the image data acquired by the infrared camera is used as input to initialize the current frame and threshold;

[0013] The infrared time-domain differential event trigger mechanism is defined as outputting an event when the radiation change of each pixel exceeds the comparison threshold. The present invention sets the initial moment scene as the first reference frame for judging event triggering. The threshold at the initial moment is set as a constant, and then positive feedback adjustment is performed according to the trigger rate.

[0014] In S102, the infrared differential event frame sequence is generated;

[0015] S102.1 Trigger events and calculate trigger rates

[0016] The polarity of the infrared differential event indicates the direction of the time domain change of the infrared radiation, that is, the direction of the image grayscale change. The grayscale of the current frame is differentiated from that of the reference frame. If the grayscale increment of the pixel exceeds the threshold, the pixel independently triggers a positive polarity event p(x,y)=1; if the grayscale decrease exceeds the threshold, the pixel triggers a negative polarity event p(x,y)=-1; otherwise, the event p(x,y)=0 is not triggered. The pixel that triggers the event updates the grayscale reference value to the grayscale value at the current moment, otherwise, the reference value maintains the grayscale value at the previous moment;

[0017] According to the number of trigger events n event Calculate the trigger rate as a ratio of the array size mm×nn:

[0018]

[0019] S102.2 Adaptive adjustment of contrast threshold

[0020] Determine whether the event trigger rate R exceeds the preset range [R L ,R H ]. If R>R H , then the contrast threshold C is increased linearly; if R<R L , the contrast threshold is linearly reduced. The threshold interval and linear change coefficient can be set according to empirical values. The updated contrast threshold is the trigger threshold of the next event frame, and the contrast threshold is adjusted by the new trigger rate. The process is repeated until the last frame of the scene, and the positive and negative polarity events are integrated into an infrared differential event frame sequence and then output.

[0021] In S103, the spatial domain filtering operator is constructed to remove noise events, and the output of S102 is used as the input of S103;

[0022] Establish a (2r+1)×(2r+1) filter window S with pixel (x,y) as the center x,y (r), where r is the radius of the filter window and the coordinates of the window center are (r+1,r+1). Construct the spatial domain filter operator:

[0023]

[0024] Where (u,v)∈S x,y (r). x,y The event polarity p in (r) is weighted:

[0025]

[0026] Calculate the 1-norm ||J(x,y)||1, if ||J(x,y)||1<J th , then set p(x,y) = 0. Remove noise events by traversing each pixel.

[0027] In S104, the faint target centroid is extracted from the pure infrared event, and the output of S103 is used as the input of S104;

[0028] S104.1 Extracting Potential Target Events Using Optical Flow

[0029] The optical flow of the event frame is estimated, and the maximum inter-class variance algorithm is used to binarize the amplitude and direction of the optical flow, and the new template is obtained after the union is calculated. The centroid of the template is solved to obtain the center of the event area, that is, the potential target event;

[0030] S104.2 Constructing the objective function using the HOG operator

[0031] Calculate the HOG operator around the feature points extracted by S104.1:

[0032] G x (x,y)=p(x+1,y)-p(x-1,y)

[0033] G y (x,y)=p(x,y+1)-p(x,y-1)

[0034] In the formula, G x and G y are the horizontal and vertical gradients of the event polarity. The gradient magnitude G and direction α are:

[0035]

[0036] Set (w cell × cell ). Divide the n-channel histogram of the gradient direction into each cell. The degree of each channel is π / n. The integral histogram of the gradient direction is calculated by weighting the gradient size. The amplitude of the feature point (x, y) in the channel bin is as follows:

[0037]

[0038] Where 1≤i≤n,i∈S. Calculate the amplitude and value of the feature point in all channels:

[0039]

[0040] The number of channels that satisfy the non-zero amplitude is calculated as |A|, where:

[0041] A={i∈S|V i (x,y)≠0}

[0042] Construct the objective function Con HOG for:

[0043] Con HOG =k1|A|+k2V

[0044] Where k1 and k2 are linear coefficients. After calculating the objective function for each feature point, normalization is performed and non-maximum suppression is used to remove redundant feature points.

[0045] In S105, the characteristic events extracted in S104 are screened to remove noise events and dynamic background events;

[0046] S105.1 Time-space correlation to remove abnormal events

[0047] Select the current time t k , the previous moment t k-1 and the next moment t k+1 The feature point set of the current moment u=(x,y) is used to calculate the spatiotemporal correlation. The Euclidean distance between the feature point u=(x,y) at the current moment and the feature points at the previous and next moments is searched respectively:

[0048] d k,k-1 =||u k ,u k-1 ||2,d k,k+1 =||u k ,u k+1 ||2

[0049] Define the distance weighting function as con ST , where r1, r2, r3 and r4 are distance parameters:

[0050]

[0051] Define the feature point u at the current moment k The spatiotemporal correlation coefficient between all feature points in the previous and next frames is:

[0052] Con ST (k) = ∑[con ST (d k,k-1 )+con ST (d k,k+1)]

[0053] If the threshold interval of the spatiotemporal correlation coefficient [Con L ,Con H ], then filter out the corresponding feature points;

[0054] S105.2 Filtering abnormal events based on statistics

[0055] The output coordinates of S105.1 are sorted and divided into four equal quartiles. The first quartile (25%) is defined as Q1, the third quartile (75%) is defined as Q3, and the expression of the interquartile range is as follows:

[0056] IQR=Q3-Q1

[0057] If IQR>0, the values ​​outside the interval [Q1-k×IQR,Q3+k×IQR] are defined as outliers, and k is the coefficient. If IQR=0, the mean μ and standard deviation σ of the characteristic coordinate sequence are calculated, defined in the interval (u-μ) / σ|>Z th The values ​​outside the range are outliers and are filtered out. The feature points obtained after outlier removal are the centroid coordinates of the target event.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The adaptive threshold infrared differential event frame method proposed in the present invention can obtain event frames flexibly, dynamically and efficiently for different targets and detection scenarios, balance the target detection rate and error trigger rate, improve the capture efficiency of moving targets, and reduce redundant information by suppressing static background.

[0060] 2. Aiming at the needs of space-based detection, the infrared dim target detection algorithm based on pure event frames proposed in the present invention can fully exploit the spatiotemporal characteristics of infrared differential event frames, achieve high detection rate and low false alarm rate of dim targets, and the performance of the algorithm is efficient and stable, with high engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flow chart of an adaptive differential event detection method for infrared moving targets according to the present invention;

[0062] Figure 2 A flow chart of a method for adaptive differential event detection of infrared moving targets provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of space-based ground observation of aerial targets, infrared differential event frames and target detection results provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0065] like Figure 1 , 2 As shown, the present invention discloses an adaptive differential event detection method for infrared moving targets. It includes the following steps:

[0066] S101, initializing the current frame and threshold;

[0067] The present invention supports two types of input: scene simulation and infrared image sequence. This embodiment takes the space-based downward-looking aerial target detection scene as an example and uses the infrared image sequence as input. The present invention sets the image at the initial moment as the first reference frame for judging event triggering. In this embodiment, the initial threshold is 0, and all pixels with grayscale changes trigger events.

[0068] S102, infrared differential event detection with adaptive threshold;

[0069] In order to adapt to the complex and changing high-intensity background, the present invention proposes an infrared differential event detection method that adaptively adjusts the contrast threshold based on the trigger rate:

[0070] S102.1 Trigger events and calculate trigger rates

[0071] The polarity of the infrared differential event indicates the direction of the time domain change of the infrared radiation, that is, the direction of the image grayscale change. The grayscale difference between the current frame and the reference frame is performed. If the grayscale increment exceeds the threshold, the pixel independently triggers a positive polarity event p(x,y)=1; if the grayscale reduction exceeds the threshold, the pixel triggers a negative polarity event p(x,y)=-1; otherwise, the event p(x,y)=0 is not triggered. The pixel that triggers the event updates the grayscale reference value to the grayscale value at the current moment, otherwise, the reference value maintains the grayscale value at the previous moment. According to the number of trigger events n event Calculate the trigger rate as a ratio of the array size mm×nn:

[0072]

[0073] S102.2 Adaptive adjustment of contrast threshold

[0074] Determine whether the event trigger rate R exceeds the preset range [R L ,R H ], in this embodiment, the trigger rate range is 1% to 10%. If the trigger rate is higher than the upper limit, that is, R>R H , then the contrast threshold C is increased linearly; if the trigger rate is lower than the lower limit, that is, R<R L, then the contrast threshold is linearly reduced. In this embodiment, the linear change coefficient is set to 0.5. The updated contrast threshold is the trigger threshold of the next event frame, and the contrast threshold is adjusted by the new trigger rate. The process is repeated until the last frame of the scene, and the positive and negative polarity events are integrated into an infrared differential event frame sequence and then output. The infrared differential event frame is as follows: Figure 3 As shown in (b) in .

[0075] S103, infrared event frame preprocessing;

[0076] The infrared event frame outputted by S102 includes target events, dynamic background events and noise events. In order to facilitate subsequent target event feature extraction, event frame preprocessing is required.

[0077] Establish a (2r+1)×(2r+1) filter window S with pixel (x,y) as the center x,y (r), where r is the radius of the filter window and the coordinates of the window center are (r+1,r+1). Construct the spatial domain filter operator:

[0078]

[0079] Where (u,v)∈S x,y (r). x,y The event polarity p in (r) is weighted:

[0080]

[0081] Calculate the 1-norm ||J(x,y)||1. In this embodiment, the threshold J is set th =7. If the 1-norm is lower than the threshold, that is, ||J(x,y)||1<J th , then filter out the event and set p(x,y) = 0. By traversing each pixel, remove the noise event.

[0082] S104, target event feature extraction;

[0083] S104.1 Extracting Potential Target Events Using Optical Flow

[0084] Optical flow characterizes the instantaneous speed of the pixel movement of an object on the observation plane, so it is possible to distinguish target events from noise events based on the difference in motion characteristics. The optical flow is estimated for the event frame to obtain the amplitude and direction of the optical flow. The maximum inter-class variance algorithm is used to binarize the amplitude and direction of the optical flow, and a new template is obtained after calculating the union. The centroid of the template is solved to extract the center of the event area, which is the potential target event.

[0085] S104.2 Constructing the objective function using the HOG operator

[0086] Calculate the HOG operator around the feature points extracted by S104.1, that is, the size and direction of the polarity change:

[0087] G x (x,y)=p(x+1,y)-p(x-1,y)

[0088] G y (x,y)=p(x,y+1)-p(x,y-1)

[0089] In the formula, G x and G y They are the horizontal gradient and vertical gradient of the event polarity. The grayscale gradient magnitude G and direction α at the pixel are:

[0090]

[0091] Set (w cell × cell ). In this embodiment, it is necessary to detect small-sized targets, and let w cell =5. Divide the gradient direction into n-channel histograms for each cell, the degree of each channel is π / n, and n=9 is set in this embodiment. The integral histogram of the gradient direction is calculated weighted by the gradient size. The amplitude of the feature point (x, y) in the channel bin is calculated as follows:

[0092]

[0093] Where 1≤i≤n,i∈S. Calculate the amplitude and value of the feature point in all channels:

[0094]

[0095] The number of channels that satisfy the non-zero amplitude is calculated as |A|, where:

[0096] A={i∈S|V i (x,y)≠0}

[0097] Construct the objective function Con based on the extracted HOG features HOG :

[0098] Con HOG =k1|A|+k2V

[0099] Where k1 and k2 are linear coefficients. The objective function is calculated for each feature point and then normalized. In order to avoid repeated extraction of feature points in the target event area, non-maximum suppression is used to remove redundancy.

[0100] S105, removing abnormal events;

[0101] S105.1 Time-space correlation to remove abnormal events

[0102] Select the current time t k , the previous moment t k-1 and the next moment t k+1 The feature point set of the current moment u=(x,y) is used to calculate the spatiotemporal correlation. The Euclidean distance between the feature point u=(x,y) at the current moment and the feature points at the previous and next moments is searched respectively:

[0103] d k,k-1 =||u k ,u k-1 ||2,d k,k+1 =||u k ,u k+1 ||2

[0104] In this embodiment, the distance weighted function is defined as:

[0105]

[0106] The current feature point u k The spatiotemporal correlation coefficient between all feature points in the previous and next frames is defined as:

[0107] Con ST (k) = ∑[con ST (d k,k-1 )+con ST (d k,k+1 )]

[0108] Set the threshold interval of the spatiotemporal correlation coefficient [Con L ,Con H ], if it exceeds this interval, the corresponding feature points are filtered out;

[0109] S105.2 Filtering abnormal events based on statistics

[0110] The feature point coordinate sequence extracted by S105.1 is arranged into four equal quartiles, and the first quartile (25%) is defined as Q1, and the third quartile (75%) is defined as Q3. The expression of the interquartile range is as follows:

[0111] IQR=Q3-Q1

[0112] If IQR>0, the value outside the interval [Q1-k×IQR, Q3+k×IQR] is defined as an abnormal value, and k is a coefficient. ST To set k, if Con ST <0.6, k = 0.6; if Con ST>0.9, k=0.8; otherwise k=0.7. If IQR=0, calculate the mean μ and standard deviation σ of the characteristic coordinate sequence, defined in the interval |(u-μ) / σ|>Z th The values ​​outside of this range are considered abnormal values ​​and are filtered out. In this embodiment, Z th is 1. The feature points obtained after outlier removal are the centroid coordinates of the target event. The detection results are as follows Figure 3 As shown in (c) in .

[0113] Simulation environment: MATLAB R2021b.

[0114] Test input: infrared image sequence of space-based downward-looking aerial target detection scene.

[0115] Output results: infrared differential event frame sequence, target event center of mass coordinates.

[0116] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the technical solution of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. An adaptive differential event detection method for infrared moving targets, characterized in that: It includes the following steps: S101, initializing the current frame and threshold; S102, infrared differential event detection with adaptive threshold; S103, infrared event frame preprocessing; S104, target event feature extraction; S105. Remove abnormal events.

2. The adaptive differential event detection method for infrared moving targets according to claim 1 is characterized in that: In S101, an infrared image is acquired as input using an infrared camera, and a current frame and a threshold are initialized; The infrared time-domain differential event trigger mechanism is defined as outputting an event when the radiation change of each pixel exceeds the contrast threshold. The initial scene is set as the first reference frame for judging the event triggering, and the threshold at the initial moment is set as a constant, and then positive feedback adjustment is performed according to the trigger rate.

3. The adaptive differential event detection method for infrared moving targets according to claim 2 is characterized in that: In S102, the infrared differential event frame sequence is generated; S102.1 Trigger events and calculate trigger rates The polarity of the infrared differential event indicates the direction of the time domain change of the infrared radiation, that is, the direction of the grayscale change of the input image; the grayscale of the current frame is differentiated from the grayscale of the reference frame. If the grayscale increment of the pixel exceeds the threshold, the pixel independently triggers a positive polarity event p(x, y) = 1; if the grayscale reduction exceeds the threshold, the pixel triggers a negative polarity event p(x, y) = -1; otherwise, the event p(x, y) = 0 is not triggered; the pixel that triggers the event updates the grayscale reference value to the grayscale value at the current moment, otherwise, the reference value maintains the grayscale value at the previous moment; According to the number of trigger events n event Calculate the trigger rate as a ratio of the array size mm×nn: S102.2 Adaptive adjustment of contrast threshold Determine whether the event trigger rate R exceeds the preset range [R L ,R H ]; if R>R H , then the contrast threshold C is increased linearly; if R<R L , the contrast threshold is linearly reduced; the threshold range and linear change coefficient can be set according to the empirical value; the updated contrast threshold is the trigger threshold of the next event frame, and the contrast threshold is adjusted by the new trigger rate, and the process is repeated until the last frame scene, and the positive and negative polarity events are integrated into the infrared differential event frame sequence and then output.

4. The adaptive differential event detection method for infrared moving targets according to claim 3 is characterized in that: In S103, a spatial domain filtering operator is constructed to remove noise events, and the output of S102 is used as the input of S103; Establish a (2r+1)×(2r+1) filter window S with pixel (x,y) as the center x,y (r), where r is the radius of the filter window and the coordinates of the window center are (r+1,r+1); construct the spatial domain filter operator: Where (u,v)∈S x,y (r); for S x,y The event polarity p in (r) is weighted: Calculate the 1-norm ||J(x,y)||1, if ||J(x,y)||1<J th , then p(x,y)=0; remove noise events by traversing each pixel.

5. The adaptive differential event detection method for infrared moving targets according to claim 4 is characterized in that: Extracting the faint target centroid from the pure infrared event as described in S104, and using the output of S103 as the input of S104; S104.1 Extracting Potential Target Events Using Optical Flow The optical flow of the event frame is estimated, and the amplitude and direction of the optical flow are binarized using the maximum inter-class variance algorithm. The new template is obtained after the union is calculated. Solve the centroid of the template to obtain the center of the event area, that is, the potential target event; S104.2 Constructing the objective function using the HOG operator Calculate the HOG operator around the feature points extracted by S104.1: G x (x,y)=p(x+1,y)-p(x-1,y) G y (x,y)=p(x,y+1)-p(x,y-1) In the formula, G x and G y are the horizontal and vertical gradients of event polarity respectively; the gradient magnitude G and direction α are: Set (w cell × cell ) rectangular cells; divide the n-channel histogram of the gradient direction for each cell, the degree of each channel is π / n, and the integral histogram of the gradient direction is calculated weighted by the gradient size; the amplitude of the feature point (x, y) in the channel bin is as follows: Where 1≤i≤n,i∈S; calculate the amplitude and value of the feature point in all channels: The number of channels that satisfy the non-zero amplitude is calculated as |A|, where: A={i∈S|V i (x,y)≠0} Construct the objective function Con HOG for: Con HOG =k1|A|+k2V Where k1 and k2 are linear coefficients respectively. After calculating the objective function for each feature point, normalization is performed and non-maximum suppression is used to remove redundant feature points.

6. The adaptive differential event detection method for infrared moving targets according to claim 5 is characterized in that: In S105, the characteristic events extracted in S104 are screened to remove noise events and dynamic background events; S105.1 Time-space correlation to remove abnormal events Select the current time t k , the previous moment t k-1 and the next moment t k+1 Calculate the spatiotemporal correlation of the feature point set; The Euclidean distance between the current feature point u=(x,y) and the feature points before and after is searched: d k,k-1 =||u k ,u k-1 ||2,d k,k+1 =||u k ,u k+1 ||2 Define the distance weighting function as con ST , where r1, r2, r3 and r4 are distance parameters: Define the feature point u at the current moment k The spatiotemporal correlation coefficient between all feature points in the previous and next frames is: Con ST (k)=∑[con ST (d k,k-1 )+con ST (d k,k+1 )] If the threshold interval of the spatiotemporal correlation coefficient [Con L ,Con H ], then filter out the corresponding feature points; S105.2 Filtering abnormal events based on statistics The output coordinates of S105.1 are sorted and divided into four equal quartiles. The first quartile (25%) is defined as Q1, the third quartile (75%) is defined as Q3, and the expression of the interquartile range is as follows: IQR=Q3-Q1 If IQR>0, the values ​​outside the interval [Q1-k×IQR,Q3+k×IQR] are defined as outliers, and k is the coefficient; if IQR=0, the mean μ and standard deviation σ of the characteristic coordinate sequence are calculated, defined in the interval (u-μ) / σ|>Z th The values ​​outside are outliers and are filtered out; the feature points obtained after outlier removal are the centroid coordinates of the target event.

Images