Spatial weak target identification and tracking method based on event stream quasi-gray features

Abstract

The invention discloses a spatial weak target identification and tracking method based on event stream quasi-gray features. The method comprises the following steps: firstly, collecting asynchronous event stream data of a moving target by using an event camera, and filtering by using an event denoising method based on a sliding window to remove noisy points in the data; secondly, on the basis of event stream data characteristics, constructing a quasi-gray model, counting the spatial distribution of the frequency of event data on a phase plane, and extracting a centroid point of a target cluster by using a Gaussian mixture model added with a quasi-gray characteristic value; and finally, tracking the motion of a target point between continuous frames by using a Hungary matching algorithm based on event stream data, thereby realizing detection and tracking of a moving target. According to the method designed by the invention, the target quasi-gray feature is constructed through binary data expression of the event stream, and the mass center point is solved by using the Gaussian mixture model added with the quasi-gray feature value, so that the recognition capability of the event camera on a weak moving point target is improved, and the method has great potential in capturing and tracking application of a maneuvering target.

Description

Technical Field

[0001] The present invention relates to the field of space target recognition, and specifically, to a method for recognizing and tracking space weak targets based on event stream pseudo-gray features. Background Art

[0002] Currently, in space-based target detection systems, visual sensors are commonly used to analyze the collected continuous frame image sequences to determine information such as the motion speed, direction, and trajectory of the target, so as to realize the analysis and understanding of the target's motion behavior. Especially when recognizing highly maneuverable targets, due to the limited sampling frequency of image frames, it is easy to cause the loss of target information between frames; if the frame rate is increased, since each frame image contains a large amount of redundant information, the computational burden is increased; at the same time, the irradiation background in the space environment is complex, and due to the dim target and background stray light interference, it is often impossible to effectively recognize space targets. It can be seen that when using traditional frame image sensors to detect space maneuvering targets, there are mainly deficiencies such as the loss of information changes between frames, large redundancy of interference information, and limited dynamic range. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for recognizing and tracking space weak targets based on event stream gray fitting, aiming to improve the tracking accuracy and efficiency of high-dynamic and weak targets in a space environment with stray light interference.

[0004] The technical solution for realizing the purpose of the present invention is as follows:

[0005] A method for recognizing and tracking space weak targets based on event stream pseudo-gray features, comprising the following steps:

[0006] (1) Using an event camera to collect asynchronous event stream data of a high-speed moving target;

[0007] (2) Reading the original data generated by the event camera in step (1), and using an event denoising method based on a sliding window for filtering to remove the noise points in the data;

[0008] (3) Counting the frequency of occurrence of each event point in the data after the filtering process in step (2), constructing a pseudo-gray model, using a Gaussian mixture model to process the event stream data containing pseudo-gray features, and using the expectation maximization algorithm to estimate the parameters of the Gaussian mixture model to obtain the centroid points of each target cluster;

[0009] (4) Using the Hungarian algorithm to match the centroid points of the target clusters in the continuous time frames obtained in step (3);

[0010] First, calculate the Manhattan distance between all pairs of points, and store these distances in the distance cost matrix D t1t2 , and use the Hungarian algorithm for the distance matrix Dt1t2 Process it to find the optimal centroid matching scheme; after completion of the matching, if there are some centroid points that fail to be successfully matched, further process these unmatched centroid points according to the matching result.

[0011] Compared with the prior art, the present invention has the following remarkable advantages: (1) A new bionic neural vision sensor (i.e., event camera) is adopted to detect and track space maneuvering targets. Different from traditional frame cameras that capture images at a fixed speed and output a strobe-synchronized image sequence, the event camera measures the brightness change of each pixel asynchronously based on scene dynamics, rather than sampling the absolute brightness within a fixed clock, and the output event stream encodes the time, position, and polarity of the brightness change. This enables the event camera to achieve a microsecond-level time resolution for information acquisition of maneuvering space targets, solving the problem of information loss between frames caused by insufficient traditional image frame rate. At the same time, since the event camera is only sensitive to light intensity changes, the output event stream only contains information about moving targets, reducing the data volume by nearly 60% compared with traditional frame images, reducing the power consumption of information transmission, and improving the efficiency of data processing. Moreover, the dynamic range of the event camera is as high as 120 dB, enhancing the recognition ability of space dim targets and the inclusiveness of the stray light environment. It can be seen that the application of the new event camera will contribute to improving the ability to detect space targets in complex environments. (2) Focus on the centroid extraction part in space target detection. Since the event frame only shows whether each pixel has triggered an event within the integration time. In order to extract useful visual information from these binary events, it is necessary to accumulate all the events at the same pixel point within a certain time window and count the number of times they occur. Then, by normalizing, the gray value of each pixel is assigned to simulate a traditional gray image, and based on this, the feature points required for target recognition are extracted, the target recognition algorithm is optimized, and its performance in complex backgrounds and high-dynamic weak target conditions is improved. This quasi-gray technology can not only effectively utilize the advantages of the high time resolution and dynamic range of the event camera, but also provide a novel solution for visual information processing in dynamic and high-speed scenarios. Description of the Drawings

[0012] Figure 1 It is a flow chart of a method for recognizing space weak targets based on event stream quasi-gray features of the present invention.

[0013] Figure 2 It is a schematic diagram of the event trigger count mapping of the present invention.

[0014] Figure 3 It is a histogram of the trigger count of a star point collected by the present invention within a certain time.

[0015] Figure 4 It is a schematic diagram of a turntable experimental device used in the present invention. Detailed Embodiment

[0016] The present invention proposes an algorithm for spatial weak target recognition and tracking based on event stream pseudo - gray - scale features. This algorithm successfully simulates a traditional gray - scale image by statistically counting the occurrence frequencies of events within a certain time window and mapping these frequencies to different gray - scale values. It uses the Gaussian mixture model to obtain the centroids of each target cluster and adopts a feature - point matching algorithm based on the Hungarian matching algorithm to achieve precise matching of targets between consecutive frames. The method of the present invention shows significant superiority in dealing with high - dynamic and fast - changing scenarios. The effectiveness of this algorithm in terms of accuracy and processing speed can provide strong technical support for the future application of event cameras in dynamic environments.

[0017] A method for spatial weak target recognition and tracking based on event stream pseudo - gray - scale features of the present invention includes the following steps:

[0018] (1) Using an event camera to collect high - frequency asynchronous event stream data of a high - speed moving target;

[0019] (2) Reading the event stream data collected by the event camera in step (1), and using a sliding - window - based event denoising method for filtering to remove the noise points in the data;

[0020] (3) Counting the occurrence frequencies of each event point in the data after filtering in step (2), constructing a pseudo - gray - scale model, processing it using the Gaussian mixture model, and using the expectation - maximization algorithm to estimate the parameters of the Gaussian mixture model to obtain the centroid of each target cluster;

[0021] (4) Using the Hungarian algorithm to match the centroid points of the target clusters in consecutive time frames obtained in step (3). First, calculate the Manhattan distance between all point pairs, store these distances in the distance cost matrix and use the Hungarian algorithm to process the distance matrix to find the optimal centroid matching scheme. After completion of the matching, there may be some centroid points that fail to be successfully matched, and these unmatched centroid points are further processed according to the matching results;

[0022] Furthermore, in step (3), the statistical process of obtaining the occurrence frequency of each event point is as follows: where, δ(x - x i , y - y i ) is the Kronecker delta function, indicating that when the coordinates (x, y) and (x i , y i ) are the same, δ = 1, otherwise δ = 0.

[0023] The calculation method of the gray - scale value is as follows:

[0024]

[0025] Among them, max_count represents the maximum frequency of occurrence among all coordinate points in the entire dataset. When the frequency of a certain point is 1, in order to facilitate differentiation from the background, a fixed gray value of 200 is directly assigned to it; for points with a frequency greater than 1, the method of linear interpolation is used, such that the higher the frequency, the smaller the gray value.

[0026] In step (3), for the event stream e within a period of time i =[x i , y i , p i , t] data, the Gaussian mixture model algorithm is used for processing. Among them, e i ∈E n (n = 1, 2…, N), E n is the set of events within a fixed period of time, n is the index of the set, and i is the index of the event within the fixed period of time (i = 1, 2,…, n). w i =[x i , y i , g i T is the trigger pixel coordinate of the event and its corresponding pseudo-gray value. According to the definition of the multi-dimensional Gaussian distribution, for any pixel w in the three-dimensional sample space E n , the probability density function is: i In the formula: μ is the mean vector of the three-dimensional Gaussian distribution, Σ is the 3×3 covariance matrix, p(w) is the probability density function. The Gaussian distribution is determined by the mean vector μ for its position, and the covariance matrix Σ determines the size, direction, and specific shape of the three-dimensional Gaussian distribution.

[0027]

[0028] The Gaussian mixture model can be regarded as the fusion of multiple single Gaussian models. Assuming that there are multiple targets in the event-based data, the projection of each target on the three-dimensional plane conforms to the Gaussian distribution, and the probability density functions of k Gaussian distributions are linearly combined and added to obtain the Gaussian mixture model:

[0029]

[0030]

[0031] Among them, μ j and Σ j are the mean vector and covariance matrix of the j-th Gaussian distribution (j = 1, 2,…, k), a j >0 is the mixing coefficient of the j-th Gaussian distribution, and the sum of all a j is 1:

[0032] For the event w i ​i The posterior probability of the j-th distribution is as follows:

[0033]

[0034] p(j|w i ) is the posterior probability that the event w i is generated by the j-th Gaussian distribution, denoted here as γ ij (j = 1, 2, …, k). When the Gaussian mixture distribution is known, GMM divides E n into k Gaussian distributions C = {C 1 , C 2 , …, C k}, and the Gaussian distribution label λ i of each event w i is determined as follows:

[0035] λ i = argmax γ ij · j ∈ {1, 2, ..., k}

[0036] In the formula, the event w i with the maximum posterior probability is assigned to the k-th Gaussian distribution C k .

[0037] The parameters μ j , Σ j , a j are updated using maximum likelihood estimation, and are obtained from the Gaussian distribution label and γ ij = p(j|w i ) as follows:

[0038]

[0039] From the above derivation, it is the EM algorithm of the Gaussian mixture model: In each iteration, first calculate the posterior probability γ ij (j = 1, 2, …, k) (E-step) of each event belonging to each Gaussian distribution according to the current parameters, and then update the model parameters μ j , Σ j , a j , (j = 1, 2, …, k) (M-step).

[0040] After solving the expectation of the objective function and performing expectation maximization iterations, the finally obtained mean vector matrix μ is in the form of:

[0041]

[0042] where each row represents the mean vector of the Gaussian distribution of a target cluster obtained by the Gaussian density probability model algorithm, (x j , yj ) represents the centroid of the feature points of the target cluster that conforms to the j-th Gaussian distribution model.

[0043] Further, in step (4), the calculation method of the Manhattan distance between all pairs of points is as follows: Where, and respectively represent the abscissa and ordinate of the centroid , and and represent the coordinates of the centroid .

[0044] In order to register two sets of coordinates at different times, these distances are stored in the distance cost matrix , and the cost matrix is:

[0045]

[0046] Where m and n respectively represent the number of feature points in the feature point set of the event frame at time t 2 and time t 1 , and represents the Manhattan distance between the i-th centroid and the j-th centroid .

[0047] Use the Hungarian algorithm to process the distance matrix to find the optimal centroid matching scheme. The Hungarian algorithm determines the best matching scheme by minimizing the total distance as:

[0048]

[0049] Further, in step (4), the specific method of further processing is as follows: all centroids of the latter frame have been matched: if there are unmatched centroids in the previous frame, insert these centroids into the corresponding positions of the latter frame, and fill in the empty coordinates [0, 0] as placeholders at the unmatched positions; all centroids of the previous frame have been matched: if there are unmatched centroids in the latter frame, directly retain these unmatched centroids without additional processing; if the centroids in both frames have been successfully matched, no additional operation is required, and the matching result is directly saved.

[0050] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0051] The embodiment is a further description of the invention and a concretization of the invention's technical solution. In many cases, this part of the content is repeated with other parts, but the content of the embodiment is more detailed. At least one example of implementing the invention should be provided. The embodiment should describe in detail and specifically all the necessary conditions required for ordinary technicians in this profession to implement and reproduce the invention, such as parameters, materials, equipment, tools, etc., as well as necessary specifications and models. If new substances or self-prepared materials are used, their manufacturing methods should also be described. When describing the specific structure, each component should be marked with the same Figure 1 Unique mark.

[0052] The embodiments are often the result of further refinement of the technical solutions recorded in the claims. Please make sure that ordinary technicians in this profession can reproduce the technical solutions of the present invention based on the contents of the specific implementation methods.

[0053] See also Figure 1 , which is a specific implementation process of an algorithm for spatial weak target recognition and tracking based on event stream quasi-grayscale features disclosed by the present invention.

[0054] The pseudo-grayscale feature model used in the present invention is different from the traditional event frame and event surface diagram. Both methods abandon the density information of the event flow in three-dimensional space. Within the set integration time, the same pixel point in the phase plane will be triggered multiple times. However, in the pseudo-grayscale feature model, within the set integration time, the event triggering density of each pixel point in the image plane is different. Figure 2 , the red pixel grid in the image plane consists of four events, the yellow pixel grid consists of three pixels, the gray pixel grid consists of two pixels, and the blank pixel grid does not generate any events.

[0055] In order to intuitively represent the spatial distribution of events as an image, we take the event stream data of a certain period of time, integrate all the event points into the same image, and map different colors in the pixel unit according to the statistical number of events triggered by each pixel. Figure 3 , is a histogram of the number of triggers of a star point collected in this embodiment within a certain integration time. It can be found that it presents a Gaussian distribution, that is, the closer to the target center, the higher the number of event triggers. Considering that the number of triggers of spatial star point targets conforms to the Gaussian distribution, a Gaussian mixture model is used to extract the centroid point of the target cluster when performing target detection and recognition.

[0056] This embodiment also discloses an experimental device for spatial weak target recognition and tracking method based on event stream grayscale fitting, see Figure 4, the device fixes the event camera on a high-precision three-axis turntable through a special bracket, and fixes 8 small LED lights on the star chart board to simulate the faint target star points in space. The turntable is controlled to move only in the horizontal direction, and the rotation speeds are set to 10° / s and 20° / s respectively.

[0057] The above is a preferred embodiment of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. However, the embodiments of the present invention are not limited by the above content. For those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific embodiments and application scopes. Any changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A spatial weak target recognition and tracking method based on event stream quasi-grayscale features, characterized in that: The following steps are involved: (1) Using event cameras to collect asynchronous event stream data of high-speed moving targets; (2) reading the raw data generated by the event camera in step (1), filtering it using an event denoising method based on a sliding window to remove noise points in the data; (3) Counting the frequency of occurrence of each event point in the data after filtering in step (2), constructing a quasi-grayscale model, using a Gaussian mixture model to process the event stream data containing quasi-grayscale features, using an expectation maximization algorithm to estimate the parameters of the Gaussian mixture model, and obtaining the centroid of each target cluster; (4) using the Hungarian algorithm to match the centroid points of the target cluster in the continuous time frames obtained in step (3); First, calculate the Manhattan distance between all point pairs and store these distances in the distance cost matrix In the example, the Hungarian algorithm is used to calculate the distance matrix Processing is performed to find the optimal centroid matching solution; after the matching is completed, if there are some centroid points that have not been successfully matched, these unmatched centroid points are further processed according to the matching results.

2. The spatial weak target recognition and tracking method based on event stream grayscale fitting according to claim 1 is characterized in that: In step (3), the statistical process of obtaining the occurrence frequency of each event point is: Among them, δ(xx i ,yy i ) is the Kronecker delta function, which means that when the coordinates (x, y) and (x i ,y i ) are the same, δ=1, otherwise δ=0; The grayscale value is calculated as follows: Among them, max_count represents the maximum frequency of occurrence of all coordinate points in the entire data set.

3. The spatial weak target recognition and tracking method based on event stream grayscale fitting according to claim 1 is characterized in that: In step (3), the event stream e within a period of time is i =[x i ,y i ,p i ,t] data, and processed using Gaussian mixture model algorithm; where e i ∈E n , n=1,2…,N,E n is the set of events in a fixed time period, n is the index of the set, i is the index of the event in the fixed time period, i = 1, 2, ..., n; w i =[x i ,y i ,g i ] T is the trigger pixel coordinate of the event and its corresponding pseudo-grayscale value. According to the definition of multidimensional Gaussian distribution, the three-dimensional sample space E n Any pixel w in i The probability density function is: Where: μ is the mean vector of the three-dimensional Gaussian distribution, Σ is the 3×3 covariance matrix, p(w) is the probability density function, the Gaussian distribution is located by the mean vector μ, and the covariance matrix Σ determines the size, direction, and specific shape of the three-dimensional Gaussian distribution.

4. The spatial weak target recognition and tracking method based on event stream grayscale fitting according to claim 1 is characterized in that: In step (3), the method for obtaining the centroid of each target cluster is as follows: the Gaussian mixture model can be regarded as the fusion of multiple single Gaussian models. Assuming that there are multiple targets in the event-based data, the projection of each target on the three-dimensional plane conforms to the Gaussian distribution, and the linear combination of the probability density functions of k Gaussian distributions is added to obtain the Gaussian mixture model: Among them, μ j With Σ j is the mean vector and covariance matrix of the jth Gaussian distribution j=1,2,…,k, a j >0 is the mixing coefficient of the jth Gaussian distribution, and all a j Adding to 1: Event i The posterior probability of the jth distribution is: p(j|w i ) is the event w i The posterior probability generated by the j-th Gaussian distribution is denoted by γ ij (j=1,2,…,k); When the Gaussian mixture distribution is known, GMM will E n Divided into k Gaussian distributions C = {C1, C2, ..., C k }, each event w i Gaussian distribution label λ i Determined by: l i =argmaxγ ij ·j∈{1,2,...,k} In the formula, the event w with the maximum posterior probability is selected i Divide into the kth Gaussian distribution C k middle; Use maximum likelihood estimation to update the parameter μ j ,Σ j , a j , with Gaussian distribution labels and γ ij =p(j|w i )get: The above derivation is the EM algorithm of the Gaussian mixture model: in each iteration, the posterior probability γ of each event belonging to each Gaussian distribution is calculated based on the current parameters. ij (j=1,2,…,k)(E-step), and then update the model parameters μ j ,Σ j ,a j ,(j=1,2,…,k)(M-step); After solving the objective function expectation and expectation maximization iteration, the final mean vector matrix μ is as follows: Each row represents the Gaussian distribution mean vector of a target cluster obtained by the Gaussian density probability model algorithm, (x j ,y j ) represents the centroid of the feature points of the target cluster that conforms to the j-th Gaussian distribution model.

5. The spatial faint target recognition and tracking method based on event stream grayscale fitting according to claim 1 is characterized in that: The Manhattan distance between all point pairs in step (4) is calculated as: in, and Represents the centroid The horizontal and vertical coordinates of and Represents the centroid The horizontal and vertical coordinates of The Hungarian algorithm is used to determine the optimal centroid matching solution by minimizing the total distance, specifically:

6. The spatial weak target recognition and tracking method based on event stream grayscale fitting according to claim 1, characterized in that: In the step (4), all the centroids of the subsequent frame have been matched: if there are unmatched centroids in the previous frame, these centroids are inserted into the corresponding positions of the subsequent frame, and empty coordinates [0,0] are filled in the unmatched positions as placeholders; all the centroids of the previous frame have been matched: if there are unmatched centroids in the subsequent frame, these unmatched centroids are directly retained without additional processing; if the centroids in the two frames have been successfully matched, the matching results are directly saved.

Images