Multi-source track data fusion method and system based on LSTM network

Abstract

The invention discloses a multi-source track data fusion method and system based on an LSTM network, and is applied to the technical field of multi-source track fusion. Comprising the following steps of performing time registration on track points, constructing an input sequence of LSTM, calculating correlation probability between radar original track points and existing fusion track points through the LSTM, classifying the radar original track points, performing quality management on a track classification result based on an average correlation probability, and performing quality management on the track classification result by fusing data of a single target and a plurality of radars. A future state of the target is estimated using a filtering algorithm. According to the method, the LSTM algorithm is introduced, the contextual information of the time sequence data is used for capturing the continuity of the target in time and the relevance of the target in space, the accuracy of track relevance is effectively improved, the model is used for track identity judgment and track quality management, and therefore the performance of a track fusion system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source track fusion, and more specifically to a multi-source track data fusion method and system based on LSTM network. Background Art

[0002] Track fusion system is a system used to integrate target detection information from multiple sensors (such as radars) to generate unified target tracking information. Its main purpose is to judge the identity of independent track data from multiple radars or sensors, that is, track association, and generate fused tracks to obtain more accurate and comprehensive target track information, improve tracking accuracy and system robustness. Track association is one of the key technologies of multi-sensor information fusion. The correctness of track association directly affects the function and performance of the track fusion system. At present, track association methods can be divided into five categories according to different problem-solving frameworks: probability statistics-based association methods, uncertain information processing-based methods, target optimization constraint-based methods, signal processing-based methods, and target reference topology-based methods. However, in complex aerial target scenarios such as dense, crossed, and closely parallel, the accuracy of traditional track association has dropped significantly.

[0003] When multiple targets are densely distributed in the airspace, the tracks of different targets detected by each radar may overlap or be close. In this case, traditional track association methods may find it difficult to distinguish the tracks of different targets, resulting in association errors. When the tracks of targets intersect in the air, the track data returned by the radar may be staggered, and it is difficult for traditional algorithms to accurately determine which tracks belong to the same target. In scenarios where targets fly in parallel, the movement directions and speeds of different targets are very close, and it is difficult for traditional methods to maintain stable target distinction for a long time. Therefore, how to correctly associate the tracks of aerial targets in complex aerial target scenarios such as dense, intersecting, and closely parallel is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] In view of this, the present invention provides a multi-source track data fusion method and system based on LSTM network, introduces LSTM algorithm, uses the context information of time series data to capture the temporal continuity and spatial correlation of targets, accurately identifies and distinguishes the tracks of dense, crossed, and closely parallel targets, effectively improves the accuracy of track association, uses rule models to perform track identity judgment and track quality management, improves the track association accuracy of complex aerial target scenes such as dense, crossed, and closely parallel, thereby improving the performance of the track fusion system.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A multi-source track data fusion method based on LSTM network includes the following steps:

[0007] S1. Taking the time t of the original radar track point as the reference, the existing fused track point at time t-1 is extrapolated to time t, so that the original radar track point and the fused track point are comparable at the same time point;

[0008] S2, normalize the state vector of the radar original track point and the state vector of any track point in the fused track set, construct a time series containing both the radar original track point and the fused track point as the input sequence of LSTM, and traverse all the track points in the fused track set;

[0009] S3, calculate the probability of belonging to the same target through the correlation between the original track points of the LSTM radar and the existing fusion track points;

[0010] S4, classifying the radar original track points based on the relevant probability, the information of the radar original track points and the fused track points, and classifying the radar original track points as new fused tracks or fused tracks associated with the radar original tracks according to the relevant probability threshold;

[0011] S5, calculating the average probability of the sum of the correlation probabilities between multiple radar original track points of the target and the existing fused track points, if the average probability between the radar original track points in the radar original track of the target and the fused track points in a fused track is greater than a preset threshold, the current radar original track is associated with the fused track, if the average probability between the radar original track points in the radar original track of the target and the fused track points in all fused tracks are less than a preset threshold, then the radar original track point is judged to be the track point of the newly discovered target, a fused track file is established, and a new fused track is generated, and if there is no data update for r consecutive detection cycles, the track file is deleted;

[0012] S6. By fusing the data of multiple radars for a single target, a filtering algorithm is used to estimate the future state of the target.

[0013] Optional, the original radar track point collection in S1 is U l ={l1,l2,…,l m}, the fused track point collection is U l ={l1,l2,…,l m}, where n is the number of fused track points, m is the number of original radar track points, and the track point of fused track i at time t is The state vector of the radar l, the original track j at time t is The state vector of The existing fused track point at time t-1 Extrapolate to

[0014] Optionally, S2 specifically includes: original track point of radar The state vector and the fused track set U s Any track point in The state vector Normalization is performed to obtain the normalized state vector and For each element x in the track point state vector, its normalized value x′ is:

[0015]

[0016] Construct a time series containing both the original radar track points and the fused track points As the input sequence of LSTM:

[0017]

[0018] In the formula, is the number of radar scanning cycles contained in the track, T is the time step of one radar scanning cycle, s l (τ) is the input feature vector corresponding to time point τ, s l (τ) is the normalized radar original track point feature vector And the normalized fusion track point feature vector Combination composition:

[0019] Optional, S3 is:

[0020] S31. Process the constructed input sequence through the LSTM network to obtain the probability distribution

[0021]

[0022] In the formula, Indicates the original radar track point The fused track point with the i-th fused track The correlation probability between p and b p They are the weight matrix and bias vector of the Softmax layer respectively.

[0023] S32. Considering that relying solely on the probability threshold may lead to misjudgment of minority tracks, an entropy H(P) is used to measure the matching confidence between the radar original track points and the fused track points:

[0024]

[0025] Where H(P) represents the uncertainty of the correlation between the original radar track point and multiple fused track points. The larger the entropy value, the more uniform the distribution and the lower the confidence. The smaller the entropy value, the more likely it is that the probability is concentrated in a certain fused track point and the confidence is higher.

[0026] Optionally, S4 is specifically:

[0027] Based on the relevant probability, the preset threshold P th , entropy H(P), information of radar original track points and fused track points, classify the radar original track points. When the correlation probability between the radar original track points and the fused track points is less than the preset threshold P th Or there is no fused track, the track point is judged to be the track point of the newly discovered target, a fused track file is established, a new fused track is generated, and the current original track point is used as the track point of the new track; when the correlation probability between the radar original track point and the fused track point is greater than the preset threshold P th When considering the uncertainty of the track, the entropy H(P) of the relevant probability is calculated as an uncertainty measure to provide decision support. If the relevant probability P of the radar original track point is greater than the threshold P th And the entropy H(P) is less than the set entropy threshold H t , it is confirmed to be a fused track associated with the original radar track; otherwise, if the entropy H(P) is greater than H th , it indicates that the uncertainty of the track point is high and it is marked as a potential new target track.

[0028] Optionally, S6 is specifically as follows: by fusing the data of multiple radars of a single target, a filtering algorithm is used to generate the future state of the target, including the target position, direction of movement, and speed information. Specifically, if the fused track is a new fused track or the track points are less than 3, the original track information of the current radar is output, and the original track information of the current radar is used as the new track point of the associated fused track; if the fused track points are greater than 3, the track points before time t-1 of the fused track and the original track points of the current radar are subjected to maneuvering target filtering to generate a state vector at time t, and the state vector is used as the new track point of the associated fused track to replace the extrapolated track point at time t, and the new track point information of the fused track is output. Maneuvering target filtering includes interactive multi-model filtering algorithm, Singer model filtering algorithm, etc.

[0029] A multi-source track data fusion system based on LSTM network executes the above-mentioned multi-source track data fusion method based on LSTM network, including: a time registration module, a track probability calculation module, a track association management module, a track quality management module and a track state estimation module; the time registration module registers the fused track points, the track probability calculation module calculates the correlation probability between the radar original track points and the existing fused track points, the track association management module classifies the radar original track points, the track quality management module performs quality management on the classification results of the track, and the track state estimation module estimates the future state of the target.

[0030] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a multi-source track data fusion method and system based on LSTM network, which has the following beneficial effects:

[0031] 1. Improved accuracy of track association: The present invention uses the LSTM algorithm to calculate the correlation probability between the original radar track and the fused track, which greatly improves the accuracy of track association. It is more robust and accurate than traditional methods, enabling the system to provide more accurate target tracking information in complex environments;

[0032] 2. Real-time and high efficiency of the system: The present invention uses the LSTM algorithm to quickly process data from multiple radars and realize real-time track fusion. When the amount of track point data is large, the system can still maintain an efficient calculation speed to ensure the timeliness of the fusion results;

[0033] 3. System stability and reliability: The present invention adopts a track quality management mechanism. Through statistical analysis of relevant probabilities, it effectively prevents the impact of single relevant probability fluctuations, enhances the stability of the system in tracking targets, cleans up track data that has not been updated for a long time, avoids data redundancy, improves system resource utilization efficiency, and complies with engineering applications;

[0034] 4. Social and economic benefits: The application of this invention can significantly improve the efficiency and accuracy of the aerial target tracking system, and is applicable to the fields of national defense, air traffic management, etc. The efficient and accurate target tracking capability will help improve airspace safety, reduce accident risks, and thus bring significant social and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 It is a flow chart of the multi-source track data fusion method of the present invention;

[0037] Figure 2 This is a schematic diagram of track fusion deployment in an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of the LSTM network structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The embodiment of the present invention discloses a multi-source track data fusion method based on LSTM network. Figure 1 As shown, the following steps are included:

[0041] S1. Taking the time t of the original radar track point as the reference, the existing fused track point at time t-1 is extrapolated to time t, so that the original radar track point and the fused track point are comparable at the same time point;

[0042] S2, normalize the state vector of the radar original track point and the state vector of any track point in the fused track set, construct a time series containing both the radar original track point and the fused track point as the input sequence of LSTM, and traverse all the track points in the fused track set;

[0043] S3, calculate the correlation probability between the original radar track points and the existing fusion track points through LSTM to determine whether they belong to the same target;

[0044] S4, classifying the radar original track points based on the relevant probability, the information of the radar original track points and the fused track points, and classifying the radar original track points as new fused tracks or fused tracks associated with the radar original tracks according to the relevant probability threshold;

[0045] S5, calculating the average probability of the sum of the correlation probabilities between multiple radar original track points of the target and the existing fused track points, if the average probability between the radar original track points in the radar original track of the target and the fused track points in a fused track is greater than a preset threshold, the current radar original track is associated with the fused track, if the average probability between the radar original track points in the radar original track of the target and the fused track points in all fused tracks are less than a preset threshold, then the radar original track point is judged to be the track point of the newly discovered target, a fused track file is established, and a new fused track is generated, and if there is no data update for r consecutive detection cycles, the track file is deleted;

[0046] S6. By fusing the data of multiple radars for a single target, a filtering algorithm is used to estimate the future state of the target.

[0047] In the embodiment of the present invention, the deployment of radar is as follows: Figure 2 As shown in the figure, there are 6 radars, with the track fusion center as the center of the circle, and the distance between the track fusion center is no more than 5km. Each radar can independently detect multiple targets in the air. The radar scans the air targets according to the period T. Each detected target generates a data vector within the period, that is, a track point, including: target ID, distance, azimuth, pitch angle, detection time, etc. The set of track points of the target at each moment is called a track. Each radar can obtain its own geographical location through the positioning device, and send the target data and position data of each detection to the track fusion center through the communication network. The fusion center performs multi-radar multi-target track fusion.

[0048] In the embodiment of the present invention, S6 adopts one of an interactive multi-model filtering algorithm and a Singer model filtering algorithm.

[0049] Furthermore, the collection of radar original track points in S1 is U l ={l1,l2,…,l m}, the fused track point collection is U l ={l1,l2,…,l m}, where n is the number of fused track points, m is the number of original radar track points, and the track point of fused track i at time t is The state vector of the radar l, the original track j at time t is The state vector of The existing fused track point at time t-1

[0050] Extrapolate to

[0051] The input of time registration is a certain radar original track point, multiple or zero fused track points. Time registration is the prerequisite for subsequent track association. Only when the time is consistent can the correct association judgment be made between a certain radar original track point and a fused track point. The data after time registration is transmitted to the track correlation probability judgment and track association management.

[0052] Furthermore, S2 is specifically: for the original radar track point The state vector and the fused track set U s Any track point in The state vector Normalization is performed to obtain the normalized state vector and For each element x in the track point state vector, its normalized value x′ is:

[0053]

[0054] Since the elements in the state vectors of different track points may have different orders of magnitude, direct LSTM processing will cause the elements with larger values ​​to dominate the model training, which will cause the model to be biased towards these elements with larger values. In order to avoid the influence of element scale differences, normalization processing is required;

[0055] In order to make full use of the ability of LSTM to process sequence data, the dynamic relationship and evolution pattern between the original radar track and the fused track are captured to the maximum extent, and a time series containing both the original radar track points and the fused track points is constructed. As the input sequence of LSTM:

[0056]

[0057] In the formula, is the number of radar scan cycles contained in the track, s l (τ) is the input feature vector corresponding to time point τ, s l (τ) is the normalized radar original track point feature vector And the normalized fusion track point feature vector Combination composition:

[0058] In the embodiment of the present invention, the structure of the LSTM network is as follows: Figure 3As shown in the figure, the core components of the LSTM network are the memory cell (Cell State Update) and three gating mechanisms: Forget Gate, Input Gate, and Output Gate. Through these gating mechanisms, LSTM can effectively retain or discard historical information in the time series, thereby adapting to the modeling needs of long-term dependencies in the time series.

[0059] The forget gate determines which past information needs to be forgotten at the current moment. The input gate determines which new information needs to be written into the memory unit. Updating the state of the memory unit is the core of LSTM and is responsible for storing information. The output gate determines the hidden state at the current moment and the final output.

[0060] In this embodiment of the present invention, the loss function of LSTM is defined as a negative log-likelihood function:

[0061]

[0062] In the formula, is the indicator function, when the radar original track point With fused track points It takes 1 when they belong to the same target, otherwise it takes 0. The optimization process uses the back propagation algorithm to adjust the parameters of the model by minimizing the loss function to maximize the correlation probability between the original track point and its true corresponding fusion track point.

[0063] Furthermore, S3 is specifically:

[0064] S31, the hidden state h of the previous moment of the forget gate t-1 and the input sequence at the current time Perform weighted summation and use the Sigmoid activation function to achieve information forgetting and retention:

[0065] f t =σ(W f ·[h t-1 ,s l (t)]+b f )

[0066] Where W f is the weight matrix of the forget gate, b f is the bias vector of the forget gate, σ(·) represents the Sigmoid activation function;

[0067] S32, the input gate generates a gate variable i that determines the write ratio through the Sigmoid activation function t :

[0068] i t =σ(Wi ·[h t-1 ,s l (t)]+b i )

[0069] Generate a candidate memory content through a tanh activation function

[0070]

[0071] Where W i and b i are the weight matrix and bias vector of the input gate, W C and b C are the weight matrix and bias vector of the candidate memory content, respectively. tanh(·) represents the hyperbolic tangent activation function.

[0072] S33, update the memory unit to store the new memory unit state C t :

[0073]

[0074] S34, the output gate generates the control variable o through the Sigmoid function t :

[0075] o t =σ(W o ·[h t-1 ,s l (t)]+b o )

[0076] Combined memory cell state C t , after scaling by the tanh function, we get the hidden state h t :

[0077] h t =o t ⊙tanh(C t )

[0078] Where W o and b o They are the weight matrix and bias vector of the output gate respectively;

[0079] S35, through a Softmax layer h t Adjusted to a probability distribution, it measures the possibility that the original radar track point and the fused track point belong to the same target:

[0080]

[0081] In the formula, Indicates the original radar track point The fused track point with the i-th fused track The correlation probability between p and b p They are the weight matrix and bias vector of the Softmax layer respectively.

[0082] Further, S4 is specifically: based on the relevant probability and the preset threshold P th , entropy H(P), information of radar original track points and fused track points, classify the radar original track points. When the correlation probability between the radar original track points and the fused track points is less than the preset threshold P th Or there is no fused track, the track point is judged to be the track point of the newly discovered target, a fused track file is established, a new fused track is generated, and the current original track point is used as the track point of the new track; when the correlation probability between the radar original track point and the fused track point is greater than the preset threshold P th When considering the uncertainty of the track, the entropy H(P) of the relevant probability is calculated as an uncertainty measure to provide decision support. If the relevant probability P of the radar original track point is greater than the threshold P th And the entropy H(P) is less than the set entropy threshold H t , it is confirmed to be a fused track associated with the original radar track; otherwise, if the entropy H(P) is greater than H th , it indicates that the uncertainty of the track point is high and it is marked as a potential new target track.

[0083] Furthermore, S6 is specifically as follows: by fusing the data of multiple radars of a single target, a filtering algorithm is used to generate the future state of the target, including the target position, direction of movement, and speed information. Specifically, if the fused track is a new fused track or the track points are less than 3, the original track information of the current radar is output, and the original track information of the current radar is used as the new track point of the associated fused track; if the fused track points are greater than 3, the track points before time t-1 of the fused track and the original track points of the current radar are subjected to maneuvering target filtering to generate a state vector at time t, and the state vector is used as the new track point of the associated fused track to replace the extrapolated track point at time t, and the new track point information of the fused track is output. Maneuvering target filtering includes interactive multi-model filtering algorithm, Singer model filtering algorithm, etc.

[0084] and Figure 1Corresponding to the method described, an embodiment of the present invention further discloses a multi-source track data fusion system based on an LSTM network, which executes the above-mentioned multi-source track data fusion method based on an LSTM network, including: a time alignment module, a track probability calculation module, a track association management module, a track quality management module and a track state estimation module; the time alignment module aligns the fused track points, the track probability calculation module calculates the correlation probability between the radar original track points and the existing fused track points, the track association management module classifies the radar original track points, the track quality management module performs quality management on the classification results of the track, and the track state estimation module estimates the future state of the target.

[0085] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0086] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-source track data fusion method based on LSTM network, characterized in that: The following steps are involved: S1. Taking the time t of the original radar track point as the reference, the existing fused track point at time t-1 is extrapolated to time t, so that the original radar track point and the fused track point are comparable at the same time point; S2, normalize the state vector of the radar original track point and the state vector of any track point in the fused track set, construct a time series containing both the radar original track point and the fused track point as the input sequence of LSTM, and traverse all the track points in the fused track set; S3, using LSTM to determine the correlation between the original radar track points and the existing fused track points, and calculate the probability of whether they are the same target; S4, classifying the radar original track points based on the relevant probability, the information of the radar original track points and the fused track points, and classifying the radar original track points as new fused tracks or fused tracks associated with the radar original tracks according to the relevant probability threshold; S5, calculating the average probability of the sum of the correlation probabilities between multiple radar original track points of the target and the existing fused track points, if the average probability between the radar original track points in the radar original track of the target and the fused track points in a fused track is greater than a preset threshold, the current radar original track is associated with the fused track, if the average probability between the radar original track points in the radar original track of the target and the fused track points in all fused tracks are less than a preset threshold, then the radar original track point is judged to be the track point of the newly discovered target, a fused track file is established, and a new fused track is generated, and if there is no data update for r consecutive detection cycles, the track file is deleted; S6. By fusing the data of multiple radars for a single target, a filtering algorithm is used to estimate the future state of the target.

2. According to a multi-source track data fusion method based on LSTM network according to claim 1, it is characterized in that: The collection of original radar track points in S1 is U l ={l1,l2,…,l m }, the fused track point collection is U l ={l1,l2,…,l m }, where n is the number of fused track points, m is the number of original radar track points, and the track point of fused track i at time t is The state vector of the radar l, the original track j at time t is The state vector of The existing fused track point at time t-1 Extrapolate to 3. The multi-source track data fusion method based on LSTM network according to claim 1 is characterized in that: S2 is specifically: for the original radar track point The state vector and the fused track set U s Any track point in The state vector Normalization is performed to obtain the normalized state vector and For each element x in the track point state vector, its normalized value x′ is: Construct a time series containing both the original radar track points and the fused track points As the input sequence of LSTM: In the formula, is the number of radar scan cycles contained in the track, s l (τ) is the input feature vector corresponding to time point τ, s l (τ) is the normalized radar original track point feature vector And the normalized fusion track point feature vector Combination composition:

4. The multi-source track data fusion method based on LSTM network according to claim 1 is characterized in that: S3 is specifically: S31. Process the constructed input sequence through the LSTM network to obtain the probability distribution In the formula, Indicates the original radar track point The fused track point with the i-th fused track The correlation probability between p and b p They are the weight matrix and bias vector of the Softmax layer respectively. S32. Considering that relying solely on the probability threshold may lead to misjudgment of minority tracks, an entropy H(P) is used to measure the matching confidence between the radar original track points and the fused track points: Where H(P) represents the uncertainty of the correlation between the original radar track point and multiple fused track points. The larger the entropy value, the more uniform the distribution and the lower the confidence. The smaller the entropy value, the more likely it is that the probability is concentrated in a certain fused track point and the confidence is higher.

5. The multi-source track data fusion method based on LSTM network according to claim 1 is characterized in that: S4 is specifically: based on the relevant probability, the preset threshold P th , entropy H(P), information of radar original track points and fused track points, classify the radar original track points. When the correlation probability between the radar original track points and the fused track points is less than the preset threshold P th Or there is no fused track, the track point is judged to be the track point of the newly discovered target, a fused track file is established, a new fused track is generated, and the current original track point is used as the track point of the new track; when the correlation probability between the radar original track point and the fused track point is greater than the preset threshold P th When considering the uncertainty of the track, the entropy H(P) of the relevant probability is calculated as an uncertainty measure to provide decision support. If the relevant probability P of the radar original track point is greater than the threshold P th And the entropy H(P) is less than the set entropy threshold H t , it is confirmed to be a fused track associated with the original radar track; otherwise, if the entropy H(P) is greater than H th , it indicates that the uncertainty of the track point is high and it is marked as a potential new target track.

6. The multi-source track data fusion method based on LSTM network according to claim 1 is characterized in that: S6 specifically involves fusing data from multiple radars of a single target and using a filtering algorithm to generate the future state of the target, including the target position, movement direction, and speed information.

7. A multi-source track data fusion system based on LSTM network, characterized in that: A multi-source track data fusion method based on an LSTM network is implemented as described in any one of claims 1-6, comprising: a time alignment module, a track probability calculation module, a track association management module, a track quality management module and a track state estimation module; the time alignment module aligns the fused track points, the track probability calculation module calculates the correlation probability between the radar original track points and the existing fused track points, the track association management module classifies the radar original track points, the track quality management module performs quality management on the classification results of the tracks, and the track state estimation module estimates the future state of the target.

Images