Asynchronous robust track association method based on time coding and graph representation

Abstract

This invention relates to an asynchronous robust track association method based on time coding and graph representation, belonging to the field of distributed radar data processing technology. It includes an intra-track representation module and an inter-track representation module. The intra-track representation module extracts spatiotemporal features between different points within a track, while the inter-track representation module extracts geometric features between tracks from the same sensor within the track representation space. This method establishes a general model for generalized track association tasks. Through end-to-end training, it can simultaneously solve multi-source track association and interrupted track association using a single model, and exhibits better generalization performance and association accuracy under asynchronous and systematic bias conditions.

Description

Technical Field

[0001] This invention relates to an asynchronous robust track association method based on time coding and graph representation, belonging to the field of distributed radar data processing technology. Background Technology

[0002] Target trajectory association is a key technology in distributed fusion systems, aiming to determine whether trajectories reported by different sensors originate from the same target. Accurate trajectory association is fundamental to trajectory segment completion and homogeneous trajectory fusion, improving the accuracy and efficiency of multi-source situational awareness. Ultimately, it yields a clear, unified, and complete multi-source fused situational awareness. However, in complex electromagnetic environments, timestamps of different trajectories are often asynchronous due to missed detections, sampling frequency, startup time, communication delays, and time synchronization errors. Furthermore, target measurements may contain random noise and constant or asymptotic systemic biases due to calibration deviations and platform attitude errors. The combination of these factors reduces the effectiveness of trajectory association and limits the quality of high-level situational awareness fusion. Therefore, in multi-source distributed fusion systems, trajectory association faces challenges due to the combined effects of random noise, systemic bias, and asynchronicity.

[0003] Currently, a large amount of research has been conducted on the track association problem. For asynchronous tracks, time alignment techniques are often used to obtain synchronous data through fitting, extrapolation, or prediction to meet the requirements of fusion centers. However, due to additional errors caused by target maneuvers and alignment performance, it increases system complexity and reduces fusion quality. Related methods directly use asynchronous tracks for association, but the constructed track similarity measurement function is not unbiased for asynchronous track data from the same source, has high computational complexity, and performs poorly in association for short tracks. For synchronous track association with systematic bias, better association performance has been achieved by manually designing track features robust to systematic bias. However, these features are easily affected by measurement noise and lack the ability to actively extract and filter effective features. Currently, there is a lack of representative research results on deep learning-based track association methods, and even more lack of systematic research results on asynchronous track and robust association problems. Summary of the Invention

[0004] This invention addresses the generalized track association problem under the combined influence of random noise, systematic bias, and asynchronicity, proposing an asynchronous robust track association method based on time coding and graph representation. This method establishes a general model for the generalized track association task. Through end-to-end training, it can simultaneously solve multi-source track association and interrupted track association using a single model, exhibiting better generalization performance and association accuracy under asynchronous and systematic bias conditions.

[0005] The asynchronous robust track association method based on time coding and graph representation of the present invention is characterized by including: an intra-track representation module and an inter-track representation module, wherein the intra-track representation module is used to extract the spatiotemporal features between different points within the track, and the inter-track representation module is used to extract the geometric features between tracks of the same sensor within the track representation space, specifically including the following steps:

[0006] Step 1: Apply domain normalization to the track point data;

[0007] Step 2: Simulate asynchronous association scenarios using various dot masking strategies;

[0008] Step 3: Input the dot masked dots into the track representation module to obtain the track representation;

[0009] Step 4: Input the intra-track representation into the inter-track representation module to obtain the inter-track representation;

[0010] Step 5: Calculate the loss function and train the network model;

[0011] Step 6: After the network model is trained, the similarity between tracks is calculated directly using the output track representations, and track association is completed using methods such as nearest neighbor or optimal decision.

[0012] Preferably, the calculation method for applying domain normalization to the track point data in step 1 is as follows:

[0013] In a two-dimensional scene, the k-th point trace of target i of sensor s is represented as: This is the timestamp corresponding to the point, where For location field information, If the information is in the velocity domain, then the position domain is normalized to...

[0014]

[0015]

[0016] Where x avg y avg x std and y std p represents the average and standard deviation of the two sensors in the position dimension. std It is the standard deviation of the positional domain.

[0017] Preferably, the dot masking strategy in step 2 includes:

[0018] (1) Random dot mask, which covers each dot in the training dataset with a given probability, to simulate asynchronous tracks caused by missed detections;

[0019] (2) Asynchronous track masking: n consecutive points of synchronous track are regarded as a group of points, which can be divided into multiple groups of points, each containing n points. In the asynchronous test dataset, for sensor 1, the last m-1 points of each group of points are masked. At the same time, for each group of points in sensor 2, except for the m-th point, other track points are masked. When m=1, they are synchronous tracks. When m=2,...,n, they are asynchronous tracks. The closer m is to (n+2) / 2, the greater the time difference and the degree of asynchrony. It is used to simulate asynchronous tracks caused by different power-on times.

[0020] (3) Discontinuous point masking: In each synchronized track pair, 2n+m consecutive points are randomly selected and other points are masked. Then, in sensor 1, the first n+m points of the selected points are masked, and in sensor 2, the last n+m points are masked. After the above processing, the same source tracks do not overlap in the time domain, and the time interval between tracks is m points. The larger m is, the greater the difficulty of discontinuous association. This is used to simulate the discontinuous association scenario of tracks.

[0021] Preferably, the track-based representation module in step 3 includes three steps: time encoding, point feature extraction, and track feature extraction.

[0022] Step 3.1: Ignoring sensor tag s and target tag i, the time encoding of the k-th track point is represented as follows:

[0023]

[0024] Here, f is a hyperparameter, considered as the fundamental frequency. f should be taken as a large value, and f < 1 / T w d model It is the dimension of time encoding, d = 1, 2, ..., d model / 2;

[0025] Step 3.2: Use a single fully connected layer to directly extract point features; multiple target points can be processed in parallel.

[0026] Step 3.3: After concatenating the time code and the point features, input them into the recurrent neural network to extract features and obtain the in-track representation.

[0027] Preferably, the inter-track representation module in step 4 includes two steps: track map construction and inter-track feature extraction.

[0028] Step 4.1: Treat the in-track representation of each track as one node. All target tracks observed by a sensor constitute the node set. Construct a directed graph using the K-nearest neighbor (KNN) method with thresholds;

[0029] Step 4.2: After constructing the track map, graph convolution is used to obtain inter-track features. Various graph convolutions can be used. After obtaining the inter-track features, multiple 1x1 point convolutions are used to further expand the output dimension, enrich the inter-track feature extraction, and obtain the final inter-track representation. LayerNorm is used to normalize the entire feature map between convolutional layers, and the ReLU function is used to improve the non-linear fitting ability of the model.

[0030] Preferably, step 4.1 specifically involves: calculating the Euclidean distance between each node and other nodes in the representation space; selecting nodes with distances less than a threshold as candidate nodes; if the number of candidate nodes is greater than K, establishing edges between the K candidate nodes with the smallest distances; if the number of candidate nodes is less than or equal to K, establishing edges with each candidate node; and adding self-loops to each node to obtain the directed trajectory map for each sensor. in Let be the set of edges.

[0031] Preferably, the method for calculating the loss function in step 5 is as follows:

[0032]

[0033] Where margin > 0 and It is a representation of the track from different sensors, track j + It is a co-originating track of track i. It is the heterogeneous track with the smallest distance to track i, N pair It represents the number of homologous track pairs.

[0034] This invention presents an asynchronous robust track association method based on time coding and graph representation, supporting the design of neural network models for generalized track association tasks, such as multi-source track association and track segment association. The proposed intra-track representation module embeds timestamps into the intra-track representation through constructed time coding, enabling the model to simultaneously solve multi-source asynchronous track association and track segment association tasks without any time alignment. The proposed inter-track representation module uses a thresholded KNN to build a track graph in the track representation space and extracts inter-track representations based on graph convolution and point convolution. Finally, track association is achieved by directly calculating the distance between inter-track representations. Domain normalization and various point masking strategies are used before data input to enhance the model's generalization performance. The proposed method exhibits good performance in asynchronous track robust association scenarios. Attached Figure Description

[0035] Figure 1 The flowchart of the asynchronous robust track association method based on time coding and graph representation of this invention is shown below. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] Combination Figure 1 The flowchart illustrating the implementation of this invention provides a further detailed description of the invention.

[0038] This embodiment presents an asynchronous robust track association method based on time encoding and graph representation. The overall framework is a dual-twin network structure, primarily comprising an intra-track representation module and an inter-track representation module. The intra-track representation module extracts spatiotemporal features between different points within a track, while the inter-track representation module extracts geometric features between tracks from the same sensor within the track representation space. The intra-track representation module includes three steps: time encoding, point feature extraction, and track feature extraction. By embedding timestamps into point features, asynchronous intra-track representations are obtained, enabling the model to solve generalized track association tasks, such as multi-source track association and discontinuous track association. The inter-track representation module includes track graph construction and inter-track feature extraction. Track graph construction uses a thresholded K-order nearest neighbor method to construct a track graph in the track representation space, followed by graph convolution and multiple point convolutions to obtain inter-track representations. Finally, the inter-track representations are directly used to calculate the similarity between tracks, completing the track association process. Furthermore, domain normalization and dot masking are applied before data input to enhance the model's generalization performance for various asynchronous correlation scenarios.

[0039] Consider two sensor systems that can detect and track multiple targets separately in a distributed architecture. After stable tracking, sensor system s obtains the target trajectory i as T. i s ,s=1,2,i=1,2,…,N s , where N s This represents the number of target tracks observed by sensor s. Each track consists of multiple trackpoints, i.e. in The track point is at the timestamp The state of the object, including its position, speed, and even acceleration or attribute information. It is the length of the flight path.

[0040] An asynchronous robust track association method based on time coding and graph representation includes the following steps:

[0041] Step 1: Apply domain normalization to the track point data;

[0042] Track point data generally includes multiple domains such as position, velocity, and attribute, each with a different value range and physical meaning, and should be normalized separately. Taking a two-dimensional scene as an example, the k-th point of target i on sensor s is represented as... in For location field information, This refers to velocity domain information. Furthermore, for different dimensions within the same domain, such as... and After normalization, the point sequence should possess translation and rotation invariance to accommodate possible track data augmentation methods (such as rotation). Therefore, data from different dimensions within the same domain should be scaled at the same scale. Thus, Domain Normalization (DN) is proposed to preprocess the data.

[0043] First, the mean and standard deviation of the target state are calculated across different dimensions. Then, the mean and standard deviation are fused according to the domain. Finally, the target track data is normalized for each dimension based on the standard deviation of the domain and the mean of the dimensions. Position domain normalization can be described as follows.

[0044]

[0045]

[0046] where x avg y avg x std and y std p represents the average and standard deviation of the two sensors in the position dimension. std This is the standard deviation for the position domain. Normalization for other domains is similar to that for the position domain. Domain-normalized track data are consistent across different dimensions within the same domain, which helps improve the model's generalization performance.

[0047] Step 2: Simulate asynchronous association scenarios using various dot masking strategies;

[0048] By employing different point masking strategies to simulate different asynchronous scenarios, the point masking of the track can also reduce the number of points in the track and improve the training speed.

[0049] (1) Random dot mask, which masks each dot in the training dataset with a given probability, to simulate asynchronous tracks caused by missed detections.

[0050] (2) Asynchronous track masking: n consecutive points of a synchronous track are considered as a group of points, which can be divided into multiple groups. Each group contains n points. In the asynchronous test dataset, for sensor 1, the last m-1 points of each group are masked. Meanwhile, for each group of points in sensor 2, all track points except the m-th point are masked. When m=1, they are synchronous tracks; when m=2,…,n, they are asynchronous tracks. The closer m is to (n+2) / 2, the greater the time difference and the degree of asynchrony, used to simulate asynchronous tracks caused by different power-on times.

[0051] (3) Discontinuous point masking: In each synchronized track pair, 2n+m consecutive points are randomly selected, and the other points are masked. Then, in sensor 1, the first n+m points of the selected points are masked, and in sensor 2, the last n+m points are masked. After the above processing, the same source tracks do not overlap in the time domain, and the time interval between tracks is m points. The larger m is, the greater the difficulty of discontinuous correlation, which is used to simulate the scenario of discontinuous track correlation.

[0052] Step 3: Input the dot masked dots into the track representation module to obtain the track representation;

[0053] The proposed in-track representation module includes three steps: time encoding, point feature extraction, and track feature extraction.

[0054] 3.1 The time code consists of sine and cosine functions of different frequencies. The time code for the k-th track point is...

[0055]

[0056] Where f is a hyperparameter, which can be considered as the fundamental frequency, and d model It is the dimension of time encoding, d = 1, 2, ..., d model / 2.

[0057] To ensure the effectiveness of time encoding, on the one hand, the associated window T w The time code within the time frame should not repeat; on the other hand, the fundamental frequency f should not be too small to flatten the time code. Therefore, f should be a large value, and f < 1 / T. w By embedding time coding, the model can solve generalized track association tasks, such as multi-source track association and discontinuous track association.

[0058] 3.2 Point feature extraction: Point features are extracted directly using a single fully connected layer, and multiple target points can be processed in parallel.

[0059] 3.3 Track feature extraction: The time code and the point features are concatenated and then input into the recurrent neural network to extract features to obtain the track representation. Since the position and motion information in the point and the asynchronous time information in the time code are embedded, the obtained track representation contains rich asynchronous track spatiotemporal information.

[0060] Step 4: Input the intra-track representation into the inter-track representation module to obtain the inter-track representation;

[0061] The track representation module in step 4 includes two steps: track map construction and track feature extraction.

[0062] Step 4.1: Treat the in-track representation of each track as one node. All target tracks observed by a sensor constitute the node set. Construct a directed graph using the K-nearest neighbor (KNN) method with thresholds;

[0063] Step 4.2: After constructing the track map, graph convolution is used to obtain inter-track features. Various graph convolutions can be used. After obtaining the inter-track features, multiple 1x1 point convolutions are used to further expand the output dimension, enrich the inter-track feature extraction, and obtain the final inter-track representation. LayerNorm is used to normalize the entire feature map between convolutional layers, and the ReLU function is used to improve the non-linear fitting ability of the model.

[0064] Preferably, step 4.1 specifically involves: calculating the Euclidean distance between each node and other nodes in the representation space; selecting nodes with distances less than a threshold as candidate nodes; if the number of candidate nodes is greater than K, establishing edges between the K candidate nodes with the smallest distances; if the number of candidate nodes is less than or equal to K, establishing edges with each candidate node; and adding self-loops to each node to obtain the directed trajectory map for each sensor. in Let be the set of edges.

[0065] Step 5: Calculate the loss function and train the network model;

[0066] For the track association task, it is necessary to minimize the distance between the representations of similar tracks and maximize the distance between the representations of dissimilar tracks. To meet these requirements, a loss function is proposed as follows:

[0067]

[0068] Where margin > 0 and It is a representation of the track from different sensors, track j + It is a co-originating track of track i. It is the heterogeneous track with the smallest distance to track i, N pair It represents the number of homologous track pairs.

[0069] Step 6: After the network model is trained, the similarity between tracks is calculated directly using the output track representations, and track association is completed using methods such as nearest neighbor or optimal decision.

[0070] This invention presents an asynchronous robust track association method based on time coding and graph representation, supporting the design of neural network models for generalized track association tasks, such as multi-source track association and track segment association. The proposed intra-track representation module embeds timestamps into the intra-track representation through constructed time coding, enabling the model to simultaneously solve multi-source asynchronous track association and track segment association tasks without any time alignment. The proposed inter-track representation module uses a thresholded KNN to build a track graph in the track representation space and extracts inter-track representations based on graph convolution and point convolution. Finally, track association is achieved by directly calculating the distance between inter-track representations. Domain normalization and various point masking strategies are used before data input to enhance the model's generalization performance. The proposed method exhibits good performance in asynchronous track robust association scenarios.

Claims

1. An asynchronous robust track association method based on time coding and graph representation, characterized in that... It includes an intra-track representation module and an inter-track representation module. The intra-track representation module is used to extract the spatiotemporal features between different points within a track, and the inter-track representation module is used to extract the geometric features between tracks from the same sensor within the track representation space. Specifically, it includes the following steps: Step 1: Apply domain normalization to the track point data; Step 2: Simulate asynchronous association scenarios using various dot masking strategies; Step 3: Input the dot masked dots into the track representation module to obtain the track representation; Step 4: Input the intra-track representation into the inter-track representation module to obtain the inter-track representation; Step 5: Calculate the loss function and train the network model; Step 6: After the network model is trained, the similarity between tracks is calculated directly using the output track representations, and track association is completed using the nearest neighbor or optimal decision method. The track-based representation module in step 3 includes three steps: time encoding, point feature extraction, and track feature extraction. Step 3.1: Ignore sensor tags and target tags , No. The time code for each waypoint is represented as follows: ； in, It is a hyperparameter, considered as the fundamental frequency. The larger value should be taken, and , It is the dimension of time encoding. ; Step 3.2: Extract point features directly using a single fully connected layer, and perform parallel computation on multiple target point traces; Step 3.3: After concatenating the time code and the point features, input them into a recurrent neural network to extract features and obtain the in-track representation; The track representation module in step 4 includes two steps: track map construction and track feature extraction. Step 4.1: Treat the in-track representation of each track as one node. All target tracks observed by a sensor constitute the node set. A directed graph is constructed using the K-nearest neighbor (KNN) method with thresholds; Step 4.2: After constructing the track map, graph convolution is used to obtain inter-track features. Various graph convolutions are used. After obtaining the inter-track features, multiple 1x1 point convolutions are used to further expand the output dimension, enrich the inter-track feature extraction, and obtain the final inter-track representation. LayerNorm is used to normalize the entire feature map between convolutional layers, and the ReLU function is used to improve the non-linear fitting ability of the model.

2. The asynchronous robust track association method based on time coding and graph representation according to claim 1, characterized in that... The calculation method for applying domain normalization to the track point data in step 1 is as follows: In a two-dimensional scene, the sensor goal The The trace of a dot is represented as ,in For location field information, If the information is in the velocity domain, then the position domain is normalized as follows: ； in , , , It is the average value and standard deviation of the two sensors in the position dimension. It is the standard deviation of the position domain. For sensors Number of observed target tracks It is the length of the flight path.

3. The asynchronous robust track association method based on time coding and graph representation according to claim 1, characterized in that... The dot masking strategy described in step 2 includes: (1) Random dot mask, which covers each dot in the training dataset with a given probability, to simulate asynchronous tracks caused by missed detection; (2) Asynchronous track masking: n consecutive points of a synchronous track are considered as a group of points, and are divided into multiple groups, each containing n points. In the asynchronous test dataset, for sensor 1, the last m-1 points of each group are masked. At the same time, for each group of points in sensor 2, except for the m-th point, other track points are masked. When m=1, they are synchronous tracks; when m=2, ... When n is equal to n, they are asynchronous tracks. The closer m is to (n+2) / 2, the greater the time difference and the degree of asynchrony. This is used to simulate asynchronous tracks caused by different power-on times. (3) Discontinuous point masking: In each synchronized track pair, 2n+m consecutive points are randomly selected and other points are masked. Then, in sensor 1, the first n+m points of the selected points are masked, and in sensor 2, the last n+m points are masked. After the above processing, the same source tracks do not overlap in the time domain, and the time interval between tracks is m points. The larger m is, the greater the difficulty of discontinuous association. It is used to simulate the discontinuous association scenario of tracks.

4. The asynchronous robust track association method based on time coding and graph representation as described in claim 1, characterized in that... Step 4.1 specifically involves: calculating the Euclidean distance between each node and other nodes in the representation space; selecting nodes with distances less than a threshold as candidate nodes; if the number of candidate nodes is greater than K, establishing edges between the K candidate nodes with the smallest distances; if the number of candidate nodes is less than or equal to K, establishing edges with each candidate node and adding self-loops to each node to obtain the directed trajectory map for each sensor. ,in Let be the set of edges.

5. The asynchronous robust track association method based on time coding and graph representation as described in claim 1, characterized in that... The method for calculating the loss function in step 5 is as follows: ； in, , and It is a representation of the track between different sensors, track It is a flight path The same origin of the trajectory, Is it to the flight path? The alien trajectory with the shortest distance It represents the number of homologous track pairs.

Images