Automatic augmentation method for track data for track association tasks

Abstract

This invention relates to an automatic trajectory data augmentation method for trajectory association tasks, belonging to the field of distributed radar data processing technology. First, a situational representation framework is constructed, using a point representation module, a trajectory representation module, and a situational representation module to sequentially extract target point representation, target trajectory representation, and target situational representation. Second, for synchronous trajectory association tasks, augmentation operation sets are designed at the point layer, trajectory layer, and situational layer, including copying, deleting, rotating, translating, and scaling. The target situational representation vectors from the two sensors are concatenated and input into the trajectory data augmentation operation selection module. Based on a reinforcement learning framework, the method trains and learns augmentation operations and corresponding magnitudes that can improve the performance of the trajectory association network model, achieving automatic trajectory data augmentation. This provides richer and more diverse data samples for the network model for trajectory association tasks, improving the association performance of the deep learning-based trajectory association network model.

Description

Technical Field

[0001] This invention relates to an automatic trajectory data augmentation method for trajectory association tasks, belonging to the field of distributed radar data processing technology. Background Technology

[0002] Training a neural network-based dual-sensor trajectory association model requires a large amount of multi-source, simultaneous spatial trajectory data. This data is relatively scarce and suffers from imbalances in quantity. Currently, there is still a lack of systematic data augmentation methods.

[0003] Accurate target trajectory association is a crucial prerequisite for distributed multi-source fusion, aiming to determine whether trajectories reported by different sensors originate from the same target. Precise trajectory association forms the basis for trajectory segment completion and homogeneous trajectory fusion, improving the accuracy of multi-source situational awareness, enhancing recognition efficiency, and ultimately obtaining a clear, unified, and complete multi-source fused situational awareness. Existing trajectory datasets are mostly simulation or semi-simulation datasets, leading to overfitting issues in trajectory association network models trained on these datasets, resulting in poor adaptability to real-world association scenarios. This is mainly due to the limited factors considered in simulation data, mismatches between error and parameter settings and application scenarios, and imbalances in sample size regarding target motion patterns and target distribution density.

[0004] Data augmentation is an effective solution for improving data diversity. The field of computer vision offers numerous data augmentation methods, including cropping, translation, rotation, random noise, and random erasure. Different augmentation methods produce varying effects. To rationally integrate multiple augmentation methods and construct efficient augmentation strategies, several automatic augmentation methods have been proposed, including AutoAugment, Fast AutoAugment, RandAugment, and TrivialAugment. Similarly, in the text domain, there are word-oriented augmentation methods such as random swapping and random deletion, and sentence-oriented augmentation methods such as back-translation. Based on these, an automatic text augmentation method based on Fast AutoAugment has also been proposed. Currently, research on augmentation methods for track data is limited, and systematic solutions are lacking. In track association scenarios, tracks from the same source typically appear in pairs, and tracks consist of multiple point traces, fundamentally different from image and text data in terms of data structure. Furthermore, track association performance is affected by various factors such as sensor measurement errors, target location distribution, and sensor position. Therefore, a systematic solution for efficient data augmentation is needed. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the existing technology and provide an automatic trajectory data augmentation method for trajectory association tasks. By training and learning augmentation operations and corresponding magnitudes that can improve the performance of trajectory association network models, the method achieves automatic trajectory data augmentation, provides richer and more diverse data samples for network models for trajectory association tasks, and improves the association performance of deep learning-based trajectory association network models.

[0006] The present invention provides an automatic trajectory data augmentation method for trajectory association tasks, characterized by the following steps:

[0007] Step 1: Normalize the target situation data obtained from the two sensors. The normalized target situation data from the s-th sensor is S. s ={T i,s i = 1, ..., M s}, where M s T represents the number of targets observed by the s-th sensor, where s is 1 or 2, indicating the sensor number. i,s ={P k,i,s k = 1, ..., N i,s} represents the trajectory data of the i-th target in the s-th sensor, where N i,s P represents the number of points on the i-th target track in the s-th sensor. k,i,s For the target point data of the i-th target track in the s-th sensor, P k,i,s It includes information such as position, speed, and heading, with each dimension's information normalized to [-1.0, 1.0].

[0008] Step 2: Transfer the data P of each point to the database. k,i,s The input is fed into the point trace representation module to obtain the corresponding target point trace representation vector. The parameters of each dot representation module are shared, where k = 1, ..., N i,s , i = 1, ..., M s , s = 1, 2;

[0009] Step 3: Represent the multiple point trace vectors of each target trajectory in chronological order. The data is sequentially input into the trajectory representation module to obtain the target trajectory representation vector. The parameters of each trajectory representation module are shared, where i = 1, ..., M s , s = 1, 2;

[0010] Step 4: Represent the target trajectory vectors corresponding to the s-th sensor. As vertices, an undirected graph G is constructed using the K-nearest neighbor method. s ={V s Es}, where E s Let be the set of edges. Let V be the set of vertices. s The trajectory representation vector containing all targets observed by the s-th sensor The undirected graph G s The input is fed into the situation representation module based on a graph neural network to obtain the target situation representation vector. The parameters of each situation characterization module are shared with each other;

[0011] Step 5: Establish a track data augmentation operation selection module based on the time series depth model. This module selects augmentation operations sequentially from the situation layer data augmentation operation set, the track layer data augmentation operation set, and the point layer data augmentation operation set, selecting two augmentation operations each time. Each augmentation operation contains two values: the first value is the augmentation operation number, and the second value is the predicted augmentation operation magnitude. The predicted augmentation operation magnitude is then processed by the tanh activation function and mapped to the initial magnitude range of the corresponding augmentation operation.

[0012] Step 6: For any neural network model to be trained for the trajectory association task, referred to as the trajectory association network model, after extracting a batch of situational samples, select trajectory data augmentation operations for each situational sample data in the order of Step 1 to Step 5, and use these operations to perform data augmentation. Then, input the augmented data into the trajectory association network model to be trained. Use the loss value output by the trajectory association network model as the feedback signal of the trajectory data augmentation operation selection module. The purpose of the trajectory data augmentation operation selection module is to maximize the loss value of the task association model by selecting augmentation operations through training. After training converges, the optimized trajectory data augmentation operations and their corresponding action amplitudes are obtained.

[0013] Step 7: Enhance each situational data in the training dataset according to the optimized common track data augmentation operations and their corresponding action amplitudes, and use the augmented data to train the task model, thereby improving the correlation performance of the task model.

[0014] Preferably, the set of situational layer data augmentation operations in step 5 includes:

[0015] Add Gaussian white noise with a standard deviation equal to the amplitude to all target point data observed by the first sensor; add Gaussian white noise with a standard deviation equal to the amplitude to all target point data observed by the second sensor; add horizontal or vertical translation with an amplitude equal to the position of all target points observed by both sensors; add rotation with an amplitude equal to the position and heading of all target points observed by both sensors; add scaling with an amplitude equal to the position and velocity of all target points observed by both sensors. The initial amplitude range can be set to [0.0,0.1], [0.0,0.1], [-0.2,0.2], [-180,180] deg, [0.9,1.1] respectively, but the initial amplitude range is not limited to the above values.

[0016] Preferably, the set of track layer data augmentation operations in step 5 includes:

[0017] Randomly copy target tracks observed by any sensor according to the amplitude multiple of the total number of tracks; randomly delete target tracks observed by any sensor with a probability of amplitude value; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly translate their positions horizontally or vertically within the range of [-0.1, 0.1]; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly rotate their positions and headings within the range of [-180, 180] degrees; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly rotate their positions and velocities within the range of [0.9, 1.1]. Scaling is performed within a range, with initial amplitude ranges that can be set to [0.0, 5.0], [0.0, 1.0], [0.0, 1.0], [0.0, 1.0], and [0.0, 1.0], respectively. The initial amplitude range is not limited to the above values. For copied target tracks, translation, rotation, and scaling need to be added randomly. If a pair of source tracks is copied, the same translation, rotation, and scaling are added to the source track pair, and the corresponding label is added to the correct association label set. For deleted target tracks, if they belong to a pair of source tracks, the corresponding label is deleted from the correct association label set.

[0018] Preferably, the set of point layer data augmentation operations in step 5 includes: randomly adding Gaussian white noise with a standard deviation of amplitude to the target point data according to a given probability; randomly deleting the target point data with a probability of amplitude; and replacing the deleted target point data with the neighboring point data through linear interpolation or two-point linear extrapolation estimation. The initial amplitude range can be set to [0.0, 0.1] and [0.0, 0.2] respectively, and the initial amplitude range is not limited to the above values.

[0019] The present invention provides an automatic trajectory data augmentation method for trajectory association tasks, which has the following advantages:

[0020] 1. For synchronous track association tasks, enhanced operation sets are designed from the point layer, track layer and situation layer respectively. The simulation scenarios of different sensor performance, different sensor positions, different target position distribution, false alarms, missed alarms and target tracking omissions are simulated. The richness and complexity of target situation data samples are improved, which helps to improve the association performance of deep learning-based track association network models.

[0021] 2. Traditional methods of randomly selecting augmentation operations and randomly determining the magnitude of augmentation operations are inefficient. This paper proposes an automatic augmentation method for track data for track association tasks, which can automatically select augmentation operations that improve model performance and thus improve data augmentation efficiency. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the implementation of the automatic trajectory data augmentation method for trajectory association tasks according to the present invention.

[0023] Figure 2 A schematic diagram of the trajectory representation;

[0024] Figure 3 This is a schematic diagram of the situation. Detailed Implementation

[0025] To better understand and implement the invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not the entire structure.

[0026] Assume two sensor systems can detect and track multiple targets in a distributed architecture, forming target tracks which are then reported to a fusion center. The fusion center synchronizes all target track data observed by the two sensors in time, obtaining target situational data for track association tasks. A pair of identical track numbers serves as a correct association label. A set of correct association labels, comprising all correct association labels, is combined with the target situational data to form a data sample. Identical track pairs refer to target tracks belonging to the same target observed by both sensors. After accumulating multiple data samples, a target situational dataset for track association tasks is obtained.

[0027] The input samples of a deep learning-based track association network model (hereinafter referred to as the track association network model) for track association tasks are generally all target tracks observed by paired sensors in the same area within the same time period. The output association results are compared with the set of correctly associated labels, and the loss function is calculated. Through iterative training, the track association network model that minimizes the loss function is obtained, and the model can be used to complete the track association task. However, the association performance of the track association network model is related to the diversity of target situation data samples. The model trained with small samples has poor generalization performance. When sensor performance is different, sensor positions change, or common observation areas change, the association performance deteriorates significantly. In complex scenarios, there may also be problems such as false alarms, missed alarms, and missed target tracking. In the above situations, the samples are usually small, which further limits the performance improvement of the track association network model. Therefore, it is necessary to perform data augmentation operations on the existing target situation data samples to increase the diversity and complexity of the samples, thereby providing data support for improving the performance of the track association network model. For the common synchronous track association problem under the condition of only random error, augmentation operation sets can be designed from the situation layer, track layer, and point track layer respectively.

[0028] The set of situational data augmentation operations includes five augmentation operations:

[0029] (1) Add Gaussian white noise with a standard deviation of amplitude to all target point trace data observed by the first sensor. The initial amplitude range can be set to, but is not limited to, [0.0, 0.1].

[0030] (2) Add Gaussian white noise with a standard deviation of amplitude to all target point trace data observed by the second sensor. The initial amplitude range can be set to, but is not limited to, [0.0, 0.1].

[0031] (3) Add a horizontal or vertical translation of the magnitude of the target point traces observed by the two sensors to the position of all target points. The initial amplitude range can be set to, but is not limited to, [-0.2, 0.2].

[0032] (4) Add a rotation of magnitude to the position and heading of all target points observed by the two sensors. The initial amplitude range can be set to, but is not limited to, [-180, 180] deg.

[0033] (5) Add an amplitude scaling to the position and velocity of all target points observed by the two sensors. The initial amplitude range can be set to, but is not limited to, [0.9, 1.1].

[0034] The track layer data augmentation operation set includes 5 augmentation operations:

[0035] (1) Randomly copy the target trajectory observed by any sensor according to the amplitude multiple of the total number of trajectories. The initial amplitude range can be set to, but is not limited to, [0.0, 5.0].

[0036] (2) Randomly delete the target tracks observed by any sensor with the probability of amplitude. The initial amplitude range can be set to, but is not limited to, [0.0, 1.0].

[0037] (3) Randomly select a pair of co-originating or non-co-originating tracks with a probability of amplitude and randomly translate their positions in the horizontal or vertical direction within the range of [-0.1, 0.1]. The initial amplitude range can be set to, but is not limited to, [0.0, 1.0].

[0038] (4) Randomly select a pair of co-originating or non-co-originating tracks with probability based on amplitude, and randomly rotate their position and heading within the range of [-180, 180] deg. The initial amplitude range can be set to, but is not limited to, [0.0, 1.0].

[0039] (5) Randomly select homogeneous or non-homogeneous track pairs with probability based on amplitude value and randomly scale their positions and velocities within the range of [0.9, 1.1]. The initial amplitude range can be set to, but is not limited to, [0.0, 1.0].

[0040] Note that for copied target tracks, translation, rotation, and scaling need to be added randomly. If a pair of track pairs from the same source is copied, the same translation, rotation, and scaling are added to the track pair from the same source, and the corresponding labels are added to the set of correct associated labels. For deleted target tracks, if they belong to a pair of track pairs from the same source, the corresponding labels are deleted from the set of correct associated labels.

[0041] The set of data augmentation operations for the trace layer includes two augmentation operations:

[0042] (1) Add Gaussian white noise with a standard deviation of amplitude to the target point trace data randomly according to the given probability. The initial amplitude range can be set to, but is not limited to, [0.0, 0.1].

[0043] (2) Delete the target point data randomly with the amplitude as the probability and replace the deleted target point data with the neighboring point data through linear interpolation or two-point linear extrapolation estimation. The initial amplitude range can be set to, but is not limited to, [0.0, 0.2].

[0044] After obtaining the set of augmentation operations, multiple sets of data augmentation operations can be selected from different levels of data augmentation operation sets. An amplitude is randomly determined within the initial amplitude range, and augmentation operations with the corresponding amplitude are used to augment the target situation data sequentially in the order of situation layer, track layer, and point track layer. The set of correct associated labels is modified accordingly. Then, the augmented target situation data is used to train the track association network model, improving the performance of the model by increasing the diversity and complexity of the sample data. However, different augmentation operations and amplitudes have varying abilities to improve model performance. The traditional method of randomly selecting augmentation operations and randomly determining their amplitudes is inefficient and cannot maximize model performance. Therefore, an automatic track data augmentation method for track association tasks is proposed. This method first uses a graph neural network to extract the representation vector of the target situation data, and then uses reinforcement learning to automatically learn and determine the augmentation operations and corresponding amplitudes with high data augmentation efficiency. The specific method is as follows:

[0045] Step 1: First, normalize all target situation data in the target situation dataset to obtain the normalized target situation data of the s-th sensor as S. s ={T i,s i = 1, ..., M s}, where M s Let T be the number of targets observed by the s-th sensor, where s = 1, 2 represents the sensor number, and T is the number of targets observed by the s-th sensor. i,s ={P k,i,s k = 1, ..., N i,s} represents the trajectory data of the i-th target in the s-th sensor, where N i,s P represents the number of points on the i-th target track in the s-th sensor. k,i,s For the target point data of the i-th target track in the s-th sensor, P k,i,s It includes information such as position, speed, and heading, and the information in each dimension is normalized to [-1.0, 1.0].

[0046] Step 2: As Figure 2 As shown, each point data P k,i,s The input is fed into the point trace representation module to obtain the corresponding target point trace representation vector. The parameters of each dot representation module are shared, where k = 1. … N i,s i = 1, … M s , s=1,2.

[0047] Step 3: As Figure 2 As shown, multiple point traces of each target trajectory are represented by vectors in chronological order. The data is sequentially input into the trajectory representation module to obtain the target trajectory representation vector. The parameters of each trajectory representation module are shared, where i = 1, … M s , s=1,2.

[0048] Step 4: As Figure 3 As shown, the representation vectors of all target tracks corresponding to the s-th sensor are... As vertices, an undirected graph G is constructed using the K-nearest neighbor method. s ={V s E s}, where E s Let be the set of edges. Let V be the set of vertices. s The trajectory representation vector containing all targets observed by the s-th sensor The undirected graph G s The input is fed into the situation representation module based on a graph neural network to obtain the target situation representation vector. The parameters of each situation characterization module are shared.

[0049] Step 5: As Figure 1 As shown, a trajectory data augmentation operation selection module is established based on a time series depth model. This module selects augmentation operations sequentially from the situation layer data augmentation operation set, the trajectory layer data augmentation operation set, and the point trace layer data augmentation operation set, selecting two augmentation operations each time. Each augmentation operation contains two values: the first value is the augmentation operation number, and the second value is the predicted augmentation operation magnitude. The predicted augmentation operation magnitude is then processed by the tanh activation function and mapped to the initial magnitude range of the corresponding augmentation operation.

[0050] Step 6: As Figure 1 As shown, for any neural network model to be trained for the trajectory association task (hereinafter referred to as the trajectory association network model), after extracting a batch of situation samples, each situation sample data is selected for trajectory data augmentation operation in the order of steps 1 to 5, and these operations are used to perform data augmentation. Then, the augmented data is input into the trajectory association network model to be trained. The loss value output by the trajectory association network model is used as the feedback signal of the trajectory data augmentation operation selection module. The purpose of the trajectory data augmentation operation selection module is to maximize the loss value of the task association model by selecting augmentation operations through training. After the training converges, the optimized augmentation operations and their corresponding magnitudes are obtained.

[0051] Step 7: Enhance each situational data in the training dataset according to the optimized common track data augmentation operations and their corresponding action amplitudes, and use the augmented data to train the task model, thereby improving the correlation performance of the task model.

[0052] The above description is merely a preferred method of the present invention and is not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all embodiments here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A method for automatic augmentation of track data for track association tasks, characterized in that... Includes the following steps: Step 1: Normalize the target situation data obtained from the two sensors. The normalized target situation data from the s-th sensor is S. s ={T i,s i = 1, ..., M s }, where M s T represents the number of targets observed by the s-th sensor, where s is 1 or 2 and represents the sensor number. i,s ={P k,i,s k = 1, ..., N i,s } represents the trajectory data of the i-th target in the s-th sensor, where N i,s P represents the number of points on the i-th target track in the s-th sensor. k,i,s For the target point data of the i-th target track in the s-th sensor, P k,i,s It includes information such as position, speed, and heading, with each dimension's information normalized to [-1.0, 1.0]. Step 2: Transfer the data P of each point to the data P. k,i,s The input is fed into the point trace representation module to obtain the corresponding target point trace representation vector. The parameters of each dot representation module are shared, where k = 1, ..., N i,s , i = 1, ..., M s , s = 1, 2; Step 3: Represent the multiple point trace vectors of each target trajectory in chronological order. The data is sequentially input into the trajectory representation module to obtain the target trajectory representation vector. The parameters of each trajectory representation module are shared, where i = 1, … M s , s = 1, 2; Step 4: Represent the target trajectory vectors corresponding to the s-th sensor. As vertices, an undirected graph G is constructed using the K-nearest neighbor method. s ={V s E s }, where E s Let be the set of edges. Let V be the set of vertices. s The trajectory representation vector containing all targets observed by the s-th sensor The undirected graph G s The input is fed into the situation representation module based on a graph neural network to obtain the target situation representation vector. The parameters of each situation characterization module are shared with each other; Step 5: Establish a track data augmentation operation selection module based on the time series depth model. This module selects augmentation operations sequentially from the situation layer data augmentation operation set, the track layer data augmentation operation set, and the point layer data augmentation operation set, selecting two augmentation operations each time. Each augmentation operation contains two values: the first value is the augmentation operation number, and the second value is the predicted augmentation operation magnitude. The predicted augmentation operation magnitude is then processed by the tanh activation function and mapped to the initial magnitude range of the corresponding augmentation operation. Step 6: For any neural network model to be trained for the trajectory association task, referred to as the trajectory association network model, after extracting a batch of situational samples, select trajectory data augmentation operations for each situational sample data in the order of Step 1 to Step 5, and use these operations to perform data augmentation. Then, input the augmented data into the trajectory association network model to be trained. Use the loss value output by the trajectory association network model as the feedback signal of the trajectory data augmentation operation selection module. The purpose of the trajectory data augmentation operation selection module is to maximize the loss value of the task association model by selecting augmentation operations through training. After training converges, the optimized trajectory data augmentation operations and their corresponding action amplitudes are obtained. Step 7: Enhance each situational data in the training dataset according to the optimized common track data augmentation operations and their corresponding action amplitudes, and use the augmented data to train the task model, thereby improving the correlation performance of the task model.

2. The automatic trajectory data augmentation method for trajectory association tasks according to claim 1, characterized in that, The set of situational layer data augmentation operations mentioned in step 5 includes: Add Gaussian white noise with a standard deviation equal to the amplitude to all target point data observed by the first sensor; add Gaussian white noise with a standard deviation equal to the amplitude to all target point data observed by the second sensor; add horizontal or vertical translation with an amplitude equal to the position of all target points observed by both sensors; add rotation with an amplitude equal to the position and heading of all target points observed by both sensors; add scaling with an amplitude equal to the position and velocity of all target points observed by both sensors. The initial amplitude range can be set to [0.0,0.1], [0.0,0.1], [-0.2,0.2], [-180,180] deg, [0.9,1.1] respectively, but the initial amplitude range is not limited to the above values.

3. The automatic trajectory data augmentation method for trajectory association tasks according to claim 1, characterized in that, The set of track layer data augmentation operations mentioned in step 5 includes: Randomly copy target tracks observed by any sensor according to the amplitude multiple of the total number of tracks; randomly delete target tracks observed by any sensor with a probability of amplitude value; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly translate their positions horizontally or vertically within the range of [-0.1, 0.1]; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly rotate their positions and headings within the range of [-180, 180] degrees; randomly select homogeneous track pairs or heterogeneous tracks with a probability of amplitude value and randomly rotate their positions and velocities within the range of [0.9, 1.1]. Scaling is performed within a range, with initial amplitude ranges that can be set to [0.0, 5.0], [0.0, 1.0], [0.0, 1.0], [0.0, 1.0], and [0.0, 1.0], respectively. The initial amplitude range is not limited to the above values. For copied target tracks, translation, rotation, and scaling need to be added randomly. If a pair of source tracks is copied, the same translation, rotation, and scaling are added to the source track pair, and the corresponding label is added to the correct association label set. For deleted target tracks, if they belong to a pair of source tracks, the corresponding label is deleted from the correct association label set.

4. The automatic trajectory data augmentation method for trajectory association tasks according to claim 1, characterized in that, The set of data augmentation operations for the point trace layer in step 5 includes: randomly adding Gaussian white noise with a standard deviation of magnitude to the target point trace data according to a given probability; randomly deleting the target point trace data with a probability of magnitude; and replacing the deleted target point trace data with the neighboring point traces through linear interpolation or two-point linear extrapolation estimation. The initial magnitude range can be set to [0.0, 0.1] and [0.0, 0.2] respectively, and the initial magnitude range is not limited to the above values.

Images