Automatic track data enhancement method for track association task

Abstract

The invention relates to an automatic track data enhancement method for a track association task, and belongs to the technical field of distributed radar data processing. According to the method, firstly, a situation characterization framework is constructed, and target plot characterization, target track characterization and target situation characterization are sequentially extracted through a plot characterization module, a track characterization module and a situation characterization module. Secondly, aiming at a synchronous track association task, respectively designing enhanced operation sets from a trace point layer, a track layer and a situation layer, including copying, deleting, rotating, translating, zooming and the like; target situation representation vectors of the two sensors are connected and then input into a track data enhancement operation selection module, and on the basis of a reinforcement learning framework, automatic enhancement of track data is realized through an enhancement operation capable of improving the performance of a track association network model and a corresponding amplitude through training learning; richer and diverse data samples are provided for the network model facing the track association task, and the association performance of the track association network model based on deep learning is improved.

Description

Technical Field

[0001] The invention relates to an automatic track data enhancement method for a track association task, and belongs to the technical field of distributed radar data processing. Background Art

[0002] When training a dual-sensor track association model based on a neural network, a large amount of multi-source simultaneous and spatial track data is required. However, these data are relatively scarce and have problems such as imbalance in quantity. Currently, there is still a lack of systematic data enhancement methods.

[0003] Accurate target track association is an important prerequisite for distributed multi-source fusion, which aims to determine whether the tracks reported by different sensors come from the same target. Accurate track association is the basis of track segment completion and same-source track fusion. It improves the accuracy of multi-source situational awareness, improves recognition efficiency, and ultimately obtains a clear, unified, and complete multi-source fusion situation. Most of the existing track data sets are simulation data sets or semi-simulation data sets, which makes the track association network model trained based on these data have a certain degree of overfitting problem and poor adaptability to real association scenarios. The main reason is that there are fewer factors considered in the simulation data, or the error and parameter settings are mismatched with the application scenario, and the number of samples is unbalanced in terms of target motion mode, target distribution density, etc.

[0004] Data augmentation technology is an effective solution to improve data diversity. There are many data augmentation methods in the field of computer vision, including shearing, translation, rotation, random noise, random erasure, etc. Different augmentation methods have different effects. In order to reasonably integrate multiple augmentation methods and construct efficient augmentation strategies, a variety of automatic augmentation methods are further proposed, including AutoAugment, Fast AutoAugment, RandAugment, TrivialAugment, etc. Correspondingly, in the text field, there are word-oriented augmentation methods such as random exchange and random deletion, and sentence-oriented augmentation methods such as back translation. On this basis, a text automatic augmentation method is also proposed based on Fast AutoAugment. At present, there are few studies on augmentation methods for track data, and there is a lack of systematic solutions. In the track association scenario, homologous tracks usually appear in pairs, and the tracks are composed of multiple point tracks, which are essentially different from image data and text data in data structure. In addition, the track association performance is affected by multiple factors such as sensor measurement error, target position distribution, sensor position, etc. Therefore, it is necessary to propose a systematic solution for efficient data augmentation. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the above-mentioned prior art and provide a method for automatic enhancement of track data for track association tasks. By training and learning the enhancement operations and corresponding amplitudes that can improve the performance of the track association network model, the automatic enhancement of track data is realized, and more abundant and diverse data samples are provided for the network model for track association tasks, thereby improving the correlation performance of the track association network model based on deep learning.

[0006] The present invention provides a method for automatically enhancing track data for a track association task, which is special in that it comprises the following steps:

[0007] Step 1: Normalize the target situation data obtained by the two sensors. The normalized target situation data of the sth sensor is S s ={T i,s ,i=1,…,M s}, where M s is the number of targets observed by the sth sensor, s is 1 or 2, indicating the sensor number, T i,s = {P k,i,s ,k=1,…,N i,s} is the track data of the i-th target in the s-th sensor, where N i,s is the number of points of the ith target track in the sth sensor, P k,i,s is the kth target point track data of the ith target track in the sth sensor, P k,i,s Contains information such as position, speed, and heading. The information of each dimension is normalized to [-1.0, 1.0];

[0008] Step 2: Set each point data P k,i,s Input into the point trace representation module to obtain the corresponding target point trace representation vector The parameters of each point characterization module are shared with each other, where k = 1, ..., N i,s ,i=1,…,M s , s=1,2;

[0009] Step 3: Vectorize the multiple points of each target track in chronological order Input into the track characterization module in sequence to obtain the target track characterization vector The parameters of each track characterization module are shared with each other, where i = 1, ..., M s , s=1,2;

[0010] Step 4: Characterize the track vectors of all targets corresponding to the sth sensor As a vertex, an undirected graph G is established based on the K nearest neighbor method. s = {V s ,Es}, where E s is the set of edges, is the vertex set, V s Contains the track representation vector of all targets observed by the sth sensor The undirected graph G s Input into the situation representation module based on graph neural network to obtain the target situation representation vector The parameters of each situation representation module are shared with each other;

[0011] Step 5: Based on the time series deep model, a track data enhancement operation selection module is established. This module selects enhancement operations from the situation layer data enhancement operation set, the track layer data enhancement operation set, and the point track layer data enhancement operation set in turn. Two enhancement operations are selected each time. Each enhancement operation contains two values. The first value is the number of the enhancement operation, and the second value is the predicted enhancement operation amplitude. The predicted enhancement operation amplitude is mapped to the initial amplitude range of the corresponding enhancement operation after the tanh activation function.

[0012] Step 6: For any neural network model to be trained for track association tasks, referred to as track association network model, after extracting a batch of situation samples, select track data enhancement operations for each situation sample data in the order of steps 1 to 5, and use these operations to perform data enhancement, and then input the enhanced data into the track association network model to be trained, and use the loss value output by the track association network model as the feedback signal of the track data enhancement operation selection module. The purpose of the track data enhancement operation selection module is to make the selected enhancement operation through training so that the loss value of the task association model can be increased as much as possible. After the training converges, the optimized track data enhancement operation and its corresponding action amplitude are obtained;

[0013] Step 7: According to the optimized common track data enhancement operations and their corresponding motion amplitudes, each situation data in the training data set is enhanced, and the enhanced data is used to train the task model to improve the association performance of the task model.

[0014] Preferably, the situation layer data enhancement operation set in step 5 includes:

[0015] Gaussian white noise with a standard deviation of amplitude is added to all target point trace data observed by the first sensor, Gaussian white noise with a standard deviation of amplitude is added to all target point trace data observed by the second sensor, a horizontal or vertical translation of the amplitude is added to the position of all target point traces observed by the two sensors, a rotation of the amplitude is added to the position and heading of all target point traces observed by the two sensors, and a scaling of the amplitude is added to the position and speed of all target point traces observed by the two sensors. The initial amplitude range can be set to [0.0, 0.1], [0.0, 0.1], [-0.2, 0.2], [-180, 180] deg, [0.9, 1.1], respectively, and the initial amplitude range is not limited to the above values.

[0016] Preferably, the track layer data enhancement operation set in step 5 includes:

[0017] Randomly copy the target track observed by any sensor according to the amplitude times of the total number of tracks, randomly delete the target track observed by any sensor with the amplitude as probability, randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly translate their position horizontally or vertically in the range of [-0.1,0.1], randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly rotate their position and heading in the range of [-180,180]deg, randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly rotate their position and speed in the range of [0.9,1.1 ] range, the initial amplitude range can be set to [0.0,5.0], [0.0,1.0], [0.0,1.0], [0.0,1.0], [0.0,1.0] respectively, and the initial amplitude range is not limited to the above values. For the copied target tracks, translation, rotation and scaling need to be added randomly. If a homologous track pair is copied, the same translation, rotation and scaling are added to the homologous track pair, and the corresponding label is added to the correctly associated label set; for the deleted target track, if it belongs to the homologous track pair, the corresponding label is deleted from the correctly associated label set.

[0018] Preferably, the point layer data enhancement operation set in step 5 includes: randomly adding Gaussian white noise with a standard deviation of amplitude to the target point data with a given probability, randomly deleting the target point data with the amplitude as probability, and using adjacent points to replace the deleted target point data through linear interpolation or two-point linear extrapolation estimation. The initial amplitude range can be set to [0.0, 0.1], [0.0, 0.2] respectively, and the initial amplitude range is not limited to the above values.

[0019] The present invention provides a method for automatically enhancing track data for track association tasks, and the beneficial effects are as follows:

[0020] 1. Aiming at the synchronous track association task, enhanced operation sets are designed from the point track layer, track layer, and situation layer respectively, simulating scenarios with different sensor performance, different sensor positions, different target position distributions, false alarms, missed alarms, and target tracking omissions. This improves the richness and complexity of the target situation data samples, which helps to improve the association performance of the track association network model based on deep learning.

[0021] 2. The traditional method of randomly selecting enhancement operations and randomly determining the amplitude of enhancement operations is inefficient. An automatic track data enhancement method for track association tasks is proposed, which can automatically select enhancement operations that improve model performance and improve data enhancement efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of the implementation of the method for automatically enhancing track data for track association tasks of the present invention;

[0023] Figure 2 It is a schematic diagram of track representation;

[0024] Figure 3 A schematic diagram of the situation representation. DETAILED DESCRIPTION

[0025] In order to better understand and implement, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It is also necessary to explain that, for ease of description, only parts related to the present invention are shown in the accompanying drawings, rather than all structures.

[0026] Assume that two sensor systems can detect and track multiple targets in a distributed architecture, form target tracks and report to the fusion center. After the fusion center synchronizes all target track data observed by the two sensors, it obtains the target situation data for the track association task. The number pair of a homologous track pair is a correct association label. The correct association label set consisting of all correct association labels is combined with the target situation data to form a data sample. The homologous track pair refers to the target track belonging to the same target observed by the two sensors. After accumulating multiple data samples, the target situation data set for the track association task is obtained.

[0027] The input samples of the deep learning-based track association network model (hereinafter referred to as the track association network model) for track association tasks are generally all target tracks observed by paired sensors in the same area in the same time period. The output association results are compared with the correct association label set, and the loss function is calculated. The track association network model that minimizes the loss function is obtained through iterative training. The track association task can be completed using this model. However, the association performance of the track association network model is related to the diversity of the target situation data samples. The generalization performance of the model trained under small sample conditions is poor. When the sensor performance is different, the sensor position changes, or the public observation area changes, the association performance deteriorates significantly. In complex scenarios, there may also be problems such as false alarms, missed alarms, and target tracking leaks. The samples in the above situations are usually small, which further limits the performance improvement of the track association network model. For this reason, it is necessary to perform data enhancement operations on the existing target situation data samples to increase the diversity and complexity of the samples, thereby providing data support for improving the performance of the track association network model. For the common problem of synchronous track association under the condition of only random errors, the enhancement operation sets can be designed from the situation layer, track layer, and point track layer respectively.

[0028] The situation layer data enhancement operation set includes 5 enhancement operations:

[0029] (1) Add Gaussian white noise with a standard deviation of amplitude to all target point trace data observed by the first sensor. The initial amplitude range can be set to but is not limited to [0.0, 0.1].

[0030] (2) Add Gaussian white noise with a standard deviation of amplitude to all target point trace data observed by the second sensor. The initial amplitude range can be set to but is not limited to [0.0, 0.1].

[0031] (3) Add a horizontal or vertical translation of the amplitude to the positions of all target points observed by the two sensors. The initial amplitude range can be set to but is not limited to [-0.2, 0.2].

[0032] (4) A rotation with an amplitude is added to the position and heading of all target points observed by the two sensors. The initial amplitude range can be set to but not limited to [-180, 180] degrees.

[0033] (5) Amplitude scaling is added to the positions and velocities of all target points observed by the two sensors. The initial amplitude range can be set to but is not limited to [0.9, 1.1].

[0034] The track layer data enhancement operation set includes 5 enhancement operations:

[0035] (1) Randomly replicate the target track observed by any sensor according to the amplitude multiple of the total number of tracks. The initial amplitude range can be set to but not limited to [0.0,5.0].

[0036] (2) The target track observed by any sensor is randomly deleted with probability based on the amplitude. The initial amplitude range can be set to but not limited to [0.0, 1.0].

[0037] (3) A homologous track pair or a non-homologous track is randomly selected with the amplitude as probability and their positions are randomly translated horizontally or vertically in the range of [-0.1, 0.1]. The initial amplitude range can be set to but not limited to [0.0, 1.0].

[0038] (4) A homologous track pair or a non-homologous track is randomly selected with the amplitude as probability and its position and heading are randomly rotated within the range of [-180, 180] degrees. The initial amplitude range can be set to but not limited to [0.0, 1.0].

[0039] (5) Randomly select homologous track pairs or non-homologous track pairs with amplitude as probability and randomly scale their positions and velocities in the range of [0.9, 1.1]. The initial amplitude range can be set to but not limited to [0.0, 1.0].

[0040] Note that for the copied target tracks, translation, rotation and scaling need to be added randomly. If a homologous track pair is copied, the same translation, rotation and scaling are added to the homologous track pair, and the corresponding labels are added to the correctly associated label set; for the deleted target tracks, if they belong to the homologous track pair, the corresponding labels are deleted from the correctly associated label set.

[0041] The point layer data enhancement operation set includes 2 enhancement operations:

[0042] (1) According to a given probability, Gaussian white noise with a standard deviation of amplitude is randomly added to the target point data. The initial amplitude range can be set to but not limited to [0.0, 0.1].

[0043] (2) The target point data is randomly deleted with probability based on the amplitude and the deleted target point data is replaced by the neighboring points through linear interpolation or two-point linear extrapolation estimation. The initial amplitude range can be set to but not limited to [0.0, 0.2].

[0044] After obtaining the set of enhancement operations, multiple data enhancement operation sets can be selected from the data enhancement operation sets at different levels, and an amplitude can be randomly determined within the initial amplitude range. The enhancement operation with the corresponding amplitude is used to enhance the target situation data in the order of situation layer, track layer, and point track layer, and the correct associated label set is modified accordingly. Then, the enhanced target situation data is used to train the track association network model, and the performance of the track association network model is improved by increasing the diversity and complexity of the sample data. However, different enhancement operations and amplitudes have different abilities to increase model performance. The above-mentioned traditional method of randomly selecting enhancement operations and randomly determining the enhancement operation amplitude is inefficient and cannot maximize the improvement of model performance. Therefore, an automatic track data enhancement method for track association tasks is proposed. This method first uses a graph neural network to extract the representation vector of the target situation data, and then uses a reinforcement learning method to automatically learn and determine the enhancement operations and corresponding amplitudes with higher data enhancement efficiency. The specific method is as follows:

[0045] Step 1: First, normalize all target situation data in the target situation data set to obtain the normalized target situation data of the sth sensor as S s ={T i,s ,i=1,…,M s}, where M s is the number of targets observed by the sth sensor, s=1,2 represents the sensor number, T i,s = {P k,i,s ,k=1,…,N i,s} is the track data of the i-th target in the s-th sensor, where N i,s is the number of points of the ith target track in the sth sensor, P k,i,s is the kth target point track data of the ith target track in the sth sensor, P k,i,s Contains information such as position, speed, and heading. The information in each dimension is normalized to [-1.0, 1.0].

[0046] Step 2: If Figure 2 As shown, each point data P k,i,s Input into the point trace representation module to obtain the corresponding target point trace representation vector The parameters of each point characterization module are shared with each other, where k = 1, … ,N i,s , i=1, … ,M s , s=1,2.

[0047] Step 3: If Figure 2 As shown, multiple point traces of each target track are represented by vectors in time sequence. Input into the track characterization module in sequence to obtain the target track characterization vector The parameters of each track characterization module are shared with each other, where i=1, … ,M s , s=1,2.

[0048] Step 4: Figure 3 As shown, all target tracks corresponding to the sth sensor are represented by vectors As a vertex, an undirected graph G is established based on the K nearest neighbor method. s = {V s ,E s}, where E s is the set of edges, is the vertex set, V s Contains the track representation vector of all targets observed by the sth sensor The undirected graph G s Input into the situation representation module based on graph neural network to obtain the target situation representation vector The parameters of each situation characterization module are shared with each other.

[0049] Step 5: Figure 1 As shown in the figure, a track data enhancement operation selection module is established based on the time series deep model. The module selects enhancement operations from the situation layer data enhancement operation set, the track layer data enhancement operation set, and the point track layer data enhancement operation set in turn. Two enhancement operations are selected each time. Each enhancement operation contains two values. The first value is the number of the enhancement operation, and the second value is the predicted enhancement operation amplitude. The predicted enhancement operation amplitude is mapped to the initial amplitude range of the corresponding enhancement operation after passing through the tanh activation function.

[0050] Step 6: Figure 1 As shown, for any neural network model to be trained for track association tasks (hereinafter referred to as track association network model), after extracting a batch of situation samples, track data enhancement operations are selected for each situation sample data in the order of steps 1 to 5, and these operations are used for data enhancement, and then the enhanced data is input into the track association network model to be trained, and the loss value output by the track association network model is used as the feedback signal of the track data enhancement operation selection module. The purpose of the track data enhancement operation selection module is to make the selected enhancement operation through training so that the loss value of the task association model can be increased as much as possible. After the training converges, the optimized enhancement operation and its corresponding amplitude are obtained;

[0051] Step 7: According to the optimized common track data enhancement operations and their corresponding motion amplitudes, each situation data in the training data set is enhanced, and the enhanced data is used to train the task model to improve the association performance of the task model.

[0052] The above is only a preferred method of the present invention, and is not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for automatic enhancement of track data for track association tasks, characterized in that The following steps are involved: Step 1: Normalize the target situation data obtained by the two sensors. The normalized target situation data of the sth sensor is S s ={T i,s ,i=1,…,M s }, where M s is the number of targets observed by the sth sensor, s is 1 or 2, indicating the sensor number, T i,s = {P k,i,s ,k=1,…,N i,s } is the track data of the i-th target in the s-th sensor, where N i,s is the number of points of the ith target track in the sth sensor, P k,i,s is the kth target point track data of the ith target track in the sth sensor, P k,i,s Contains information such as position, speed, and heading. The information of each dimension is normalized to [-1.0, 1.0]; Step 2: Set each point data P k,i,s Input into the point trace representation module to obtain the corresponding target point trace representation vector The parameters of each point characterization module are shared with each other, where k = 1, ..., N i,s ,i=1,…,M s , s=1,2; Step 3: Vectorize the multiple points of each target track in chronological order Input into the track characterization module in sequence to obtain the target track characterization vector The parameters of each track characterization module are shared with each other, where i=1, … ,M s , s=1,2; Step 4: Characterize the track vectors of all targets corresponding to the sth sensor As a vertex, an undirected graph G is established based on the K nearest neighbor method. s = {V s ,E s }, where E s is the set of edges, is the vertex set, V s Contains the track representation vector of all targets observed by the sth sensor The undirected graph G s Input into the situation representation module based on graph neural network to obtain the target situation representation vector The parameters of each situation representation module are shared with each other; Step 5: Based on the time series deep model, a track data enhancement operation selection module is established. This module selects enhancement operations from the situation layer data enhancement operation set, the track layer data enhancement operation set, and the point track layer data enhancement operation set in turn. Two enhancement operations are selected each time. Each enhancement operation contains two values. The first value is the number of the enhancement operation, and the second value is the predicted enhancement operation amplitude. The predicted enhancement operation amplitude is mapped to the initial amplitude range of the corresponding enhancement operation after the tanh activation function. Step 6: For any neural network model to be trained for track association tasks, referred to as track association network model, after extracting a batch of situation samples, select track data enhancement operations for each situation sample data in the order of steps 1 to 5, and use these operations to perform data enhancement, and then input the enhanced data into the track association network model to be trained, and use the loss value output by the track association network model as the feedback signal of the track data enhancement operation selection module. The purpose of the track data enhancement operation selection module is to make the selected enhancement operation through training so that the loss value of the task association model can be increased as much as possible. After the training converges, the optimized track data enhancement operation and its corresponding action amplitude are obtained; Step 7: According to the optimized common track data enhancement operations and their corresponding motion amplitudes, each situation data in the training data set is enhanced, and the enhanced data is used to train the task model to improve the association performance of the task model.

2. A method for automatically enhancing track data for track association tasks according to claim 1, characterized in that: The set of situation layer data enhancement operations in step 5 includes: Gaussian white noise with a standard deviation of amplitude is added to all target point trace data observed by the first sensor, Gaussian white noise with a standard deviation of amplitude is added to all target point trace data observed by the second sensor, a horizontal or vertical translation of the amplitude is added to the position of all target point traces observed by the two sensors, a rotation of the amplitude is added to the position and heading of all target point traces observed by the two sensors, and a scaling of the amplitude is added to the position and speed of all target point traces observed by the two sensors. The initial amplitude range can be set to [0.0, 0.1], [0.0, 0.1], [-0.2, 0.2], [-180, 180] deg, [0.9, 1.1], respectively, and the initial amplitude range is not limited to the above values.

3. A method for automatically enhancing track data for track association tasks according to claim 1, characterized in that: The track layer data enhancement operation set in step 5 includes: Randomly copy the target track observed by any sensor according to the amplitude times of the total number of tracks, randomly delete the target track observed by any sensor with the amplitude as probability, randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly translate their position horizontally or vertically in the range of [-0.1,0.1], randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly rotate their position and heading in the range of [-180,180]deg, randomly select the same source track pair or non-homologous track with the amplitude as probability and randomly rotate their position and speed in the range of [0.9,1.1 ] range, the initial amplitude range can be set to [0.0,5.0], [0.0,1.0], [0.0,1.0], [0.0,1.0], [0.0,1.0] respectively, and the initial amplitude range is not limited to the above values. For the copied target tracks, translation, rotation and scaling need to be added randomly. If a homologous track pair is copied, the same translation, rotation and scaling are added to the homologous track pair, and the corresponding label is added to the correctly associated label set; for the deleted target track, if it belongs to the homologous track pair, the corresponding label is deleted from the correctly associated label set.

4. A method for automatically enhancing track data for track association tasks according to claim 1, characterized in that: The point layer data enhancement operation set in step 5 includes: randomly adding Gaussian white noise with a standard deviation of amplitude to the target point data according to a given probability, randomly deleting the target point data with the amplitude as probability, and using adjacent points to replace the deleted target point data through linear interpolation or two-point linear extrapolation estimation. The initial amplitude range can be set to [0.0, 0.1], [0.0, 0.2] respectively, and the initial amplitude range is not limited to the above values.

Images