Track basic model training and application method

Abstract

The invention relates to a track basic model training and applying method, and belongs to the technical field of information fusion processing and artificial intelligence crossing. According to the method, a track basic model is constructed, and a task input module and a task output module are added, so that specific application of a downstream task scene is met. The method comprises the following steps: firstly, taking unsupervised track prediction as a pre-training task, constructing track data, predicting a track state by using a track basic model and a track prediction task input and output module, and pre-training the track basic model; the method comprises the following steps: pre-training a plurality of downstream tasks, constructing downstream task models including a track generation model, a track association model and a track identification model, performing fine adjustment on the downstream task models on the basis of pre-training, and finally applying the track basic model to various downstream tasks, thereby solving the problem of complex multi-scene target track processing.

Description

Technical Field

[0001] The invention relates to a method for training and applying a track basic model, and belongs to the cross-technical field of information fusion processing and artificial intelligence. Background Art

[0002] Target track processing is to establish the corresponding relationship between each frame of sensor measurement data and different real targets, connect the measurement information from the same target at different times, and obtain accurate and reliable target state information such as position and speed through filtering estimation, so as to finally realize the real-time, continuous and accurate monitoring and mastering of individual targets. The existing target track processing methods can be divided into two categories: probability and statistics target track processing methods and intelligent target track processing methods. The probability and statistics target track processing methods started in the 1960s. They mainly adopted the idea of ​​probability and statistics, with data interconnection and Kalman filtering as the core. The main technologies include the nearest neighbor algorithm, the probabilistic nearest neighbor algorithm, the probabilistic data association algorithm, the comprehensive probabilistic data association algorithm, the joint probabilistic data association algorithm, the all-neighbor fuzzy clustering data association algorithm, the optimal Bayesian algorithm, Kalman filtering, Kalman filtering extension method, random finite set, etc. However, the probability and statistics target track processing methods need manual tuning, and there are few adjustable parameters and limited generalization solution space. They can only solve the single scene target tracking problem, and have poor ability to process complex Yangtze River target tracks. The intelligent target track processing method started around 2012, mainly using artificial intelligence ideas to solve the target track processing problem. Its main technologies include the JDIF (Joint detection and identification feature) method based on the two-stage detector Faster-RCNN framework, and the JDE (Joint detection and embeddings) method based on the one-stage detector YOLOv3

[52] . In addition, some researchers have proposed using recurrent neural networks (RNNs) such as long short-term memory (LSTM), gated recurrent unit (GRU), and Transformer networks to process radar tracks, and have achieved remarkable research results. However, the intelligent target track processing method mainly replaces some track processing modules intelligently under the traditional framework, and its ability to improve is limited. It is still difficult to directly apply it to complex multi-scenario target track processing. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for training and applying a track basic model. The present invention is based on the research paradigm of artificial intelligence algorithms, data, and training, utilizes large-scale track data, and adopts an unsupervised track prediction pre-training strategy to provide a powerful pre-trained track basic model, thereby providing a unified solution for track downstream tasks such as track generation, track association, and track identification, thereby greatly reducing the complexity of the system and effectively solving the problem of complex multi-scenario target track processing.

[0004] The present invention provides a method for training and applying a track basic model, which is special in that it comprises the following steps:

[0005] Step 1: Build a basic track model, including input embedding module, mask module, and track basic feature fusion module. Add task input module and task output module for downstream tasks. Use the track basic model pre-training and downstream task fine-tuning mode to meet the specific application requirements of downstream task scenarios.

[0006] Step 2: Take unsupervised track prediction as the pre-training task, build the track prediction task input module and the track prediction task output module, and together with the track basic model, form a track prediction model. According to the set model input and model output, widely collect various types of track data, perform random sampling, preprocessing, maximum track length completion, variable frequency sub-track sampling on the track, and build the track data required for pre-training of the track basic model;

[0007] Step 3: Construct the input embedding module of the track basic model, including the track embedding module, the time embedding module, and the global position embedding module. The variable frequency sub-track tensor, the time tensor, and the global position feature tensor are used as the input of each embedding module respectively. The track embedding, time embedding, and global position embedding output by each module are added together to obtain the input embedding E.

[0008] Step 4: Construct a mask module of the track basic model, including a causal mask and a completion mask, and add the causal mask and the completion mask to obtain a mask M;

[0009] Step 5: Construct the track basic feature fusion module of the track basic model, including the normalization layer, the mask multi-head attention layer, and the attention fully connected layer. Use the input embedding E and the mask M as the input of the track basic feature fusion module to obtain the high-dimensional track embedding K f , K f It is the input of the track prediction model;

[0010] Step 6: Embed K with high-dimensional trajectory f As the input of the track prediction task output module, the track status corresponding to the time to be predicted is output Calculate the loss and update the model parameters to pre-train the basic track model;

[0011] Step 7: For the downstream task of track generation, the track basic model is used as the backbone model, and the input module and regression output module of the downstream task of track generation are added to build the downstream task model and the downstream task dataset of track generation. Through model fine-tuning, the track basic model is applied to different downstream tasks of track generation. The downstream tasks of track generation include track completion, track filtering, and track prediction.

[0012] Step 8: For the track association downstream tasks, the track basic model is used as the backbone model, and the input module and classification output module of the track association downstream tasks are added to build the track association downstream task model and track association downstream task data set. Through model fine-tuning, the track basic model is applied to different track association downstream tasks. The track association downstream tasks include point-to-point association, interruption association, and multi-source association.

[0013] Step 9: For the downstream task of track identification, take the track basic model as the backbone model, add the input module and classification output module of the downstream task of track identification, build the downstream task model of track identification and the downstream task data set of track identification, and apply the track basic model to different downstream tasks of track identification through model fine-tuning. The downstream tasks of track identification include track initiation and track classification.

[0014] Preferably, the step 1 specifically includes the following sub-steps:

[0015] Step 1.1: Build a basic track model, pre-train the basic track model, and save the pre-trained model;

[0016] Step 1.2: Construct corresponding task input modules and task output modules according to different downstream tasks, and combine them with the track basic model to form a downstream task model;

[0017] Step 1.3: Use the training data of the downstream task to fine-tune the downstream task model, update the downstream task model parameters, and save the model after fine-tuning;

[0018] Step 1.4: Use the saved downstream task model to complete different downstream tasks.

[0019] Preferably, the step 2 specifically includes the following sub-steps:

[0020] Step 2.1: Collect different types of tracks through various sensors and signal sources, including but not limited to various track sources such as land, sea, air, and space. The track consists of multiple track points, including time, position, speed and other information, and saves them as the original track data set;

[0021] Step 2.2: The track prediction task input module traverses the original track data set, randomly samples the original track, and forms several sub-tracks;

[0022] Step 2.3: Perform differential processing on the temporal characteristics of the sampled sub-tracks, using forward differential or backward differential, and add 0 time at the first sampling point;

[0023] Step 2.4: Split network input and predicted labels. If the number of sampling points of the sub-track is k s +1, then the first k s sampling points as network input and saved as network input set D in , kth s +1 sampling point as the predicted label, saved as the predicted label set D label ;

[0024] Step 2.5: Save the sub-track global position features and traverse the network input set D in For each sub-track in

[0025] Calculate the mean of the longitude and latitude of all sampling points respectively as the global position feature of the sub-track and save it as the global position feature set P;

[0026] Step 2.6: Expand the latitude and longitude positions and traverse the network input set D in And the predicted label set D label For each sub-track in , multiply the longitude and latitude in the sub-track by λ to expand them proportionally, where λ is the position expansion coefficient;

[0027] Step 2.7: Standardize the position features of the track using z-score standardization and traverse the network input set D in For each sub-track in , calculate the mean and standard deviation of the longitude and latitude of each track, subtract the corresponding mean from the longitude and latitude of each sampling point and divide by the standard deviation to obtain the standardized longitude and latitude;

[0028] Step 2.8: Add position differential features and traverse the network input set D in For each sub-track in , use forward difference or backward difference, add 0 value as padding at the first sampling point, and splice the new longitude and latitude features into the track;

[0029] Step 2.9: Variable frequency sub-track sampling, traversing the network input set D in For each sub-track in the , set the number of variable frequency sub-tracks n, and set the corresponding n variable frequency sampling steps s according to n. v , except for the first frequency conversion sampling step of 1, the subsequent n-1 sampling steps are arranged from small to large according to prime numbers. For each sampling step sv , starting from the first sampling point of the sub-track to the end of the sub-track traversal, the corresponding variable frequency sub-track is obtained. For sub-tracks with a length less than k max The variable frequency sub-track is completed by adding 0 elements at the end, so that the length of each variable frequency sub-track after completion is equal to k max Finally, merge the n variable frequency sub-tracks to obtain the variable frequency sub-track set G; Step 2.10: Merge the network input set G and the predicted label set D respectively label , the global position feature set P is obtained, and the corresponding set becomes a tensor as the pre-training data of the track basic model;

[0030] Step 2.11: The output module of the track prediction task is a mapping model. A feasible solution is a fully connected layer that outputs the track state corresponding to the time to be predicted.

[0031] Preferably, the step 3 specifically includes the following sub-steps:

[0032] Step 3.1: Construct a track embedding module and expand the dimension of the variable frequency sub-track tensor G to D;

[0033] Step 3.2: Construct a time embedding module, whose input is the time tensor U, whose shape is [B, 1, 1]. According to different downstream task models, different time tensors U are selected. For the track generation model, the time tensor is the track time to be generated. For the track association model, the time tensor is the time corresponding to the last track point. For the track recognition model, the time tensor is the time corresponding to the last track point. Finally, the dimension of U is expanded to D.

[0034] Step 3.3: Construct a global position embedding module and expand the dimension of the global position feature tensor P to D; Step 3.4: The input embedding module consists of a track embedding module, a time embedding module, and a global position embedding module. Add the calculation results of the track embedding module, the time embedding module, and the global position embedding module to obtain the input embedding E.

[0035] Preferably, the step 4 specifically includes the following sub-steps:

[0036] Step 4.1: Construct the causal mask M c , whose shape is [k max ,k max ], where the values ​​of the elements on and below the main diagonal are 0, and the remaining values ​​are negative infinity;

[0037] Step 4.2: Construct the completion mask M p , whose shape is [k max ,k max ], network input tensor D inEach sub-track in has a completion mask M p , M p In each row of elements in , the value of the element position corresponding to the uncompleted sub-track is 0, and the value of the completed element position is negative infinity; Step 4.3: The mask module consists of a causal mask and a completion mask. c and the completion mask M p Add them together to get the mask M.

[0038] Preferably, the step 5 specifically includes the following sub-steps:

[0039] Step 5.1: Construct a normalization layer to calculate the mean and standard deviation of each sampling point in the input embedding in all feature dimensions, subtract the corresponding mean from all features of the sampling point and divide by the standard deviation to obtain the normalized tensor;

[0040] Step 5.2: Construct a masked multi-head attention layer. The input of the masked multi-head attention layer is the input embedding after the normalization layer. Mask M, position encoding strategy, use the multi-head attention network to build a masked multi-head attention layer, and get the output J of the masked multi-head attention layer;

[0041] Step 5.3: Construct an attention fully connected layer, use multiple fully connected networks to process the output J of the masked multi-head attention layer, and obtain the output U of the attention fully connected layer;

[0042] Step 5.4: Construct the track basic feature fusion layer. Each track basic feature fusion layer includes a masked multi-head attention layer and an attention fully connected layer. Each masked multi-head attention layer or attention fully connected layer is connected to a normalization layer.

[0043] Step 5.5: Connect n track basic feature fusion layers in series to obtain a track basic feature fusion module;

[0044] Step 5.6: The input embedding E and mask M are used as the input of the track basic feature fusion module. The output K of the track basic feature fusion module is f It is a high-dimensional trajectory embedding, which serves as the input of the output module of the trajectory prediction task.

[0045] Preferably, step 6 specifically includes the following sub-steps:

[0046] Step 6.1: Use the track prediction task output module to embed the high-dimensional track into K f Mapped to the track state corresponding to the time to be predicted Its input is a high-dimensional trajectory embedding K f The last sampling point of The characteristic dimension of D label The characteristic dimensions are the same;

[0047] Step 6.2: Use regression loss calculation and D label The difference between them is calculated and the gradient descent algorithm is used to update the model parameters to realize the pre-training of the track basic model. After the pre-training, the trained track basic model is saved.

[0048] Preferably, the step 7 specifically includes the following sub-steps:

[0049] Step 7.1: The input module of the downstream task of track generation is used to construct the input data required by the track generation task and use it as the input of the track basic model;

[0050] Step 7.2: The regression output module of the downstream task of trajectory generation is a mapping model that embeds the high-dimensional trajectory into K f Mapped to the generated track status Its input is a high-dimensional trajectory embedding K f The last sampling point of the output dimension is the same as D label The feature dimension is the same as that of and D label The difference between

[0051] Step 7.3: According to the requirements of different track generation downstream tasks, determine the output of the track generation downstream task input module and the output of the track generation downstream task regression output module. The track prediction task is the downstream task used in the pre-training of the track basic model, and its settings are the same as the pre-training. For the track completion task, the output of the input module is the track to be completed and the time to be completed, and the output of the output module is the state of the track to be completed. For the track filtering task, the output of the input module is the track to be filtered and the track end time, and the output of the output module is the filtered track state. For the track prediction task, the output of the input module is the track to be predicted and the track prediction time, and the output of the output module is the predicted track state.

[0052] Preferably, the step 8 specifically includes the following sub-steps:

[0053] Step 8.1: The input module of the track association downstream task is used to construct the input data required by the track association task and use it as the input of the track basic model;

[0054] Step 8.2: The classification output module of the downstream task of track association is a mapping model with an output dimension of 2, which embeds the high-dimensional track into K f Mapping to Association Class Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction is calculated using the classification loss. and the difference between the associated label R;

[0055] Step 8.3: According to the requirements of different track association downstream tasks, determine the output of the track association downstream task input module and the output of the track association downstream task classification output module. For the point association task, the output of the input module is the track to be associated and the track point to be associated, and the output of the output module is the binary association category. For the interruption association task, the output of the input module is the track where the interruption occurs, and the output of the output module is the binary association category. For the multi-source association task, the output of the input module is the track from different sources, and the output of the output module is the binary association category.

[0056] Preferably, the step 9 specifically includes the following sub-steps:

[0057] Step 9.1: The input module of the downstream task of track identification is used to construct the input data required by the track identification task and use it as the input of the track basic model;

[0058] Step 9.2: The classification output module of the downstream task of track recognition is a mapping model with an output dimension of P, where P represents the number of categories to be recognized. f Mapping to categories

[0059] Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction category is calculated using the classification loss and the difference between the true category C;

[0060] Step 9.3: According to the requirements of different track identification downstream tasks, determine the output of the track identification downstream task input module and the output of the track identification downstream task classification output module. For the track start task, it is necessary to determine whether there is a target, so the output of the input module is the track to be started, and the output of the output module is a binary classification category. For the track classification task, it is necessary to determine which type of target exists, so the output of the input module is the track to be classified, and the output of the output module is a multi-classification category.

[0061] A method for training and applying a basic track model of the present invention is based on the research paradigm of artificial intelligence algorithms, data, and training, utilizes large-scale track data, and adopts an unsupervised track prediction pre-training strategy to provide a powerful pre-trained track basic model, thereby providing a unified solution for track downstream tasks such as track generation, track association, and track identification, thereby greatly reducing the complexity of the system and effectively solving the problem of complex multi-scenario target track processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The present invention is a flow chart of a method for training and applying a track basic model. DETAILED DESCRIPTION

[0063] A method for training and applying a track-based model in this embodiment assumes that there are N target tracks, forming a track set Φ = {T1, T2, ..., T N},in

[0064]

[0065] represent the time, longitude, and latitude of the kth sampling point of the ith track, respectively. i The number of sampling points of the ith track is represented, 80% of the total number of tracks N is divided into a training set, and 20% is divided into a test set. The method for training and applying a track basic model of the present invention comprises the following steps:

[0066] Step 1: Build a basic track model, including input embedding module, mask module, and track basic feature fusion module. Add task input module and task output module for downstream tasks. Use the track basic model pre-training and downstream task fine-tuning mode to meet the specific application requirements of downstream task scenarios.

[0067] Step 1.1: Build a basic track model, pre-train the basic track model, and save the pre-trained model;

[0068] Step 1.2: Construct corresponding task input modules and task output modules according to different downstream tasks, and combine them with the track basic model to form a downstream task model;

[0069] Step 1.3: Use the training data of the downstream task to fine-tune the downstream task model, update the downstream task model parameters, and save the model after fine-tuning;

[0070] Step 1.4: Use the saved downstream task model to complete different downstream tasks.

[0071] Step 2: Take unsupervised track prediction as the pre-training task, build the track prediction task input module and the track prediction task output module, and together with the track basic model, form a track prediction model. According to the set model input and model output, widely collect various types of track data, perform random sampling, preprocessing, maximum track length completion, variable frequency sub-track sampling on the track, and build the track data required for pre-training of the track basic model;

[0072] Step 2.1: Collect different types of tracks through various sensors and signal sources, including but not limited to various track sources such as land, sea, air, and space. The track consists of multiple track points, including time, position, speed and other information, and saves them as the original track data set;

[0073] Step 2.2: The track prediction task input module traverses the original track data set and predicts the track T i Perform random sampling to form several sub-tracks t j , i represents the index of the track, j represents the index of the sub-track, and sets the minimum sub-track length k min , maximum sub-track length k max , sampling step length s, the sampling process is, starting from the first sampling point of the track to the kth sampling point i -k min The sampling point ends, and the sampling starting point m is moved backward by the step length s each time, and the sub-track sampling length is set to k. s , calculated according to the following formula

[0074] k s =randint(k min ,min(k max ,k i -m-1)) (2)

[0075] Where randint(a,b) represents a function that returns an integer in the range [a,b]. Then, according to the track sampling length k s The track T is evenly spaced i To perform sampling, a total of k samples are required s +1 track point, when the traversal is completed, save all sub-tracks t j , forming a sub-track set Φ s ={t1, t2, ...}, the parameter included in this step is the minimum sub-track length k min , maximum sub-track length k max , sampling step s, a feasible parameter setting scheme is k min =5, k max =1000, s=5;

[0076] Step 2.3: Perform differential processing on the time characteristics of the track, using the forward difference method to traverse the sub-track set Φ s For each sub-track in the track, the time of the next sampling point of the track is subtracted from the time of the previous sampling point, and 0 time is added to the first sampling point. The time feature after time feature difference processing is

[0077] for

[0078] Step 2.4: Split network input and prediction labels, traverse the sub-track set Φ s For each sub-track in , if the number of sampling points of the sub-track is k s +1, then the first k s sampling points as network input and saved as network input set Din , kth s +1 sampling point as the predicted label, saved as the predicted label set D label ;

[0079] Step 2.5: Save the sub-track global position features and traverse the network input set D in For each sub-track in , calculate the mean of the longitude and latitude of all sampling points respectively as the global position feature of the sub-track and save it as the global position feature set P;

[0080] Step 2.6: Expand the latitude and longitude positions and traverse the network input set D in And the predicted label set D label For each sub-track in the , multiply the longitude and latitude in the sub-track by λ to expand it proportionally. λ is the position expansion coefficient. A feasible choice is λ=10 6 ;

[0081] Step 2.7: Standardize the position features of the track using z-score standardization and traverse the network input set D in For each sub-track in , calculate the mean and standard deviation of the longitude and latitude of each track, subtract the corresponding mean from the longitude and latitude of each sampling point and divide by the standard deviation to obtain the standardized longitude and latitude Taking longitude as an example, the calculation process of z-score standardization is shown as follows:

[0082]

[0083] Among them, μ x represents the mean of longitude, σ x Represents the standard deviation of longitude. ε is a very small constant. A feasible value is ε = 10 -9 ;

[0084] Step 2.8: Add position differential features, use forward differential method to traverse the network input set D in For each sub-track in , the longitude and latitude of the last sampling point of the track are subtracted from the longitude and latitude of the previous sampling point, and a 0 value is added as padding at the first sampling point. The longitude feature after position feature difference processing is The latitude feature is Splice the new longitude and latitude features into the track;

[0085] Step 2.9: Variable frequency sub-track sampling, traversing the network input set D in For each sub-track in the , set the number of variable frequency sub-tracks n, and set the corresponding n variable frequency sampling steps s according to n. v, except for the first frequency conversion sampling step of 1, the subsequent n-1 sampling steps are arranged from small to large according to prime numbers. For each sampling step s v , starting from the first sampling point of the sub-track to the end of the sub-track traversal, the corresponding variable frequency sub-track is obtained. For sub-tracks with a length less than k max The variable frequency sub-track is completed by adding 0 elements at the end, so that the length of each variable frequency sub-track after completion is equal to k max Finally, the n variable frequency sub-tracks are merged to obtain the variable frequency sub-track set G. The parameter included in this step is the number of variable frequency sub-tracks n. A feasible parameter setting scheme is n = 50;

[0086] Step 2.10: Merge the variable frequency sub-track set G and the predicted label set D respectively label , the global position feature set P, and the new batch dimension is obtained. The shape of the merged G is [B,n×k max ,5], after the merger D label The shape of is [B, 1, 3], and the shape of the merged global position feature set P is [B, 1, 2]. The corresponding set becomes a tensor as the pre-training data of the track basic model.

[0087] Step 2.11: The output module of the track prediction task is a mapping model. A feasible solution is a fully connected layer that outputs the track state corresponding to the time to be predicted.

[0088] Step 3: Construct the input embedding module of the track basic model, including the track embedding module, the time embedding module, and the global position embedding module. The variable frequency sub-track tensor, the time tensor, and the global position feature tensor are used as the input of each embedding module respectively. The track embedding, time embedding, and global position embedding output by each module are added together to obtain the input embedding E.

[0089] Step 3.1: Construct the track embedding module and use the fully connected layer to expand the dimension of the variable frequency sub-track tensor G to D, as shown in the following formula:

[0090] f g (G) = GW g +b g (6)

[0091] Among them, f g (·) represents the track embedding module, W g and b g are the learnable parameters of the track embedding module;

[0092] Step 3.2: Construct a time embedding module, whose input is the time tensor U, whose shape is [B, 1, 1]. Different time tensors U are selected according to different downstream task models. For the track generation model, the time tensor is the track time to be generated. For the track association model, the time tensor is the time corresponding to the last track point. For the track recognition model, the time tensor is the time corresponding to the last track point. The fully connected layer is used to expand the dimension of U to D, as shown in the following formula:

[0093] f u (U)=UW u +b u (7)

[0094] Among them, f u (·) represents the temporal embedding module, W u and b u are the learnable parameters of the track embedding module;

[0095] Step 3.3: Construct the global position embedding module and use a fully connected layer to expand the dimension of the global position feature tensor P to D, as shown in the following formula.

[0096] f p (P) = PW p +b p (8)

[0097] Where fp(·) represents the temporal embedding module, W p and b p are the learnable parameters of the track embedding module;

[0098] Step 3.4: Add the calculation results of the track embedding module, time embedding module, and global position embedding module to obtain the input embedding E, as shown in the following formula.

[0099] E=f d (G)+f u (U)+f p (P) (9)

[0100] Step 4: Construct a mask module of the track basic model, including a causal mask and a completion mask, and add the causal mask and the completion mask to obtain a mask M;

[0101] Step 4.1: Construct the causal mask M c , whose shape is [k max ,k max ], where the values ​​of the elements on and below the main diagonal are 0, and the remaining values ​​are negative infinity, as shown in the following formula,

[0102]

[0103] Where -inf means negative infinity;

[0104] Step 4.2: Construct the completion mask M p , whose shape is [k max ,k max ], network input tensor D in Each sub-track in has a completion mask M p , M p In each row of elements, the value of the uncompleted element position corresponding to the sub-track is 0, and the value of the completed element position is negative infinity, as shown in the following formula.

[0105]

[0106] Step 4.3: Set the causal mask M c and the completion mask M p Add them together to get the mask M.

[0107] Step 5: Construct the track basic feature fusion module of the track basic model, including the normalization layer, the mask multi-head attention layer, and the attention fully connected layer. Use the input embedding E and the mask M as the input of the track basic feature fusion module to obtain the high-dimensional track embedding K f , K f It is the input of the track prediction model;

[0108] Step 5.1: Construct a normalization layer and use the layer normalization method to calculate the mean and standard deviation of each sampling point in the input tensor in all feature dimensions. Subtract the corresponding mean from all features of the sampling point and divide by the standard deviation to obtain the normalized tensor, as shown in the following formula:

[0109]

[0110] in, represents the normalized input embedding, E[b,k,d] represents the input embedding of the bth batch number, the kth sampling point, and the dth feature, μ L represents the normalized mean of the layer, σ L represents the normalized standard deviation of the layer;

[0111] Step 5.2: Construct a masked multi-head attention layer. The input of the masked multi-head attention layer is the input embedding after layer normalization. Mask M, Position Encoding Strategy,First, input embedding Through three independent mapping functions, they are mapped into query matrix Q, key matrix K, and value matrix V, respectively, as shown in the following formula:

[0112]

[0113] Among them, W Qrepresents the query mapping matrix, W K represents the key mapping matrix, W V represents the value mapping matrix, which are all learnable network parameters, and, in order to calculate multi-head attention, the number of attention heads is set to H, and the dimension D is reduced to And add the attention head dimension based on the original tensor shape, the number of dimensions is H, and calculate the query matrix Q, key matrix K, and value matrix V for each attention head;

[0114] Step 5.3: Add position coding to the query matrix Q and the key matrix K. Rotational position coding, fixed sinusoidal position coding or learnable position coding can be used.

[0115] Step 5.4: Calculate the inner product between the query matrix Q and the key matrix K, add it to the mask M, and calculate the attention weight A through the SoftMax function, as shown in the following formula,

[0116]

[0117] Step 5.5: Calculate the matrix product of the attention weights A and the value matrix V and add it to the input embedding E to get the output J of the masked multi-head attention layer, as shown in the following formula,

[0118] J=AV (19)

[0119] Step 5.6: Add the output J of the masked multi-head attention layer and the input embedding E to get the residual output H of the masked multi-head attention layer, as shown in the following formula,

[0120] H=J+E (20)

[0121] Step 5.7: Merge the residual outputs H of the masked multi-head attention layer of the H attention heads to obtain the merged attention output H;

[0122] Step 5.8: Construct the attention fully connected layer. The attention fully connected layer consists of three fully connected layers, denoted as f1(·), f2(·), and f3(·). The learnable parameters of each fully connected layer are W1, W2, and W3, and there is no bias term. Its input is the merged attention output after layer normalization. The first fully connected layer f1(·) and the third fully connected layer f3(·) map the input to the hidden dimension Q. A feasible choice of Q is Q=4D. Then, the output of the first fully connected layer is element-wise multiplied with the output of the third fully connected layer after the SiLU nonlinear activation function to obtain the output U of the attention fully connected layer, as shown in the following formula:

[0123]

[0124] Among them, SiLU(·) represents the SiLU nonlinear activation function, and ⊙ represents element-wise multiplication;

[0125] Step 5.9: The output U and Add together to get the residual output K of the attention fully connected layer, as shown in the following formula:

[0126]

[0127] Step 5.10: Construct the track basic feature fusion layer. Each track basic feature fusion layer includes a masked multi-head attention layer and an attention fully connected layer. Each masked multi-head attention layer or attention fully connected layer is connected to a normalization layer.

[0128] Step 5.11: Connect n track basic feature fusion layers in series to obtain a track basic feature fusion module;

[0129] Step 5.12: The input embedding E and mask M are used as the input of the track basic feature fusion module, and the output K of the track basic feature fusion module is f It is a high-dimensional trajectory embedding, which serves as the input of the output module of the trajectory prediction task.

[0130] Step 6: Embed K with high-dimensional trajectory f As the input of the track prediction task output module, the track status corresponding to the time to be predicted is output Calculate the loss and update the model parameters to pre-train the basic track model;

[0131] Step 6.1: Use the track prediction task output module to embed the high-dimensional track into K f Mapped to the track state corresponding to the time to be predicted Its input is a high-dimensional trajectory embedding K f The last sampling point of The characteristic dimension of D label The characteristic dimensions are the same;

[0132] Step 6.2: Use regression loss calculation and D label A feasible regression loss is SmoothL1Loss. The gradient descent algorithm is used to update the model parameters to realize the pre-training of the track basic model. After the pre-training, the trained track basic model is saved.

[0133] Step 7: For the downstream task of track generation, the track basic model is used as the backbone model, and the input module and regression output module of the downstream task of track generation are added to build the downstream task model and the downstream task dataset of track generation. Through model fine-tuning, the track basic model is applied to different downstream tasks of track generation. The downstream tasks of track generation include track completion, track filtering, and track prediction.

[0134] Step 7.1: The input module of the downstream task of track generation is used to construct the input data required by the track generation task and use it as the input of the track basic model;

[0135] Step 7.2: The regression output module of the downstream task of trajectory generation is a mapping model. A feasible solution is a fully connected layer to embed the high-dimensional trajectory into K f Mapped to the generated track status Its input is a high-dimensional trajectory embedding K f The last sampling point of The characteristic dimension of D label The feature dimensions are the same as shown below:

[0136]

[0137] Among them, K f [:,-1,:] means extract K f The last sampling point, W g and b g Represents the learnable parameters of the track generation model, calculated using regression loss and D label The difference between them is shown in the following formula:

[0138]

[0139] Among them, a feasible regression loss is SmoothL1Loss, express and D label SmoothL1Loss between express The F norm of SmoothL1Loss is used as the model loss to update the model parameters through the stochastic gradient descent algorithm;

[0140] Step 7.3: According to the requirements of different track generation downstream tasks, determine the output of the track generation downstream task input module and the output of the track generation downstream task regression output module. The track prediction task is the downstream task used in the pre-training of the track basic model, and its settings are the same as the pre-training. For the track completion task, the output of the input module is the track to be completed and the time to be completed, and the output of the output module is the state of the track to be completed. For the track filtering task, the output of the input module is the track to be filtered and the track end time, and the output of the output module is the filtered track state. For the track prediction task, the output of the input module is the track to be predicted and the track prediction time, and the output of the output module is the predicted track state.

[0141] Step 8: For the track association downstream tasks, the track basic model is used as the backbone model, and the input module and classification output module of the track association downstream tasks are added to build the track association downstream task model and track association downstream task data set. Through model fine-tuning, the track basic model is applied to different track association downstream tasks. The track association downstream tasks include point-to-point association, interruption association, and multi-source association.

[0142] Step 8.1: The input module of the track association downstream task is used to construct the input data required by the track association task and use it as the input of the track basic model;

[0143] Step 8.2: The classification output module of the downstream task of track association is a mapping model. A feasible solution is a fully connected layer with an output dimension of 2, which embeds the high-dimensional track into K f Mapping to Association Class Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction is calculated using the classification loss. The difference between the associated label R is shown in the following formula:

[0144]

[0145] Among them, a feasible classification loss is cross entropy loss, express The cross entropy loss between and R, P represents the number of categories, which is 2 in the track association model. express The element value of the pth category, r p Represents the element value of the pth category after R is one-hot encoded;

[0146] Step 8.3: According to the requirements of different track association downstream tasks, determine the output of the track association downstream task input module and the output of the track association downstream task classification output module. For the point association task, the output of the input module is the track to be associated and the track point to be associated, and the output of the output module is the binary association category. For the interruption association task, the output of the input module is the track where the interruption occurs, and the output of the output module is the binary association category. For the multi-source association task, the output of the input module is the track from different sources, and the output of the output module is the binary association category.

[0147] Step 9: For the downstream task of track identification, the track basic model is used as the backbone model, and the input module and classification output module of the downstream task of track identification are added to build the downstream task model and the downstream task data set of track identification. Through model fine-tuning, the track basic model is applied to different downstream tasks of track identification. The downstream tasks of track identification include track initiation and track classification.

[0148] Step 9.1: The input module of the downstream task of track identification is used to construct the input data required by the track identification task and use it as the input of the track basic model;

[0149] Step 9.2: The classification output module of the downstream task of track recognition is a mapping model. A feasible solution is a fully connected layer with an output dimension of P, where P represents the number of categories to be recognized. The high-dimensional track is embedded in K f Mapping to categories Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction category is calculated using the classification loss The difference between the real category C is shown in the following formula:

[0150]

[0151] Among them, a feasible classification loss is cross entropy loss, express and the cross entropy loss between C, express The element value of the pth category, c p Represents the element value of the pth category after one-hot encoding of C;

[0152] Step 9.3: According to the requirements of different track identification downstream tasks, determine the output of the track identification downstream task input module and the output of the track identification downstream task classification output module. For the track start task, it is necessary to determine whether there is a target, so the output of the input module is the track to be started, and the output of the output module is a binary classification category. For the track classification task, it is necessary to determine which type of target exists, so the output of the input module is the track to be classified, and the output of the output module is a multi-classification category.

Claims

1. A method for training and applying a track-based model, characterized in that: The following steps are involved: Step 1: Build a basic track model, including input embedding module, mask module, and track basic feature fusion module. Add task input module and task output module for downstream tasks. Use the track basic model pre-training and downstream task fine-tuning mode to meet the specific application requirements of downstream task scenarios. Step 2: Take unsupervised track prediction as the pre-training task, build the track prediction task input module and the track prediction task output module, and together with the track basic model, form a track prediction model. According to the set model input and model output, widely collect various types of track data, perform random sampling, preprocessing, maximum track length completion, variable frequency sub-track sampling on the track, and build the track data required for pre-training of the track basic model; Step 3: Construct the input embedding module of the track basic model, including the track embedding module, the time embedding module, and the global position embedding module. The variable frequency sub-track tensor, the time tensor, and the global position feature tensor are used as the input of each embedding module respectively. The track embedding, time embedding, and global position embedding output by each module are added together to obtain the input embedding E. Step 4: Construct a mask module of the track basic model, including a causal mask and a completion mask, and add the causal mask and the completion mask to obtain a mask M; Step 5: Construct the track basic feature fusion module of the track basic model, including the normalization layer, the mask multi-head attention layer, and the attention fully connected layer. Use the input embedding E and the mask M as the input of the track basic feature fusion module to obtain the high-dimensional track embedding K f , K f It is the input of the track prediction model; Step 6: Embed K with high-dimensional trajectory f As the input of the track prediction task output module, the track status corresponding to the time to be predicted is output Calculate the loss and update the model parameters to pre-train the basic track model; Step 7: For the downstream task of track generation, the track basic model is used as the backbone model, and the input module and regression output module of the downstream task of track generation are added to build the downstream task model and the downstream task dataset of track generation. Through model fine-tuning, the track basic model is applied to different downstream tasks of track generation. The downstream tasks of track generation include track completion, track filtering, and track prediction. Step 8: For the track association downstream tasks, the track basic model is used as the backbone model, and the input module and classification output module of the track association downstream tasks are added to build the track association downstream task model and track association downstream task data set. Through model fine-tuning, the track basic model is applied to different track association downstream tasks. The track association downstream tasks include point-to-point association, interruption association, and multi-source association. Step 9: For the downstream task of track identification, take the track basic model as the backbone model, add the input module and classification output module of the downstream task of track identification, build the downstream task model of track identification and the downstream task data set of track identification, and apply the track basic model to different downstream tasks of track identification through model fine-tuning. The downstream tasks of track identification include track initiation and track classification.

2. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 1 specifically includes the following sub-steps: Step 1.1: Build a basic track model, pre-train the basic track model, and save the pre-trained model; Step 1.2: Construct corresponding task input modules and task output modules according to different downstream tasks, and combine them with the track basic model to form a downstream task model; Step 1.3: Use the training data of the downstream task to fine-tune the downstream task model, update the downstream task model parameters, and save the model after fine-tuning; Step 1.4: Use the saved downstream task model to complete different downstream tasks.

3. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 2 specifically includes the following sub-steps: Step 2.1: Collect different types of tracks through various sensors and signal sources, including but not limited to various track sources such as land, sea, air, and space. The track consists of multiple track points, including time, position, speed and other information, and saves them as the original track data set; Step 2.2: The track prediction task input module traverses the original track data set, randomly samples the original track, and forms several sub-tracks; Step 2.3: Perform differential processing on the temporal characteristics of the sampled sub-tracks, using forward differential or backward differential, and add 0 time at the first sampling point; Step 2.4: Split network input and predicted labels. If the number of sampling points of the sub-track is k s +1, then the first k s sampling points as network input and saved as network input set D in , kth s +1 sampling point as the predicted label, saved as the predicted label set D label ; Step 2.5: Save the sub-track global position features and traverse the network input set D in For each sub-track in , calculate the mean of the longitude and latitude of all sampling points respectively as the global position feature of the sub-track and save it as the global position feature set P; Step 2.6: Expand the latitude and longitude positions and traverse the network input set D in And the predicted label set D label For each sub-track in , multiply the longitude and latitude in the sub-track by λ to expand them proportionally, where λ is the position expansion coefficient; Step 2.7: Standardize the position features of the track using z-score standardization and traverse the network input set D in For each sub-track in , calculate the mean and standard deviation of the longitude and latitude of each track, subtract the corresponding mean from the longitude and latitude of each sampling point and divide by the standard deviation to obtain the standardized longitude and latitude; Step 2.8: Add position differential features and traverse the network input set D in For each sub-track in , use forward difference or backward difference, add 0 value as padding at the first sampling point, and splice the new longitude and latitude features into the track; Step 2.9: Variable frequency sub-track sampling, traversing the network input set D in For each sub-track in the , set the number of variable frequency sub-tracks n, and set the corresponding n variable frequency sampling steps s according to n. v , except for the first frequency conversion sampling step of 1, the subsequent n-1 sampling steps are arranged from small to large according to prime numbers, For each sampling step s v , starting from the first sampling point of the sub-track to the end of the sub-track traversal, the corresponding variable frequency sub-track is obtained. For sub-tracks with a length less than k max The variable frequency sub-track is completed by adding 0 elements at the end, so that the length of each variable frequency sub-track after completion is equal to k max Finally, merge the n variable frequency sub-tracks to obtain the variable frequency sub-track set G; Step 2.10: Merge the network input set G and the predicted label set D respectively label , the global position feature set P is obtained, and the corresponding set becomes a tensor as the pre-training data of the track basic model; Step 2.11: The output module of the track prediction task is a mapping model. A feasible solution is a fully connected layer that outputs the track state corresponding to the time to be predicted.

4. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 3 specifically includes the following sub-steps: Step 3.1: Construct a track embedding module and expand the dimension of the variable frequency sub-track tensor G to D; Step 3.2: Construct a time embedding module, whose input is the time tensor U, whose shape is [B, 1, 1]. According to different downstream task models, different time tensors U are selected. For the track generation model, the time tensor is the track time to be generated. For the track association model, the time tensor is the time corresponding to the last track point. For the track recognition model, the time tensor is the time corresponding to the last track point. Finally, the dimension of U is expanded to D. Step 3.3: Construct a global position embedding module and expand the dimension of the global position feature tensor P to D; Step 3.4: The input embedding module consists of a track embedding module, a time embedding module, and a global position embedding module. Add the calculation results of the track embedding module, the time embedding module, and the global position embedding module to obtain the input embedding E.

5. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 4 specifically includes the following sub-steps: Step 4.1: Construct the causal mask M c , whose shape is [k max ,k max ], where the values ​​of the elements on and below the main diagonal are 0, and the remaining values ​​are negative infinity; Step 4.2: Construct the completion mask M p , whose shape is [k max ,k max ], network input tensor D in Each sub-track in has a completion mask M p , M p In each row of elements in , the value of the element position corresponding to the uncompleted sub-track is 0, and the value of the completed element position is negative infinity; Step 4.3: The mask module consists of a causal mask and a completion mask. c and the completion mask M p Add them together to get the mask M.

6. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 5 specifically includes the following sub-steps: Step 5.1: Construct a normalization layer to calculate the mean and standard deviation of each sampling point in the input embedding in all feature dimensions, subtract the corresponding mean from all features of the sampling point and divide by the standard deviation to obtain the normalized tensor; Step 5.2: Construct a masked multi-head attention layer. The input of the masked multi-head attention layer is the input embedding after the normalization layer. Mask M, position encoding strategy, use the multi-head attention network to build a masked multi-head attention layer, and get the output J of the masked multi-head attention layer; Step 5.3: Construct an attention fully connected layer, use multiple fully connected networks to process the output J of the masked multi-head attention layer, and obtain the output U of the attention fully connected layer; Step 5.4: Construct the track basic feature fusion layer. Each track basic feature fusion layer includes a masked multi-head attention layer and an attention fully connected layer. Each masked multi-head attention layer or attention fully connected layer is connected to a normalization layer. Step 5.5: Connect n track basic feature fusion layers in series to obtain a track basic feature fusion module; Step 5.6: The input embedding E and mask M are used as the input of the track basic feature fusion module. The output K of the track basic feature fusion module is f It is a high-dimensional trajectory embedding, which serves as the input of the output module of the trajectory prediction task.

7. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 6 specifically includes the following sub-steps: Step 6.1: Use the track prediction task output module to embed the high-dimensional track into K f Mapped to the track state corresponding to the time to be predicted Its input is a high-dimensional trajectory embedding K f The last sampling point of The characteristic dimension of D label The feature dimensions are the same; Step 6.2: Use regression loss calculation and D label The difference between them is calculated and the gradient descent algorithm is used to update the model parameters to realize the pre-training of the track basic model. After the pre-training, the trained track basic model is saved.

8. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 7 specifically includes the following sub-steps: Step 7.1: The input module of the downstream task of track generation is used to construct the input data required by the track generation task and use it as the input of the track basic model; Step 7.2: The regression output module of the downstream task of trajectory generation is a mapping model that embeds the high-dimensional trajectory into K f Mapped to the generated track status Its input is a high-dimensional trajectory embedding K f The last sampling point of the output dimension is the same as D label The feature dimension is the same as that of and D label The difference between Step 7.3: According to the requirements of different track generation downstream tasks, determine the output of the track generation downstream task input module and the output of the track generation downstream task regression output module. The track prediction task is the downstream task used in the pre-training of the track basic model, and its settings are the same as the pre-training. For the track completion task, the output of the input module is the track to be completed and the time to be completed, and the output of the output module is the state of the track to be completed. For the track filtering task, the output of the input module is the track to be filtered and the track end time, and the output of the output module is the filtered track state. For the track prediction task, the output of the input module is the track to be predicted and the track prediction time, and the output of the output module is the predicted track state.

9. A method for training and applying a track-based model as claimed in claim 1, characterized in that: The step 8 specifically includes the following sub-steps: Step 8.1: The input module of the track association downstream task is used to construct the input data required by the track association task and use it as the input of the track basic model; Step 8.2: The classification output module of the downstream task of track association is a mapping model with an output dimension of 2, which embeds the high-dimensional track into K f Mapping to Association Class Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction is calculated using the classification loss. and the associated label R The difference between Step 8.3: According to the requirements of different track association downstream tasks, determine the output of the track association downstream task input module and the output of the track association downstream task classification output module. For the point association task, the output of the input module is the track to be associated and the track point to be associated, and the output of the output module is the binary association category. For the interruption association task, the output of the input module is the track where the interruption occurs, and the output of the output module is the binary association category. For the multi-source association task, the output of the input module is the track from different sources, and the output of the output module is the binary association category.

10. A method for training and applying a track-based model according to claim 1, characterized in that: The step 9 specifically includes the following sub-steps: Step 9.1: The input module of the downstream task of track identification is used to construct the input data required by the track identification task and use it as the input of the track basic model; Step 9.2: The classification output module of the downstream task of track recognition is a mapping model with an output dimension of P, where P represents the number of categories to be recognized. f Mapping to categories Its input is a high-dimensional trajectory embedding K f The last sampling point of the prediction category is calculated using the classification loss and the difference between the true category C; Step 9.3: According to the requirements of different track identification downstream tasks, determine the output of the track identification downstream task input module and the output of the track identification downstream task classification output module. For the track start task, it is necessary to determine whether there is a target, so the output of the input module is the track to be started, and the output of the output module is a binary classification category. For the track classification task, it is necessary to determine which type of target exists, so the output of the input module is the track to be classified, and the output of the output module is a multi-classification category.

Images