A track identification and classification method based on multi-level residual cross attention

Abstract

The application relates to the field of signal processing and discloses a track recognition and classification method based on multi-level residual cross attention. The method is composed of a global feature extraction network and a classification network, wherein the global feature extraction network is sequentially stacked by multiple residual cross attention modules, each level of the modules includes a residual feature extraction module and a cross attention module, and is respectively used for extracting local features and modeling global dependence; the multi-level structure enhances the feature expression capability, and improves the modeling effect of global features and time sequence relationship in complex track data. The model adopts an end-to-end training mode, combines a cross entropy loss function, and optimizes the classification performance. Compared with a traditional method, the application can effectively extract long-distance dependence and global feature expression in track data, simultaneously enhances the stability and training efficiency of feature transmission through a residual mechanism, is suitable for track target recognition and classification tasks in a complex environment, and has good robustness and expansibility.

Description

Technical Field

[0001] This invention belongs to the field of signal processing, specifically relating to a track recognition and classification method based on multi-level residual cross-attention, which is used to accurately identify target types. It is particularly suitable for efficient and accurate identification and classification of target track data, thus significantly improving the efficiency and safety of air traffic management. Background Technology

[0002] With the continuous increase in air traffic volume, traditional track identification and classification methods are no longer sufficient to meet the needs of modern air traffic management. Existing track identification and classification methods mainly rely on traditional machine learning algorithms, such as Support Vector Machines (SVM) and Random Forests. These methods have the following problems when processing complex track data:

[0003] Feature extraction is difficult: Traditional machine learning methods require manual design and extraction of features, which often makes it difficult to capture effective feature information when faced with high-dimensional and non-linear trajectory data.

[0004] Limited ability to process long-series data: Track data is usually time series data, and traditional machine learning methods have difficulty capturing long-term dependencies in the sequence when processing long-series data.

[0005] Insufficient generalization ability: Traditional machine learning methods have poor generalization ability when faced with unseen trajectory patterns and are prone to overfitting.

[0006] High computational complexity: Traditional machine learning methods have high computational complexity when processing large-scale track data, making it difficult to meet the needs of real-time processing. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a track recognition and classification method based on multi-level residual cross-attention, which overcomes the problems of insufficient long-range dependency modeling capability and inadequate global feature representation in traditional track recognition.

[0008] The technical solution adopted in this invention is as follows:

[0009] A track recognition and classification method based on multi-level residual cross-attention specifically includes the following steps:

[0010] S1: Acquire target trajectory data, preprocess it, and divide it into training and test sets;

[0011] S2: Construct a multi-level residual cross-attention structure to extract global features from the preprocessed target trajectory data;

[0012] S3: Input the extracted global features into the classification network for target track classification and recognition, and map the input global features to the target category label;

[0013] S4: Supervised training of the constructed network model is performed using labeled training set samples to obtain a trained classification model; the constructed network model includes a multi-level residual cross-attention structure and a classification network.

[0014] S5: Input the target trajectory data to be identified in the test set into the trained classification model for classification, and output the target trajectory identification result.

[0015] Furthermore, the preprocessing steps in S1 are as follows:

[0016] S101: Remove outliers whose target location suddenly jumps to a location far from the trajectory within a set time.

[0017] S102: For target trajectory data lacking position or velocity information, the missing data is filled in based on nearby time data using interpolation.

[0018] S103: Remove data points where the speed is 0 for consecutive periods.

[0019] Furthermore, the multi-level residual cross-attention structure constructed in S2 is composed of multiple residual cross-attention modules with the same structure stacked sequentially. Each level of residual cross-attention module includes a residual feature extraction module and a cross-attention module; the overall structure is represented as follows:

[0020] ;

[0021] Where ResAttBlock(l) represents the residual cross-attention module of the l-th layer, which is used to perform local semantic enhancement and global dependency modeling by taking the output F(l-1) of the previous layer as input features; L is the number of residual cross-attention modules;

[0022] The specific processing procedure of the residual cross-attention module for the input features is as follows:

[0023] The input features are first enhanced by the residual feature extraction module, which includes two one-dimensional convolutional layers, Conv1 and Conv2, and a ReLU activation function. Skip connections are also introduced. The enhanced feature Fr(l) is obtained by adding the input to the convolution result, and the calculation formula is as follows:

[0024] ;

[0025] After initial feature enhancement, the feature enters the cross-attention module. The cross-attention module maps the enhanced feature Fr(l) to query Q, key K, and value V vectors, respectively. Then, it calculates the dependencies between features based on the scaling dot product attention mechanism. The calculation is as follows:

[0026] ;

[0027] ;

[0028] In the formula, W Q W K and W V For the learnable weight matrix, d a For attention feature dimensions;

[0029] The output of the cross-attention module The residual cross-attention module output Fr(l) is added element-wise to the residual feature extraction module output Fr(l) to form a residual connection, which yields the output of the current layer's residual cross-attention module. The calculation formula is as follows:

[0030] .

[0031] Furthermore, the goal of the classification network in S3 is to map the input global feature vector to the label of the target category. The classification network structure formula is as follows:

[0032] ;

[0033] In the formula, As a global feature, The classification results; The classification network model consists of three fully connected layers and two activation layers, learning data features layer by layer. The activation layers enable the network to have non-linear mapping capabilities, and the softmax function is used to transform the network output into the probability of each class for multi-class classification.

[0034] Furthermore, the specific training process in S4 is as follows:

[0035] S401: Input labeled training set samples into the built network model for supervised training, and output the label prediction of the training samples.

[0036] S402: Calculate the loss function between the predicted label and the true label using the cross-entropy loss function; assume the number of target categories is... The probability distribution of the model output is The real label is The cross-entropy loss function is expressed as:

[0037] ;

[0038] By continuously adjusting network parameters and optimizing the network model through the backpropagation algorithm, the model can learn the key features of the target trajectory data and the output results match the real target labels.

[0039] The advantages of this invention compared to the prior art are:

[0040] This invention effectively mines long-range dependencies and global feature representations in track data by constructing a multi-level residual cross-attention structure, significantly improving the accuracy of target recognition and classification. This structure balances local feature extraction and global modeling, overcoming the limitations of traditional methods such as insufficient long-sequence modeling capabilities and inadequate global feature extraction. Simultaneously, the residual connections between modules enhance the stability of feature transfer and training efficiency, making it suitable for track recognition tasks in complex and dynamic scenarios, and possessing good stability, robustness, and practical value.

[0041] This invention utilizes multi-level stacking and residual connections to effectively capture multi-scale features and long-range dependencies in flight track data, enhancing feature representation and training stability. Compared to traditional methods, this invention eliminates the need for manual feature design, possesses stronger automatic feature learning and generalization capabilities, and exhibits high computational efficiency and robustness in large-scale flight track data processing. It better meets the dual demands of modern air traffic management for both accuracy and real-time performance in flight track identification. Attached Figure Description

[0042] Figure 1 This is a flowchart of a trajectory recognition and classification method based on multi-level residual cross-attention provided in an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the principle of a track recognition and classification method based on multi-level residual cross-attention provided by an example of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0045] like Figure 1 As shown, this invention provides a method for track recognition and classification based on multi-level residual cross-attention, which specifically includes the following steps:

[0046] S1: Acquire target trajectory data, perform preprocessing to resolve outlier issues, and divide the data into training and test sets.

[0047] The preprocessing of the input target track data in S1 to address outlier issues includes operations such as removing outliers, filling in missing data points, and removing points with consecutive speeds of 0. The specific preprocessing steps are as follows:

[0048] S101: Remove outliers whose target location suddenly jumps to a location far from the trajectory within a set time.

[0049] S102: For target trajectory data lacking position or velocity information, the missing data is filled in based on nearby time data using interpolation.

[0050] S103: Remove data points with a continuous speed of 0 to avoid target track data points that remain stationary for too long affecting subsequent trajectory analysis.

[0051] S2: Construct a multi-level residual cross-attention structure to extract global features from the preprocessed target trajectory data, fully capture its long-distance dependence and multi-scale features, and improve the global semantic modeling capability.

[0052] The multi-level residual cross-attention structure constructed in the global feature extraction part of the target trajectory data in S2 is composed of multiple residual cross-attention modules with the same structure stacked sequentially. Each level of residual cross-attention module contains a residual feature extraction module and a cross-attention module, which improves the feature representation capability of the network while taking into account both local feature modeling and global dependency modeling; the specific structure is shown below:

[0053] The multi-level residual cross-attention structure is composed of multiple residual cross-attention modules with the same structure stacked sequentially. Each level of the residual cross-attention module contains a residual feature extraction module and a cross-attention module; the overall structure is represented as follows:

[0054] ;

[0055] Wherein, ResAttBlock(l) represents the residual cross-attention module of the l-th layer, which is used to take the output F(l-1) of the previous layer as input features for local semantic enhancement and global dependency modeling. The first layer residual cross-attention module takes the preprocessed target trajectory data as input features; L is the number of residual cross-attention modules. Through the continuous residual connection mechanism between modules, cross-layer information is effectively preserved, the gradient vanishing problem in deep networks is alleviated, and the transmission efficiency and training stability of deep features are improved.

[0056] The specific processing procedure of the residual cross-attention module for the input features is as follows:

[0057] The input features are first enhanced by the residual feature extraction module. This module includes two one-dimensional convolutional layers, Conv1 and Conv2, and a ReLU activation function. Skip connections are introduced to add the input to the convolution result, extracting short-term local dependencies, thereby enhancing the underlying semantic representation, mitigating gradient vanishing, and obtaining the enhanced feature Fr(l). The calculation formula is as follows:

[0058] ;

[0059] After initial feature enhancement, the feature enters the cross-attention module. The cross-attention module maps the enhanced feature Fr(l) to query Q, key K, and value V vectors, respectively. Then, it calculates the dependencies between features based on the scaling dot product attention mechanism. The calculation is as follows:

[0060]

[0061] ;

[0062] In the formula, W Q W K and W V For the learnable weight matrix, d a For attention feature dimensions;

[0063] The output of the cross-attention module The residual cross-attention module output Fr(l) is added element-wise to the residual feature extraction module output Fr(l) to form a residual connection, which yields the output of the current layer's residual cross-attention module. The calculation formula is as follows:

[0064] .

[0065] By using a continuous residual connection mechanism between modules, cross-layer information is effectively preserved, alleviating the gradient vanishing problem in deep networks and improving the transmission efficiency and training stability of deep features.

[0066] The aforementioned structure achieves layer-by-layer enhancement from low-level local feature modeling to high-level semantic representation, exhibiting good scalability and expressive power. The feature representation output from the Lth layer will serve as input to subsequent track recognition or classification networks, effectively improving the model's ability to identify complex track behaviors. In practical applications, this method fully combines the local perception capability of convolutional operations with the global dependency modeling capability of attention mechanisms. Compared to traditional structures, it can better capture complex motion patterns and long-distance behavioral dependencies in track sequences, improving the accuracy and robustness of track recognition tasks.

[0067] S3: Input the extracted global features into the classification network for target track classification and recognition, and map the input global features to the target category label.

[0068] The formula for the classification network structure in S3 is as follows:

[0069]

[0070] ;

[0071] In the formula, As a global feature, The classification results; The classification network model consists of three fully connected layers and two activation layers, learning data features layer by layer. The activation layers enable the network to have non-linear mapping capabilities, and the softmax function is used to transform the network output into the probability of each class for multi-class classification.

[0072] The functions of each layer are as follows:

[0073] Input layer: receives latent feature vectors;

[0074] The first fully connected layer (DenseLayer): This layer weights the input features and extracts the features. It is assumed that this layer contains N1 neurons.

[0075] The second fully connected layer (DenseLayer) further processes the output of the previous layer, and is set to N2 neurons;

[0076] The third fully connected layer (DenseLayer): This layer continues to extract higher-order features, and is set to N3 neurons;

[0077] After each fully connected layer, a non-linear activation function (such as ReLU) is used to introduce non-linear mapping capabilities, enabling the network to learn more complex patterns.

[0078] Two activation functions are added between the input and output layers, with each activation function followed by a fully connected layer. The activation function formula is as follows: ;

[0079] The final layer is a Softmax layer used for multi-class classification tasks, which transforms the network's output into the probability of each class; the specific formula is as follows:

[0080] ;

[0081] In the formula, and These are the output values ​​of the i-th and j-th categories, respectively. The Softmax function converts these outputs into probabilities of the categories.

[0082] The goal of a classification network is to map the input feature vector to the label of the target category.

[0083] S4: Supervised training of the constructed network model is performed using labeled training set samples to obtain a trained classification model; the constructed network model includes a multi-level residual cross-attention structure and a classification network.

[0084] The specific training process in S4 is as follows:

[0085] S401: Input labeled training set samples into the built network model for supervised training, and output the label prediction of the training samples.

[0086] S402: The cross-entropy loss function is used to calculate the loss between the predicted and true labels. This process is typically achieved by minimizing the loss function. For multi-class classification tasks, the commonly used loss function is cross-entropy loss. Assume the number of target classes is... The probability distribution of the model output is The real label is The cross-entropy loss function is expressed as:

[0087] ;

[0088] By continuously adjusting network parameters and optimizing the network model through the backpropagation algorithm, the model can learn the key features of the target trajectory data and the output results match the real target labels.

[0089] S5: Input the target trajectory data to be identified in the test set into the trained classification model for classification, and output the target trajectory identification result.

[0090] In summary, this invention effectively extracts long-range dependencies and global feature representations from track data through a deeply stacked residual cross-attention module, improving the model's ability to recognize complex track patterns. This method overcomes the shortcomings of traditional track recognition methods, such as insufficient long-range dependency modeling and inadequate global feature representation. Furthermore, it enhances the stability of feature transfer and training efficiency through a residual mechanism. Ultimately, it achieves efficient, stable, and accurate classification of track targets, meeting the high-precision and robustness requirements of modern air traffic management for intelligent recognition.

Claims

1. A track identification and classification method based on multi-level residual cross attention, characterized in that, Specifically comprising the following steps: S1: obtaining target track data, preprocessing, and dividing training set and test set; S2: constructing a multi-level residual cross attention structure to extract global features from the preprocessed target track data; S3: inputting the extracted global features into a classification network for target track classification and recognition, and mapping the input global features to target category labels; S4: using labeled training set samples to supervise the training of the built network model, and obtaining a trained classification model; wherein the built network model includes a multi-level residual cross attention structure and a classification network; S5: inputting the target track data to be identified in the test set into the trained classification model for classification, and outputting the identification result of the target track; The multi-level residual cross attention structure constructed in S2 is sequentially stacked by a plurality of residual cross attention modules with the same structure, and each level of residual cross attention module includes a residual feature extraction module and a cross attention module; the overall structure is represented as: ； wherein, ResAttBlock(l) represents the first l residual cross-attention module of the layer, used for outputting the F ( l- 1) local semantic enhancement and global dependency modeling as input features, wherein the first residual cross-attention module takes the preprocessed target track data as input features; L is the number of residual cross-attention modules. The specific processing process of the residual cross attention module for input features is as follows: The input features are first enhanced by a residual feature extraction module, which includes two one-dimensional convolution layers Conv 1and Conv 2and a ReLU activation function, and a skip connection is introduced to add the input and the convolution result to obtain enhanced features Fr(l) The calculation formula is as follows: ； The preliminary feature enhancement enters a cross-attention module, and the cross-attention module calculates the dependency between the enhanced features Fr(l) are respectively mapped into query Q, key K and value V vectors, and then the dependency between the features is calculated based on a scaled dot-product attention mechanism , and the calculation is as shown below: ； ； wherein W Q , W K and W V are learnable weight matrices, d a is the attention feature dimension; The output of the cross-attention module is added to the output of the residual feature extraction module The output of the cross-attention module is added to the output of the residual feature extraction module Fr(l) An element-wise addition is performed to form a residual connection, and the output result of the residual cross-attention module of the current layer is obtained, and the calculation formula is as follows: 。 2. The track identification and classification method based on multi-level residual cross attention according to claim 1, characterized in that, The preprocessing step in S1 is as follows: S101: removing wild points of target position jumping to a set range position away from the track within a set time; S102: filling the missing data based on the adjacent time data through interpolation method for target track data with missing position or speed information; S103: removing data points with continuous speed of 0.

3. The track identification and classification method based on multi-level residual cross attention according to claim 1, characterized in that, The target of the classification network in S3 is to map the input global feature vector to the target category label, and the classification network structure formula is as follows: ； In the formula, is a global feature, is a classification result; is a classification network model, including three layers of full connection layers and two layers of activation layers, learning data features layer by layer; The activation layer enables the network to have nonlinear mapping capability, and the output of the network is converted into the probability of each category through the Softmax function for multi-classification.

4. The track identification and classification method based on multi-level residual cross attention according to claim 1, characterized in that, The specific training process in S4 is as follows: S401: inputting the labeled training set samples into the built network model for supervised training, and outputting the label prediction of the training samples; S402: Calculate the loss function between the predicted label and the real label by using the cross-entropy loss function; assuming that the number of target categories is , the probability distribution output by the model is , the real label is , and the cross-entropy loss function is represented as: ； Through the back propagation algorithm, the network parameters are continuously adjusted and the network model is optimized, so that the model can learn the key features of the target track data, and the output result matches the real target label.

Images