System for recognizing aircraft maneuvers on basis of deep learning

Abstract

The present invention relates to the field of machine learning, and relates to a system for recognizing aircraft maneuvers on the basis of deep learning. The system is aimed at tasks of understanding tactical actions in air combat games, and performs operations of detection, recognition, analysis, etc., for independent and continuous maneuvering actions of an aircraft. The recognition and analysis system comprises a data collection module, a data pre-processing module, a data normalization module, a model training module, a model prediction module, and a result visualization module. Aircraft maneuver trajectory data is collected by means of an aircraft maneuver trajectory collection apparatus in a sample data collection module, and then the collected aircraft maneuver trajectory data is preprocessed in the data preprocessing module and the data normalization module, so that the aircraft maneuver trajectory data meets requirements during model training and prediction. For new continuous-maneuver samples, each independent maneuver in the samples and corresponding start and end time are predicted by using a continuous-maneuver recognition model which is obtained by means of training; and finally, the result visualization module generates recognition visualization results of predicted maneuver samples.

Description

Aircraft maneuver recognition system based on deep learning Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an aircraft maneuver recognition system based on deep learning. Background Art

[0002] Aircraft maneuvers are tactically significant flight maneuvers performed by an aircraft during flight. Identifying and analyzing these maneuvers is a crucial component of tactical maneuver understanding in air combat games. It helps pilots determine enemy aircraft movements and proactively implement corresponding tactical countermeasures. Therefore, combining deep learning with effective methods to identify aircraft maneuvers has significant scientific and practical implications.

[0003] In air combat, analyzing and understanding pilots' tactical maneuvers is crucial for improving skills and enhancing performance. Accurately understanding a pilot's tactical maneuvers is extremely challenging. This requires identifying and identifying individual tactical maneuvers within recorded flight trajectories, thereby understanding the pilot's tactical intent. This process requires domain experts with specialized knowledge to analyze and understand the pilot's actions by watching extended flight videos, which is extremely time-consuming and labor-intensive.

[0004] Traditional solutions to the problem of maneuver trajectory generation and following are mostly based on a library of maneuver trajectory libraries. These methods search for standard tactical maneuver trajectories, interpolate and approximate these trajectories based on the aircraft's state variables, and generate a recommended trajectory tailored to the current situation. This approach requires a large library of standard tactical maneuver trajectories and requires detailed analysis of different maneuvers, resulting in limited generalization.

[0005] Detecting and identifying independent tactical maneuvers within a flight trajectory is challenging. This is primarily due to two challenges. Firstly, due to differences in aircraft models and pilots' individual operating habits, the trajectory of the same tactical maneuver can vary significantly and in length. Furthermore, trajectory parameters (such as speed, direction, acceleration, etc.) constantly change, resulting in significant trajectories for the same tactical maneuver, posing a challenge to the recognition model. Secondly, a flight trajectory is a complex sequence consisting of multiple tactical maneuvers. Therefore, in addition to being able to distinguish between different tactical maneuvers, the recognition model must first locate the tactical maneuver within the continuous flight trajectory before it can recognize the maneuver. This is a simultaneous detection and classification task.

[0006] Summary of the Invention

[0007] In this invention, we use artificial intelligence technology to identify and annotate recorded flight trajectories and convert flight trajectories into sequences of tactical actions, thereby helping field experts better analyze and understand pilots' tactical actions, improve analysis efficiency, and enhance analysis quality.

[0008] The present invention aims to provide a deep learning-based aircraft maneuver recognition system. This system, specifically designed for tactical maneuver understanding tasks in air combat games, detects, recognizes, and analyzes both independent and continuous aircraft maneuvers. The recognition and analysis system comprises a data acquisition module, a data preprocessing module, a data normalization module, a model training module, a model prediction module, and a result visualization module. The aircraft maneuver trajectory acquisition device in the sample data acquisition module collects aircraft maneuver trajectory data. Subsequently, the collected aircraft maneuver trajectory data is preprocessed in the data preprocessing and data normalization modules to meet the requirements for model training and prediction. In the model training module, a deep learning-based aircraft maneuver recognition model is trained to generate two main components: an independent maneuver recognition model and a continuous maneuver recognition model. For new aircraft maneuver samples, the model prediction module uses the trained independent maneuver recognition model to predict the sample's maneuver category. For new continuous maneuver samples, the trained continuous maneuver recognition model is used to predict each independent maneuver in the sample and its corresponding start and end times. Finally, the result visualization module generates a recognition visualization of the predicted maneuver sample.

[0009] To address the shortage of continuous maneuver trajectory data samples, a method of data simulation and synthesis is used to generate simulated continuous maneuver trajectory training data from independent maneuver data samples. A prediction system based on deep learning methods uses this training data to learn the general spatiotemporal characteristics of aircraft maneuver trajectories and use them to predict the maneuver category probabilities of new maneuver samples. For each input continuous trajectory, the system predicts the individual maneuvers of the new maneuver sample and generates predictions for each maneuver, including the maneuver category, start time, and end time. The maneuver recognition results are visualized through charts.

[0010] This paper proposes an aircraft maneuver recognition system based on deep learning, which includes the following modules:

[0011] 1. Data Acquisition Module: This module is used to collect the maneuver trajectory data of the aircraft. For independent maneuver trajectory data, it includes the maneuver trajectory data and the corresponding maneuver category label; for continuous maneuver trajectory data, it includes the category label, start time, and end time of each sub-maneuver in the trajectory;

[0012] The maneuver trajectory data mainly refers to the six-degree-of-freedom trajectory data samples generated by the aircraft during flight. A single maneuver trajectory data sample contains data from several time steps generated by the aircraft during flight. In each time step, it contains the aircraft's movement freedom characteristics along the preset spatial rectangular coordinate axes (or equivalent longitude, latitude, and altitude characteristics) and the rotation freedom characteristics around these three coordinate axes;

[0013] The maneuver category label, that is, whether the input maneuver data belongs to one of the preset maneuver categories. The preset maneuver categories include terminal climb, climb profile, dive, exit dive, half-buoy flip, bucket, half roll, large barrel roll, rectangular barrel roll, small barrel roll, level bank turn, level load turn, ascending sharp turn, descending sharp turn, tail maneuver, tail serpentine, tail turn, tail out turn, 39 side, offset maneuver, post-launch offset, jump attack, ejection (straight), ejection (offset), serpentine (straight), serpentine (offset), stealth, tracking and surveillance, hovering (standby), hovering (delayed), etc.

[0014] The maneuver identification tag is a list of maneuver trajectories Y identified by inputting continuous maneuver trajectories. is the label matrix of the continuous trajectory, t is the number of time steps divided by the trajectory, y i =[y i,1 ,y i,2 ,…y i,m ], m is the number of categories of continuous maneuver trajectories, y i,j represents the label of the j-th maneuver at the i-th time step within the trajectory, and y i,j =[c i,j ,s i,j ,e i,j ], c i,j is the binary classification label of the j-th maneuver in time step i, c i,j =1 means that there is an m-th maneuver in the t time step, which is one of the maneuver category labels, s i,j is the starting time of the j-th maneuver in time step i, e i,j The termination time of the j-th type of maneuver within the i-th time step, s i,j 、e i,j Normalized to a floating point number between 0 and 1, it represents the proportion of the maneuver start and end in the corresponding time step.

[0015] 2. Data preprocessing module: The data preprocessing module is used to process the aircraft maneuver trajectory data. For the input maneuver trajectory data, it is mainly to perform data enhancement, and use the linear interpolation method to repair the samples with missing data, and align the data to make the trajectory length uniform. Finally, the aircraft maneuver trajectory data for subsequent model training and prediction is obtained as the input data for subsequent model training and testing. The single maneuver trajectory feature of the aircraft (including independent maneuver trajectory and continuous maneuver trajectory) is expressed as F = [f1, f2, ..., f n ], where f i represents the characteristics of the aircraft maneuver trajectory at the i-th time step, n represents the total number of n time steps in each trajectory; the characteristics of each time step fi is a 6-dimensional vector, i.e. f i =[x i ,y i ,z i ,p i ,h i ,r i ], where x i 、y i 、z i is the degree of freedom of movement of the aircraft along the three spatial rectangular coordinate axes of x, y, and z, p i 、h i 、r i is the rotational freedom characteristic around these three coordinate axes; for independent maneuver trajectories, the trajectory label is processed as the maneuver category label Y = [y1, y2, ..., y m ], as the label for subsequent model training and testing, where y i =1 means that the input maneuver trajectory is identified as the i-th category among the preset m categories, y i = 0 means that the input maneuver trajectory does not belong to the i-th category; for continuous maneuver trajectories, the input maneuver trajectory needs to be pre-divided into t time steps first, and then the trajectory label is processed into a maneuver identification label Y = [y1, y2, ..., y t ],in is the label matrix of the continuous trajectory, y i represents the recognition result of the i-th time step within the maneuver duration.

[0016] The data enhancement includes augmentation of sample data, including but not limited to the following commonly used data enhancement techniques:

[0017] Time Shift: Moves the entire trajectory forward or backward by a certain amount of time. This can help simulate changes in the start or end time of a trajectory, which may be due to changes in traffic patterns, weather conditions, or other factors.

[0018] Spatial perturbations: Introducing small changes to trajectory coordinates, such as adding noise or random offset points. This can help simulate changes in the actual path of a moving object.

[0019] Rotation and Reflection: This involves rotating or reflecting the trajectory to create a change in the direction or orientation of the moving object. This can help simulate changes in flight direction or changes in the environment.

[0020] Scaling: This involves rescaling the track to simulate changes in the speed or distance traveled by a moving object. This can help simulate changes in the speed of an object, such as when the target is moving faster or slower than usual.

[0021] Jitter: This involves randomly perturbing the trajectory points to create variations in the motion of a moving object. This can help simulate changes in the object's behavior, such as when it is more erratic than usual.

[0022] 3. Data normalization module: The data preprocessing module is used to perform data normalization operations on the obtained aircraft maneuver trajectory characteristics, and finally obtain a data set for subsequent model training and testing. Each data sample in the data set includes the normalized aircraft maneuver trajectory characteristics and the category label corresponding to the maneuver. The data set consists of two parts: a training set and a test set.

[0023] Because continuous maneuver sample training data is relatively small and expensive to collect, it's difficult to provide sufficient training data for the continuous maneuver recognition model. However, independent maneuver data samples are relatively abundant. Therefore, using independent maneuver data to assist in the training of the continuous maneuver recognition model can be considered. However, independent maneuver trajectory data differs from continuous maneuver trajectory data in terms of data distribution and cannot be directly used for training the continuous maneuver recognition model. Instead, it should be spliced ​​together to form a coherent continuous maneuver trajectory data set by combining several segments of independent maneuver trajectory data and applying a mean shift to each segment. This approach can alleviate the problem of insufficient continuous maneuver data samples and reduce the difficulty of model training.

[0024] The training set of independent maneuver recognition consists of 90% of the dataset samples of all independent maneuver trajectory datasets, and the test set consists of 10% of the dataset samples of independent maneuver trajectory; the training set of continuous maneuver recognition consists of all the synthetic datasets of independent maneuver datasets and 70% of the real continuous maneuver trajectory dataset, and the test set consists of 30% of the real continuous maneuver trajectory dataset.

[0025] Data standardization refers to mapping clinical data features to a standard normal distribution using the following formula to avoid increasing the difficulty of model training due to large differences in data ranges;

[0026] In the above formula, f i represents the i-th feature in the corresponding aircraft maneuver trajectory data feature F, μ i represents the mean of the features of all trajectory points in the i-th maneuver trajectory, σ i represents the variance of the features of all trajectory points in the i-th maneuver trajectory; ε is the numerical error term to avoid σ i = 0, a numerical overflow problem occurs. i It represents the standard eigenvalue of the i-th feature after normalization. The final normalized maneuver trajectory data feature is expressed as X = [x1, x2, ..., x n ]; and express the category probability of independent maneuver trajectory data as

[0027] 4. Deep learning model training module: The deep learning model training module uses a deep learning-based method to train and construct an aircraft maneuver trajectory recognition model for subsequent aircraft maneuver trajectory recognition. The aircraft maneuver trajectory recognition model is divided into two parts: an independent maneuver trajectory recognition model and a continuous maneuver trajectory recognition model. The independent maneuver recognition model is responsible for accepting a single maneuver trajectory input and outputting the maneuver trajectory category, while the continuous maneuvering model accepts a continuous maneuver trajectory input composed of multiple independent maneuvers and outputs the category and start and end time of each component maneuver. Both models are trained using an iterative training method. For a given input training set data sample, it is first determined whether the input data sample is independent maneuver data or continuous maneuver data. When the input data sample is independent maneuver data, the independent maneuver trajectory model is used to identify the maneuver category and the start and end time of the model, and the model is trained and updated according to the label. When the input data sample is continuous maneuver data, the continuous maneuver trajectory model is used to identify the category and start and end time of each component maneuver, and the model is trained and updated according to the label. Finally, when the recognition accuracy of the aircraft maneuver trajectory recognition model meets the preset threshold requirement, training is stopped.

[0028] The independent maneuver recognition model uses the loss function during the training process. Where m is the number of independent maneuver categories, y i is the i-th component in the data label, is the probability vector output by the independent maneuver recognition model The continuous maneuver recognition model uses the loss function during training. in is the category loss of the j-th maneuver at the i-th time step, which is specifically expressed as follows:

[0029] In the above formula, λ is a hyperparameter that balances category loss and time loss, c i,j is the binary classification label of the j-th maneuver in time step i, c i,j =1 means that there is the mth type of maneuver in the t time step, s i,j is the starting time of the j-th maneuver in time step i, e i,j The end time of the j-th maneuver within the i-th time step.

[0030] 5. In the independent maneuver recognition model, a TarNet-based independent maneuver detection model is constructed and trained by deep learning methods for subsequent independent maneuver probability prediction. The independent maneuver detection model training method adopts an iterative training method. For a given pre-processed and standardized independent maneuver training set data sample X = [x1, x2, ..., x n ] and input it into the neural network model based on the TarNet architecture. Among them, the TarNet neural network uses a network structure based on the self-attention mechanism to model the spatiotemporal features in the maneuver trajectory data. It has the advantages of being able to process time series data of different lengths and frequencies without alignment or interpolation, being able to calculate the output of all time steps in parallel, and being able to learn complex and nonlinear patterns in time series data. During the training process, the network first adds the position information in the time series data to the input vector through the position encoding layer. In this layer, the input vector can be encoded using methods such as sine functions, learning-based encoding, and time features; subsequently, the encoder layer is composed of a stack of multiple sub-attention encoder modules, each of which contains a multi-head self-attention sublayer, a feedforward neural network sublayer, a residual connection sublayer, and a layer normalization sublayer; finally, the output layer converts the features learned by the encoder layer into a target feature vector, and methods such as fully connected layers, convolutional layers, and activation functions can be used.

[0031] The TarNet-based neural network model maps the input independent maneuver trajectories into a high-dimensional feature space, learns to identify independent maneuver categories in the high-dimensional space, and converts the features into maneuver classification probability vectors. Through the loss function Perform error back propagation so that Continue to approach the true probability vector Y. Finally, when the accuracy of the independent maneuver recognition model meets the requirements, stop training.

[0032] 6. In the continuous maneuver detection model, a continuous maneuver detection model based on the Yolo architecture is constructed and trained through a deep learning method for subsequent continuous maneuver prediction. The continuous maneuver detection model training method adopts an iterative training method. In the continuous maneuver detection model, the boundary detection problem of the maneuver trajectory is transformed into a regression problem of the maneuver boundary, rather than an independent maneuver recognition problem based on a sliding window. This method uses different neurons to learn the maneuver category, start time, and end time respectively, and obtains the output matrix The output matrix is ​​then converted into the final recognition maneuver list through data post-processing.

[0033] For a given preprocessed and standardized input training set data sample X=[x1,x2,…,x n] and input it into the convolutional neural network based on the Yolo architecture. The convolutional neural network mainly consists of an encoding layer, a feature extraction layer composed of multiple convolutional sublayers, and an output layer. The encoding layer mainly increases the dimension of the input data features and maps the low-dimensional 6-DOF trajectory data to a high-dimensional feature space for encoding; the feature extraction layer consists of multiple convolutional sublayers, residual sublayers, and pooling layers, and uses convolution and pooling operations to learn the feature representation of the maneuver from the high-dimensional space; the output layer maps the learned feature representation to the final The output matrix of constitutes the recognition result including maneuver category and start and end time information.

[0034] Prediction results is the label matrix of continuous trajectories, where y i represents the recognition result of the i-th time step during the maneuver duration. By using the mean square error loss function To make the prediction results Continuously approaching the true label Y. i,j is the binary classification label of the j-th maneuver in time step i, c i,j =1 means that there is the mth type of maneuver in the t time step, s i,j is the starting time of the j-th maneuver in time step i, e i,j The end time of the j-th maneuver within the i-th time step.

[0035] 7. Model prediction module: In this module, the independent maneuver recognition model and continuous maneuver recognition model trained in the deep learning model training module are used to identify and predict new data samples.

[0036] When using the trained independent maneuver recognition model to identify the input new independent maneuver trajectory data sample, it is only necessary to input the data preprocessing module and the data normalization module to obtain the data feature X=[x1,x2,…,x n ], the extracted maneuver trajectory data features are directly input into the independent maneuver recognition model, and the trained independent maneuver recognition model can output the probability vector of each maneuver category corresponding to the maneuver trajectory data through the neural network based on the TarNet architecture Each component in the vector is the probability of being identified as the corresponding maneuver category.

[0037] When using the trained continuous maneuver recognition model to identify the input new independent maneuver trajectory data sample, it is only necessary to input the data preprocessing module and the data normalization module to obtain the data feature X=[x1,x2,…,x n], the extracted maneuver trajectory data features are directly input into the continuous maneuver recognition model, and the trained independent maneuver recognition model can output the recognition matrix of maneuver trajectory data through the neural network based on Yolo architecture exist middle, represents the recognition result of the i-th time step within the maneuver duration, including 3m recognition results between 0 and 1. Under our preset maneuver categories, there are m = 30 maneuver categories, so y i The 3m=90 vector components of correspond to 30 maneuver categories. That is, for the jth maneuver A j , corresponding to Three components of data. Among them, Is a binary classification flag indicating whether there is maneuver A at time i j ,and It indicates the corresponding maneuver start and end time. 1 o'clock Therefore, after obtaining the output recognition matrix Finally, a step of data post-processing is required to remove invalid outputs and organize them to obtain the final recognition results.

[0038] 8. Visualization module: The visualization module generates recognition visualization results of predicted maneuver samples.

[0039] This invention uses deep learning technology to identify and label aircraft maneuver trajectories, convert independent maneuver trajectories into maneuver categories, and convert continuous maneuver trajectories into a list of maneuver components, thereby helping field experts better analyze and understand pilot tactical actions, improve analysis efficiency, and enhance analysis quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of an aircraft maneuver recognition system based on deep learning;

[0041] Figure 2 Workflow diagram of the aircraft maneuver identification system based on deep learning. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention clearer, the specific embodiments of the present invention are described below in conjunction with the embodiments and drawings so that those skilled in the art can better understand the present invention. It should be noted that the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and are not intended to limit the scope of the invention claimed for protection. All other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present invention.

[0043] As shown in Figure 1, the present invention provides a deep learning-based aircraft maneuver identification system, which takes aircraft maneuver trajectory data as input and outputs aircraft maneuver identification results. The system includes a data acquisition module, a data preprocessing module, a data standardization module, a model training module for deep learning methods, a model prediction module, and a visualization module. The data acquisition module is used to collect aircraft maneuver trajectory data. For independent maneuver trajectory collection, the system includes the maneuver trajectory data and the corresponding maneuver category label; for continuous maneuver trajectory data, the system includes the category label, start time, and end time of each sub-maneuver in the trajectory.

[0044] The data preprocessing module processes the aircraft maneuver trajectory data. It primarily performs data augmentation on the input maneuver trajectory data, uses linear interpolation to repair missing data samples, and aligns the data to uniform trajectory lengths. Ultimately, the resulting aircraft maneuver trajectory data is used for subsequent model training and prediction.

[0045] The data normalization module is used to perform data normalization on the obtained aircraft maneuver trajectory features F, ultimately generating a dataset for subsequent model training and testing. For independent maneuver datasets, each data sample in the dataset includes the normalized aircraft maneuver trajectory features and the corresponding class label for the maneuver. For continuous maneuver datasets, each data sample in the dataset includes the normalized aircraft maneuver trajectory features and the corresponding recognition result matrix for the maneuver. The dataset consists of two parts: a training set and a test set.

[0046] The deep learning model training module uses a deep learning-based method to train and construct an aircraft maneuver trajectory recognition model for subsequent aircraft maneuver trajectory recognition. The aircraft maneuver trajectory recognition model is divided into two parts: an independent maneuver trajectory recognition model and a continuous maneuver trajectory recognition model. The independent maneuver recognition model is responsible for accepting a single maneuver trajectory input and outputting the maneuver trajectory category, while the continuous maneuver recognition model accepts a continuous maneuver trajectory input composed of multiple independent maneuvers and outputs the category and start and end time of each component sub-maneuver.

[0047] The model prediction module uses the independent maneuvering trajectory recognition model and the continuous maneuvering trajectory recognition model trained in the deep learning model training module to respectively recognize and predict the independent maneuvering trajectory samples and the continuous maneuvering trajectory samples in the test set.

[0048] The visualization module generates a recognition visualization result of the predicted maneuver sample and outputs the recognition result of the maneuver trajectory data to be predicted in the form of a chart.

[0049] Specifically, as shown in FIG2 , the workflow of the deep learning-based aircraft maneuver recognition system includes:

[0050] S1. Data Collection: The data collection module collects the aircraft's maneuver trajectory data. For independent maneuver trajectory data, it includes the maneuver trajectory data and the corresponding maneuver category label; for continuous maneuver trajectory data, it includes the category label, start time, and end time of each sub-maneuver in the trajectory.

[0051] S2. Independent maneuvering data preprocessing: Preprocess the aircraft independent maneuvering data, including data enhancement, removal of samples with missing data, and desensitization of sensitive information. For independent maneuvering trajectory data that is too long, the data is uniformly downsampled, and for independent maneuvering trajectory data that is too short, the data is padded or oversampled. After preprocessing the aircraft independent maneuvering data features, a clinical data feature representation that can be used for subsequent model training and prediction is obtained; the aircraft independent maneuvering data labels also need to be preprocessed and processed into maneuvering category labels from 0 to m (the preset number of maneuvering categories); the independent maneuvering data feature representation and the independent maneuvering data labels constitute the aircraft independent maneuvering data set;

[0052] S3. Continuous maneuvering data preprocessing: Preprocess the aircraft continuous maneuvering data, including data enhancement, removal of samples with missing data, and desensitization of sensitive information. For real continuous maneuvering trajectory data, evenly downsample the overly long data, and fill or oversample the overly short continuous maneuvering trajectory data; for synthetic continuous maneuvering trajectory data, randomly select several maneuvering trajectories from the independent maneuvering trajectory dataset for downsampling, perform mean shift, and then splice to obtain synthetic continuous maneuvering trajectory data. After continuous maneuvering data preprocessing, a continuous trajectory data feature representation that can be used for subsequent model training and prediction is obtained; the continuous trajectory identification label is preprocessed and processed into an identification result matrix; the continuous maneuvering data feature representation and the continuous maneuvering data label constitute the aircraft continuous maneuvering dataset;

[0053] S4. Standardization of independent and continuous maneuvering data: Standardize the data features of the independent and continuous maneuvering datasets to transform the distribution of the maneuvering trajectory data features into a standard normal distribution.

[0054] S5. Divide the training set and test set: The training set of independent maneuver recognition consists of 90% of the dataset samples of all independent maneuver trajectory datasets, and the test set consists of 10% of the dataset samples of independent maneuver trajectories; the training set of continuous maneuver recognition consists of all the synthetic datasets of independent maneuver datasets and 70% of the real continuous maneuver trajectory dataset, and the test set consists of 30% of the real continuous maneuver trajectory dataset.

[0055] S6. Deep Learning-Based Aircraft Maneuver Identification Model Training: The deep learning model training module uses deep learning-based methods to train and construct an aircraft maneuver trajectory identification model for subsequent aircraft maneuver trajectory identification. The aircraft maneuver trajectory identification model is divided into two parts: an independent maneuver trajectory identification model and a continuous maneuver trajectory identification model. The independent maneuver identification model accepts a single maneuver trajectory input and outputs the maneuver trajectory category, while the continuous maneuver identification model accepts a continuous maneuver trajectory composed of multiple independent maneuvers as input and outputs the category and start and end times of each component maneuver.

[0056] Among them, the independent maneuver recognition model uses the loss function during the training process Where m is the number of independent maneuver categories, y i is the i-th component in the data label, is the probability vector output by the independent maneuver recognition model The i-th component of ;

[0057] The continuous maneuver recognition model uses the loss function during training. in is the category loss of the j-th maneuver at the i-th time step, which is specifically expressed as follows:

[0058] In the above formula, λ is a hyperparameter that balances category loss and time loss, c i,j is the binary classification label of the j-th maneuver in time step i, c i,j =1 means that there is the mth type of maneuver in the t time step, s i,j is the starting time of the j-th maneuver in time step i, e i,j The end time of the j-th maneuver within the i-th time step.

[0059] Both models are trained using an iterative training method. For a given input training set data sample, the first step is to determine whether the input data sample is independent maneuver data or continuous maneuver data. If the input data sample is independent maneuver data, the independent maneuver trajectory model is used to identify the maneuver category and the start and end time of each component maneuver, and the model is trained and updated according to the label. If the input data sample is continuous maneuver data, the continuous maneuver trajectory model is used to identify the maneuver category and start and end time of each component maneuver, and the model is trained and updated according to the label. Finally, when the recognition accuracy of the aircraft maneuver trajectory recognition model meets the preset threshold requirement, training stops.

[0060] S7, Maneuvering trajectory data prediction and recognition: When performing model prediction, it is only necessary to input the data preprocessing module and the data normalization module to obtain the data features X=[x1,x2,…,xn ], the extracted maneuver trajectory data features are directly input into the continuous maneuver recognition model, and the trained independent maneuver recognition model can output the recognition matrix of maneuver trajectory data through the neural network based on Yolo architecture

[0061] S8. Visualization of maneuver recognition results: The visualization module generates recognition visualization results of the predicted maneuver samples and outputs the recognition results of the maneuver trajectory data to be predicted in the form of a chart.

[0062] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with that in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless defined as herein. The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Aircraft maneuver identification system based on deep learning, characterized by: Aiming at the task of understanding tactical actions in air combat games, the independent and continuous maneuvers of aircraft are detected, identified and analyzed; the identification and analysis system includes data acquisition module, data preprocessing module, data normalization module, model training module, model prediction module and result visualization module; The aircraft maneuver trajectory data is collected by the aircraft maneuver trajectory collection device in the sample data collection module. Subsequently, the collected aircraft maneuver trajectory data is preprocessed in the data preprocessing module and the data normalization module to meet the requirements of model training and prediction. In the model training module, an aircraft maneuver recognition model is trained based on the deep learning method, which includes two main parts: an independent maneuver recognition model and a continuous maneuver recognition model. For new aircraft maneuver samples, the model prediction module uses the trained independent maneuver recognition model to predict the maneuver category of the sample. For new continuous maneuver samples, the trained continuous maneuver recognition model is used to predict each independent maneuver in the sample and the corresponding start and end time. Finally, the result visualization module generates the recognition visualization result of the predicted maneuver sample.

2. The system according to claim 1, characterized in that To address the problem of insufficient continuous maneuver trajectory data samples, the simulated continuous maneuver trajectory training data is generated using data simulation synthesis method through independent maneuver data samples; the prediction system based on deep methods learns the general spatiotemporal characteristics of the aircraft maneuver trajectory through the training data, which is used to predict the maneuver category probability of new maneuver samples. For the input continuous trajectory, the various component maneuvers of the new maneuver sample are predicted, and the predicted recognition results of the maneuver category, start time, and end time of each maneuver are generated; and the maneuver recognition results are visualized by drawing charts.

3. The system according to claim 2, characterized in that The system includes the following modules: data acquisition module, data preprocessing module, data standardization module, deep learning model training module, independent maneuver recognition model, continuous maneuver detection model, model prediction module, and visualization module.

4. The system according to claim 3, characterized in that Data acquisition module: The data acquisition module is used to collect the maneuver trajectory data of the aircraft; for independent maneuver trajectory collection, it includes the maneuver trajectory data and the corresponding maneuver category label; for continuous maneuver trajectory data, it includes the category label, start time, and end time of each sub-maneuver in the trajectory; The maneuvering trajectory data mainly refers to the six-degree-of-freedom trajectory data samples generated by the aircraft during flight; a single maneuvering trajectory data sample includes data of several time steps generated by the aircraft during flight; in each time step, it includes the characteristics of the degree of freedom of movement of the aircraft along the direction of the preset spatial rectangular coordinate axis and the characteristics of the degree of freedom of rotation around the three coordinate axes; The maneuver category label, i.e., whether the input maneuver data belongs to one of the preset maneuver categories; wherein the preset maneuver categories include final climb, climb profile, dive, exit dive, half-bucket flip, bucket, half roll, large barrel roll, rectangular barrel roll, small barrel roll, horizontal slope turn, horizontal load turn, ascending sharp turn, descending sharp turn, tail maneuver, tail serpentine, tail turn, tail outward turn, three-nine side facing, offset maneuver, offset after launch, jump attack, ejection straight, ejection offset, serpentine straight, serpentine offset, stealth detour, tracking and monitoring, hovering standby, hovering delay category; The maneuver identification tag is a list of maneuver trajectories Y composed of continuous maneuver trajectories identified as input; is the label matrix of the continuous trajectory, t is the number of time steps divided by the trajectory, y i =[y i,1 ,y i,2 ,…y i,m ], m is the number of categories of continuous maneuvering trajectories, y i,j represents the label of the j-th maneuver at the i-th time step in the trajectory, and y i,j =[c i,j ,s i,j ,e i,j ], c i,j is the binary classification label of the j-th maneuver in the i-th time step, c i,j = 1 means that there is an m-th maneuver in the t time step, which is one of the maneuver category labels, s i,j is the starting time of the j-th maneuver in the i-th time step, e i,j The termination time of the j-th maneuver in the i-th time step, s i,j 、e i,j It is normalized to a floating point number between 0 and 1, representing the proportion of the maneuver start and end in the corresponding time step.

5. The system according to claim 3, characterized in that Data preprocessing module: The data preprocessing module is used to process the aircraft maneuver trajectory data. For the input maneuver trajectory data, it mainly performs data enhancement, uses the linear interpolation method to repair the samples with missing data, and aligns the data to make the trajectory length uniform. Finally, the aircraft maneuver trajectory data for subsequent model training and prediction is obtained as the input data for subsequent model training and testing. The single maneuver trajectory feature of the aircraft is expressed as F = [f1, f2, …, f n ], where f i represents the characteristics of the aircraft maneuver trajectory at the i-th time step, n represents the total number of n time steps in each trajectory; the characteristic f of each time step i is a 6-dimensional vector, i.e. f i =[x i ,y i ,z i ,p i ,h i ,r i ], where x i ,y i 、z i is the freedom of movement of the aircraft along the three spatial rectangular coordinate axes of x, y, and z, p i 、h i 、r i is the rotational freedom characteristic around these three coordinate axes; for independent maneuvering trajectories, the trajectory labels are processed as maneuvering category labels Y = [y1, y2, …, y m ], as the label for subsequent model training and testing, where y i = 1 means that the input maneuver trajectory is identified as the i-th category among the preset m categories, y i = 0 means that the input maneuver trajectory does not belong to the i-th category; for continuous maneuver trajectories, the input maneuver trajectory needs to be pre-divided into t time steps first, and then the trajectory label is processed into a maneuver identification label Y = [y1, y2, …, y t ],in is the label matrix of continuous trajectory, y i represents the recognition result of the ith time step within the maneuver duration; The data enhancement includes augmentation of sample data, including but not limited to the following commonly used data enhancement techniques: Time shift: Move the entire trajectory forward or backward by a certain amount of time; this can help simulate changes in the start or end time of a trajectory, which may be due to changes in traffic patterns, weather conditions, or other factors; Spatial perturbations: introducing small changes in trajectory coordinates, adding noise or random offset points; can help simulate changes in the actual path of a moving object; Rotation and reflection: This involves rotating or reflecting the trajectory to create a change in direction or orientation of the moving object; this can help simulate changes in flight direction or changes in the environment; Scaling: This involves rescaling the track to simulate changes in the speed or distance traveled of a moving object. This can help simulate changes in the speed of an object, when the target is moving faster or slower than usual. Jitter: This involves randomly perturbing the track points to create variations in the motion of a moving object; can help simulate changes in an object's behavior when it is more irregular than usual.

6. The system according to claim 3, characterized in that Data standardization module: The data preprocessing module is used to perform data standardization operations on the obtained aircraft maneuver trajectory features, and finally obtain a data set for subsequent model training and testing. Each data sample in the data set includes the normalized maneuver trajectory features of the aircraft and the category label corresponding to the maneuver. The data set consists of a training set and a test set. Among them, because the number of training data of continuous maneuvering samples is small and the collection cost is high, it is difficult to provide sufficient training data for the continuous maneuvering recognition model, while the number of independent maneuvering data samples is relatively large, so it is possible to consider using independent maneuvering data to assist the training of the continuous maneuvering recognition model; however, the independent maneuvering trajectory data and the continuous maneuvering trajectory data are different in data distribution, and cannot be directly used for the training of the continuous maneuvering recognition model. Instead, several segments of independent maneuvering trajectory data should be spliced ​​together through splicing and synthesis, and mean shifts should be made for different independent trajectories to form coherent continuous maneuvering trajectory data; in this way, the problem of insufficient continuous maneuvering data samples can be alleviated and the difficulty of model training can be reduced; The training set of independent maneuver recognition consists of 90% of the data set samples of all independent maneuver trajectory data set samples, and the test set consists of 10% of the data set samples of independent maneuver trajectory; the training set of continuous maneuver recognition consists of all the independent maneuver data set synthetic data set and 70% of the real continuous maneuver trajectory data set, and the test set consists of 30% of the real continuous maneuver trajectory data set; The data standardization refers to mapping the clinical data features to a standard normal distribution through the following formula to avoid increasing the difficulty of model training due to large differences in data ranges; In the above formula, f i represents the i-th feature in the feature F of the corresponding aircraft maneuver trajectory data, μ i represents the mean of the features of all trajectory points in the i-th maneuver trajectory, σ i represents the variance of the features of all trajectory points in the i-th maneuver trajectory; ε is the numerical error term to avoid σ i = 0, a numerical overflow occurs; x i represents the standard eigenvalue of the i-th feature after normalization. The final normalized maneuver trajectory data feature is expressed as X = [x1, x2, ..., x n ]; and the category probability of independent maneuver trajectory data is expressed as 7. The system according to claim 3, characterized in that Deep learning model training module: The deep learning model training module trains and constructs an aircraft maneuver trajectory recognition model through a deep learning-based method for subsequent aircraft maneuver trajectory recognition; the aircraft maneuver trajectory recognition model is divided into two parts: an independent maneuver trajectory recognition model and a continuous maneuver trajectory recognition model, wherein the independent maneuver recognition model is responsible for accepting a single maneuver trajectory input and outputting a maneuver trajectory category, while the continuous maneuver accepts a continuous maneuver trajectory input composed of multiple independent maneuvers and outputs the category and start and end time of each component maneuver; both models are trained using an iterative training method, for a given input training set data sample, first determine whether the input data sample is an independent maneuver data or a continuous maneuver data, when the input data sample is an independent maneuver data, use the independent maneuver trajectory model to identify the maneuver category, and train and update the model according to the label; when the input data sample is a continuous maneuver data, use the continuous maneuver trajectory model to identify each component maneuver category and start and end time, and train and update the model according to the label; finally, when the recognition accuracy of the aircraft maneuver trajectory recognition model meets the preset threshold requirement, stop training; The independent maneuver recognition model uses the loss function during the training process. Where m is the number of independent maneuver categories, y i is the i-th component in the data label, is the probability vector output by the independent maneuver recognition model The continuous maneuver recognition model uses the loss function during training. in is the category loss of the j-th maneuver at the i-th time step, which is specifically expressed as follows: In the above formula, λ is a hyperparameter that balances the category loss and time loss, c i,j is the binary classification label of the j-th maneuver in the i-th time step, c i,j = 1 means that there is the mth type of maneuver in the t time step, s i,j is the starting time of the j-th maneuver in the i-th time step, e i,j The end time of the j-th maneuver in the i-th time step.

8. The system according to claim 3, characterized in that In the independent maneuver recognition model, an independent maneuver detection model based on the TarNet architecture is constructed and trained by a deep learning method for subsequent independent maneuver probability prediction; the independent maneuver detection model training method adopts an iterative training method; for a given preprocessed and standardized independent maneuver training set data sample X = [x1, x2, ..., x n ], and input it into the neural network model based on the TarNet architecture; wherein, the TarNet neural network uses a network structure based on the self-attention mechanism to model the spatiotemporal features in the maneuver trajectory data, and has the advantages of being able to process time series data of different lengths and frequencies without alignment or interpolation, being able to calculate the output of all time steps in parallel, and being able to learn complex and nonlinear patterns in time series data; during the training process, the network first adds the position information in the time series data to the input vector through the position encoding layer, where the input vector can be encoded using sinusoidal functions, learning encoding, and time feature methods; subsequently, the encoder layer is stacked with multiple sub-attention encoder modules, each of which contains a multi-head self-attention sublayer, a feedforward neural network sublayer, a residual connection sublayer, and a layer normalization sublayer; finally, the output layer converts the features learned by the encoder layer into a target feature vector, and can use a fully connected layer, a convolutional layer, and an activation function method; The TarNet-based neural network model maps the input independent maneuver trajectories into a high-dimensional feature space, learns to identify independent maneuver categories in the high-dimensional space, and converts the features into maneuver classification probability vectors. Through the loss function Perform error back propagation so that Continue to approach the true probability vector Y; finally, when the accuracy of the independent maneuver recognition model meets the requirements, stop training.

9. The system according to claim 3, characterized in that In the continuous maneuver detection model, a continuous maneuver detection model based on the Yolo architecture is constructed and trained through a deep learning method for subsequent continuous maneuver prediction; The continuous maneuver detection model training method adopts an iterative training method; in the continuous maneuver detection model, the boundary detection problem of the maneuver trajectory is transformed into a regression problem of the maneuver boundary, rather than an independent maneuver recognition problem based on a sliding window; this method uses different neurons to learn the category, start time, and end time of the maneuver respectively, and obtains an output matrix The output matrix is ​​then converted into the final recognition maneuver list through data post-processing; For a given preprocessed and standardized input training set data sample X = [x1, x2, ..., x n ], and input it into the convolutional neural network based on the Yolo architecture; the convolutional neural network is mainly composed of an encoding layer, a feature extraction layer composed of multiple convolutional sublayers, and an output layer; the encoding layer mainly increases the dimension of the input data features and maps the low-dimensional 6-DOF trajectory data to a high-dimensional feature space for encoding; the feature extraction layer is composed of multiple convolutional sublayers, residual sublayers, and pooling layers, and uses convolution and pooling operations to learn the feature representation of the maneuver from the high-dimensional space; the output layer maps the learned feature representation to the final The output matrix of the ,forms the recognition result including the maneuver category, start and end time information; Prediction results is the label matrix of continuous trajectories, where y i represents the recognition result of the i-th time step within the maneuver duration; by using the mean square error loss function To make the prediction results Continuously approaching the true label Y; where c i,j is the binary classification label of the j-th maneuver in the i-th time step, c i,j = 1 means that there is the mth type of maneuver in the t time step, s i,j is the starting time of the j-th maneuver in the i-th time step, e i,j The end time of the j-th maneuver in the i-th time step.

10. The system according to claim 3, characterized in that Model prediction module: In this module, the independent maneuver recognition model and continuous maneuver recognition model trained in the deep learning model training module are used to recognize and predict new data samples; When the trained independent maneuver recognition model is used to recognize the input new independent maneuver trajectory data sample, it is only necessary to input the data preprocessing module and the data standardization module to obtain the data feature X=[x1,x2,…,x n ], the extracted maneuver trajectory data features are directly input into the independent maneuver recognition model. The trained independent maneuver recognition model can output the probability vector of each maneuver category corresponding to the maneuver trajectory data through a neural network based on the TarNet architecture. Each component in the vector is the probability of being identified as the corresponding maneuver category; When the trained continuous maneuver recognition model is used to recognize the input new independent maneuver trajectory data sample, it is only necessary to input the data preprocessing module and the data standardization module to obtain the data feature X = [x1, x2, ..., x n ], the extracted maneuver trajectory data features are directly input into the continuous maneuver recognition model, and the trained independent maneuver recognition model can output the recognition matrix of maneuver trajectory data through the neural network based on the Yolo architecture exist middle, represents the recognition result of the ith time step within the duration of the maneuver, including 3m recognition results between 0 and 1; under our preset maneuver category, there are m = 30 maneuver categories, so y i The 3m = 90 vector components of correspond to 30 types of maneuvers; that is, for the jth maneuver A j , corresponding to Three components of data; among them, is a binary classification flag indicating whether there is maneuver A at time i j ,and It indicates the corresponding maneuver start and end time. It should be noted that only when 1 hour Therefore, after obtaining the output recognition matrix Finally, a step of data post-processing is required to remove invalid outputs and sort out the final recognition results. Visualization Module: The visualization module generates recognition visualization results of predicted maneuver samples.

Images