A method for transonic speed prediction based on multi-task transformer

Abstract

This invention belongs to the fields of artificial intelligence and time series prediction technology, and relates to a post-stall prediction method based on a multi-task Transformer. This invention mainly studies the patterns of aerodynamic time series data during aircraft post-stall processes at high angles of attack, and proposes a multi-task Transformer network structure capable of simultaneously performing parameter prediction and state classification. The raw data, after relevant processing, is fed into the neural network for training and testing. The resulting model, through comparative experiments with small sample training data, shows improved accuracy compared to similar algorithms, which is beneficial for ensuring flight safety and improving aircraft execution efficiency. This invention, through improvements to the deep learning model Transformer, can simultaneously complete prediction and classification tasks, and achieves superior results compared to similar algorithms, demonstrating promising application prospects.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and time series prediction technology, and relates to a post-stall prediction method based on multi-task Transformer. Background Technology

[0002] In early aircraft design, the post-stall region was considered a useless flight area, and the aerodynamic characteristics of aircraft in this region were often ignored by designers. It wasn't until the 1970s that the German company MBB began researching the possibility of fighter jets performing maneuvers in this "useless" region and their air combat effectiveness. In the early 1980s, Dr. Herbst of Germany first proposed the concept of "supermaneuverability," clearly pointing out that post-stall maneuvers could significantly improve combat efficiency and survivability, emphasizing that future fighter jets needed the ability to perform controllable post-stall maneuvers even above the stall angle of attack. As a result, the maneuverability of fighter jets in the post-stall region attracted widespread attention from aircraft design engineers. my country has explicitly stated that its next-generation fighter jets must possess controllable maneuverability at high angles of attack. However, my country's thrust vectoring technology is not yet mature enough. Therefore, accurately reflecting the unsteady aerodynamic characteristics of aircraft at high angles of attack, accurately predicting deviation behavior, and thus providing a reference for aircraft control system design to achieve controllable flight at high angles of attack has become an urgent problem to be solved.

[0003] Currently, research on unsteady aerodynamic characteristics of aircraft at high angles of attack mainly relies on wind tunnel simulations. Because unsteady aerodynamics involves numerous influencing parameters, it is essential to design appropriate experimental methods and analyze and process data to establish an unsteady aerodynamic model that meets engineering accuracy requirements. This model must accurately represent the correct aerodynamic forces and moments during aircraft maneuvers, thereby enabling accurate analysis of the aircraft's aerodynamic performance at high angles of attack and prediction of the possibility of aerodynamic instability and deviation. A prerequisite for successful testing is the ability to design reasonable and reliable wind tunnel experiments to obtain the necessary aerodynamic database for aerodynamic modeling, which significantly hinders the engineering application of this method.

[0004] With the rise of artificial intelligence technology, the exploration of unsteady aerodynamic characteristics in academic fields both domestically and internationally has been deepening. From BP and RNN to fuzzy logic and SVM, many scholars have used these methods in recent years, combined with specific scenario data, and have made some progress. However, these methods have drawbacks such as large computational models, low accuracy, and poor portability. In recent years, the Transformer model architecture has stood out, achieving amazing results in many fields, especially in the field of time series prediction, where it has shown great application potential and research value in the study of unsteady aerodynamics at high angles of attack. Summary of the Invention

[0005] This invention mainly studies the aerodynamic time series data of aircraft during post-stall at high angles of attack, and proposes a multi-task Trsansformer network structure that can simultaneously perform parameter prediction and state classification. The raw data is processed and fed into the neural network for training and testing. The resulting model, through comparative experiments, shows improved accuracy compared to similar algorithms with small sample training data, which is beneficial to ensuring flight safety and improving aircraft execution efficiency.

[0006] Technical solution of the present invention: 1. A post-stall prediction method based on a multi-task Transformer, characterized by comprising the following steps: S1, Data Acquisition Simulation data of a certain type of aircraft was collected and used as the raw data for multi-dimensional data. The specific format is as follows, and it is saved in the form of a txt file. Table 1 Original Data Format

[0007] S2, Data Processing S21, Data Preprocessing Use the pandas library to extract the raw data from the txt file and store it in a dataframe structure; S22, Data Standardization Min-Max Scaling is used to normalize the preprocessed data to generate a dataframe; S23. Generate sequence data Perform a groupby operation on the dataframe generated in the previous step to obtain the time series data. S24, Constructing Data Each data point in the time series data obtained in the previous step is segmented and processed, and then integrated together to form the input dataset, parameter prediction and state classification ground truth dataset of the Transformer codec. The datasets formed above are shuffled and sorted in the same order, and then divided into training, validation and test sets in a ratio of 1:3:6. S3, Multi-task Transformer Neural Network Training For time-series data during aircraft post-stall processes, the aircraft state is classified into three parts—model encoder, model decoder, and multi-task generator—while predicting state parameters in subsequent time series data. S31, Construction Model Encoder The model encoder performs feature upscaling on the current flight state parameter sequence, then adds position encoding information to it, and finally extracts the high-level flight state parameter sequence representation of the current flight state parameter sequence by entering the Transformer Encoder; S32, Constructing a Model Decoder The structure of the model decoder is consistent with that of the model encoder. It requires feature upscaling of the predicted sequence, adding positional encoding information, and finally extracting features from the preceding flight state parameter sequence through the Transformer Decoder to predict the subsequent flight state parameter sequence, thus obtaining the final predicted flight state parameter sequence. ; S33, Multitasking Generator After obtaining the final predicted flight state parameter sequence, the multi-task generator is used to complete the prediction of the flight state parameters for the main task and the classification of the flight state for the auxiliary task.

[0008] Furthermore, the dataset generated in S2 is input into the model encoder and model decoder respectively. In the multi-task generator, the outputs of the prediction task and the classification task are obtained. The loss values ​​of the two tasks are calculated using the RMSE and Softmax functions respectively, and then backpropagated. The overall loss function of the model is as follows: LOSS = LOSS1 + α * LOSS2 Where LOSS1 is the loss value for classification results and LOSS2 is the loss value for prediction results. α is an adjustment coefficient, which is used to adjust the weight of the loss values ​​to obtain the optimal model.

[0009] Furthermore, step S22 uses Min-Max Scaling to normalize the preprocessed data, in the form of...

[0010] X represents the standardized data, and X represents the original data.

[0011] Furthermore, the S23 time series data consists of a list with a length equal to the number of txt files. Each list contains a dataframe, which corresponds to the data in the txt file. The data in each dataframe is then extracted and horizontally concatenated to obtain the complete time series data, which has dimensions [N, K]. Here, N represents the number of items in the time series data, i.e., the number of rows in the dataframe, and K represents the selected K-dimensional features.

[0012] Furthermore, the segmentation and truncation process for each data item in S24 is as follows: [0,0+P] data items are taken from each data item as the input of the Transformer encoder, and the [P,P+Q]th item is taken as the true value of the predicted data. The input of the Transformer decoder is composed of start_symobol (an array of K 1s, in the form of [1,1,1...,1]) and the true value of the predicted data after removing the first row. For the state classification task, the "flag" field in the [P,P+Q]th item constitutes the true value of the category data.

[0013] Furthermore, S31 specifically refers to: the current flight state parameter sequence being represented as follows: Where batch_size is the batch size. This represents the number of state points in the current flight state parameter sequence. To obtain a high-dimensional flight state parameter sequence, a Linear1 layer is used to perform feature upscaling on the current flight state parameter sequence. (1) in For a fully connected layer, a high-dimensional flight state parameter sequence ; This is the high-dimensional parameter space after feature extraction.

[0014] Use Position Embdedding to add position encoding information to the advanced flight state parameter sequence; After passing through the position encoding module, a high-dimensional flight state parameter sequence containing position information is obtained: (4) Among them, the high-dimensional flight state parameter sequence with position encoding ; After obtaining the high-dimensional flight state parameter sequence with position encoding Then, the Transformer Encoder module is used to extract the sequence of high-level flight state parameters; The overall process can be described as follows: (8) Among them, the advanced flight status parameter sequence .

[0015] Furthermore, the formula for the Position Embedding layer is: (2) (3)

[0016] in For the th flight sequence 1 sequence point For the first Each dimension.

[0017] Furthermore, the Transformer Encoder uses two layers, each consisting of a multi-head attention network and an FFN network.

[0018] Furthermore, the formula for multi-head attention is:

[0019] (5) Where h is the number of heads in multi-head attention. , , , , . The formula is: (6).

[0020] Furthermore, the formula for the FFN network is: (7) in , .

[0021] Furthermore, S32 specifically involves: during the multi-step prediction process, the predicted flight state parameter sequence output by the model needs to be input into the model decoder, and the predicted flight state parameter sequence is represented as follows: Where batch_size is the batch size, and q is the number of state points in the predicted flight state parameter sequence. To obtain the state parameter sequence, a Linear2 layer is used to perform feature upscaling on the predicted flight state parameter sequence, resulting in a high-dimensional predicted flight state parameter sequence. (9) Linear2 is a fully connected layer. High-dimensional prediction of flight state parameter sequences ; The Position Embedding layer in the model encoder is used to predict the flight state parameter sequence in high dimension. Provide location encoding information to obtain a high-dimensional predicted flight state parameter sequence containing location encoding information: (10) Obtain a high-dimensional predicted flight state parameter sequence with position encoding. Then, the Transformer Decoder module is used to generate the final predicted flight state parameter sequence. The Transformer Decoder process is the same as the Transformer Encoder process and can be described as the final predicted flight state parameter sequence. ,

[0022] (11).

[0023] Furthermore, S33 specifically involves: for the main task, using a Linear3 algorithm to perform depth calculation to obtain an intermediate vector. (12) in , is a fully connected layer, and is the intermediate vector of the main task. .

[0024] Next, the Status Predictor is used to predict the state. (13) in It is a fully connected layer, and the output vector of the main task is... .

[0025] For auxiliary tasks, a Linear4 algorithm is used for depth calculation to obtain an intermediate vector: (14) in , is a fully connected layer, and serves as the intermediate vector for the auxiliary task. .

[0026] Finally, TypeClssify is used to perform state classification to obtain the output vector of the auxiliary task: (15) in This is a fully connected layer, where type_num represents the number of flight state types. (Auxiliary task output) .

[0027] The beneficial effects of this invention are: This invention primarily studies the aerodynamic time-series data of aircraft during post-stall at high angles of attack, proposing a multi-task Transformer network structure capable of simultaneously performing parameter prediction and state classification. The raw data, after processing, is fed into the neural network for training and testing. Comparative experiments show that, with limited training data, the model achieves improved accuracy compared to similar algorithms, thus contributing to flight safety and improved aircraft execution efficiency. This invention, through improvements to the Transformer deep learning model, can simultaneously perform prediction and classification tasks, achieving superior results compared to similar algorithms and demonstrating promising application prospects. Attached Figure Description

[0028] Figure 1 Overall flowchart; Figure 2 Multi-task Tranformer network architecture diagram; Figure 3 Mean square error of each state feature when predicting 30 steps; Figure 4 The mean square error of each state parameter when predicting 60 steps; Figure 5 The mean square error of each state parameter when predicting 90 steps; Figure 6 Data partitioning in the example; Figure 7 The input data and truth data are constructed in the example. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments: To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] To facilitate the subsequent description of this embodiment, the variable names that may be used are explained as follows.

[0031] src In the Seq2Seq structure, the input vector is the Transformer encoder. Typically, the dimensions are [batch_size, P, T]. `batch_size` is the batch size for model training, `P` is the sequence length of the input vector (the length of the first P states), and `T` is the dimension of the elements in the sequence (the number of features selected). tgt `tgt` is the input vector of the Tranformer decoder during training. `tgt` is formed by concatenating `start_symbol` and `tgt_y[:-1]`. The dimensions are [batch_size, Q, T]. `batch_size` is the batch size for model training, `Q` is the sequence length of the input vector (corresponding to the predicted Q-step spin state), and `T` is the dimension of the elements in the sequence (how many features are selected).

[0032] gd GD represents the true value corresponding to the predicted result. In classification tasks, GD is composed of the spin class of each state of tgt_y, with dimensions [batch_size, Q, class_num], where class_num is the number of spin classes.

[0033] tgt_y tgt_y is the intermediate data that constitutes tgt and gd. Its dimension is [batch_size, Q, T], and it represents the actual state values ​​of the last Q states.

[0034] In multi-task scenarios, tgt_y serves as the ground truth for the state prediction task, and together with the model's state prediction output_2, the loss value is calculated to optimize the model parameters.

[0035] start_symbol The start symbol is required for the first test in the decoder. During training, `start_symbol` and `tgt_y[:-1]` constitute the decoder input `tgt`. ​​During testing, the input for the first test is `start_symbol`. Its dimensions are [batch_size, 1, T], and all values ​​in the vector are 1.

[0036] tgt' `tgt'` is the input vector to the Tranformer decoder during testing. In the first input step of classification prediction, `tgt'` consists only of `start_symbol`, with dimensions [batch_size, 1, T]. During multi-step classification prediction, the output vector from the last step of the previous time step is concatenated to the end of `tgt'`, changing the dimensions to [batch_size, 2, T]. At the Q-th prediction step, the dimensions of `tgt'` are [batch_size, Q, T].

[0037] output The output represents the model's Q-step classification prediction, which consists of two parts: the classification prediction value `output_1` and the state prediction value `output_2`. `output_1` has dimensions [batch_size, Q, class_num], where `class_num` represents the number of spin types. `output_2` has dimensions [batch_size, Q, T]. In a single-task model, the loss value can be obtained by calculating `output_1` and the state prediction (GD), and the optimizer can then be used to optimize the model parameters.

[0038] In the multi-task model, the loss value is obtained by calculating the loss function using the classification task output_1 and gd, and the loss value is obtained by calculating the loss function using the state prediction task output_2 and tgt_y. The two loss values ​​are used together to optimize the model parameters.

[0039] P In the corresponding src, the length of the sequence input to the Trnaformer encoder (using the first P state points to predict the next Q state points).

[0040] Q In the corresponding tgt, the length of the sequence input to the Transormer decoder (using the first P state points to predict the next Q state points).

[0041] T The feature dimension of each state in the sequence corresponding to src and tgt is selected as T features.

[0042] like Figure 1 The diagram shown is a flowchart of a post-stall prediction method based on a multi-task Transformer according to an embodiment of the present invention, including: S1, Data Acquisition Due to the specific nature of the application scenario, this invention uses simulation data from a certain type of machine. The original data is multi-dimensional and is presented in the following format, saved as a txt file.

[0043] Table 1 Original Data Format

[0044] In this embodiment, alpha, beta, p, q, r (angle of attack, sideslip angle, roll rate, pitch rate, yaw rate) and flag (status flag) from Table 1 are selected as data input.

[0045] S2, Data Processing S21, Data Preprocessing The pandas library is used to extract the raw data from the txt file and store it in a dataframe structure, with each column corresponding to the feature name in Table 1.

[0046] S22, Data Standardization Because the eigenvalues ​​of different features have different dimensions, the weights of some highly important features may be suppressed, affecting the model's performance. This invention uses Min-Max Scaling to normalize the preprocessed data, in the form of...

[0047] X represents the standardized data, and X represents the original data. S23. Generate sequence data The dataframes generated in the previous data standardization step are grouped by a list, with a length equal to the number of txt files. Each list contains a dataframe corresponding to the data in the txt file. The data in each dataframe is then extracted and concatenated horizontally to obtain the complete time series data with dimensions [N, T]. Here, N represents the number of items in the time series data, i.e., the number of rows in the dataframe, and T represents the selected T-dimensional features.

[0048] S24, Constructing Data In the time-series data obtained in the previous step, taking one data point as an example, the [0,0+P]th data point is taken as the input of the Transformer encoder, and the [P,P+Q]th item is taken as the true value of the predicted data. The input of the Transformer decoder is composed of start_symobol and the true value of the predicted data minus the first row. For the state classification task, the "flag" field in the [P,P+Q]th item constitutes the true value of the class data. In this way, all data points are integrated together to form the input dataset of the Transformer encoder and decoder, and the true value dataset for parameter prediction and state classification.

[0049] Furthermore, to better extract data features, the datasets formed above are shuffled and sorted in the same order, and divided into training, validation, and test sets in a ratio of 1:3:6.

[0050] In this embodiment, taking a single time-series data point as an example, the data is constructed as follows: Figure 6 , Figure 7The data in the range [0,0+P] is taken as the src data, and the data in the range [P,P+Q] is taken as the tgt_y data. tgt is composed of start_symobol and tgt_y[:-1], where start_symbol is a vector of length T with all values ​​being 1. In the classification task, gd is composed of the complete tgt_y. Each gd item corresponds to the class state of the current result, and the range is [0,1], where 0 represents normal and 1 represents abnormal (post-stall).

[0051] S3, Multi-task Transformer Neural Network Training The multi-task Transformer network structure proposed in this invention, targeting time-series data during aircraft post-stall processes, not only predicts state parameters in subsequent time series but also classifies the aircraft's state. Employing a shared attention mechanism, it consists of three parts: a model encoder, a model decoder, and a multi-task generator. The network structure is as follows: Figure 2 . S31, Construction Model Encoder The model encoder performs feature upscaling on the current flight state parameter sequence, then adds position encoding information to it, and finally extracts a high-level flight state parameter sequence representation of the current flight state parameter sequence by entering the Transformer Encoder.

[0052] The current flight state parameter sequence is represented as follows: Where batch_size is the batch size, and p is the number of state points in the current flight state parameter sequence. These are state parameters. The current flight state parameter sequence is up-dimensioned using a Linear1 layer to obtain a high-dimensional flight state parameter sequence. (1) in For a fully connected layer, a high-dimensional flight state parameter sequence .

[0053] Since the advanced flight state parameter sequence does not provide time-encoded information, and the Transformer cannot process time-series data, Position Embedding is used to add position-encoded information to the advanced flight state parameter sequence. The formula for the Position Embedding layer is: (2) (3)

[0054] Where pos is the pos-th sequence point in the th flight sequence. For the first Each dimension.

[0055] After passing through the position encoding module, a high-dimensional flight state parameter sequence containing position information is obtained: (4) Among them, the high-dimensional flight state parameter sequence with position encoding .

[0056] After obtaining the high-dimensional flight state parameter sequence T2 with position encoding, the Transformer Encoder module is used to extract the high-level flight state parameter sequence. This invention uses a two-layer Transformer Encoder, each layer consisting of a multi-head attention network and an FFN network. The formula for the multi-head attention is:

[0057] (5) Where h is the number of heads in multi-head attention. , , , , The formula for Attention is: (6) The formula for the FNN network following the multi-head attention layer is: (7) in , .

[0058] The overall process can be described as follows: (8) Among them, the advanced flight status parameter sequence . S32, Constructing a Model Decoder The structure of the model decoder is consistent with that of the model encoder. It is necessary to perform feature upscaling on the predicted sequence, then add position encoding information to it, and finally extract the final predicted flight state parameter sequence representation of the first p steps of predicted flight state parameter sequence by entering the Transformer Decoder.

[0059] In the multi-step prediction process, the predicted flight state parameter sequence output by the model needs to be concatenated and input into the model decoder. The predicted flight state parameter sequence is represented as follows: Where batch_size is the batch size, and q is the number of state points in the predicted flight state parameter sequence. These are state parameters. A Linear2 layer is used to perform feature upscaling on the predicted flight state parameter sequence, resulting in a high-dimensional predicted flight state parameter sequence. (9) Linear2 is a fully connected layer. High-dimensional prediction of flight state parameter sequences .

[0060] The Position Embedding layer in the model encoder is used to predict the flight state parameter sequence in high dimension. Provide location encoding information to obtain a high-dimensional predicted flight state parameter sequence containing location encoding information: (10) Obtain a high-dimensional predicted flight state parameter sequence with position encoding. Then, the Transformer Decoder module is used to generate the final predicted flight state parameter sequence. The Transformer Decoder process is consistent with the Transformer Encoder process and can be described as follows: (11) Final predicted flight state parameter sequence .

[0061] S33, Multitasking Generator After obtaining the final predicted flight state parameter sequence, the multi-task generator is used to complete the prediction of the flight state parameters for the main task and the classification of the flight state for the auxiliary task.

[0062] For the main task, a Linear3 algorithm is used for depth calculation to obtain the intermediate vector: (12) in , is a fully connected layer, and is the intermediate vector of the main task. .

[0063] Next, the Status Predictor is used to predict the state. (13) in It is a fully connected layer, and the output vector of the main task is... .

[0064] For auxiliary tasks, a Linear4 algorithm is used for depth calculation to obtain an intermediate vector: (14) in , is a fully connected layer, and serves as the intermediate vector for the auxiliary task. .

[0065] Finally, TypeClssify is used to perform state classification to obtain the output vector of the auxiliary task: (15) in This is a fully connected layer, where type_num represents the number of flight state types. (Auxiliary task output) .

[0066] The dataset generated in S2 is input into the model encoder and decoder, respectively. In the multi-task generator, the outputs for the prediction and classification tasks are obtained. The loss values ​​for each task are calculated using the RMSE and Softmax functions, respectively, and then backpropagated. The overall loss function of the model is as follows: LOSS = LOSS1 + α * LOSS2 Where LOSS1 is the loss value for classification results and LOSS2 is the loss value for prediction results. α is an adjustment coefficient, which is used to adjust the weight of the loss values ​​to obtain the optimal model.

[0067] In this embodiment, the relevant network parameters are set as shown in the table below.

[0068] Table 5 Neural Network Parameter Table

[0069] To test the performance of the multi-task Transformer model proposed in this invention, the experiment mainly compares its performance with that of two existing methods and two single-task versions of this method. These four methods are: (1) Seq2Seq(Linear): The two-layer linear model is used in the modeling of unsteady aerodynamics at large angles of attack. The key point of the modeling process is to adjust the weights to minimize the error between the output layer and the actual value. We applied the two-layer Linear to the Seq2Seq structure to obtain the method Seq2Seq(Linear).

[0070] (2) LSTM: This is a method for predicting and modeling aircraft stall using LSTM. LSTM recurrent neural networks can learn the unique temporal evolution of various related parameters before stall and can predict potential stalls long in advance, thus effectively providing stall warnings.

[0071] (3) Transformer(task1): This is a single-task version of the model, obtained by removing the flight state classification function from the multi-task generator module. This method only retains the flight state parameter prediction function in the model.

[0072] (4) Transformer(task2): This is another single-task version of the model, obtained by removing the flight state parameter prediction function from the multi-task generator module. This method only retains the flight state classification function in the model.

[0073] The flight state parameters used in this invention consist of 5-dimensional parameters (angle of attack, sideslip angle, roll rate, pitch rate, and yaw rate) and 1-dimensional flight state labels (marking normal and abnormal state values). The mean squared error (RMSE) and accuracy (Acc) are used to evaluate the performance of all methods. For the primary task—flight state parameter prediction—RMSE is used to measure the error between the model's predicted values ​​and the actual values, quantifying the model's performance. For the auxiliary task—flight state classification—accuracy is used to measure the error between the model's classification results and the actual values. Accuracy is calculated by dividing the total number of correctly classified flight states by the total number of unclassified flight states.

[0074] Table 2 shows the sum of mean squared error and accuracy when predicting 30 steps. Table 2 rmse and acc in predicting 30 steps

[0075] As shown in Table 2, in the experiment with 30 prediction steps, our method achieved the best performance of 0.576 in the main task flight state parameter prediction. Compared with the Seq2Seq (Linear) algorithm and the LSTM algorithm, the mean squared error of the multi-task Transformer was reduced by 0.195 and 0.159 respectively, demonstrating the superiority of the Transformer model in the multi-task architecture; compared with Transformer (T2), the mean squared error decreased by 0.064, proving that the multi-task architecture has a positive effect on improving the prediction performance of the main task flight state parameters.

[0076] In the auxiliary task of flight state classification, our method achieves a 0.037 improvement in accuracy compared to Seq2Seq (Linear) and LSTM, demonstrating the superiority of the Transformer model when using the same multi-task architecture. Our method achieves the same best flight state classification accuracy of 0.944 as Transformer (T1). This proves that the multi-task architecture used in our model achieves the best performance in flight state classification while improving the prediction performance of flight state parameters.

[0077] Table 3 shows the mean square error and accuracy when predicting for 60 steps. Table 3 rmse and acc in predicting 60 steps

[0078] Table 4 shows the mean square error and accuracy when predicting 90 steps. Table 4 rmse and acc in predicting 90 steps

[0079] As shown in Tables 3 and 4, in experiments with 60 and 90 prediction steps, our method still achieves the best performance in the main task of predicting flight state parameters, with mean squared errors of 0.994 and 1.347, respectively. Compared to the Seq2Seq (Linear) and LSTM algorithms, our method reduces the mean squared error by 0.493 and 0.44, respectively, at 60 prediction steps; and by 0.73 and 0.809, respectively, at 90 prediction steps, demonstrating the superiority of using the Transformer model in a multi-task architecture. Compared to Transformer (T2), the mean squared errors decrease by 0.053 and 0.046, respectively, at 60 and 90 prediction steps, proving that the multi-task architecture has a positive effect on improving the flight state prediction performance.

[0080] In the auxiliary task of flight state classification, with 60 prediction steps, our method achieves an accuracy improvement of 0.037 and 0.045 compared to Seq2Seq(Linear) and LSTM, respectively; with 90 prediction steps, our method achieves an accuracy improvement of 0.04 and 0.055 compared to Seq2Seq(Linear) and LSTM, respectively, demonstrating the superiority of the multi-task Transformer model under the same multi-task architecture. The accuracy achieved by our model shows a slight decrease of 0.001 compared to the accuracy achieved by Transformer(T1) at both 60 and 90 prediction steps.

[0081] Regarding state prediction, the error ratios for different prediction step sizes are as follows: Figure 3 , 4 5 For the prediction of flight state parameters for the main mission, this paper further analyzes the parameter prediction performance of various methods, and the results are as follows: Figure 3 , Figure 4 , Figure 5 As shown. For α, p, and r features, the multi-task Transformer achieved stable and optimal results in experiments with 30, 60, and 90 prediction steps. For β and q features, LSTM achieved the best results in experiments with 30, 60, and 90 prediction steps, but the multi-task Transformer achieved results close to LSTM, with the RMSE being at most 0.023 higher for β features and at most 0.007 higher for r features. For l features, except for Seq2Seq (Linear), which performed poorly, the other methods achieved similar results.

[0082] Overall, the multi-task Transformer demonstrates a significant performance improvement over the comparative methods in predicting flight state parameters, particularly in α, p, and r (angle of attack, roll rate, and yaw rate). It also exhibits superior performance, approaching the best method, in β and q (slip angle and pitch rate).

Claims

1. A post-stall prediction method based on a multi-task Transformer, characterized in that, Includes the following steps: S1, Data Acquisition Simulation data of a certain type of aircraft was collected and used as the raw data for multi-dimensional data. The specific format is as follows, and it is saved in the form of a txt file. Table 1 Original Data Format S2, Data Processing S21, Data Preprocessing Use the pandas library to extract the raw data from the txt file and store it in a dataframe structure; S22, Data Standardization Min-Max Scaling is used to normalize the preprocessed data to generate a dataframe; S23. Generate sequence data Perform a groupby operation on the dataframe generated in the previous step to obtain the time series data. S24, Constructing Data Each data point in the time series data obtained in the previous step is segmented and processed, and then integrated together to form the input dataset, parameter prediction and state classification ground truth dataset of the Transformer codec. The datasets formed above are shuffled and sorted in the same order, and then divided into training, validation and test sets in a ratio of 1:3:

6. S3, Multi-task Transformer Neural Network Training For time-series data during aircraft post-stall processes, the aircraft state is classified into three parts—model encoder, model decoder, and multi-task generator—while predicting state parameters in subsequent time series data. S31, Construction Model Encoder The model encoder performs feature upscaling on the current flight state parameter sequence, then adds position encoding information to it, and finally extracts the high-level flight state parameter sequence representation of the current flight state parameter sequence by entering the Transformer Encoder; S32, Constructing a Model Decoder The structure of the model decoder is consistent with that of the model encoder. It requires feature upscaling of the predicted sequence, adding positional encoding information, and finally extracting features from the preceding flight state parameter sequence through the Transformer Decoder to predict the subsequent flight state parameter sequence, thus obtaining the final predicted flight state parameter sequence. ; S33, Multitasking Generator After obtaining the final predicted flight state parameter sequence, the multi-task generator is used to complete the prediction of the flight state parameters for the main task and the classification of the flight state for the auxiliary task.

2. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, The dataset generated in S2 is input into the model encoder and model decoder respectively. In the multi-task generator, the outputs of the prediction task and the classification task are obtained. The loss values ​​of the two tasks are calculated using the RMSE and Softmax functions respectively, and then backpropagated. The overall loss function of the model is as follows: LOSS = LOSS1 + α * LOSS2 Where LOSS1 is the loss value of the classification result, LOSS2 is the loss value of the prediction result; α is the adjustment coefficient, which is used to adjust the weight of the loss value to obtain the optimal model.

3. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, S22 uses Min-Max Scaling to normalize the preprocessed data, in the form of... in: X represents the standardized data, and X represents the original data.

4. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, The S23 time series data consists of a list with a length equal to the number of txt files. Each list contains a dataframe, which corresponds to the data in the txt file. The data in each dataframe is then extracted and horizontally concatenated to obtain the complete time series data, which has dimensions [N, K]. Here, N represents the number of items in the time series data, i.e., the number of rows in the dataframe, and K represents the selected K-dimensional features.

5. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, The segmentation and truncation process for each data item in S24 is as follows: [0,0+P] data items are taken from each data item as the input of the Transformer encoder, and the [P,P+Q]th item is taken as the true value of the predicted data. The input of the Transformer decoder is composed of start_symobol (an array of K 1s, in the form of [1,1,1,...,1]) and the true value of the predicted data after removing the first row. For the state classification task, the "flag" field in the [P,P+Q]th item constitutes the true value of the category data.

6. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, Specifically, S31 refers to the current flight state parameter sequence as follows: Where batch_size is the batch size. This represents the number of state points in the current flight state parameter sequence. To obtain a high-dimensional flight state parameter sequence, a Linear1 layer is used to perform feature upscaling on the current flight state parameter sequence. (1) in For a fully connected layer, a high-dimensional flight state parameter sequence ; The high-dimensional parameter space after feature extraction; Use Position Embdedding to add position encoding information to the advanced flight state parameter sequence; After passing through the position encoding module, a high-dimensional flight state parameter sequence containing position information is obtained: (4) Among them, the high-dimensional flight state parameter sequence with position encoding ; After obtaining the high-dimensional flight state parameter sequence with position encoding Then, the Transformer Encoder module is used to extract the sequence of high-level flight state parameters; The overall process can be described as follows: (8) Among them, the advanced flight status parameter sequence .

7. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, The Transformer Encoder uses two layers, each consisting of a multi-head attention network and an FFN network.

8. The post-stall prediction method based on multi-task Transformer according to claim 7, characterized in that, The formula for multi-head attention is: (5) Where h is the number of heads in multi-head attention. , , , , ; The formula is: (6) Furthermore, the formula for the FFN network is: (7) in , .

9. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, Specifically, S32 involves inputting the predicted flight state parameter sequence output by the model into the model decoder during the multi-step prediction process. The predicted flight state parameter sequence is represented as follows: Where batch_size is the batch size. q To predict the number of state points in the flight state parameter sequence, t To obtain the state parameter sequence, a Linear2 layer is used to perform feature upscaling on the predicted flight state parameter sequence, resulting in a high-dimensional predicted flight state parameter sequence. (9) Linear2 is a fully connected layer. High-dimensional prediction of flight state parameter sequences ; The Position Embedding layer in the model encoder is used to predict the flight state parameter sequence in high dimension. Provide location encoding information to obtain a high-dimensional predicted flight state parameter sequence containing location encoding information: (10) Obtain a high-dimensional predicted flight state parameter sequence with position encoding. Then, the Transformer Decoder module is used to generate the final predicted flight state parameter sequence. The Transformer Decoder process is consistent with the Transformer Encoder and can be described as the final predicted flight state parameter sequence. , (11) 。 10. The post-stall prediction method based on multi-task Transformer according to claim 1, characterized in that, Specifically, S33 involves: for the main task, using a Linear3 vector to perform depth calculations to obtain an intermediate vector. (12) in , is a fully connected layer, and is the intermediate vector of the main task. ; Next, the Status Predictor is used to predict the state. (13) in It is a fully connected layer, and the output vector of the main task is... ; For auxiliary tasks, a Linear4 algorithm is used for depth calculation to obtain an intermediate vector: (14) in , is a fully connected layer, and serves as the intermediate vector for the auxiliary task. ; Finally, TypeClssify is used to perform state classification to obtain the output vector of the auxiliary task: (15) in This is a fully connected layer, where type_num represents the number of flight state types, and it is used for auxiliary task output. .

Images