Helicopter flight action recognition method based on flight parameters

Abstract

A method for recognizing helicopter flight actions based on flight parameters relates to the field of flight action recognition. The present invention is to solve the problems of low recognition accuracy and waste of resources in existing helicopter action recognition methods. The present invention includes: obtaining flight parameter data of a helicopter, and marking flight action labels for the flight parameter data; preprocessing the flight parameter data marked with flight action labels to obtain preprocessed flight parameter data; combining features of the preprocessed flight parameter data to obtain a feature data set, and dividing the feature data set into a training set, a verification set, and a test set; using the training set to train a BiLSTM‑gMLP deep neural network model; using the verification set and the test set to verify and test the trained BiLSTM‑gMLP deep neural network model to obtain a flight action recognition model. The flight parameter data to be identified is input into the flight action recognition model to obtain a recognition result. The present invention is used to identify helicopter flight actions.

Description

Technical Field

[0001] The invention relates to the field of flight action recognition, and in particular to a helicopter flight action recognition method based on flight parameters. Background Art

[0002] As a kind of flying tool with flexible maneuverability, high convenience and strong adaptability to climate and environment, the movement law of helicopter is relatively complex, and different operation methods are required for different actions. The flight operation of helicopter has many coordinated actions and is difficult. Traditional flight training mainly relies on flight instructors to fly. In the initial stage, the coordination between helicopter operation and response action is poor, and it is easy to form habitual actions, slow growth of flight technology and low training efficiency. According to the investigation of researchers, in addition to equipment quality and climate factors, improper flight operation is one of the important causes of flight accidents. In order to improve the flight technology level of helicopter pilots and ensure the safe flight of helicopters, it is necessary to accurately identify and evaluate the flight actions of helicopters. Helicopter flight action recognition refers to the technology of automatically identifying a series of helicopter flight modes, which has important applications in many fields. However, due to the chaos of helicopter flight data and the professional complexity of the recognition process, helicopters face many difficulties and challenges.

[0003] The current helicopter action recognition method mainly adopts the helicopter action recognition method based on expert system. This method recognizes flight actions by compiling action feature criterion library. However, this method has the problems of long time consumption, mechanical feature criterion, and reliance on expert experience, resulting in poor multi-dimensional data fusion effect and poor generalization ability, thus making the helicopter action recognition accuracy low. At the same time, the action recognition process consumes a lot of manpower and material resources, resulting in a waste of resources. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of low recognition accuracy and waste of resources in the existing helicopter action recognition methods, and propose a helicopter flight action recognition method based on flight parameters.

[0005] The specific process of the helicopter flight action recognition method based on flight parameters is as follows:

[0006] Acquire helicopter flight parameter data of the action to be identified, perform format conversion on the helicopter flight parameter data of the action to be identified, input the format-converted helicopter flight parameter data of the action to be identified into a helicopter flight action recognition model, and obtain the helicopter flight action;

[0007] The helicopter flight action recognition model is obtained by:

[0008] Step 1: Obtain the flight parameter data of the helicopter and label the flight action labels for the flight parameter data;

[0009] Step 2: preprocessing the flight parameter data marked with the flight action label to obtain the preprocessed flight parameter data;

[0010] Step 3: Combine the preprocessed flight parameter data to obtain a feature data set, and divide the feature data set into a training set, a validation set, and a test set;

[0011] Step 4: Use the training set to train the BiLSTM-gMLP deep neural network model to obtain a trained BiLSTM-gMLP deep neural network model;

[0012] Step 5. Use the validation set to verify the trained BiLSTM-gMLP deep neural network model to obtain the accuracy of the trained BiLSTM-gMLP deep neural network model. If the accuracy of the trained BiLSTM-gMLP deep neural network model is less than the first preset threshold, adjust the hyperparameters of the BiLSTM-gMLP deep neural network model, and return to step 4 for retraining until the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the preset first threshold, and obtain the optimal BiLSTM-gMLP network; if the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the first preset threshold, the currently trained BiLSTM-gMLP deep neural network model is the optimal BiLSTM-gMLP network, and then execute step 6;

[0013] Step 6. Use the optimal BiLSTM-gMLP network in the test set to test and obtain the accuracy of the optimal BiLSTM-gMLP network. If the accuracy of the optimal BiLSTM-gMLP network is less than the second accuracy threshold, further adjust the hyperparameters of the optimal BiLSTM-gMLP network while increasing the number of samples in the training set, and return to step 4 until the accuracy of the optimal BiLSTM-gMLP network is greater than or equal to the second accuracy threshold to obtain a helicopter flight action recognition model.

[0014] Furthermore, the format conversion of the helicopter flight parameter data of the action to be identified is specifically as follows:

[0015] Firstly, the helicopter flight parameter data of the to-be-identified action is divided by sliding based on a preset fixed-length time window to obtain multiple groups of to-be-identified flight parameter data;

[0016] Then, normalizing or standardizing the multiple groups of flight parameter data to be identified to obtain normalized or standardized flight parameter data to be identified;

[0017] Finally, the standardized or normalized flight parameter data to be identified is converted into a tensor form to obtain the flight parameter data to be identified after format conversion.

[0018] Furthermore, the step 1 of obtaining the flight parameter data of the helicopter and labeling the flight action labels for the flight parameter data is specifically as follows:

[0019] First, the flight parameter data of the helicopter is obtained, and the critical values ​​of each flight parameter data in different time periods are obtained based on the sliding window;

[0020] Then, based on the critical values ​​of each flight parameter data in different time periods, a flight action label is set for the flight parameter data;

[0021] The flight actions include: level flight, ascent, descent, turning, and hovering;

[0022] Finally, the labels of the flight parameter data are verified. If all the labels of the flight parameter data are correctly labeled, step 2 is executed; if there are incorrectly labeled labels, the incorrectly labeled labels are modified, and then step 2 is executed.

[0023] Furthermore, the flight parameter data of the helicopter includes: speed, pitch angle, roll angle, yaw angle, and radio altitude.

[0024] Furthermore, the flight parameter data marked with the flight action label in step 2 is preprocessed to obtain a preprocessed flight parameter data set, specifically:

[0025] Step 21: performing category imbalance processing on the flight parameter data marked with flight action labels; Step 22: performing normalization or standardization processing on the flight parameter data after category imbalance processing to obtain a preprocessed flight parameter data set;

[0026] Among them, the category of flight parameter data greater than the preset category threshold is the majority category, and the category of flight parameter data less than the preset category threshold is the minority category.

[0027] Furthermore, the flight parameter data marked with flight action labels in step 21 is subjected to category imbalance processing, specifically:

[0028] Data enhancement processing is performed on the flight parameter data of the minority categories in the flight parameter data marked with flight action labels, and data reduction processing is performed on the flight parameter data of the majority categories in the flight parameter data marked with flight action labels.

[0029] Furthermore, the data enhancement is implemented using a DBA algorithm.

[0030] Furthermore, the data attenuation is achieved by using an interval sampling method.

[0031] Furthermore, the normalization of the flight parameter data after the category imbalance processing is implemented by using the Min-Max normalization method.

[0032] Furthermore, the BiLSTM-gMLP network includes: a BiLSTM subnetwork, a gMLP subnetwork, a time feature extraction layer, a concat feature connection layer, and a fully connected layer;

[0033] The BiLSTM sub-network includes: LSTM units and GRU units;

[0034] The LSTM unit is used to process the flight parameter data and input the obtained features into the GRU unit and the temporal feature extraction layer;

[0035] The GRU unit is used to process the features output by the LSTM, and input the features output by the GRU unit into the temporal feature extraction layer;

[0036] The time feature extraction layer is used to obtain the maximum feature and average feature of the LSTM unit output feature, the maximum feature and average feature of the GRU unit output feature, and combine the maximum feature and average feature of the LSTM unit output feature, the maximum feature and average feature of the GRU unit output feature, to obtain the time feature in the flight parameter data, and input the time feature to the Concat feature connection layer;

[0037] The gMLP sub-network is used to extract the spatial features of the flight parameter data and input the spatial features into the Concat feature connection layer;

[0038] The Concat feature connection layer is used to perform Concat feature connection on the time feature and the space feature, and input the connected time feature and space feature into the fully connected layer;

[0039] The fully connected layer is used to output the probability of the helicopter's flight action category according to the connected temporal features and spatial features.

[0040] The beneficial effects of the present invention are:

[0041] The present invention inputs the newly acquired flight parameter data into the helicopter action recognition model, and uses the trained and optimized model to accurately identify the various flight action categories during the helicopter flight process. The helicopter action recognition model proposed by the present invention can meet the needs under different flight conditions and scenarios, improve the accuracy of helicopter flight action recognition, and thus effectively improve the flight safety and efficiency of the helicopter. At the same time, the helicopter action recognition model proposed by the present invention is used for action recognition, and there is no need to compile an action feature criterion library, and there is no need for the flight instructor to observe the flight action of the helicopter at every moment, thereby reducing the waste of manpower and material resources and saving resources. By using the present invention to monitor and identify flight actions in real time, the crew can grasp the flight status information more timely, make decisions and adjust the flight plan in time, so as to better ensure the safety of the pilot and the aircraft. The helicopter action recognition method proposed by the present invention can be widely used in aspects such as helicopter flight training, unmanned driving and autonomous control, and has provided important support for the development of the aviation industry. At the same time, it improves the safety performance and operating efficiency of the helicopter, and has broad application prospects and market potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is the overall flow chart of the present invention;

[0043] Figure 2 A schematic diagram of the categories of helicopter flight actions of the present invention;

[0044] Figure 3 A schematic diagram of the unbalanced processing of helicopter flight parameter data categories according to the present invention;

[0045] Figure 4 A schematic diagram of a method for labeling helicopter flight action categories according to the present invention;

[0046] Figure 5 It is a schematic diagram of the weights of various features of helicopter flight parameter data of the present invention;

[0047] Figure 6 The BiLSTM-gMLP model intention for helicopter flight action recognition of the present invention;

[0048] Figure 7 The figure shows the performance comparison between the BiLSTM-gMLP model and other models. DETAILED DESCRIPTION

[0049] Specific implementation method: Figure 1 As shown, the helicopter flight action recognition method based on flight parameters in this embodiment has the following specific process:

[0050] Acquire helicopter flight parameter data of the action to be identified, convert the format of the helicopter flight parameter data of the action to be identified, and input the helicopter flight parameters of the action to be identified after the format conversion into a helicopter flight action recognition model to obtain the helicopter flight action;

[0051] The specific format conversion of the helicopter flight parameter data of the action to be identified is as follows:

[0052] Firstly, the helicopter flight parameter data of the action to be identified is divided based on the sliding of a time window with a preset fixed length to obtain multiple sets of flight parameter data to be identified; the time window length is set to 11, and the window sliding step is set to 5;

[0053] Then, normalizing or standardizing the multiple groups of flight parameter data to be identified to obtain standardized or normalized flight parameter data to be identified;

[0054] Finally, using the PyTorch framework, the standardized or normalized flight parameter data to be identified is converted into a tensor with a shape of (M, S, N). Among them, M is the number of samples, S is the number of feature variables, and N is the amount of data processed at each time step;

[0055] The helicopter flight action recognition model is obtained by:

[0056] Step 1: Obtain the flight parameter data of the helicopter and label the flight action labels for the flight parameter data;

[0057] First, read the helicopter's flight parameter data and calculate the critical values ​​of each flight parameter in different time periods based on the sliding window:

[0058] Read the flight parameter data of the helicopter, analyze and interpret the actual meaning of each feature in the flight parameter data and the degree of factors affecting the helicopter's flight movements; flight parameters refer to various physical quantities and signals collected by flight parameter instruments during the helicopter's flight process, reflecting the helicopter's flight status and performance. These parameters include but are not limited to the helicopter's speed, altitude, pitch angle, roll angle, yaw angle, pitch angular velocity, roll angular velocity, yaw angular velocity and radio altitude in a fixed reference system, as shown in Table 1-2, and each feature has a clear practical meaning.

[0059] Table 1

[0060]

[0061] Table 2

[0062]

[0063] According to the parameter change rules of different flight actions of the helicopter in the flight parameter data, the critical value of each flight parameter data feature is calculated. The critical value of the flight parameter refers to the minimum or maximum parameter value that can distinguish different flight actions, that is, the flight parameter interval value of two different flight actions. The parameter change rules of the flight parameter data are shown in Table 3.

[0064] Table 3

[0065]

[0066] The present invention can fully utilize flight parameter data to identify the characteristics and laws of helicopters under different flight actions, so as to better understand the flight process and state of the helicopter. At the same time, since the collected flight parameters and characteristics are all expressed as physical quantities under a fixed reference system, the differences and connections between different flight actions can be more accurately analyzed and compared, which is conducive to the accurate identification and evaluation of helicopter flight actions.

[0067] Then, flight action labels are annotated for flight parameter data based on sliding windows and programs, and the 3D rendering platform is used to annotate and verify flight action categories of auxiliary lines;

[0068] In order to distinguish the flight actions, a combination of sliding window and program structure is used to effectively mark the categories of helicopter flight parameter data in different time periods.

[0069] The 3D rendering platform is used to demonstrate the flight characteristics and movements of the helicopter in different flight postures. With the help of visual flight trajectory analysis, the flight characteristics and movements of the helicopter can be evaluated more accurately, the visibility and operability of the helicopter flight data can be improved, and the flight action category can be further assisted in the labeling, which can promote more accurate analysis and identification of the data. The 3D rendering platform refers to a software system that can generate a three-dimensional model of the helicopter and draw the flight trajectory based on the flight parameter data. This rendering platform includes but is not limited to Unity, Unreal Engine and other alternative rendering engines.

[0070] According to the mission type and flight phase of the actual helicopter flight, the flight process is divided into different flight actions, including but not limited to helicopter level flight, ascent, descent, turn, hovering and other categories of any action, such as Figure 2 As shown in the table, level flight refers to the helicopter flying forward in a horizontal posture; ascending action refers to the helicopter ascending vertically while moving forward; descending action refers to the helicopter descending vertically while moving forward; circling action refers to the helicopter rotating around a fixed point; hovering action refers to the helicopter remaining motionless in the vertical direction; flight action labels are shown in Table 4:

[0071] Table 4

[0072] Flying action Level flight rise Downward Turning Hover Label 0 1 2 3 4

[0073] By dividing and labeling the flight data, the characteristics and rules of each flight action can be described more accurately, which can facilitate the identification and analysis of flight actions. The schematic diagram of the flight action category labeling method is shown in the figure. Figure 4 As shown in the figure, a sliding window is used to select a fixed-length time window data, and slide down at the same step length to read all the data in sequence. The central value of each feature in each time window is calculated, and the differences between the time windows are compared to determine the critical value of each flight parameter feature, and the action category of the helicopter flight parameter data is marked. Then, the Unreal Engine is used to verify the marked flight parameter data. If the marking error is verified, the marked data is modified and step 2 is executed.

[0074] These labeled data also provide the necessary data foundation for subsequent deep learning-based methods, so that different flight actions can be predicted and identified more accurately. The present invention uses comprehensive calculation and visual display to carefully classify and label the actual flight process, which can better adapt to the different application scenarios and specific target mission requirements of helicopters, improve the understanding and mastery of the flight process, and thus more accurately and comprehensively evaluate the flight actions of helicopters.

[0075] Step 2: Preprocess the flight parameter data with the flight action labels to obtain the preprocessed flight parameter data, specifically:

[0076] Step 21: Perform category imbalance processing on the flight parameter data with flight action labels:

[0077] When processing labeled flight parameter data, the data is made more standardized and reasonable by optimizing the processing method, so that it can be better passed to the deep neural network for learning and analysis. Among them, the category imbalance processing method can expand or reduce the majority and minority category samples to achieve a balanced effect of the imbalanced data distribution and improve the generalization and accuracy of the model. Specifically:

[0078] Data enhancement is performed on the flight parameter data of a few categories whose values ​​are less than a preset category threshold in the flight parameter data labeled with flight action labels, and data reduction is performed on the flight parameter data of the majority categories whose values ​​are greater than the preset category threshold;

[0079] The data enhancement is achieved by using a DBA algorithm that can enhance time series data; the data reduction is achieved by using an interval sampling method;

[0080] Since the number of multi-category samples is relatively high and the difference between each frame of data is small, we adopt the interval sampling method to reduce the amount of multi-category data. For the small number of minority category data, we use the data enhancement method to increase the number of opposing minority category data in reverse; these processing operations help improve the generalization and accuracy of the model, such as Figure 3 shown.

[0081] Step 22: Perform Min-Max normalization or standardization on the flight parameter data with flight action labels after category imbalance processing to obtain pre-processed flight parameter data:

[0082] Normalization or standardization in preprocessing operations can scale feature data to the same scale to avoid the impact of numerical differences between features on model training and recognition. The application of these preprocessing methods helps to improve the accuracy, reliability and usability of the model, and also provides more complete and accurate data support for the recognition and analysis of helicopter flight movements, better adapting to the needs of actual application scenarios.

[0083] The present invention adopts the Min-Max normalization method to achieve the scaling of feature data. This can avoid the influence of numerical differences between features on model training and recognition, and improve the training and prediction effects of the model, as shown in Tables 5-6, Table 5 is the data before normalization, and Table 6 is the data after normalization;

[0084] Table 5

[0085]

[0086] Table 6

[0087]

[0088] Step 3: Perform feature selection and feature combination on the preprocessed flight parameter data to obtain a feature data set, and divide the feature data set into a training set, a validation set, and a test set in proportion, specifically:

[0089] Step 31: Perform feature selection in the preprocessed flight parameter data to obtain the flight parameters after feature selection:

[0090] Firstly, the influence of different flight parameter features in the preprocessed flight parameter data on flight action recognition is obtained based on the method of comparing different groups of features:

[0091] The different types of preprocessed flight parameter data are sorted according to the order of data acquisition, and then one type of flight parameter features are removed from the back to the front to obtain multiple flight parameter data sets; each flight parameter data set is used to train LSTM, gMLP, and BiLSTM-gMLP in turn to obtain the accuracy of each trained LSTM, gMLP, and BiLSTM-gMLP. From an overall perspective, the influence of different flight parameter features in the preprocessed flight parameter data on flight action recognition is determined according to the accuracy of each trained LSTM, gMLP, and BiLSTM-gMLP model; as shown in Table 7.

[0092] Table 7

[0093]

[0094] Then, a feature selection algorithm is used to perform feature selection on the flight parameter data to obtain the flight parameter data after feature selection;

[0095] The Filter filtering method is used to assign weights to the preprocessed flight parameter data, and then the preprocessed flight parameter data corresponding to the weight greater than the weight threshold are retained to obtain a roughly screened flight parameter data set;

[0096] In the flight parameter data set after coarse screening, 1 to n-1 flight parameter data are removed in sequence (n is the total number of flight parameter data in the flight parameter data set after coarse screening) to form multiple flight parameter sub-datasets. The LSTM, gMLP, and BiLSTM-gMLP trained by each flight parameter sub-dataset are used, combined with the accuracy of the trained LSTM, gMLP, and BiLSTM-gMLP, or the flight parameter data after feature selection. As shown in Table 8, the accuracy of the LSTM model and BiLSTM-gML model trained for the flight parameter sub-dataset corresponding to the 6th row has increased significantly, and the accuracy of the trained gMLP has decreased slightly, and this slight decrease in accuracy can be ignored. Therefore, the flight parameter data in the second to last row, namely speed, pitch angle, roll angle, yaw angle, and radio altitude, are finally selected.

[0097] Table 8

[0098]

[0099] The feature selection method is to eliminate redundant features and unimportant features from the preprocessed flight parameters, including: filter filtering method, wrapping method, embedding method and other commonly used feature selection algorithms, or correlation analysis, information gain, chi-square test and other methods. In addition, the most relevant features of the flight action can also be selected from the flight parameter data by manual screening methods based on expert knowledge or machine learning. This method can improve the training efficiency and accuracy of subsequent models and avoid problems such as overfitting and underfitting.

[0100] The present invention adopts the Filter filtering method to assign weights to each dimension of features, and then selects the best feature subset to improve the performance of the model, thereby screening out features that have a greater impact on helicopter action recognition, such as Figure 5 This method can avoid losing too much key information in the feature selection process and more accurately describe the influence of each feature on the helicopter flight action.

[0101] Step 32: Combine the flight parameter data after feature selection in step 31, and then convert the format of the combined flight parameter data to obtain a feature data format that is suitable for the model input port:

[0102] The format conversion of the flight parameter data after feature combination is specifically as follows:

[0103] Firstly, the flight parameter data after feature combination is divided based on the sliding of a time window with a preset fixed length to obtain multiple sets of flight parameter data; the time window length is set to 11, and the window sliding step is set to 5;

[0104] Then, normalizing or standardizing the multiple sets of flight parameter data to obtain standardized or normalized flight parameter data;

[0105] Finally, using the PyTorch framework, the standardized or normalized flight parameter data is converted into a tensor of shape (M, S, N). Among them, M is the number of samples, S is the number of feature variables, and N is the amount of data processed at each time step. Make sure that the dimension and shape of the data match the input requirements of the model.

[0106] In this step, different feature selection methods will result in different feature combinations. By rationally selecting and optimizing feature combinations, we can improve our ability to understand and analyze data, improve the accuracy and reliability of subsequent model training and decision-making, and thus improve the performance of the model. At the same time, by continuously optimizing feature combinations, we can better explore the potential correlation and regularity of data, obtain more accurate flight reference data analysis results, improve the accuracy of helicopter flight maneuvers recognition and analysis, and enhance flight safety and efficiency.

[0107] Feature combination methods include artificial feature combination, experimental feature combination and other methods. Feature combination refers to the splicing or fusion of selected features according to certain rules or methods to form a new feature vector. Feature data set refers to the feature vector combination that can distinguish different flight actions to the greatest extent and improve the accuracy and efficiency of recognition and evaluation.

[0108] Step 3: Divide the feature data set into training set, validation set and test set according to the proportion:

[0109] The feature data set is divided according to the 6:2:2 rule to obtain the training set, validation set, and test set.

[0110] Step 4: Use the training set to train the BiLSTM-gMLP deep neural network model to obtain a trained BiLSTM-gMLP deep neural network model;

[0111] The BiLSTM-gMLP deep neural network model structure includes: a BiLSTM improved bidirectional long short-term memory network (BiLSTM sub-network), an average and maximum feature extraction layer, a gMLP gated multi-layer perceptron network (gMLP sub-network), a time feature extraction layer, a Concat feature connection layer, and a fully connected layer;

[0112] The improved BiLSTM bidirectional long short-term memory network includes: LSTM unit and GRU unit;

[0113] The LSTM unit is used to process the flight parameter data and input the obtained features into the GRU unit and the time feature extraction layer. LSTM (Long Short-Term Memory) is a variant of the recurrent neural network that captures and transmits long-term dependencies by using gating units. It can effectively learn and store important information in the input sequence, avoid problems such as gradient disappearance and gradient explosion, and thus better process time series data.

[0114] The GRU unit is responsible for processing the features of the LSTM output and inputting the features of the GRU output into the temporal feature extraction layer. GRU (Gated Recurrent Unit) is also a variant of a recurrent neural network, but it uses a simpler gating structure. The GRU layer can better capture contextual information in the sequence by selectively updating and resetting its hidden state.

[0115] The time feature extraction layer is used to obtain the average and maximum values ​​of the output features of the LSTM unit, the average and maximum values ​​of the output features of the GRU unit, combine the average and maximum values ​​of the output features of the LSTM unit, and the average and maximum values ​​of the output features of the GRU unit, obtain the time features in the flight parameter data, and input the time features into the Concat feature connection layer;

[0116] The temporal feature extraction layer combines the operations of averaging and maximizing to extract useful and representative features from the output sequence of the BiLSTM model, taking into account the global trend and key information of the flight parameter data. Such feature representation has stronger expressive power in subsequent tasks and more effectively captures the temporal features in the flight parameter data.

[0117] The gMLP network is used to capture the spatial features of the flight parameter data, and input the spatial features into the Concat feature connection layer;

[0118] The Concat feature connection layer is used to perform Concat feature combination on the temporal feature and the spatial feature, and input the combined feature into the fully connected layer;

[0119] The fully connected layer is used to process the combined features using a softmax function, and use neurons to output the probability of the flight action category pre-set for each helicopter;

[0120] The fully connected layer uses 5 neurons and takes the maximum output neuron as the input flight action label.

[0121] The core of the BiLSTM-gMLP deep neural network model includes a modified bidirectional BiLSTM model, a gMLP model, an average and maximum feature extraction layer, and a fully connected layer. The first layer of the bidirectional BiLSTM layer uses an LSTM layer, which uses gated units to capture and transmit long-term dependencies. It can effectively learn and store important information in the input sequence, avoid problems such as gradient disappearance and gradient explosion, and thus better process time series data. The second layer uses a GRU layer. Compared with LSTM, GRU has fewer parameters and a simpler architecture. By selectively updating and resetting its hidden state, it can better capture contextual information in the sequence and improve the training efficiency of the model. The average and maximum feature extraction layers are connected after the bidirectional BiLSTM layer. Combined with the operations of finding the average and maximum values, useful and representative features can be extracted from the output sequence of the BiLSTM model. The global trend and key information of the flight parameter data are comprehensively considered, and the temporal features in the flight parameter data are more effectively captured for further training. At the same time, gMLP is a global multi-layer perceptron that uses the gMLP layer to capture the spatial features of the flight parameter data. The entire architecture greatly reduces the complexity of the model. Finally, the temporal and spatial features extracted by the two models are combined by concat, and the combined features are passed to the fully connected layer, and the probability of each helicopter's pre-set flight action category is output through the softmax function. In this layer, we use 5 neurons and take the maximum output neuron as the input flight action label.

[0122] The BiLSTM-gMLP deep neural network model is a deep learning model that combines a bidirectional long short-term memory network (BiLSTM) and a gated multilayer perceptron (gMLP). It is very effective in processing time series data and capturing the dynamic changes and complex relationships of flight actions. The use of the BiLSTM-gMLP deep neural network model can better adapt to the characteristics and laws of different flight actions, improve the generalization ability and recognition accuracy of the model, and provide more efficient and accurate support for flight control and guidance. Figure 6 The BiLSTM-gMLP hybrid model used in the present invention is demonstrated. The BiLSTM model is used to extract features from time series data, extract the relevant laws of flight parameter data in the time dimension, and obtain the time characteristics of the data. The gMLP model is used to extract features from spatial series data, extract the relevant laws of flight parameter data in the spatial dimension, and obtain the spatial characteristics of the data. The time characteristics and spatial characteristics are fused to obtain the recognition results of different helicopter actions. Therefore, the BiLSTM-gMLP deep neural network model can learn the time and space characteristics in the flight parameter data more comprehensively, thereby improving the accuracy and efficiency of helicopter flight action recognition and analysis.

[0123] Step 5. Use the validation set to verify the trained BiLSTM-gMLP deep neural network model to obtain the accuracy of the trained BiLSTM-gMLP deep neural network model. If the accuracy of the trained BiLSTM-gMLP deep neural network model is less than the first preset threshold, adjust the hyperparameters of the BiLSTM-gMLP deep neural network model and retrain until the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the preset first threshold, and obtain the optimal BiLSTM-gMLP network; if the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the first preset threshold, the currently trained BiLSTM-gMLP deep neural network model is the optimal BiLSTM-gMLP network, and then execute step 6;

[0124] Repeated training and tuning are performed through training sets and validation sets. Repeated training and tuning refers to the use of methods such as gradient descent, cross entropy loss function, and stochastic gradient descent. First, the model is trained using the training set. The error between the model's predicted value and the true label is calculated using the gradient descent method and the cross entropy loss function, and the model parameters are updated through back propagation.

[0125] Use the validation set to evaluate the performance of the model and calculate the accuracy of the model on the validation set. If the accuracy is lower than the set threshold, proceed to the next step of tuning. Adjust the model's hyperparameters based on the accuracy and performance of the validation set. Hyperparameters include learning rate, batch size, size of hidden layers, etc. You can try different hyperparameter combinations, compare their accuracy on the validation set, and choose the best performing combination. Retrain the model using the training set, using the updated hyperparameters and parameters.

[0126] Step 6: Use the optimal BiLSTM-gMLP network of the test set to test and obtain the accuracy of the optimal BiLSTM-gMLP network. If the accuracy of the optimal BiLSTM-gMLP network is less than the second accuracy threshold, increase the number of samples in the training set and adjust the hyperparameters of the optimal BiLSTM-gMLP network, and return to step 4 until the accuracy of the optimal BiLSTM-gMLP network is greater than or equal to the second accuracy threshold, and obtain a helicopter flight action recognition model;

[0127] Use the test set to evaluate the final performance of the model. Pass the test set into the trained model. The data that comes out is the flight parameter data file with the flight action annotated after the model prediction. Then use the 3D visualization platform to verify the accuracy of the output flight parameter file to obtain an estimate of the real performance of the model. If the accuracy is lower than the threshold: you can try to further adjust the model's hyperparameters and increase the amount of training data. Then retrain, verify, and evaluate the model until the expected accuracy is achieved. Repeating the training, tuning, and verification process can gradually improve the accuracy and efficiency of the model, making it better approximate real flight actions.

[0128] Example: To verify the beneficial effects of the present invention, the present invention uses a variety of models to conduct action recognition experiments on helicopter flight parameter data, uses the training set to train deep neural network models such as LSTM, gMLP and BiLSTM-gMLP for model parameter update and learning, uses the validation set to adjust hyperparameters, and preliminarily evaluates the capabilities of each model. The trained deep neural network models such as LSTM, gMLP and BiLSTM-gMLP are obtained; the helicopter action recognition model proposed above is continuously trained and tuned, and the test set is used to finally evaluate the accuracy of the model, and finally a helicopter flight action recognition model is obtained. The experimental results are as follows: Figure 7 As shown, Figure 7 The comparison results of the performance of the fusion model described in the present invention and other models are shown.

[0129] By conducting recognition experiments on multiple groups of helicopter flight parameter data, we found that under the same experimental conditions, the fusion model of the present invention has higher accuracy and stronger robustness, and can more comprehensively and accurately evaluate the flight action of the helicopter. At the same time, compared with other models, the fusion model proposed in the present invention also has higher generalization performance, can adapt to various helicopter flight action scenes, and show better practical value and application prospects. Therefore, the flight parameters to be identified in the present invention are input into the helicopter action recognition model, and the trained and optimized model can accurately identify the flight actions of various categories during the helicopter flight process, and output the corresponding recognition results. As the model is continuously improved, it can meet the needs of different flight conditions and scenarios, and improve the adaptability and accuracy of the model. As a result, the flight safety and efficiency of the helicopter are effectively improved. By real-time monitoring and identification of flight actions, the crew can grasp the flight status information more timely, so as to make better decisions and adjust the flight plan to ensure the safety of the pilot and the aircraft.

Claims

1. A helicopter flight action recognition method based on flight parameters, characterized in that The specific process of the method is: obtaining helicopter flight parameter data of the action to be identified, performing format conversion on the helicopter flight parameter data of the action to be identified, inputting the helicopter flight parameter data of the action to be identified after format conversion into a helicopter flight action recognition model, and obtaining the helicopter flight action; The helicopter flight action recognition model is obtained by: Step 1: Obtain the flight parameter data of the helicopter and label the flight action labels for the flight parameter data; Step 2: preprocessing the flight parameter data marked with the flight action label to obtain the preprocessed flight parameter data; Step 3: Combine the preprocessed flight parameter data to obtain a feature data set, and divide the feature data set into a training set, a validation set, and a test set; Step 4: Use the training set to train the BiLSTM-gMLP deep neural network model to obtain a trained BiLSTM-gMLP deep neural network model; The BiLSTM-gMLP network includes: a BiLSTM subnetwork, a gMLP subnetwork, a time feature extraction layer, a concat feature connection layer, and a fully connected layer; The BiLSTM sub-network includes: LSTM units and GRU units; The LSTM unit is used to process the flight parameter data and input the obtained features into the GRU unit and the temporal feature extraction layer; The GRU unit is used to process the features output by the LSTM, and input the features output by the GRU unit into the temporal feature extraction layer; The time feature extraction layer is used to obtain the maximum feature and average feature of the LSTM unit output feature, the maximum feature and average feature of the GRU unit output feature, and combine the maximum feature and average feature of the LSTM unit output feature, the maximum feature and average feature of the GRU unit output feature, to obtain the time feature in the flight parameter data, and input the time feature to the Concat feature connection layer; The gMLP sub-network is used to extract the spatial features of the flight parameter data and input the spatial features into the Concat feature connection layer; The Concat feature connection layer is used to perform Concat feature connection on the time feature and the space feature, and input the connected time feature and space feature into the fully connected layer; The fully connected layer is used to output the probability of the helicopter's flight action category according to the connected temporal features and spatial features; Step 5. Use the validation set to verify the trained BiLSTM-gMLP deep neural network model to obtain the accuracy of the trained BiLSTM-gMLP deep neural network model. If the accuracy of the trained BiLSTM-gMLP deep neural network model is less than the first preset threshold, adjust the hyperparameters of the BiLSTM-gMLP deep neural network model, and return to step 4 for retraining until the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the preset first threshold, and obtain the optimal BiLSTM-gMLP network; if the accuracy of the trained BiLSTM-gMLP deep neural network model is greater than or equal to the first preset threshold, the currently trained BiLSTM-gMLP deep neural network model is the optimal BiLSTM-gMLP network, and then execute step 6; Step 6. Use the test set to test the optimal BiLSTM-gMLP network to obtain the accuracy of the optimal BiLSTM-gMLP network. If the accuracy of the optimal BiLSTM-gMLP network is less than the second accuracy threshold, further adjust the hyperparameters of the optimal BiLSTM-gMLP network while increasing the number of samples in the training set, and return to step 4 until the accuracy of the optimal BiLSTM-gMLP network is greater than or equal to the second accuracy threshold to obtain a helicopter flight action recognition model.

2. The method for recognizing helicopter flight actions based on flight parameters according to claim 1, characterized in that: The format conversion of the helicopter flight parameter data for the action to be identified is specifically as follows: Firstly, the helicopter flight parameter data of the to-be-identified action is divided by sliding based on a preset fixed-length time window to obtain multiple groups of to-be-identified flight parameter data; Then, normalizing or standardizing the multiple groups of flight parameter data to be identified to obtain normalized or standardized flight parameter data to be identified; Finally, the standardized or normalized flight parameter data to be identified is converted into a tensor form to obtain the flight parameter data to be identified after format conversion.

3. The method for recognizing helicopter flight actions based on flight parameters according to claim 2, characterized in that: The step 1 of obtaining the flight parameter data of the helicopter and labeling the flight action labels for the flight parameter data is specifically as follows: First, the flight parameter data of the helicopter is obtained, and the critical values ​​of each flight parameter data in different time periods are obtained based on the sliding window; Then, based on the critical values ​​of each flight parameter data in different time periods, a flight action label is set for the flight parameter data; The flight actions include: level flight, ascent, descent, turning, and hovering; Finally, the labels of the flight parameter data are verified. If all the labels of the flight parameter data are correctly labeled, step 2 is executed; if there are incorrectly labeled labels, the incorrectly labeled labels are modified, and then step 2 is executed.

4. The method for recognizing helicopter flight actions based on flight parameters according to claim 3 is characterized in that: The flight parameter data of the helicopter include: speed, pitch angle, roll angle, yaw angle, and radio altitude.

5. The method for recognizing helicopter flight actions based on flight parameters according to claim 4, characterized in that: The flight parameter data marked with the flight action labels in step 2 are preprocessed to obtain the preprocessed flight parameter data set, specifically: Step 21: Perform category imbalance processing on the flight parameter data marked with flight action labels; Step 22: Normalize or standardize the flight parameter data after category imbalance processing to obtain a preprocessed flight parameter data set.

6. The method for recognizing helicopter flight actions based on flight parameters according to claim 5, characterized in that: The step 21 in which the flight parameter data with flight action labels are subjected to category imbalance processing is specifically as follows: Performing data enhancement processing on the minority categories of flight parameter data in the flight parameter data labeled with flight action labels, and performing data weakening processing on the majority categories of flight parameter data in the flight parameter data labeled with flight action labels; Among them, the category of flight parameter data greater than the preset category threshold is the majority category, and the category of flight parameter data less than the preset category threshold is the minority category.

7. The method for recognizing helicopter flight actions based on flight parameters according to claim 6, characterized in that: The data enhancement is implemented using the DBA algorithm.

8. The method for recognizing helicopter flight actions based on flight parameters according to claim 7, characterized in that: The data attenuation is achieved by using an interval sampling method.

9. The method for recognizing helicopter flight actions based on flight parameters according to claim 8, characterized in that: The normalization of the flight parameter data after the category imbalance processing is achieved by using the Min-Max normalization method.

Images