A method for predicting aviation sensor degradation based on long-term flight data

Abstract

The present invention discloses an aviation sensor degradation prediction method based on long time series flight data. First, data preprocessing is performed, and then the data is input into a residual module, and then into a cascade module. Finally, classification and prediction are achieved through a fully connected layer. Through experimental comparative analysis, it can be seen that the present invention has a very good effect on aviation sensor degradation prediction.

Description

Technical Field

[0001] The invention belongs to the field of aviation technology, and in particular relates to an aviation sensor degradation prediction method. Background Art

[0002] With the rapid development of the aviation industry, a large number of sensors of various types are used in the fields of aircraft manufacturing, testing and flight, aiming to accurately grasp the performance status of each component of the aircraft at each stage. The stability and reliability of the sensor are crucial to the entire process. However, due to regular calibration and the non-periodic nature of sensor degradation, it is difficult to grasp the timing of sensor removal and installation.

[0003] In view of this, it is crucial to detect the degree of degradation without removing the sensor from the aircraft. Therefore, the present invention introduces a neural network for time series data processing to predict the degradation trend of aviation sensors, aiming to judge the degree of degradation from the output of 25 sensors and mine potential features.

[0004] At present, anomaly detection mainly uses machine learning methods, including support vector machines, K-means, random trees, and naive Bayes. Among them, the support vector machine method focuses more on network security, such as the anomaly detection method proposed by Kulkarni et al. in the document "Kulkarni, A., Pino, Y., French, M., and Mohsenin, T. (2016). Real-Time Anomaly Detection Framework for Many-Core Router through Machine-Learning Techniques. ACM Journal on Emerging Technologies in Computing Systems, 13 (1), 1–22. doi: 10.1145 / 2827699". This method focuses more on efficiency, but may not be effective when facing complex data types. The K-means method is widely used in unsupervised learning, but compared with support vector machines, the accuracy achieved by this method may be lower. For example, Gaddam et al. proposed the cascaded K-means and ID3 decision tree learning method for anomaly detection in the document "SRGaddam, VVPhoha and K.S.Balagani, 'K-Means+ID3: A Novel Method for Supervised Anomaly Detection byCascading K-Means Clustering and ID3 Decision Tree Learning Methods,' in IEEE Transactions on Knowledge and Data Engineering, vol.19, no.3, pp.345-354, March2007, doi:10.1109 / TKDE.2007.44." In addition, random tree-based methods can analyze and generate sequences of multiple random tree ensembles, such as "A. Nanduri and L. Sherry, 'Anomaly detection in aircraft data using Recurrent Neural Networks (RNN), '2016 Integrated Communications Navigation and Surveillance (ICNS), 2016, pp. 5C2-1-5C2-8, doi: 10.1109 / ICNSUR V.2016.7486356.".While other algorithms are considering how to improve accuracy, naive Bayes-based methods focus on how to reduce computational overhead. Such methods can enable neural networks to be ported from high-performance servers to embedded devices, such as "M. Fliess, C. Join, and H. Sira-Ramirez, 'Non-linear estimation is easy,' Int. J. Model., Ident. Control, vol. 4, no. 1, pp. 12–27, 2008.".

[0005] However, due to the different types of sensors, complex structures, and the diversity of aviation flights, it is difficult to find a quantitative detection method to describe the degradation of all sensors. There are three main problems: first, sensors come in many types and record different parts of the aircraft, so it is difficult to search for degradation from a specific model; second, the gap between the two types of data is too small to be separated; in addition, how to deal with degradation in a timely manner is also a key issue. It is difficult for machine learning to solve the above three problems at the same time.

[0006] Experiments show that the LSTM network proposed by Graves in the paper "Graves A. "Long short-term memory [J]. Supervised sequence labelling with recurrent neural networks," 2012: 37-45." is much better than the recurrent neural network (RNN) proposed by HE et al. in the paper "K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778, doi: 10.1109 / CVPR.2016.90.". Thanks to the development of computing power, LSTM has been increasingly widely used in many classification and regression tasks. LSTM model structures such as Figure 1 As shown. Although LSTM units have achieved such exciting results, there is still a lack of work on deep learning composed of LSTM. Most works simply use LSTM as a feature extractor. By extracting features from high-dimensional data, fully connected linear and support vector machines are introduced as decoders of the model. These works have a common deficiency that they cannot perform well when faced with complex data. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the present invention provides an aviation sensor degradation prediction method based on long time series flight data. First, data preprocessing is performed, and then the data is input into a residual module, and then into a cascade module. Finally, classification and prediction are achieved through a fully connected layer. Through experimental comparative analysis, it can be seen that the present invention has a very good effect on aviation sensor degradation prediction.

[0008] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0009] Step 1: Data processing;

[0010] Step 1-1: Flight record data expressed as X t , where x i,j represents the jth value of sensor i, X t represents the tth flight;

[0011]

[0012] Step 1-2: For each flight, Y t By m y i Composition, represents the degradation of m sensors;

[0013]

[0014]

[0015] where y i It is a binary code with two values ​​0 and 1, 1 means degradation and 0 means normal; preposed data represents the mapping from X to Y;

[0016] Step 1-3: Resample the original sequence of flight record data to a fixed length, that is, equally spaced X t Each row is sampled to obtain

[0017]

[0018]

[0019] Step 1-4: Preprocess the data Encode into multiple fixed-length sequences and transfer the previous mapping relationship labels from X to Y to the fixed-length sequences;

[0020] Step 2: Input the fixed-length sequence obtained in step 1 into the residual module;

[0021] The residual module includes an LSTM unit with multiple hidden parameters, which is connected in parallel with the first layer of LTSM; the residual block with multiple layers of recursive LSTM is used to extract features and is combined with the initial features obtained by the first layer of LSTM;

[0022] Step 3: Input the features extracted from the residual module into the cascade module; the cascade module consists of three cascaded LSTM units;

[0023] Step 4: Finally, through a fully connected layer, the 32-dimensional result output in step 3 is converted into a 2-bit code to obtain the final classification prediction result.

[0024] The beneficial effects of the present invention are as follows:

[0025] The present invention can not only alleviate the problem of insufficient aviation flight data, but also alleviate the problem of poor global prediction caused by long time series data. By cascading LSTM networks and using contextual information to predict degradation rules, the degradation accuracy and efficiency of aviation sensors are greatly improved. A long-time series degradation model for multiple types of aviation sensors is constructed, the negative exponential degradation mechanism of the sensor's physical function is revealed, a multi-mode associated sensitivity analysis scheme is designed, and autonomous monitoring of multiple types of sensor functions is achieved. A joint learning method for multiple types of sensor data in complex environments is proposed, and the competitive degradation mechanism of sensor performance in complex environments is discovered, solving the problem of adaptive dynamic degradation modeling under complex disturbances. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is the structure diagram of the LSTM model.

[0027] Figure 2 It is a schematic diagram of the overall architecture of the present invention.

[0028] Figure 3 The figure is a flow chart of the method of the present invention.

[0029] Figure 4 Schematic diagram of the residual module of the method of the present invention.

[0030] Figure 5 It is a schematic diagram of the cascade module of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0032] The present invention proposes a cascade network architecture with a long short-term memory network (LSTM) and residual connections. The architecture can start from sensor recorded data, obtain the characteristics between different sensors in aviation flight, construct an aviation data set, and test the network in multiple data sets containing multiple types of sensors.

[0033] The present invention proposes a cascade network architecture based on LSTM for anomaly detection of aviation flight data. The first layer of the architecture is a residual module containing multiple hidden parameters, and its structure is a parallel connection of an LSTM unit and an LSTM combination unit, the purpose of which is to fuse the extracted features with the initial features. The data processed by the residual module will then be input into the cascade module, which is composed of three cascaded LSTM units. The last layer of the network is composed of a fully connected layer (Fully Connected Layer, FCN). The function of the FCN layer is to convert the result into a 2-bit hot encoding.

[0034] like Figure 2 and Figure 3 As shown, a method for predicting aviation sensor degradation based on long-time series flight data has the following steps:

[0035] 1: Data processing. First, the flight record data can be expressed as X t , where x i,j represents the jth value of sensor i, X t represents the tth flight.

[0036]

[0037] Second, for each flight, Y t By m y i Composition, represents the degradation of m sensors.

[0038]

[0039] Then, the original sequence is resampled to a fixed length, that is, equally spaced pairs X t Each row is sampled to obtain preprocessed data

[0040]

[0041]

[0042] Finally, the preprocessed data is encoded into multiple fixed-length sequences, and the previous mapping labels from X to Y are transferred to these sequences.

[0043] 2: If Figure 4 As shown in FIG. 1 , the data processed in step 1 is input into the residual module, which includes an LSTM unit with multiple hidden parameters, connected in parallel with the first layer, and uses a residual block with multi-layer recursive LSTM to extract features and combine them with the initial features.

[0044] 3: If Figure 5As shown in the figure, the features extracted by the residual module are then input into the cascade module, which consists of three cascaded LSTM units.

[0045] 4: Finally, there is a fully connected layer, the main function of which is to convert the 12 32-dimensional results into 2-bit encoding, which is the classification result of the entire architecture.

[0046] 5: Use the above steps to train and test all data sets, and compare and analyze the experimental results. Specific embodiment:

[0048] 1. Experimental setup.

[0049] 1) In terms of data set selection, we not only use self-built data sets, but also test the performance on two data sets: Lightning2 proposed in the paper "Eads DR, Hill D, Davis S, et al. Genetic algorithms and support vector machines for time series classification [C] / / Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation V. SPIE, 2002, 4787: 74-85." and Earthquake proposed in the paper "Berkeley Seismological Laboratory, The northern california earthquake datacenter, http: / / www.ncedc.org / ."

[0050] The detailed comparison of the three datasets is shown in Table 1:

[0051] Table 1 Dataset comparison

[0052] Dataset length Training set Test Set Earthquake 512 322 139 Lightning2 637 60 61 The present invention 300000 350 350

[0053] 2) In terms of evaluation indicators. Due to the task of the model, it focuses more on accuracy. In addition, the task is a binary classification, and its output uses one hot encoding as the output of the model. The loss function uses the binary cross entropy loss (Binary Cross Entropy, BCE) proposed by Zhang et al. in the literature "Zhang Z, Sabuncu M. "Generalized cross entropy loss for training deep neural networks with noisy labels [J]." Advances in neural information processing systems, 2018, 31." to evaluate the gap between the output and the actual label, instead of using the mean square error (MSE) loss, which has better performance in prediction rather than classification. When using MSE loss, the output of the model is close to a neutral result. Because the one hot encoding here does not represent the value to be predicted, it represents the classification result, which means that each bit of the encoding is mutually exclusive. Therefore, in the present invention, BCE loss is used to evaluate the gap between the output and the actual label.

[0054] 3) Details. The original data is sampled into multiple sequences of length 1024 and labeled. During the training process, the model of the present invention is trained on a 6GB GTX 1660 super GPU with a batch size of 16 or 32, and the model is trained using a combined optimization strategy, which includes the stochastic gradient descent (SGD) proposed by Robbins et al. in the document "Robbins H, Monro S. "A stochastic approximation method [J]." The annals of mathematical statistics, 1951: 400-407." and the adaptive momentum estimation (Adam) proposed by Kingma et al. in the document "Kingma DP, Ba J. "Adam: A method for stochastic optimization [J]." arXiv preprint arXiv: 1412.6980, 2014." The initial learning rate is 0.0001. For the testing process, each sequence put into the model is counted. Through the classification summary, the accuracy of the entire test set is obvious, and then the performance of the model on this data set is evaluated.

[0055] 2. Ablation experiment

[0056] 1) To prove the validity of the model of the present invention, the data set and the comparison data set are tested to prove that the model has strong generalization ability. Although the model may not achieve the best performance on other data sets, it still has certain competitiveness and the number of parameters is much less. On the self-built data set, the present invention respectively experiments on the MLP proposed by Murtagh in the document "Murtagh F. "Multilayer perceptrons for classification and regression [J]." Neurocomputing, 1991, 2 (5-6): 183-197.", the SVM proposed by Noble in the document "Noble W S. "What is a support vector machine? [J]." Nature biotechnology, 2006, 24 (12): 1565-1567.", RNN and LSTM models.

[0057] The experimental results are compared in Table 2:

[0058] Table 2 Comparison of experimental results

[0059] method BCEloss Accuracy F1 score Recall Accuracy SVM 0.632 55.20％ 70.10％ 82.80％ 60.80％ MLP 0.681 67.10％ 71.80％ 85.70％ 61.80％ RNN 0.585 63.90％ 75.10％ 91.20％ 63.80％ LSTM 0.537 68.90％ 78.90％ 98.10％ 66.00％ The present invention 0.458 80.00％ 87.70％ 95.80％ 80.90％

[0060] 2) In the comparison data set, the model of the present invention also achieved good results and has fewer parameters. As shown in Table 3:

[0061] Table 3 Experimental results of comparative datasets

[0062] method Lightning2 Earthquakes ResNet 77.00％ 71.20％ FCN 73.90％ 72.70％ STSF 72.46％ 76.91％ The present invention 76.67％ 78.10％

[0063] 3) Moreover, the generalization ability of the model of the present invention is strong. And compared with the method with the same parameters, the model has higher accuracy. As shown in Table 4:

[0064] Table 4 Comparison of different architectures

[0065] method BCE Loss Accuracy No residual module 0.548 73.10％ No cascade module 0.478 78.90 Complete Architecture 0.458 80.00％

[0066] 3. Experimental analysis

[0067] The method of predicting aviation sensor degradation from time series learning proposed in the present invention has been proven to be effective and effective through experiments, and has good performance in comparison with other data sets and network architectures with similar parameters.

[0068] The main innovative features of the present invention are:

[0069] 1. Propose a cascaded residual long short-term memory (CR-LSTM) network to process long-term aviation data. The CR-LSTM network is used to process long-term aviation data and detect anomalies in aviation data, achieving relatively good results.

[0070] 2. LSTM was first proposed to be applied to anomaly detection in aviation flight data.

Claims

1. A method for predicting aviation sensor degradation based on long-term flight data, characterized in that: The steps include: Step 1: Data processing; Step 1-1: Flight record data expressed as X t , where x i,j represents the jth value of sensor i, X t represents the tth flight; Step 1-2: For each flight, Y t By m y i Composition, represents the degradation of m sensors; where y i It is a binary code with two values ​​0 and 1, 1 means degradation and 0 means normal; preposed data represents the mapping from X to Y; Step 1-3: Resample the original sequence of flight record data to a fixed length, that is, equally spaced X t Each row is sampled to obtain Step 1-4: Preprocess the data Encode into multiple fixed-length sequences and transfer the previous mapping relationship labels from X to Y to the fixed-length sequences; Step 2: Input the fixed-length sequence obtained in step 1 into the residual module; The residual module includes an LSTM unit with multiple hidden parameters, which is connected in parallel with the first layer of LTSM; the residual block with multiple layers of recursive LSTM is used to extract features and is combined with the initial features obtained by the first layer of LSTM; Step 3: Input the features extracted from the residual module into the cascade module; the cascade module consists of three cascaded LSTM units; Step 4: Finally, through a fully connected layer, the 32-dimensional result output in step 3 is converted into a 2-bit code to obtain the final classification prediction result.

Images