A method for predicting the remaining useful life of an engine based on improved HRHN
Through the improved HRHN model, the use of exogenous data to replace historical observations and optimize network parameters, the prediction difficulties of traditional models under missing data are solved, and the accurate prediction of residual service life of gas turbine engines and the capture of space-time relationships is achieved, which improves the improvement of the Prediction accuracy and robustness.
Patent Information
- Application Number
- CN202210635416.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-07
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2042-06-07
AI Technical Summary
Traditional time series prediction models are difficult to accurately predict the remaining service life of gas turbine engines without historical observations, and cannot effectively capture complex spatio-temporal relationships and instability.
Through the improved HRHN model, exogenous data are used to replace historical observations in the hierarchical attention mechanism, and combined with zero-start training method and grid search technology, network parameters are optimized to achieve residual service life prediction of gas turbine engines.
The accurate residual service life prediction of the gas turbine engine is achieved in the absence of historical observations, and the spatio-temporal relationship and instability of the time series are better captured, improving the accuracy and robustness of the prediction.
Smart Images
Figure CN114971052B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of remaining useful life prediction, and in particular to a method for predicting the remaining useful life of an engine based on an improved HRHN. Background Art
[0002] With the development of science and technology, aircraft have become a common means of transportation. The safety and endurance of aircraft are very important. Once any accident occurs, the chance of survival is extremely slim, which will have a huge impact on life safety and economic losses. Therefore, it is of great significance to be able to accurately predict the remaining service life of gas turbine engines.
[0003] Traditional time series processing is usually linear, while the problems that need to be predicted in reality are often very complex, containing unstable, irregular, noisy problems, and are usually nonlinear. With the development of deep learning, it has been applied in different fields, promoting the remaining service life prediction as one of the core technologies of aviation systems. The remaining service life prediction is to predict the remaining service life given the data of different sensor degradation before the engine is scrapped. As a new time prediction model, HRHN models from both spatial and temporal aspects, and can adaptively select the time information of historical data and the correlation characteristics of past exogenous data. It is better than the neural network of traditional time series prediction and achieves the purpose of capturing the long-term time dependency between historical exogenous data. However, HRHN predicts the future based on historical observations. This type of model is very suitable for stock price prediction. In the test set, when historical observations are missing and only historical exogenous data are left, predicting the observations is still an unresolved problem. Therefore, it is of great application value to consider the interaction between time and space of historical exogenous data and make predictions in the absence of historical observations. Summary of the invention
[0004] The purpose of the present invention is to propose an engine remaining service life prediction method based on improved HRHN to solve the problem of predicting the remaining service life of a gas turbine.
[0005] The technical solution to achieve the purpose of the present invention is: a method for predicting the remaining useful life of an engine based on an improved HRHN, comprising the following steps:
[0006] Step 1, obtain two-dimensional data of a gas turbine engine gradually degraded from a good state to scrapped over time to construct a training set, and perform preprocessing to obtain three-dimensional data including degradation process, time lag order, and number of sensors;
[0007] Step 2, using the degradation data of each sensor of the engine to replace the remaining service life of the gas turbine engine in the hierarchical attention mechanism of HRHN, and obtaining an improved HRHN model for the remaining service life prediction of the gas turbine engine;
[0008] Step 3, use the zero-start training method to train the improved HRHN model, and use the grid search technology to obtain the optimal parameters of the network;
[0009] Step 4, pre-processing the gas turbine engine sensor data for which the remaining useful life is to be predicted, inputting the improved HRHN model, and completing the remaining useful life prediction.
[0010] Further, in step 1, two-dimensional data of a gas turbine engine gradually degraded from a good state to a scrapped state over time is obtained to construct a training set, and preprocessed to obtain three-dimensional data including degradation process, time lag order, and number of sensors. The specific method is as follows:
[0011] Step 1.1: Draw a trend graph of all sensor data of each engine in the training set over time. Based on the trend of the graph, discard the sensors that have no effect on engine degradation. Finally, get the sensor data that affects engine degradation. This data is recorded from the brand new state until the engine is completely scrapped.
[0012] Step 1.2: Assume that the remaining service life decreases linearly to zero. Select a suitable time node for the sensor data that affects engine degradation. Before this time, the degradation of any engine is not obvious. Cut it off from here. That is, if the remaining service life of the engine is greater than the node value, it is set to the node value. Finally, the remaining service life gradually decreases after this node.
[0013] Step 1.3: Add the time lag order of the sensor to the truncated data of the training set, and discard the entries with missing historical data. The data dimension changes from two-dimensional (n, m) to three-dimensional (np*(T-1), T, m), where n is the total number of sensor records in the training set, T is the lag order, m is the number of sensors, and p is the number of engines;
[0014] Step 1.4: Convert the three-dimensional data with the added time lag order into two-dimensional data (np*(T-1), T*m) by multiplying the last two dimensions, and perform Z value standardization. After standardization, restore all data dimensions to three dimensions (np*(T-1), T, m) and convert them into tensors to store high-order arrays, and finally obtain the preprocessed data.
[0015] Further, in step 2, in the hierarchical attention mechanism, the degradation data of each sensor of the engine is used to replace the remaining service life of the gas turbine engine in the hierarchical attention mechanism of HRHN, and an improved HRHN model is obtained for the remaining service life prediction of the gas turbine engine. The specific method is as follows:
[0016] In the layered attention mechanism, the historical observations X of the exogenous data are used t , replaces the historical observations in the hierarchical attention mechanism of HRHN, the context vector d t With a given exogenous variable input X t Combined, we get the time-dependent X t and d t The interaction between
[0017]
[0018] After the data is replaced, the dimension of the network lag term changes from (np*(T-1), T, 1)-dimensional y t It becomes the historical observation value X of the exogenous data with dimension (np*(T-1), T, m) retained after data preprocessing t ), after the multi-dimensional data passes through the fully connected layer, the output is still the same as the original model dimension (np*(T-1), 1) dimensional X t and d t The data of interaction between
[0019] in, It is X that needs to be learned t The corresponding weight matrix is, is the context vector d to be learned t The corresponding weight matrix is, represents the deviation, n is the total number of sensor records in the training set, T is the lag order, m is the number of sensors, and p is the number of engines.
[0020] Furthermore, in step 3, the improved HRHN model is trained using the zero-start training method, and the grid search technique is used to obtain the optimal parameters of the network. The specific method is as follows:
[0021] MSE is used as the loss function, the batch size of the training network is 8192, the time lag order is set to 16, and in the grid search, a variable learning rate is used for training. The initial learning rate is 0.01, the initial minimum mean square error is 9999, and the number of network training rounds is 10000 rounds. The model trained in the current round is saved. If the mean square error no longer decreases after 15 consecutive rounds of optimization, the iteration is terminated and the optimal parameters are obtained.
[0022] Further, in step 4, the gas turbine engine sensor data for which the remaining service life is to be predicted is preprocessed and input into the improved HRHN model to complete the remaining service life prediction. The specific method is as follows:
[0023] The processing of the gas turbine engine sensor data for predicting the remaining useful life is the same as that of the test set. After training, the model is loaded with the optimal weights, and the preprocessed data is predicted on the server. The forward inference process does not perform loss calculation and return loss. The network structure is the same as during training, and what is returned is the predicted remaining useful life of the gas turbine engine.
[0024] An engine remaining useful life prediction system based on improved HRHN realizes remaining useful life prediction based on a deep learning-based RUL prediction framework.
[0025] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the remaining useful life prediction of an engine based on an improved HRHN-based remaining useful life prediction method is realized.
[0026] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the remaining useful life prediction of an engine based on an improved HRHN-based remaining useful life prediction method is realized.
[0027] Compared with the prior art, the present invention has the following significant advantages: 1) The historical observations of exogenous data are used to replace the historical observations in the hierarchical attention mechanism of HRHN, which can predict the remaining service life of gas turbine engines when historical observations are missing. 2) An improved hierarchical attention mechanism is proposed, which can solve problems such as panel data time series prediction. 3) The spatiotemporal relationship of exogenous data is taken into account to better capture the instability of time series. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A flowchart of the steps based on the present invention.
[0029] Figure 2 This is the structural diagram of the improved version of HRHN of the present invention.
[0030] Figure 3 This is a partial sensor signal degradation image of the present invention.
[0031] Figure 4 This is a graph of the change process of the training process loss, the root mean square error (RMSE) on the test set, and the R2 on the test set of the present invention.
[0032] Figure 5 This is the prediction effect of a test set of the present invention.
[0033] Figure 6 This is a method of RMSE and R in overfitting. 2
[0034] changes. DETAILED DESCRIPTION
[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0036] The present invention provides an engine remaining service life prediction method based on improved HRHN, such as Figure 1 As shown, the following steps are included: (1) preprocessing the data set; (2) building an improved model; (3) training the model; and (4) predicting actual gas turbine engine data and outputting the prediction results.
[0037] The design and implementation of each step are introduced below.
[0038] Step 1: Obtain the two-dimensional data of the gas turbine engine gradually degrading from a good state to scrapping over time to construct a training set, and perform preprocessing to obtain three-dimensional data including the degradation process, time lag order, and number of sensors. The specific method is as follows:
[0039] Step 1.1: Plot a trend graph of all sensor data for each engine in the training set over time. Based on the trend of the graph, discard sensors that have no effect on engine degradation. Finally, get the sensor data that affects engine degradation. This data is recorded from a brand new state until the engine is completely scrapped.
[0040] Step 1.2: Assume that the remaining service life decreases linearly to zero. Select a suitable time node for the sensor data that affects engine degradation. Before this time, any engine degradation is not obvious. Truncation is performed from here, that is, if the remaining service life of the engine is greater than the node value, it is set to the node value. Finally, data is obtained that the remaining service life gradually decreases after this node.
[0041] Step 1.3: Add the time lag order of the sensor to the truncated data of the training set, and discard the entries with missing historical data. The data dimension changes from two-dimensional (n, m) to three-dimensional (np*(T-1), T, m), where n is the total number of sensor records in the training set, T is the lag order, m is the number of sensors, and p is the number of engines.
[0042] Step 1.4: The three-dimensional data with added time lag orders are uniformly converted into two-dimensional data (np*(T-1), T*m) by multiplying the last two dimensions, and the Z value is normalized. After standardization, all data dimensions are restored to three dimensions (np*(T-1), T, m) and converted into tensors to store high-order arrays, and finally the preprocessed data is obtained.
[0043] Step 2: In the hierarchical attention mechanism, the historical observations of the exogenous data (i.e., the sensor degradation data of each engine) are used to replace the historical observations in the hierarchical attention mechanism of HRHN (i.e., the remaining useful life of the gas turbine engine), and the improved HRHN model is obtained for the remaining useful life prediction. The specific method is as follows:
[0044] In the layered attention mechanism, the historical observations X of the exogenous data are used t (i.e., the sensor degradation data of each engine in the preprocessed data), replaces the historical observations in the hierarchical attention mechanism of HRHN (i.e., the remaining useful life y of the gas turbine engine t ). Context vector d t With a given exogenous variable input X t Combined, among them, It is X that needs to be learned t The corresponding weight matrix is, is the context vector d to be learned t The corresponding weight matrix is, Denotes the deviation, and obtains the time-dependent X t and d t The interaction between
[0045]
[0046] After the data is replaced, the dimension of the network lag term changes (from (np*(T-1), T, 1) dimensional y t It becomes the historical observation value X of the exogenous data with dimension (np*(T-1), T, m) retained after data preprocessing t ), after the multi-dimensional data passes through the fully connected layer, the output is still the same as the original model dimension (np*(T-1), 1) dimensional X t and d t The data of interaction between
[0047] Step 3: Use the zero-start training method to train the improved HRHN model and apply the grid search technique to obtain the optimal parameters of the network. The specific method is as follows:
[0048] MSE is used as the loss function, the batch size of the training network is 8192, the time lag order is set to 16, and in the grid search, a variable learning rate is used for training. The initial learning rate is 0.01, the initial minimum mean square error is 9999, and the number of network training rounds is 10000. The model trained in the current round is saved. If the mean square error no longer decreases after 15 consecutive rounds of optimization, the iteration is terminated and the optimal parameters are obtained.
[0049] Step 4, pre-processing the gas turbine engine sensor data for which the remaining service life is to be predicted, inputting the improved HRHN model, and completing the remaining service life prediction, the specific method is as follows:
[0050] The optimal weights are loaded into the trained model, and the preprocessed data is predicted on the server. The forward inference process does not perform loss calculation and return loss. The network structure is the same as that during training, and what is returned is the predicted remaining service life of the gas turbine engine.
[0051] The present invention proposes an engine remaining useful life prediction method based on improved HRHN, and realizes remaining useful life prediction based on the RUL prediction framework based on deep learning.
[0052] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the remaining useful life prediction of an engine based on an improved HRHN-based remaining useful life prediction method is realized.
[0053] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the remaining useful life prediction of an engine based on an improved HRHN-based remaining useful life prediction method is realized.
[0054] Example
[0055] In order to verify the effectiveness of the solution of the present invention, the FD001 data set of NASA's CMAPSS is selected for the prediction and evaluation of gas turbine engine performance. The following experiment is conducted.
[0056] Step 1: Obtain the two-dimensional data of the gas turbine engine gradually degraded from good condition to scrapped over time, and preprocess it to obtain three-dimensional data including degradation process, time lag order, and number of sensors. The FD001 data set is the operating status of 21 sensors in 3 operating modes. The brand new engine gradually wears out under various external environments until it is scrapped. The data set is divided into training set and test set. The data used for training contains 100 groups of engines, 21 sensors, and panel data of 3 operating modes consumed over time; the test set is the data of 100 groups of engines, and the service life of each group of engines is stored in a new file.
[0057] The description of the FD001 dataset is as follows:
[0058] Table 1 TrainFD001 dataset description information table
[0059]
[0060]
[0061] Table 2 Train FD001 partial data example
[0062]
[0063]
[0064] Table 3 Test FD001 dataset display
[0065]
[0066] Table 4 Partial data of remaining service life of Test FD001
[0067]
[0068] Step 1.1, make an image of the degradation of some sensor signals over time, such as Figure 3 As shown in the figure, the horizontal axis is the number of days left. During the data preprocessing process, the sensors irrelevant to the experiment were removed based on the trend of the sensor data. After the data preprocessing was completed, a total of seven sets of sensor data were removed, and information such as setting items and unit numbers were removed. After this operation, the data dimension changed from (n, 27) (n is the number of record entries) to (n, 14).
[0069] Step 1.2, from Figure 3 It can be seen that before 125 days, the wear of the engine is not obvious, and a cutoff is made here.
[0070] Step 1.3, for each unit of sensor data, add the historical sensor state to the data, and then discard the entries with missing historical data. The data dimension changes from (n, 14) to (n-100*(T-1), T, 14) (T is the order of time lag, which is set to 16 here, and 100*(T-1) is the number of entries with missing historical data); the remaining life cycle of the training data is given by the remaining time.
[0071] Step 1.4: Before training, Z-standardize the independent and dependent variables of the training and test data. After data preprocessing, convert all data into tensors to store high-order arrays.
[0072] Step 2, in the hierarchical attention mechanism, use the historical observations of exogenous data (i.e., the sensor degradation data of each engine) to replace the historical observations in the hierarchical attention mechanism of HRHN (i.e., the remaining useful life of the gas turbine engine), and obtain an improved HRHN model for predicting the remaining useful life of the gas turbine engine.
[0073] Step 3: Use the zero-start training method to train the improved HRHN model, and use the grid search technology to obtain the optimal parameters of the network. The optimization rounds of this model are set to 10,000 rounds, and the initial minimum mean square error is set to 9999. Because there will be a slight overfitting phenomenon during the training process, the early termination iteration method is used to solve it. When the mean square error of the model on the test set after each round of training is less than the previous minimum mean square error, we save the model trained in the current round. If the mean square error no longer decreases after 15 consecutive rounds of optimization, the iteration is terminated.
[0074] By performing a grid search (GridSearch) on the number of convolution layers, the number of encoder-decoder units, the encoder depth, the learning rate, and the number of units after the convolution layer in the model parameters, the specific settings are shown in the table below. All candidate parameters are traversed for tuning and the best fitting parameters are selected as the model parameters.
[0075] Table 5 Grid search parameter settings
[0076]
[0077] After the grid search, the hyperparameters of the model are finally set as follows: the number of convolutional layers is set to 3, the convolution kernel size is set to 1×5, the pooling kernel size is 1×2, the number of units after the convolution layer is set to 16, the activation layer uses the RELU function, the RHN encoder units are set to 16, the decoder units are set to 16, the depth of the RHN encoder and decoder is 2, and the time lag order is set to 16 during the modeling process.
[0078] Step 4: After preprocessing the test set, the three-dimensional data including the degradation process, time lag order, and number of sensors are input into the improved HRHN model to complete the remaining service life prediction.
[0079] The variables in the training process (training process loss, root mean square error on the test set, and R on the test set) 2 ) changes such as Figure 4 As shown, the final prediction effect on the test set is as follows Figure 5 shown.
[0080] Choosing RMSE and R 2 As an evaluation indicator. After repeated training, the best result of the model is R 2 =0.8319, RMSE = 17.04. In order to test the fitting effect of the improved HRHN, this paper compares the improved HRHN with the following time series prediction models or neural network models on the FD001 dataset of CMAPSS:
[0081] Table 6 Prediction results of different time series models
[0082]
[0083] Note: The data marked in black are the best results among all models.
[0084] By adjusting the parameters of the HRHN model, it is found that the model designed in this paper has good robustness.
[0085] Table 7 Model results under different model parameters
[0086]
[0087] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0088] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. An engine remaining useful life prediction method based on improved HRHN, It is characterized in that The steps include: Step 1, obtain two-dimensional data of a gas turbine engine gradually degraded from a good state to scrapped over time to construct a training set, and perform preprocessing to obtain three-dimensional data including degradation process, time lag order, and number of sensors; Step 2, using the degradation data of each sensor of the engine to replace the remaining service life of the gas turbine engine in the hierarchical attention mechanism of HRHN, and obtaining an improved HRHN model for the remaining service life prediction of the gas turbine engine; Step 3, use the zero-start training method to train the improved HRHN model, and use the grid search technology to obtain the optimal parameters of the network; Step 4, preprocessing the gas turbine engine sensor data for which the remaining service life is to be predicted, inputting the improved HRHN model, and completing the remaining service life prediction; Step 1: Obtain the two-dimensional data of the gas turbine engine gradually degrading from a good state to scrapping over time to construct a training set, and perform preprocessing to obtain three-dimensional data including the degradation process, time lag order, and number of sensors. The specific method is as follows: Step 1.1: Draw a trend graph of all sensor data of each engine in the training set over time. Based on the trend of the graph, discard the sensors that have no effect on engine degradation. Finally, get the sensor data that affects engine degradation. This data is recorded from the brand new state until the engine is completely scrapped. Step 1.2: Assume that the remaining service life decreases linearly to zero. Select a suitable time node for the sensor data that affects engine degradation. Before this time, the degradation of any engine is not obvious. Cut it off from here. That is, if the remaining service life of the engine is greater than the node value, it is set to the node value. Finally, the remaining service life gradually decreases after this node. Step 1.3: Add the time lag order of the sensor to the truncated data of the training set, and discard the entries with missing historical data. The data dimension changes from two-dimensional (n, m) to three-dimensional (np*(T-1), T, m), where n is the total number of sensor records in the training set, T is the lag order, m is the number of sensors, and p is the number of engines; Step 1.4: The three-dimensional data with the added time lag order is converted into two-dimensional data (np*(T-1), T*m) by multiplying the last two dimensions, and the Z value is normalized. After normalization, all data dimensions are restored to three dimensions (np*(T-1), T, m) and converted into tensors to store high-order arrays, and finally the preprocessed data is obtained; Step 2: In the hierarchical attention mechanism, the degradation data of each sensor of the engine is used to replace the remaining useful life of the gas turbine engine in the hierarchical attention mechanism of HRHN, and an improved HRHN model is obtained for the remaining useful life prediction of the gas turbine engine. The specific method is as follows: In the layered attention mechanism, the historical observations X of the exogenous data are used t , replaces the historical observations in the hierarchical attention mechanism of HRHN, the context vector d t With a given exogenous variable input X t Combined, we get the time-dependent X t and d t The interaction between After the data is replaced, the dimension of the network lag term changes from (np*(T-1), T, 1)-dimensional y t It becomes the historical observation value X of the exogenous data with dimension (np*(T-1), T, m) retained after data preprocessing t ), after the multi-dimensional data passes through the fully connected layer, the output is still the same as the original model dimension (np*(T-1), 1) dimensional X t and d t The data of interaction between in, It is X that needs to be learned t The corresponding weight matrix is, is the context vector d to be learned t The corresponding weight matrix is, represents the deviation, n is the total number of sensor records in the training set, T is the lag order, m is the number of sensors, and p is the number of engines.
2. The method for predicting the remaining useful life of an engine based on improved HRHN according to claim 1, It is characterized in that Step 3: Use the zero-start training method to train the improved HRHN model and apply the grid search technique to obtain the optimal parameters of the network. The specific method is as follows: Using MSE as the loss function, the batch size of the training network is 8192, the time lag order is set to 16, and in the grid search, a variable learning rate is used for training. The initial learning rate is 0.01, the initial minimum mean square error is 9999, and the number of training rounds of the network is 10000 rounds; The model of the current round of training is saved. If the mean square error does not decrease after 15 consecutive rounds of optimization, the iteration is terminated and the optimal parameters are obtained.
3. The engine remaining useful life prediction method based on improved HRHN according to claim 1, It is characterized in that Step 4, pre-processing the gas turbine engine sensor data for which the remaining service life is to be predicted, inputting the improved HRHN model, and completing the remaining service life prediction, the specific method is as follows: The optimal weights are loaded into the trained model, and the preprocessed data is predicted on the server. The forward inference process does not perform loss calculation and return loss. The network structure is the same as that during training, and what is returned is the predicted remaining service life of the gas turbine engine.
4. An engine remaining useful life prediction method based on improved HRHN, It is characterized in that The remaining service life prediction method of an engine based on improved HRHN as described in any one of claims 1 to 3 is used to realize the prediction of the remaining service life.
5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an accurate prediction of the remaining useful life of an engine based on the improved HRHN-based remaining useful life prediction method according to any one of claims 1 to 3 is achieved.
6. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the remaining useful life prediction of an engine based on the improved HRHN-based remaining useful life prediction method according to any one of claims 1 to 3 is implemented.