Aero-engine remaining service life prediction method based on attention fusion neural network

Abstract

The invention discloses an aero-engine remaining service life prediction method based on an attention fusion neural network, and belongs to the field of aero-engines. The method is used for modeling the performance degradation state of the aero-engine under complex working conditions and estimating the future available life of the aero-engine. In order to solve the problems of high data dimension, strong nonlinearity of a degradation process, complex time sequence dependence and the like of multiple sensors of an aero-engine, the invention proposes that an attention fusion mechanism is combined with a time convolutional network, and a gating circulation unit is introduced to form a depth prediction model; the multi-scale key degradation features can be adaptively extracted, and the long-term dependency relationship can be effectively captured, so that the accuracy and stability of life prediction are remarkably improved. The method has the advantages of being flexible in structure, high in calculation efficiency, high in generalization ability and the like, and the prediction performance superior to that of a traditional method is obtained on a public CMAPSS data set; and meanwhile, the method has good expandability and can be popularized and applied to the fields of health assessment and residual life prediction of other complex mechanical equipment.

Description

Technical Field

[0001] This invention belongs to the field of aero-engines, and specifically relates to a method for predicting the remaining service life of aero-engines. Background Technology

[0002] Aero engines are hailed as the "crown jewel" of the aviation industry, their technological level serving as a benchmark for a nation's industrial strength and driving the vigorous development of the aviation sector. As a core component of an aircraft, the aero engine provides thrust for flight. During aircraft operation, the smoothness of its engine performance directly impacts flight safety. With the increasing complexity and intelligence of industrial equipment, Prognostics and Health Management (PHM) has emerged as a new, multidisciplinary, and comprehensive technology to reduce economic losses and improve efficiency in equipment maintenance. PHM is a technology and method that monitors the health status of equipment or systems, predicts future failures, and formulates maintenance strategies. Its purpose is to improve equipment reliability, extend service life, optimize maintenance costs, and reduce unexpected failures. Aero engine PHM systems encompass condition monitoring, fault diagnosis, fault prediction, and decision support. In the aviation field, to meet the demands for improved safety, reduced unplanned downtime, lower maintenance costs, and extended engine life, researchers both domestically and internationally are continuously researching key technologies applied to aero engine PHM systems.

[0003] In PHM (Prognostics and Maintenance) technology, remaining service life (RUL) prediction is a crucial topic. Aero engines, as complex thermomechanical machines, operate in harsh environments. During use, component performance deviates from initial design requirements, resulting in performance degradation. This degradation further reduces the overall performance of the aero engine and can even lead to major safety accidents. Therefore, assessing the remaining service life of an aero engine is a key issue in ensuring flight safety. Similarly, when airlines perform aircraft maintenance, engineers calculate the deviation of component performance from factory specifications to ensure the safe and stable operation of the aero engine. Therefore, establishing intelligent and highly accurate RUL prediction models can simplify the recording and analysis of component and overall performance during maintenance, improving efficiency. Furthermore, the predictive capabilities of these models can further promote a condition-based maintenance (RUL) approach.

[0004] Previous predictions of remaining lifetime have focused on improving prediction accuracy by stacking network structures. However, how to improve the accuracy of remaining lifetime prediction without increasing model complexity through a data fusion model is a problem worth studying. Summary of the Invention

[0005] This invention addresses the problems of complex structures, limited room for improvement in accuracy, and low accuracy in predicting the remaining service life of aero-engines in existing technologies. It proposes an attention fusion neural network method for predicting the remaining service life of aero-engines.

[0006] The technical solution of the present invention: A method for predicting the remaining service life of an aero-engine based on an attention fusion neural network includes the following steps: Step 1: Feature selection of data: exclude features in the aero-engine degradation data where sensor values ​​remain constant, select the remaining sensor features, preprocess the selected sensor features using standardization and sliding window sampling methods, divide the preprocessed sensor features into two-stage degradation processes, and use the stage to which the sensor feature belongs as the label. Finally, divide the sensor feature data of the two stages into training set and validation set.

[0007] Furthermore, the standardization process is as follows: (1) in, σ represents the mean of the input data, and σ represents the standard deviation of the data. Represents the input data, This represents the standardized data.

[0008] Furthermore, the sliding window sampling method involves sampling the time-series data using a sliding window with a step size of t, converting the input data into... In the form of, This represents the model input data in the training set, where... T For sliding window size, C For the input data dimensions, The labels in the training set are generated using the two-stage degradation label generation method. The label for each sample is defined as the Remaining Useful Life (RUL) value corresponding to the last time point within the original data window.

[0009] Furthermore, the two-stage degradation includes: the first stage: the sensor recorded values ​​change slowly, exhibiting a gradual decay; the second stage: the sensor recorded values ​​change more rapidly.

[0010] Step 2: Based on the features of the training set, design a temporal convolutional network structure based on attention fusion to extract hidden layer features of aero-engine performance degradation.

[0011] (1) For one-dimensional input data The output is obtained by feeding it into a separate Temporal Convolutional Network (TCN) module. The output features of each module are concatenated along the channel dimension to obtain a feature representation that integrates multi-scale information. Subsequently, the data was compressed in the time dimension using average pooling, and the data was transformed... ; (2) Obtained through two fully connected layers Two fully connected layers are used to learn data weights during model training; (3) The weight coefficients are obtained by inputting the Softmax function. Then the weighting coefficients are compared with... Element-wise multiplication yields weighted data; The formula for calculating Softmax is as follows: (2) in, This represents the input data matrix; , and Indicates the index of the data in the matrix; (4) To reduce the data dimension, the weighted data is compressed to obtain the hidden layer features. The compression operation is to convert the data into the shape of "channel number, n, time step" according to the number of TCN networks n, and perform a summation operation in dimension 1 so that the size of the final output data is consistent with the output dimension of a single TCN module.

[0012] Furthermore, the TCN module is a neural network structure composed of convolutional layers and residual connection blocks. The convolutional layers consist of two dilated convolutions, two non-linear activation layers, and two random deactivation layers superimposed. The dilated convolution is an improvement on ordinary one-dimensional convolution; to expand the receptive field of the convolution, TCN introduces a dilation rate into the one-dimensional convolution. d This allows the convolution kernel to sample at fixed intervals along the time dimension, thus modeling long-term temporal dependencies without significantly increasing the number of parameters. The dilated convolution formula is: (3) in, Represents the kernel length. s Represents the position of the element. f Represents convolution calculation, d This represents performing convolution calculations on the data at fixed intervals. This represents the index in the input data sequence x. This represents the output data of the dilated convolution.

[0013] In addition, the specific implementation of residual connections involves converting the output of the convolutional layer into a single data structure. With the original input Perform element-wise addition: (4) in, 'o' represents the input data, 'o' represents the output data, and F(.) represents a series of convolutional transformations in the residual block. If the input and output dimensions are inconsistent, a 1×1 convolution is used to linearly map the input before summing to ensure dimension matching.

[0014] Step 3: Based on the hidden layer features output by the attention-based temporal convolutional network structure, select the gated recurrent network as the network for predicting remaining lifetime.

[0015] First, the data from the original dataset is dimensionally aligned with the hidden layer features obtained from the TCN using a fully connected layer. Then, the GRU network is used to extract cross-sensor information related to the performance degradation of aero-engines from the multi-timescale fused features. Equation (3) illustrates the calculation process of GRU: (5) in, This indicates the reset gate, used to control the degree to which historical information from the previous moment is forgotten; This indicates the update gate, used to control the fusion ratio of new and old information; This represents the candidate hidden state at the current time t; , , , , and The weights and biases representing the calculations for the reset gate, update gate, and hidden information. It is the sigmoid function. The hidden state of the current unit output. Represents input data, The function is the hyperbolic tangent. Finally, as shown in the formula, the hidden state and output of the GRU are obtained by synthesizing the update gate output and the candidate hidden state. Finally, the output of the GRU is passed through a fully connected layer to obtain the final RUL prediction value.

[0016] Step 4: Based on the PyTorch deep learning framework, build the network framework from Steps 1-3, and use the grid search method to test it on the training and validation sets to determine the optimal hyperparameters.

[0017] Step 5: Apply the obtained optimal hyperparameters to model training to obtain the optimal model, and use the optimal model to predict the remaining service life of the aero-engine.

[0018] The beneficial effects of this invention are: This invention constructs a deep learning model combining a temporal convolutional network based on attention fusion and a gated recurrent unit, which effectively extracts multi-scale key features from aero-engine degradation sequences and enhances the modeling ability for long-term dependent information. A complete data preprocessing workflow, including feature selection, standardization, sliding window construction, and two-stage degradation label generation, ensures more thorough and stable model training, improving the generalization performance of predictions. Furthermore, the optimal hyperparameters obtained through grid search further guarantee the model's reliability under different operating conditions. Experimental results and ablation analysis based on the CMAPSS dataset demonstrate that this invention significantly outperforms traditional methods in terms of remaining service life prediction accuracy, stability, and robustness. It possesses advantages such as flexible structure, strong interpretability, and good adaptability, providing efficient and reliable technical support for aero-engine health management. Attached Figure Description

[0019] Figure 1 This is a trend chart of degradation data for aero-engine units.

[0020] Figure 2 This is a schematic diagram of the sliding window method.

[0021] Figure 3 A schematic diagram showing segmented degradation of the remaining service life.

[0022] Figure 4 Schematic diagram of temporal convolutional networks Figure 5 This is a schematic diagram of the proposed attention computation framework.

[0023] Figure 6 This is a schematic diagram of the proposed model framework.

[0024] Figure 7 This is a graph showing the model's prediction results on the dataset.

[0025] Figure 8 This is a graph showing the model's prediction performance on a single unit of the dataset. Detailed Implementation

[0026] To make the objectives, techniques, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and examples.

[0027] An attention fusion neural network method for predicting the remaining service life of aero-engines is proposed. This method uses a neural network to model aero-engine data and predict its remaining service life. During the operation of an aero-engine, after each work cycle, its performance parameters exhibit certain degradation characteristics, such as… Figure 1 As shown. The core work of this patent is to extract the degree of degradation of aero-engines from sensor information and predict their remaining service life. Given the problems of noisy data, non-linear degradation trends, and inconsistent initial states and degradation processes among different aero-engines, an attention fusion neural network is designed to predict the remaining service life of aero-engines, including the following steps: Step 1: To facilitate the subsequent training of the neural network deep learning model, feature selection of the data is first required. Since some sensor values ​​remain constant throughout the engine performance degradation process, these features are redundant for the neural network input. Therefore, this study selected 14 sensor features (sensor numbers 2, 3, 4, 7, 8, 9, 11, 12, 13, 14, 15, 17, 20, and 21) for modeling and analysis.

[0028] For the selected sensor features, the training set employs standardization and sliding window sampling methods. However, the sensor data in the dataset exhibits numerical scale differences, which can negatively impact the neural network training process. Therefore, this paper uses a general normalization method to standardize each feature variable separately. This method transforms the sensor data into normalized data with a mean of 0 and a variance of 1, ensuring that the values ​​from different sensors remain at similar magnitudes.

[0029] (1) in, σ represents the mean of the input data, and σ represents the standard deviation of the data. Represents the input data, This represents the standardized data.

[0030] For single-point data containing only measurements at the current moment, models struggle to effectively extract performance degradation information. To incorporate the influence of historical data and improve model accuracy, a sliding window approach is needed to process the dataset. For example... Figure 2 As shown, time-series data is sampled using a sliding window with a step size of 1 (moving forward one time unit each time), and the input data is converted into... In the form of, This represents the model input data in the training set, where... T For sliding window size, C For the input data dimensions, The labels in the training set are generated using the two-stage degradation label generation method. Taking the engine data of 192 operating cycles in the FD001 training set as an example, 173 training samples can be generated using a sliding window of size 30. The label of each sample is defined as the Remaining Useful Life (RUL) value corresponding to the last time point in the original data window.

[0031] The method for generating labels for the training set and validation set through a two-stage degradation process is as follows: In the backpropagation process of the neural network, the model parameters need to be updated using the model output values ​​and the true labels. To ensure that the proposed model better fits the data characteristics, data trend analysis reveals that the parameter degradation of aero-engines exhibits a two-stage characteristic: in the first stage, the sensor-recorded values ​​change slowly, showing a gradual decline; in the second stage, the parameter changes accelerate significantly. The threshold between the two stages is typically set to 125 cycles. The data obtained based on this transformation method will be used as the labels for the training set, and its numerical mapping relationship is as follows: Figure 3 As shown.

[0032] Step 2: Based on the features of the training set, design a temporal convolutional network structure based on attention fusion to extract hidden layer features of aero-engine performance degradation.

[0033] (1) For one-dimensional input data The output is obtained by feeding it into a separate Temporal Convolutional Network (TCN) module. The output features of each module are concatenated along the channel dimension to obtain a feature representation that integrates multi-scale information. Subsequently, the data was compressed in the time dimension using average pooling, and the data was transformed... ; (2) Obtained through two fully connected layers Two fully connected layers are used to learn data weights during model training; (3) The weight coefficients are obtained by inputting the Softmax function. Then the weighting coefficients are compared with... Element-wise multiplication yields weighted data; The formula for calculating Softmax is as follows: (2) in, This represents the input data matrix; , and Indicates the index of the data in the matrix; (4) To reduce the data dimension, the weighted data is compressed to obtain the hidden layer features. The compression operation is to convert the data into the shape of "channel number, n, time step" according to the number of TCN networks n, and perform a summation operation in dimension 1 so that the size of the final output data is consistent with the output dimension of a single TCN module.

[0034] Furthermore, the TCN module is a neural network structure composed of convolutional layers and residual connection blocks. The convolutional layers consist of two dilated convolutions, two non-linear activation layers, and two random deactivation layers superimposed. The dilated convolution is an improvement on ordinary one-dimensional convolution; to expand the receptive field of the convolution, TCN introduces a dilation rate into the one-dimensional convolution. d This allows the convolution kernel to sample at fixed intervals along the time dimension, thus modeling long-term temporal dependencies without significantly increasing the number of parameters. The dilated convolution formula is: (3) in, k Represents the kernel length. s Represents the position of the element. f Represents convolution calculation, d This represents performing convolution calculations on the data at fixed intervals. This represents the index in the input data sequence x. This represents the output data of the dilated convolution.

[0035] In addition, the specific implementation of residual connections involves converting the output of the convolutional layer into a single data structure. With the original input Perform element-wise addition: (4) Here, x represents the input data, o represents the output data, and F(.) denotes a series of convolutional transformations in the residual block. If the input and output dimensions are inconsistent, a 1×1 convolution is used to linearly map the input before summing to ensure dimension matching. The TCN network structure is as follows: Figure 4 As shown.

[0036] Step 3: Based on the hidden layer features output by the model, select the gated recurrent unit recurrent network as the network for predicting the remaining lifetime.

[0037] First, the data from the original dataset is dimensionally aligned with the hidden layer features obtained from the TCN using a fully connected layer. Then, the GRU network is used to extract cross-sensor information related to the performance degradation of aero-engines from the multi-timescale fused features. Equation (3) illustrates the calculation process of GRU: (5) in, This indicates the reset gate, used to control the degree to which historical information from the previous moment is forgotten; This indicates the update gate, used to control the fusion ratio of new and old information; This represents the candidate hidden state at the current time t; , , , , and The weights and biases representing the calculations for the reset gate, update gate, and hidden information. It is the sigmoid function. The hidden state of the current unit output. Represents input data, The function is the hyperbolic tangent. Finally, as shown in the formula, the hidden state and output of the GRU are obtained by synthesizing the update gate output and the candidate hidden state. Finally, the output of the GRU is passed through a fully connected layer to obtain the final RUL prediction value.

[0038] In summary, the framework structure proposed in this paper is as follows: Figure 6 As shown, it comprises three main components: data preprocessing, a TCN feature extraction network incorporating an attention mechanism, and a GRU remaining lifetime prediction network.

[0039] Step 4: Based on the PyTorch deep learning framework, build the network framework from Steps 1-3, and use the grid search method to test it on the training and validation sets to determine the optimal hyperparameters.

[0040] The proposed framework consists of two independent modules. To explore the impact of different parameter variations on the model's prediction accuracy, a grid search method is used to determine the optimal hyperparameter combination. Specifically, a set of candidate values ​​is preset for each hyperparameter, and specific values ​​are selected from each candidate set to form a hyperparameter configuration scheme in each experiment. The setting of the hyperparameter candidate set is based on existing experience and related research results; for example, the number of TCN channels is searched from [15, 20, 25, 30], and the batch size is selected from [128, 256, 512, 1024]. During neural network training, mean squared error (MSE) is used as the loss function, and the Adam optimizer is used for parameter optimization. Its learning rate gradually decays with the increase of training epochs, and is multiplied by a decay coefficient of 0.9 every 10 training epochs.

[0041] Step 7: Apply the obtained optimal hyperparameters to model training to obtain the optimal model, and use the optimal model to predict the remaining service life of the aero-engine.

[0042] To evaluate the generalization error during model training, 20% of the engine units in the training set are used as a validation set (i.e., a subset is selected from the training set of the dataset for validation, while the test set remains unchanged). By observing the trend of the validation set error, it is possible to determine whether to enable the early stopping strategy and to assess overfitting.

[0043] Based on the publicly available aero-engine degradation dataset CMAPSS, performance analysis and ablation experiments were conducted on the proposed model and its corresponding modules. First, the impact of window size on the model's prediction results during the sliding window process was analyzed. Second, the impact of the core attention fusion module on the prediction accuracy of remaining service life was analyzed through ablation experiments. Finally, the final prediction results were analyzed.

[0044] In the experimental section, firstly, by adjusting the sliding window size, repeated experiments were conducted on the FD001 dataset to verify the hypothesis that increasing the window size provides more effective information and improves prediction accuracy. Secondly, ablation experiments were conducted to verify the effectiveness of the designed multi-channel attention mechanism. By calculating attention weights for the input channels of the TCN module, the model can focus on potentially effective information, which is directly reflected in the decrease in RMSE and evaluation scores on the FD002 and FD004 datasets. Specifically, the root mean square error of RUL prediction reached 16.19 on the FD002 dataset and 18.33 on the FD004 dataset. Finally, the model performance was comprehensively validated on the FD002 dataset (e.g., ...). Figure 7 , Figure 8 (As shown).

Claims

1. A method for predicting the remaining service life of an aero-engine based on an attention fusion neural network, characterized in that, Includes the following steps: Step 1: Feature selection of data: exclude features in the aero-engine degradation data where sensor values ​​remain constant, select the remaining sensor features, preprocess the selected sensor features using standardization and sliding window sampling methods, divide the preprocessed sensor features into two-stage degradation processes, and use the stage to which the sensor feature belongs as the label. Finally, divide the sensor feature data of the two stages into training set and validation set. Step 2: Based on the features of the training set, design a temporal convolutional network structure based on attention fusion to extract hidden layer features of aero-engine performance degradation; (1) For one-dimensional input data The output is obtained by feeding it into a separate temporal convolutional network (TCN) module. The output features of each module are concatenated along the channel dimension to obtain a feature representation that integrates multi-scale information. Subsequently, the data was compressed in the time dimension using average pooling, and the data was transformed... ; (2) Obtained through two fully connected layers Two fully connected layers are used to learn data weights during model training; (3) The weight coefficients are obtained by inputting the Softmax function. Then the weighting coefficients are compared with... Element-wise multiplication yields weighted data; The formula for calculating Softmax is as follows: (2) in, This represents the input data matrix; , and Indicates the index of the data in the matrix; (4) In order to reduce the data dimension, the weighted data is compressed to obtain the hidden layer features. The compression operation is to convert the data into the shape of "channel number, n, time step" according to the number of TCN network n, and perform summation operation in dimension 1 so that the size of the final output data is consistent with the output dimension of a single TCN module. Step 3: Based on the hidden layer features output by the attention-based temporal convolutional network structure, select the gated recurrent network as the network for predicting remaining lifetime. Step 4: Based on the PyTorch deep learning framework, build the network framework from Steps 1-3, and use the grid search method to test it on the training and validation sets to determine the optimal hyperparameters; Step 5: Apply the obtained optimal hyperparameters to model training to obtain the optimal model, and use the optimal model to predict the remaining service life of the aero-engine.

2. The method for predicting the remaining service life of an aero-engine based on an attention fusion neural network according to claim 1, characterized in that, In step 1, the standardization method processing procedure is as follows: (1) in, σ represents the mean of the input data, and σ represents the standard deviation of the data. Represents the input data, This represents the standardized data.

3. The method for predicting the remaining service life of an aero-engine based on an attention fusion neural network according to claim 1, characterized in that, In step 1, the sliding window sampling method involves sampling the time-series data using a sliding window with a step size of t, and converting the input data into... In the form of, This represents the model input data in the training set, where... T For sliding window size, C For the input data dimensions, The labels in the training set are generated using the two-stage degradation label generation method; the label for each sample is defined as the remaining lifetime (RUL) value corresponding to the last time point within the original data window.

4. The method for predicting the remaining service life of an aero-engine based on an attention fusion neural network according to claim 1, characterized in that, In step 1, the two-stage degradation includes: the first stage: the sensor recorded value changes slowly, showing a gradual decay; the second stage: the sensor recorded value changes rapidly.

5. The method for predicting the remaining service life of an aero-engine based on an attention fusion neural network according to claim 1, characterized in that, In step 2, the TCN module is a neural network structure composed of convolutional layers and residual connection blocks. The convolutional layers consist of two dilated convolutional layers, two nonlinear activation layers, and two random deactivation layers superimposed. The dilated convolution is an improvement on ordinary one-dimensional convolution; to expand the receptive field of the convolution, TCN introduces a dilation rate into the one-dimensional convolution. d This allows the convolution kernel to sample at fixed intervals along the time dimension, thus modeling long-term temporal dependencies without significantly increasing the number of parameters. The dilated convolution formula is: (3) in, k Represents the kernel length. s Represents the position of the element. f Represents convolution calculation, d This represents performing convolution calculations on the data at fixed intervals. This represents the index in the input data sequence x. This represents the output data of the dilated convolution; In addition, the specific implementation of residual connections involves converting the output of the convolutional layer into a single data structure. With the original input Perform element-wise addition: (4) Where x represents the input data and o represents the output data, and F(.) represents a series of convolution transformations in the residual block; if the input and output dimensions are inconsistent, the input is linearly mapped by 1×1 convolution and then added together to ensure dimension matching.

6. The method for predicting the remaining service life of an aero-engine based on an attention fusion neural network according to claim 1, characterized in that, Step 3 specifically involves: firstly, dimensional alignment of the original dataset data with the hidden layer features obtained from the TCN using a fully connected layer; then, extraction of cross-sensor information related to aero-engine performance degradation from multi-timescale fused features using the GRU network; Equation (3) illustrates the calculation process of GRU: (5) in, This indicates the reset gate, used to control the degree to which historical information from the previous moment is forgotten; This indicates the update gate, used to control the fusion ratio of new and old information; This represents the candidate hidden state at the current time t; , , , , and The weights and biases representing the calculations for the reset gate, update gate, and hidden information. It is the sigmoid function. The hidden state of the current unit output. Represents input data, The function is a hyperbolic tangent. Finally, as shown in the formula, the hidden state and output of GRU are obtained by combining the updated gate output and the candidate hidden state. Finally, the output of GRU is passed through a fully connected layer to obtain the final RUL prediction value.

Images