Method for engine life prediction based on multi-head attention

Abstract

The present application relates to the technical field of engine life prediction, in particular to an engine life prediction method based on multi-head attention, which first normalizes the original data of the aero-engine sensor and eliminates redundant features; then realizes time-space double-dimension adaptive weighting through the multi-head attention mechanism, and strengthens the key degradation information; then inputs the weighted data into the long short-term memory network to learn the long-time dependence relationship, and completes the preliminary prediction of the remaining useful life; finally, combining Monte Carlo regularization and kernel density estimation, multiple prediction values are generated and the uncertainty is quantified, and the mean value is taken as the final prediction result. Experimental verification shows that this method effectively highlights the key degradation features, reduces the influence of uncertainty, and significantly improves the prediction performance under single working condition and multiple fault modes, providing reliable data support for preventive maintenance of aero-engines.

Description

Technical Field

[0001] This invention relates to the field of engine life prediction technology, specifically to an engine life prediction method based on multi-head attention. Background Technology

[0002] As the core power unit of an aircraft, the aero-engine is characterized by its complex structure, variable operating conditions, and harsh operating environment. Its operational reliability is directly related to flight safety and operational economic benefits. Accurately predicting the remaining service life of the aero-engine is a key technical support for achieving preventive maintenance, reducing the risk of sudden failures, and optimizing the allocation of maintenance resources. It is of great practical significance for ensuring air transport safety and controlling operating costs.

[0003] Currently, the methods for predicting the remaining service life of aero-engines are mainly divided into two categories: model-driven and data-driven. Model-driven methods rely on a precise understanding of the engine degradation mechanism and describe the equipment failure process by establishing physical or mathematical models. However, the coupling relationship between the internal air path and mechanical structure of aero-engines is complex, multiple failure modes coexist, and the degradation process is highly nonlinear. This makes it extremely difficult to build a universal and accurate physical model. Moreover, the modeling process is time-consuming and laborious, and it is difficult to adapt to actual application scenarios with multiple operating conditions and multiple failure modes.

[0004] With the development of sensor and data storage technologies, data-driven prediction methods have become a research hotspot and have been widely applied in this field because they do not require modeling based on complex physical mechanisms and can predict remaining service life simply by mining degradation features from multi-sensor monitoring data. However, existing data-driven methods still have significant shortcomings in practical applications: On the one hand, different sensors contribute differently to the engine degradation process, and sensor data contains long-term dependencies in the time dimension and cross-sensor correlation information in the spatial dimension. Existing methods often only perform feature weighting for a single dimension or ignore multi-dimensional correlations, resulting in the burying of key degradation information and affecting the model's accuracy in capturing degradation trends. On the other hand, the prediction process inevitably involves random uncertainties such as data noise and sensor errors, as well as cognitive uncertainties such as insufficient optimization of model parameters and assumption bias. Existing research mostly focuses on improving the accuracy of remaining service life prediction results, lacking effective quantification of prediction uncertainties, resulting in insufficient credibility of prediction results and difficulty in providing comprehensive and reliable reference for maintenance decisions.

[0005] Therefore, how to enhance the extraction of key degradation features by optimizing data processing methods and improve the credibility of prediction results by combining effective uncertainty quantification methods has become an urgent technical problem to be solved in the field of aero-engine remaining service life prediction. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide an engine life prediction method based on multi-head attention that can improve the reliability of prediction results.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] An engine life prediction method based on multi-head attention includes the following steps:

[0009] S1: Data preprocessing: Obtain raw sensor data collected by sensors of key components of aero-engines; normalize the raw sensor data using the maximum-minimum normalization method to eliminate the influence of dimensions; identify and remove constant, non-fluctuating redundant features through data visualization analysis.

[0010] S2: Spatiotemporal dimension data weighting. The preprocessed sensor data is input into the multi-head attention mechanism. A dual-dimensional weighting model is constructed based on the multi-head attention mechanism. The sensor data is adaptively weighted from the spatial and temporal dimensions to enhance key feature information.

[0011] S3: Remaining useful life prediction. Sensor data weighted by spatiotemporal dimensions is input into the Long Short-Term Memory (LSTM) network. The LSM network learns long-term dependencies and degradation features in the data through the forget gate, input gate, output gate and state update unit to obtain a trained LSM network.

[0012] S4: Quantification of prediction uncertainty. Based on a trained long short-term memory network combined with Monte Carlo regularization, the network neurons are randomly shut down multiple times according to a preset discard rate and forward prediction is performed to generate multiple remaining lifetime prediction results. The average of the multiple remaining lifetime prediction results is taken to obtain the final point prediction result. Then, the probability density distribution of the multiple remaining lifetime prediction results is estimated by the kernel density estimation method to quantify the prediction uncertainty.

[0013] As a preferred embodiment, a further technical solution of the present invention is:

[0014] Preferably, in step S1, the formula for normalizing the original sensor data using the max-min normalization method is as follows:

[0015] ;

[0016] in, The raw data collected by the e-th engine and the i-th sensor; This represents the minimum value of the raw data of the i-th sensor during the entire operating cycle; This represents the maximum value of the raw data of the i-th sensor during the entire operating cycle; This represents the normalized data of the e-th engine and the i-th sensor, where F is the normalization identifier.

[0017] Preferably, the adaptive weight allocation of sensor data in step S2 from the spatial dimension is implemented as follows:

[0018] S201: Convert sensor data to... The matrix-form input multi-head attention mechanism, i.e., the input data ;in, This represents the time series data from the i-th input sensor, where F represents the number of features (i.e., the number of sensors), and T represents the time step. t∈[1,T]; for Perform a linear transformation to obtain three sets of intermediate feature matrices. , , ;in, For query direction intermediate matrix, This is the intermediate matrix for the bond direction. The intermediate matrix is ​​the value direction;

[0019] Through trainable parameter matrix pairs , , The mapping is performed to obtain the final query matrix used for attention calculation. Key matrix Value matrix The mapping process is as follows:

[0020] ;

[0021] ;

[0022] ;

[0023] in, , , It is a trainable parameter matrix;

[0024] S202: Calculate the query matrix AND key matrix The correlation is normalized using softmax to obtain attention weights, which are then used to derive attention weights from the weight-value matrix. Weighted, the formula is:

[0025] ;

[0026] in, The feature dimension of the key matrix in the multi-head attention mechanism; The dot product operation directly outputs the correlation score between sensor features; the higher the score, the stronger the feature correlation. The scaling factor is used; the softmax function normalizes the relevance scores, outputting attention weights that conform to a probability distribution, with the sum of all weights being 1, ensuring that features with stronger correlations correspond to larger weights. Finally, the attention weights are determined by... The dot product enhances key features;

[0027] S203: Multiple independent query matrices, key matrices, and value matrices are used to calculate and concatenate the results in parallel to highlight key degradation information between sensors;

[0028] Each group calculates its own single-head attention output independently.

[0029] ;

[0030] Where i = 1, 2, ..., h, and h is the number of heads in the multi-head attention;

[0031] After concatenating all single-head attention outputs, output the weight matrix. Mapping yields a spatial dimension-weighted result. The formula is:

[0032] ;

[0033] in, For feature splicing operations, The output weight matrix is ​​used to integrate features from multiple parallel computations to obtain a spatially weighted result with a unified dimension.

[0034] Preferably, the implementation process of adaptive weight allocation of sensor data from the time dimension in step S2 is as follows:

[0035] S211: Weighting the spatial dimensions Reorganized into a time-series input matrix according to the time step. Specifically, to The column dimensions remain unchanged. The time step information integrated during the original spatial dimension weighting process is re-decomposed and used as the row dimension, ultimately resulting in... ;

[0036] ;

[0037] in, This represents the spatially weighted result of reorganization by time step. ;

[0038] ;

[0039] Where t∈[1,T] represents the feature data at time step t, and T is the time step size;

[0040] S212: For the reassembled time-series input matrix A linear transformation is performed to obtain three sets of intermediate feature matrices along the time dimension. , , These serve as the basic feature matrices for time-dimensional queries, keys, and values, respectively; the trainable parameter matrix in the time dimension is used to... , , Perform dimension adaptation mapping to obtain the final query matrix used for time dimension attention calculation. Key matrix Value matrix The mapping process is as follows;

[0041] ;

[0042] ;

[0043] ;

[0044] Wherein, is the trainable parameter matrix in the time dimension. The initial state is initialized with random values ​​by Xavier and subsequently updated as the model is trained and optimized.

[0045] S213: Calculate the time dimension query matrix AND key matrix The correlation is normalized using softmax to obtain the temporal attention weights, which are then compared with the value matrix. Multiplying them yields the single-head time attention output, as shown in the formula;

[0046] ;

[0047] in, The dot product operation outputs correlation scores between time steps; higher scores indicate stronger feature associations across different time steps. The softmax function normalizes the correlation scores, outputting attention weights that conform to a probability distribution, with the sum of all weights equal to 1. This ensures that features with stronger associations receive larger weights. Finally, the correlation is calculated using... The dot product enhances key temporal degradation information;

[0048] S214: Employing multiple time dimensions , , Matrix parallel computation and result concatenation highlight key degradation correlations between time steps, yielding a time-weighted result. .

[0049] Preferably, in step S214, the time-weighted result The calculation process is as follows: Let the number of heads in the multi-head attention over the time dimension be... Each group independently performs the calculation process S211-S213 to obtain... Group single-head time attention output;

[0050] ;

[0051] Where i = 1, 2, ..., ;

[0052] Will The single-head temporal attention outputs of the group are concatenated along the feature dimension to obtain the dimension. The concatenated matrix is ​​then processed by the output weight matrix in the time dimension. Perform a linear mapping to obtain the time-weighted result. The formula is:

[0053] ;

[0054] in, The output weight matrix is ​​for the time dimension, and the dimension is... The initial state is initialized with random values ​​using Xavier and updated as the model is trained and optimized. The dimension is This serves as the input for subsequent Long Short-Term Memory (LSTM) networks.

[0055] Preferably, the training process of the Long Short-Term Memory network in step S3 involves weighting the feature matrix based on both spatiotemporal dimensions. The data is divided into training samples by a sliding time window. The training samples are input into a 3-layer long short-term memory network. The network learns long-term dependencies and degradation features in the data through forget gate, input gate, output gate and state update unit. The number of hidden layer neurons in each layer of the long short-term memory network is 100.

[0056] Preferably, in step S3, the Long Short-Term Memory network uses the Adam optimization algorithm for parameter optimization, with mean squared error as the loss function. The loss function is calculated using the following formula:

[0057] ;

[0058] Where n is the number of engines. Let e ​​be the predicted remaining service life of the e-th engine. Given the true value of the e-th engine, iterate the training until the loss function converges, and output the trained Long Short-Term Memory network.

[0059] Preferably, the Monte Carlo regularization-based multi-prediction sample generation process in step S4 involves adding a Dropout layer after the hidden layers of the trained Long Short-Term Memory (LSTM) network and setting a preset dropout rate. For the same test sample, keeping the network parameters unchanged, the Dropout layer randomly shuts down some neurons at least 50 times and performs forward prediction to generate multiple independent remaining lifetime prediction values, forming a prediction sample set. , where m is the number of predictions.

[0060] Preferably, the formula for calculating the final prediction result in step S4 is as follows:

[0061] ;

[0062] in, This represents the predicted final remaining service life of the e-th engine, where m is the number of predictions. Let be the remaining service life value predicted for the e-th engine in the k-th prediction. By taking the average value, the random error of the prediction is reduced and the stability of the result is improved.

[0063] Preferably, the prediction uncertainty quantification process based on kernel density estimation in step S4 is as follows:

[0064] Using the predicted sample set as input, the Gaussian kernel function is used as the kernel function for kernel density estimation. The expression for the kernel function is as follows:

[0065] ;

[0066] The probability density distribution curve of the remaining useful life prediction is obtained by kernel density estimation. Based on the distribution curve, the prediction interval [L,U] with a confidence level of 95% is determined, where L is the lower limit of the interval and U is the upper limit of the interval. The prediction uncertainty is characterized by the interval width UL and the probability that the interval contains the true value of the remaining useful life, thus completing the uncertainty quantification.

[0067] The present invention, which adopts the above technical solution, has the following prominent features compared with the prior art:

[0068] 1. This method is based on a multi-head attention mechanism, which adaptively assigns weights to the data from spatial and temporal dimensions, accurately captures key degradation features in multiple dimensions, and solves the problem of information omission caused by single-dimensional weighting.

[0069] 2. This method integrates Monte Carlo regularization and kernel density estimation, which reduces random errors through multiple rounds of prediction and quantifies the uncertainty caused by data noise and model bias, providing probability density distribution and confidence interval, thus overcoming the limitation of traditional point prediction lacking credibility support.

[0070] 3. This method combines the long-term dependent learning ability of Long Short-Term Memory Networks with the feature selection advantages of attention mechanisms. It performs well in single-condition multi-fault mode, does not rely on complex physical modeling, is highly adaptable to scarce full life cycle data, and balances engineering practicality and prediction accuracy. Attached Figure Description

[0071] Figure 1 This is a flowchart of the engine life prediction method based on multi-head attention in an embodiment of the present invention; Figure 2 This is a visualization analysis diagram of sensor data s1-s21 in an embodiment of the present invention; Figure 3 This is a schematic diagram of label settings in an embodiment of the present invention; Figure 4 This refers to the loss variation of the FD001 dataset during actual operation in this embodiment of the invention; Figure 5 This refers to the loss variation of the FD003 dataset during actual operation in this embodiment of the invention. Figure 6 This is a degradation trajectory map predicted for a portion of the engines in the FD001 test set in this embodiment of the invention; Figure 5 Figure (a) shows the predicted degradation trajectory of engine #3 in the FD001 test set; Figure 5 Figure (b) shows the predicted degradation trajectory of engine #81 in the FD001 test set; Figure 7 This is a degradation trajectory map predicted for a portion of the engines in the FD003 test set in this embodiment of the invention; Figure 6 Figure (a) shows the predicted degradation trajectory of engine #66 in the FD003 test set; Figure 6 Figure (b) shows the predicted degradation trajectory of engine #81 in the FD003 test set; Figure 8 This is a probability density distribution diagram of engine No. 1 in the FD001 test set of this invention. Detailed Implementation

[0072] The present invention will be further illustrated below with reference to specific embodiments. The purpose of this illustration is solely to provide a better understanding of the invention. Therefore, the examples given do not limit the scope of protection of the present invention.

[0073] like Figure 1 As shown in the figure, this embodiment presents an engine life prediction method based on multi-head attention, including the following steps:

[0074] S1: Data preprocessing: Obtain raw sensor data collected by sensors of key components of the aero-engine, and normalize the raw sensor data using the maximum-minimum normalization method to eliminate the influence of dimensions. Identify and remove constant and non-fluctuating redundant features through data visualization analysis.

[0075] The formula for normalizing the original sensor data using the max-min normalization method is as follows:

[0076] ;

[0077] in, The raw data collected by the e-th engine and the i-th sensor, where e∈[1,n] and i∈[1,F]; This represents the minimum value of the raw data of the i-th sensor during the entire operating cycle; This represents the maximum value of the raw data of the i-th sensor during the entire operating cycle; This represents the normalized data of the e-th engine and the i-th sensor, where F is the normalization identifier.

[0078] S2: Spatiotemporal dimension data weighting. The preprocessed sensor data is input into the multi-head attention mechanism. A two-dimensional weighting model is constructed based on the multi-head attention mechanism. The sensor data is adaptively weighted from the spatial and temporal dimensions to enhance key feature information.

[0079] (1) The process of adaptive weight allocation of sensor data from the spatial dimension is as follows:

[0080] S201: Convert sensor data to... The matrix-form input multi-head attention mechanism, i.e., the input data ;in, This represents the time series data from the i-th input sensor, where F represents the number of features (i.e., the number of sensors), and T represents the time step. t∈[1,T]; for Perform a linear transformation to obtain three sets of intermediate feature matrices. , , ;in, For query direction intermediate matrix, This is the intermediate matrix for the bond direction. The intermediate matrix is ​​the value direction;

[0081] Through trainable parameter matrix pairs , , The mapping is performed to obtain the final query matrix used for attention calculation. Key matrix Value matrix The mapping process is as follows:

[0082] ;

[0083] ;

[0084] ;

[0085] in, , , This is a trainable parameter matrix.

[0086] S202: Calculate the query matrix AND key matrix The correlation is normalized using softmax to obtain attention weights, which are then used to derive attention weights from the weight-value matrix. Weighted, the formula is:

[0087] ;

[0088] in, The feature dimension of the key matrix in the multi-head attention mechanism; The dot product operation directly outputs the correlation score between sensor features; the higher the score, the stronger the feature correlation. The scaling factor is used; the softmax function normalizes the relevance scores, outputting attention weights that conform to a probability distribution, with the sum of all weights being 1, ensuring that features with stronger correlations correspond to larger weights. Finally, the attention weights are determined by... The dot product is used to enhance key features.

[0089] S203: Multiple independent query matrices, key matrices, and value matrices are used to calculate and concatenate the results in parallel to highlight key degradation information between sensors;

[0090] Each group calculates its own single-head attention output independently.

[0091] ;

[0092] Where i = 1, 2, ..., h, and h is the number of heads in the multi-head attention;

[0093] After concatenating all single-head attention outputs, output the weight matrix. Mapping yields a spatial dimension-weighted result. The formula is:

[0094] ;

[0095] in, For feature splicing operations, The output weight matrix is ​​used to integrate features from multiple parallel computations to obtain a spatially weighted result with a unified dimension.

[0096] (2) The process of adaptive weight allocation of sensor data from the time dimension is as follows:

[0097] S211: Weighting the spatial dimensions Reorganized into a time-series input matrix according to the time step. Specifically, to The column dimensions remain unchanged. The time step information integrated during the original spatial dimension weighting process is re-decomposed and used as the row dimension, ultimately resulting in... ;

[0098] ;

[0099] in, This represents the spatially weighted result of reorganization by time step. ;

[0100] ;

[0101] Where t∈[1,T] represents the feature data at time step t, and T is the time step size.

[0102] S212: For the reassembled time-series input matrix A linear transformation is performed to obtain three sets of intermediate feature matrices along the time dimension. , , These serve as the basic feature matrices for time-dimensional queries, keys, and values, respectively; the trainable parameter matrix in the time dimension is used to... , , Perform dimension adaptation mapping to obtain the final query matrix used for time dimension attention calculation. Key matrix Value matrix The mapping process is as follows;

[0103] ;

[0104] ;

[0105] ;

[0106] Here, is the trainable parameter matrix in the time dimension. The initial state is initialized with random values ​​by Xavier and subsequently updated as the model is trained and optimized.

[0107] S213: Calculate the time dimension query matrix AND key matrix The correlation is normalized using softmax to obtain the temporal attention weights, which are then compared with the value matrix. Multiplying them yields the single-head time attention output, as shown in the formula;

[0108] ;

[0109] in, The dot product operation outputs correlation scores between time steps; higher scores indicate stronger feature associations across different time steps. The softmax function normalizes the correlation scores, outputting attention weights that conform to a probability distribution, with the sum of all weights equal to 1. This ensures that features with stronger associations receive larger weights. Finally, the correlation is calculated using... The dot product enhances key temporal degradation information.

[0110] S214: Employing multiple time dimensions , , Matrix parallel computation and result concatenation highlight key degradation correlations between time steps, yielding a time-weighted result. .

[0111] Specifically, the time-weighted results The calculation process is as follows: Let the number of heads in the multi-head attention over the time dimension be... Each group independently performs the calculation process S211-S213 to obtain... Group single-head time attention output;

[0112] ;

[0113] Where i = 1, 2, ..., ;

[0114] Will The single-head temporal attention outputs of the group are concatenated along the feature dimension to obtain the dimension. The concatenated matrix is ​​then processed by the output weight matrix in the time dimension. Perform a linear mapping to obtain the time-weighted result. The formula is:

[0115] ;

[0116] in, The output weight matrix is ​​for the time dimension, and the dimension is... The initial state is initialized with random values ​​using Xavier and updated as the model is trained and optimized. The dimension is This serves as the input for subsequent Long Short-Term Memory (LSTM) networks.

[0117] S3: Remaining service life prediction. The spatiotemporally weighted sensor data is input into a Long Short-Term Memory (LSTM) network. The LSTM network learns long-term dependencies and degradation features in the data through its forget gate, input gate, output gate, and state update unit, resulting in a trained LSTM network. The output of the LSTM network is the predicted remaining engine service life. ;

[0118] .

[0119] The training process of the LSTM network involves weighting the feature matrix in both spatiotemporal dimensions. Divide into training samples according to a sliding time window (window size 30). The training samples are input into a 3-layer LSTM network, which learns long-term dependencies and degradation features in the data through forget gate, input gate, output gate and state update unit. The number of hidden layer neurons in each LSTM network is 100.

[0120] Forgotten Gate (Controlling the retention of historical degradation information):

[0121] ;

[0122] Input gate (Control the input weights of the current sensor data):

[0123] ;

[0124] Output gate (controls the characteristic output of the current state):

[0125] ;

[0126] Candidate cell states (intermediate variables for updating cell states):

[0127] ;

[0128] Cell status update (integrating historical and current information):

[0129] ;

[0130] Hidden layer output (representation of degenerate features at the current time step):

[0131] ;

[0132] in, ∈ , t∈[1,T]; For the e-th engine at time t, there is a single-step input sample. , , , This represents the weight matrix of each gated unit in the LSTM network; , , , These are the bias terms for each gate unit in the LSTM network; It is a Sigmoid activation function with an output range of [0,1]. It is a hyperbolic tangent activation function with an output range of [-1, 1].

[0133] Furthermore, the LSTM network employs the Adam optimization algorithm for parameter optimization, using the mean squared error as the loss function. The loss function is calculated as follows:

[0134] ;

[0135] Where n is the number of engines. Let e ​​be the predicted remaining service life of the e-th engine. Given the true value of the e-th engine, iterate the training until the loss function converges, and output the trained Long Short-Term Memory network.

[0136] Specifically, the conditions for the convergence of the loss function are as follows: within 10 consecutive training iterations, the fluctuation of the MSE value on the training set does not exceed 5% of the initial MSE value; and the MSE value on the validation set does not decrease for 5 consecutive training iterations, and the difference between the MSE value on the validation set and the MSE value on the validation set is less than 3% of the initial MSE value.

[0137] S4: Quantification of prediction uncertainty. Based on a trained long short-term memory network combined with Monte Carlo regularization (MC-Dropout) method, the network neurons are randomly shut down multiple times according to a preset dropout rate and forward prediction is performed to generate multiple remaining lifetime prediction results. The average of multiple remaining lifetime prediction results is taken to obtain the final point prediction result. Then, the probability density distribution of multiple remaining lifetime prediction results is estimated by kernel density estimation method to quantify the prediction uncertainty.

[0138] Specifically, the multi-prediction sample generation process based on MC-Dropout involves adding a Dropout layer after the hidden layers of a trained Long Short-Term Memory (LSTM) network, setting a preset dropout rate. For the same test sample, keeping the network parameters constant, the Dropout layer randomly shuts down some neurons at least 50 times and performs forward predictions to generate multiple independent Remaining Lifetime (RUL) prediction values, forming a prediction sample set. , where m is the number of predictions.

[0139] The formula for calculating the final point prediction result is as follows:

[0140] ;

[0141] in, Let m be the final RUL prediction value for the e-th engine, and m be the number of predictions. Let RUL be the predicted value of the e-th engine in the k-th prediction. By taking the average value, the random error of the prediction is reduced and the stability of the result is improved.

[0142] Specifically, the process of quantifying prediction uncertainty based on kernel density estimation is as follows:

[0143] Using the predicted sample set as input, the Gaussian kernel function is used as the kernel function for kernel density estimation. The expression for the kernel function is as follows:

[0144] ;

[0145] The probability density distribution curve of the predicted RUL value is obtained by kernel density estimation. Based on the distribution curve, the prediction interval [L,U] with a confidence level of 95% is determined, where L is the lower limit of the interval and U is the upper limit of the interval. The prediction uncertainty is characterized by the interval width UL and the probability that the interval contains the true value of RUL, thus completing the uncertainty quantification.

[0146] To further validate the superior performance of the engine remaining service life prediction model obtained by our proposed method (i.e., multi-head attention mechanism + LSTM network + MC-Dropout and kernel density estimation), validation was performed on the single-condition datasets FD001 and FD003 from the C-MAPSS dataset. The hardware environment consisted of a Windows 11 operating system, 16GB of RAM, and an i5-10210U processor; the software environment used was PyCharm compiler, Python 3.8, and the PyTorch deep learning framework. The dataset used was the publicly available NASA C-MAPSS dataset, with the single-condition subsets FD001 (one fault mode) and FD003 (two fault modes) selected to validate the performance of the engine remaining service life prediction model.

[0147] Since the C-MAPSS dataset used for life prediction analysis does not exhibit a standard normal distribution trend, a maximization-minimum normalization method is used to preprocess the sensor data under different operating conditions to reduce the impact of missing values. The normalized data is then grouped according to engine number to facilitate subsequent network training and testing. A visualization analysis of some normalized sensor data is shown below. Figure 2 As shown, by Figure 2It is known that some sensor data are constant and cannot provide effective degradation information for engine RUL prediction. Therefore, redundant features are removed and only 14 key features are retained for subsequent prediction tasks, including s2, s3, s4, s7, s8, s9, s10, s12, s13, s14, s15, s17, s20, and s21, to complete data preprocessing.

[0148] Labels are essential during model training and help improve model prediction performance. Setting a threshold for labels makes the model pay more attention to data in obvious degradation stages. In this invention, the label threshold is set to 125, and the label settings are as follows: Figure 3 As shown.

[0149] The optimal parameters were determined through multiple sets of comparative experiments, and the specific settings are shown in Table 1.

[0150] Table 1

[0151]

[0152] The sensor data was then processed in batches, and the number of heads for the multi-head attention mechanism was determined based on the data input size. After comparing results over 50 cycles, the optimal number of heads was selected for weighted processing of the two-dimensional data. Table 2 shows a comparison of the prediction results. After 50 cycles, the training effect tended to stabilize. The model performed best when the number of heads in the feature dimension was 10 and the number of heads in the time dimension was 7.

[0153] Table 2

[0154]

[0155] Table 3 presents the process and results of the optimal training period for the FD001 and FD003 datasets for different model training periods. As shown in Table 3, the optimal training period for the model on the FD001 test set is 150, and the optimal training period for the model on the FD003 test set is 100.

[0156] Table 3

[0157]

[0158] The loss changes in the two datasets during actual operation are as follows: Figure 4 , Figure 5 As shown. By Figure 4 , Figure 5 It can be seen that the engine's remaining service life prediction model has a fast convergence speed, high prediction accuracy, and excellent overall performance.

[0159] To evaluate the predictive performance of the engine remaining service life prediction model in this invention, commonly used prediction model evaluation metrics are further introduced to comprehensively evaluate the prediction model. The commonly used metrics used in this embodiment include the root mean square error (RMSE) and the score. The expressions for the root mean square error (RMSE) and the score are as follows:

[0160] ;

[0161] ;

[0162] First, some prediction results from the engine remaining service life prediction model are presented. The training task is performed using a pre-trained LSTM network, and several engines are randomly selected from the test sets of the FD001 and FD003 subsets to examine the correlation between their predicted degradation trajectories and labels. The predicted degradation trajectories and labels for some engines in the FD001 and FD003 test sets are shown below. Figure 6 , Figure 7 As shown in the figure, the prediction fluctuates in the early stages, but as the number of cycles increases, the deviation between the model's predicted value and the actual value decreases, and the overall trajectory approaches the actual value.

[0163] Furthermore, to ensure the credibility and accuracy of the RUL prediction results, we used MC-Dropout and kernel density estimation methods to output 100 prediction results in the forward phase during the testing phase (the mean of the 100 results is the final prediction result) and estimated the probability density distribution of multiple results, while quantifying the prediction uncertainty. The probability density distribution of engine No. 1 in the FD001 test set is shown below. Figure 8 As shown. According to the RUL file in the dataset, the true RUL value of engine No. 1 in the FD001 test set is 112, and the model prediction result is 108. The probability density of the model prediction result is relatively high, so the prediction result is highly reliable and achieves early prediction of engine RUL.

[0164] This invention employs ablation experiments to verify the effectiveness of the two-dimensional data weighting method using the multi-head attention mechanism on the FD001 and FD003 datasets. The experiments compare the prediction performance of the LSTM network and the engine remaining service life prediction model used in this invention, highlighting the importance of data weighting in improving the model's learning ability. The ablation experiment results are shown in Table 4. Compared to the traditional LSTM, the engine remaining service life prediction model shows significant improvements in both RMSE and Score evaluation metrics. Therefore, the spatiotemporal data weighting method proposed in this invention can significantly improve the model's prediction accuracy.

[0165] Table 4

[0166]

[0167] To verify the performance of the engine remaining service life prediction model of this invention in RUL prediction, a comparative analysis was conducted with the prediction results of other models on the FD001 and FD003 datasets, as shown in Table 5. Besides the engine remaining service life prediction model mentioned in this application, the table includes nine relatively new prediction models: Support Vector Regression (SVR), Convolutional Neural Network (CNN), Long Short-Term Memory (LATM), Bayesian Structured Vector Autoregression (BSVAR), Attention-Enhanced LSTM (Attn-LSTM), CNN-LSTM Hybrid Model, Autoencoder-Multi-Scale Similarity Ensemble Model (AE-MSEN), Multi-Fault Mode-Multi-Scale Similarity Ensemble Model (MFM-MSEN), and Bayesian Optimization-Category Boosting Model (BO-CatBoost). The engine remaining service life prediction model designed in this invention significantly outperforms other models in single-condition multi-fault mode prediction, with a 15% improvement in RMSE and a 19.5% improvement in Score. According to the average prediction results, this model has the highest accuracy.

[0168] Table 5

[0169]

[0170] This invention simultaneously considers the spatiotemporal weighting of sensor data and the quantification of RUL prediction uncertainty. Based on a multi-head attention mechanism, sensor data is weighted in both time and space dimensions to highlight key degradation information. Based on an LSTM network, combined with MC-Dropout and kernel density estimation, the RUL prediction uncertainty is quantified to address the impact of uncertainty on model prediction accuracy.

[0171] The experimental results were validated using a single-condition aero-engine, demonstrating that the model has the best average prediction performance and shows significant performance improvement under single-condition multi-fault mode. At the same time, the model provides reliability of RUL prediction by quantifying prediction uncertainty, making the results more reliable. The model of this invention achieves early prediction of RUL, providing data support for engine maintenance. Future research can combine RUL prediction with maintenance decision-making to further optimize equipment maintenance strategies and better achieve intelligent equipment health management.

[0172] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the invention. All equivalent changes made based on the description and drawings of the present invention are included within the scope of the present invention.

Claims

1. A method for predicting engine life based on multi-head attention, characterized in that: Includes the following steps: S1: Data preprocessing: Obtain raw sensor data collected by sensors of key components of aero-engines; normalize the raw sensor data using the maximum-minimum normalization method to eliminate the influence of dimensions; identify and remove constant, non-fluctuating redundant features through data visualization analysis. S2: Spatiotemporal dimension data weighting. The preprocessed sensor data is input into the multi-head attention mechanism. A dual-dimensional weighting model is constructed based on the multi-head attention mechanism. The sensor data is adaptively weighted from the spatial and temporal dimensions to enhance key feature information. S3: Remaining useful life prediction. Sensor data weighted by spatiotemporal dimensions is input into the Long Short-Term Memory (LSTM) network. The LSM network learns long-term dependencies and degradation features in the data through the forget gate, input gate, output gate and state update unit to obtain a trained LSM network. S4: Quantification of prediction uncertainty. Based on a trained long short-term memory network combined with Monte Carlo regularization, the network neurons are randomly shut down multiple times according to a preset discard rate and forward prediction is performed to generate multiple remaining lifetime prediction results. The final point prediction result is obtained by averaging the multiple remaining useful life prediction results; then, the probability density distribution of the multiple remaining useful life prediction results is estimated by kernel density estimation method to quantify the prediction uncertainty.

2. The engine life prediction method based on multi-head attention according to claim 1, characterized in that: In step S1, the formula for normalizing the original sensor data using the max-min normalization method is as follows: ； in, The raw data collected by the e-th engine and the i-th sensor; This represents the minimum value of the raw data of the i-th sensor during the entire operating cycle; This represents the maximum value of the raw data of the i-th sensor during the entire operating cycle; This represents the normalized data of the e-th engine and the i-th sensor, where F is the normalization identifier.

3. The engine life prediction method based on multi-head attention according to claim 2, characterized in that: The adaptive weight allocation of sensor data from the spatial dimension in step S2 is implemented as follows: S201: Convert sensor data to... The matrix-form input multi-head attention mechanism, i.e., the input data ;in, This represents the time series data from the i-th input sensor, where F represents the number of features (i.e., the number of sensors), and T represents the time step. t∈[1,T]; for Perform a linear transformation to obtain three sets of intermediate feature matrices. , , ;in, For query direction intermediate matrix, The middle matrix for the bond direction. The intermediate matrix is ​​the value direction; Through trainable parameter matrix pairs , , The mapping is performed to obtain the final query matrix used for attention calculation. Key matrix Value matrix The mapping process is as follows: ； ； ； in, , , It is a trainable parameter matrix; S202: Calculate the query matrix AND key matrix The correlation is normalized using softmax to obtain attention weights, which are then used to derive attention weights from the weight-value matrix. Weighted, the formula is: ； in, The feature dimension of the key matrix in the multi-head attention mechanism; The dot product operation directly outputs the correlation score between sensor features; the higher the score, the stronger the feature correlation. The scaling factor is used; the softmax function normalizes the relevance scores, outputting attention weights that conform to a probability distribution, with the sum of all weights being 1, ensuring that features with stronger correlations correspond to larger weights. Finally, the attention weights are determined by... The dot product enhances key features; S203: Multiple independent query matrices, key matrices, and value matrices are used to calculate and concatenate the results in parallel to highlight key degradation information between sensors; Each group calculates its own single-head attention output independently. ； Where i = 1, 2, ..., h, and h is the number of heads in the multi-head attention; After concatenating all single-head attention outputs, output the weight matrix. Mapping yields a spatial dimension-weighted result. The formula is: ； in, For feature splicing operations, The output weight matrix is ​​used to integrate features from multiple parallel computations to obtain a spatially weighted result with a unified dimension.

4. The engine life prediction method based on multi-head attention according to claim 3, characterized in that: The implementation process of adaptive weight allocation of sensor data from the time dimension in step S2 is as follows: S211: Weighting the spatial dimensions Reorganized into a time-series input matrix according to the time step. Specifically, to The column dimensions remain unchanged. The time step information integrated during the original spatial dimension weighting process is re-decomposed and used as the row dimension, ultimately resulting in... ; ； in, This represents the spatially weighted result of reorganization by time step. ; ； Where t∈[1,T] represents the feature data at time step t, and T is the time step size; S212: For the reassembled time-series input matrix A linear transformation is performed to obtain three sets of intermediate feature matrices along the time dimension. , , These serve as the basic feature matrices for time-dimensional queries, keys, and values, respectively; the trainable parameter matrix in the time dimension is used to... , , Perform dimension adaptation mapping to obtain the final query matrix used for time dimension attention calculation. Key matrix Value matrix The mapping process is as follows; ； ； ； Wherein, is the trainable parameter matrix in the time dimension. The initial state is initialized with random values ​​by Xavier and subsequently updated as the model is trained and optimized. S213: Calculate the time dimension query matrix AND key matrix The correlation is normalized using softmax to obtain the temporal attention weights, which are then compared with the value matrix. Multiplying them yields the single-head time attention output, as shown in the formula; ； in, The dot product operation outputs correlation scores between time steps; higher scores indicate stronger feature associations across different time steps. The softmax function normalizes the correlation scores, outputting attention weights that conform to a probability distribution, with the sum of all weights equal to 1. This ensures that features with stronger associations receive larger weights. Finally, the correlation is calculated using... The dot product enhances key temporal degradation information; S214: Employing multiple time dimensions , , Matrix parallel computation and result concatenation highlight key degradation correlations between time steps, yielding a time-weighted result. .

5. The engine life prediction method based on multi-head attention according to claim 4, characterized in that: In step S214, the time-weighted result The calculation process is as follows: Let the number of heads in the multi-head attention over the time dimension be... Each group independently performs the calculation process S211-S213 to obtain... Group single-head time attention output; ； Where i = 1, 2, ..., ; Will The single-head temporal attention outputs of the group are concatenated along the feature dimension to obtain the dimension as follows: The concatenated matrix is ​​then processed by the output weight matrix in the time dimension. Perform a linear mapping to obtain the time-weighted result. The formula is: ； in, The output weight matrix is ​​for the time dimension, and the dimension is... The initial state is initialized with random values ​​using Xavier and updated as the model is trained and optimized. The dimension is This serves as the input for subsequent Long Short-Term Memory (LSTM) networks.

6. The engine life prediction method based on multi-head attention according to claim 1, characterized in that: The training process of the Long Short-Term Memory network in step S3 involves weighting the feature matrix in both the spatiotemporal dimensions. The data is divided into training samples by a sliding time window. The training samples are input into a 3-layer long short-term memory network. The long-term dependencies and degradation features in the data are learned through the forget gate, input gate, output gate and state update unit. The number of hidden layer neurons in each layer of the long short-term memory network is 100.

7. The engine life prediction method based on multi-head attention according to claim 1, characterized in that: In step S3, the Long Short-Term Memory network uses the Adam optimization algorithm for parameter optimization, with mean squared error as the loss function. The loss function is calculated as follows: ； Where n is the number of engines. Let e ​​be the predicted remaining service life of the e-th engine. Given the true value of the e-th engine, iterate the training until the loss function converges, and output the trained Long Short-Term Memory network.

8. The engine life prediction method based on multi-head attention according to claim 7, characterized in that: Step S4, the Monte Carlo regularization-based multi-prediction sample generation process, involves adding a Dropout layer after the hidden layers of the trained Long Short-Term Memory (LSTM) network and setting a preset dropout rate. For the same test sample, keeping the network parameters constant, the Dropout layer randomly shuts down some neurons at least 50 times and performs forward prediction to generate multiple independent remaining lifetime prediction values, forming a prediction sample set. , where m is the number of predictions.

9. The engine life prediction method based on multi-head attention according to claim 8, characterized in that: The formula for calculating the final prediction result in step S4 is as follows: ； in, Let m be the predicted final remaining service life of the e-th engine, and m be the prediction number. The remaining service life value of the e-th engine in the k-th prediction is used to reduce the random error of the prediction and improve the stability of the result by taking the average value.

10. The engine life prediction method based on multi-head attention according to claim 1, characterized in that: The process of quantifying the prediction uncertainty based on kernel density estimation in step S4 is as follows: Using the predicted sample set as input, the Gaussian kernel function is used as the kernel function for kernel density estimation. The expression for the kernel function is as follows: ； The probability density distribution curve of the remaining useful life prediction is obtained by kernel density estimation. Based on the distribution curve, the prediction interval [L,U] with a confidence level of 95% is determined, where L is the lower limit of the interval and U is the upper limit of the interval. The prediction uncertainty is characterized by the interval width UL and the probability that the interval contains the true value of the remaining useful life, thus completing the uncertainty quantification.

Images