Aero-engine remaining service life prediction method based on diffusion model and space-time attention mechanism

Abstract

The invention belongs to the technical field of aero-engine health management, and discloses an aero-engine remaining service life prediction method based on a diffusion model and a space-time attention mechanism. The invention provides a novel learning framework which is used for RUL prediction of an aero-engine. The framework fuses a diffusion process and an attention-based time module to achieve end-to-end accurate estimation from sensor signals to RUL. Aiming at the problem that the scarcity problem of aero-engine data limits the application of a deep learning network in the aspect of improving the RUL prediction precision, the method adopts a diffusion process to enrich the data diversity and improve the data quality. In order to improve the accuracy of RUL prediction, an attention mechanism is used to extract time features in samples and between the samples, and time information is effectively utilized, so that the capture capability of the aero-engine performance degradation process is enhanced. The RUL prediction method provided by the invention provides more reliable support for health management and maintenance decision of the aero-engine.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft engine health management and is a method for predicting the remaining useful life (RUL) of an aircraft engine. The method involves data enhancement and time series processing and can be used in technical fields such as aerospace, transportation and the like. Background Art

[0002] In modern industry, the Prognostics and Health Management (PHM) system is crucial to ensure the reliability of industrial activities. It includes key links such as anomaly detection, fault diagnosis and RUL estimation. As a key component of an aircraft, the health status of an aircraft engine is directly related to the safety and reliability of flight. RUL prediction technology can evaluate the operating status of aircraft engines and effectively prevent failures. For airlines and maintenance teams, it is an important tool that can help them develop scientific maintenance plans, detect and solve potential failures in a timely manner, and thus ensure the safety and availability of the engine.

[0003] Aircraft engine failures are usually not sudden, but are caused by the gradual wear and performance degradation of internal components during long-term operation. Key components inside the engine, such as turbine blades, bearings, and seals, operate under extreme conditions such as high temperature, high pressure, and high speed, and are prone to wear and loss, which in turn affects the performance and reliability of the engine. In order to effectively monitor and manage the degradation process of the engine, airlines usually use a variety of means to conduct regular inspections and maintenance. At present, the RUL prediction methods of aircraft engines are mainly divided into model-based methods and data-driven methods. Model-based methods rely on a deep understanding of aircraft engines to establish degradation mechanism models, such as Kalman filters and particle filters. However, given the complexity of aircraft engines, it is difficult to establish an accurate physical model. With the advancement of modern instruments and measurement technology, a large amount of monitoring data can be obtained from sensors of aircraft engines, making data-driven methods a hot topic of research. These methods attempt to directly establish a nonlinear mapping relationship between historical monitoring data and the health status of the engine. In recent years, artificial intelligence technology, especially deep learning technology, has achieved remarkable results in many fields. Neural networks can directly model sensor monitoring data and predict RUL in a data-driven manner without relying on complex physical models and expert knowledge.

[0004] Deep learning technology has shown great potential in the field of aircraft engine RUL prediction, mainly including methods based on RNN, CNN and Transformer. For example, RNN and its variants LSTM (Ma M, Mao Z. Deep Convolution-based LSTM Network for Remaining Useful Life Prediction [J], IEEE Transactions on Industrial Informatics, 2020, PP (99): 1-1. DOI: 10.1109 / TII.2020.2991796.) and GRU (R. Gong, J. Li, and C. Wang, "Remaining useful life prediction based on multisensor fusion and attention TCN-BiGRU model," IEEE Sensors J., vol. 22, no. 21, pp. 21101–21110, Nov. 2022.) have performed well in capturing the complex nonlinear relationship between input and output and the short-term correlation in the time series, and have become the main framework for predicting engine RUL. In addition, RNN is good at capturing time correlation for sequence learning, while CNN is good at extracting local features. Combining the two is also a common solution for RUL prediction. In addition, improved CNN and multi-scale CNN can improve the model's ability to learn complex features (Ren L, Sun Y, Wang H, et al. Prediction of Bearing Remaining Useful Life With Deep Convolution Neural Network [J]. IEEE Access, 2018: 13041-13049. DOI: 10.1109 / ACCESS.2018.2804930.). Transformer-based methods can also achieve more accurate RUL prediction by focusing on more critical information in monitoring data (Zhang Z, Song W, Li Q. Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction [J]. 2021. DOI: 10.48550 / arXiv.2106.15842.).

[0005] Although existing methods have achieved certain results in mining sequence data features to predict the RUL of aircraft engines, there are still challenges that need further improvement. For example, existing methods are often affected by the time dependency of the sequence, which is easily masked by redundant data or noise, and it is difficult to effectively extract the components that really play an important role in the prediction. This patent discovered a time attention mechanism that can pay special attention to the time features between samples to obtain the time correlation in the data, which can suppress the influence of redundant data or noise. In addition, due to the limitations of equipment and testing costs, it is very difficult to obtain experimental samples for the entire life cycle of aircraft engines, which directly limits the number of training samples that can be collected. For deep learning technology that relies on a large amount of data support, this limitation seriously affects the predictive performance of the RUL model. In order to alleviate this problem, the present invention enhances the diversity of data through a symmetric diffusion model, thereby increasing the number of training samples, improving the generalization ability of the model to prevent overfitting, and thus improving the predictive performance of the model. In recent years, diffusion generative models have made great achievements in the field of vision and are the most cutting-edge neural network model infrastructure in the field of image generation. DDIM (Song J, Meng C, Ermon S. Denoising Diffusion Implicit Models [C] / / International Conference on Learning Representations. 2021.) and DDPM (Nichol A, Dhariwal P. Improved Denoising Diffusion Probabilistic Models [J]. 2021. DOI: 10.48550 / arXiv.2102.09672.) improved on the basis of diffusion models have also been gradually applied to research directions such as time series (Rasul K, Seward C, Schuster I, et al. Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting [J]. 2021. DOI: 10.48550 / arXiv.2101.12072.). However, there is no research on the application of diffusion generation model to the RUL prediction task of aircraft engines. While introducing the diffusion generation model, the present invention improves the inverse process of the model, only uses the forward diffusion process to amplify data, and directly replaces the inverse process of the diffusion model through the RUL prediction network, thereby reducing the model training cost and obtaining a new prediction method for aircraft engines. The model can be expanded and applied to related technical fields such as aerospace, transportation, etc.

[0006] In summary, the present invention proposes a method for predicting the RUL of an aircraft engine that combines a symmetric diffusion process and an attention mechanism. The method uses the integration of time attention units to enhance its ability to recognize features of different time scales, so that the model can accurately identify the process of aircraft engine performance degradation, which is crucial for predicting the complex degradation process of aircraft engines. In addition, the method applies the diffusion process to the training stage of model prediction, adds noise to the input sequence and the target sequence to achieve sample amplification, increases the diversity of the data set, reduces the empirical risk in model training, improves generalization ability, and prevents overfitting. The combination of these technologies not only improves the accuracy of RUL prediction, but also enhances the generalization and robustness of the model, providing effective decision support for the health management and maintenance of aircraft engines. Summary of the invention

[0007] The present invention proposes a method for predicting the remaining useful life (RUL) of an aircraft engine, which combines a diffusion process with a temporal attention mechanism. By expanding the original data through the diffusion process, the present invention not only maintains the randomness of the data, but also reduces the training burden of the inference network. In addition, the introduction of the temporal attention unit further enhances the network's comprehensive ability to time features and dependencies between samples, thereby improving the accuracy of the prediction.

[0008] The technical solution of the present invention:

[0009] A method for predicting the remaining useful life of an aircraft engine based on a diffusion model and a spatiotemporal attention mechanism comprises the following steps:

[0010] Step 1: Process the data obtained by the sensor and construct training samples, which include training sets and test sets;

[0011] 1.1) The K-means clustering algorithm is used to conditionally classify the data obtained by the sensor, and the z-score is used to standardize the data of each category. The calculation formula is:

[0012]

[0013] Where i and j represent the i-th sensor and j-th condition classification, respectively, and ν j and τ j They represent the mean and standard deviation of the ith sensor under the jth condition classification, respectively, and x ij and α ij Respectively represent the data before and after standardization;

[0014] 1.2) Write a function to change the shape of input data. The input data includes training data and test data used for training. Generate input data of the desired shape according to the window length and offset. Use a linear degradation model or a piecewise degradation model to predict RUL and set the parameter early_rul. Obtain the shape of the specified array according to different requirements.

[0015] Step 2: Perform data enhancement based on the diffusion model. The data includes the training data set and the label data set in the training set.

[0016] 2.1) Use the forward process of the diffusion model to synchronously expand the training dataset and the label dataset; for a given sequence distribution x0~q(x0), the approximate posterior q(x1,…,x T |x0) is expressed as:

[0017]

[0018] Among them, the transition kernel q(x t ∣x t-1 ) is defined as:

[0019]

[0020] Among them, t and T represent the time step, q(x t ∣x t-1 ) means that at a given x t-1 Under the condition of x t Conditional probability distribution; represents normal distribution; x t and x t-1 is the continuous time step state during the diffusion process; is the scaling factor of the mean; β t is the noise variance at time step t, is the increasing variance schedule used to control the level of added noise; I is the identity matrix;

[0021] 2.2) By sampling the Gaussian vector, we get:

[0022]

[0023] Among them, α t =1-β t , ∈ x is the sampled Gaussian noise of the training dataset;

[0024] 2.3) In order to reduce the uncertainty brought by the generative model, the diffusion process adds the same type of noise to the training dataset and the label dataset at the same time, so that the training dataset sequence and the label dataset sequence remain consistent and intact, and the label formula is generated:

[0025]

[0026] in, β′ t =ωβ t , ω is a fixed scale parameter, ∈ y is the label dataset sampled Gaussian noise;

[0027] Step 3: Based on the training samples constructed in steps 1 and 2, a neural network model is constructed as the engine remaining life prediction model; the neural network model structure includes a time series information encoding layer, a spatiotemporal convolution layer, a multi-head attention layer, and an output layer;

[0028] 3.1) In order to solve the problem that the Transformer self-attention mechanism cannot directly capture the temporal characteristics of the input data, after the input data enters the time series information encoding layer, sine position encoding and cosine position encoding are added through the position encoding layer to provide the relative position information of the input data, thereby enhancing the network structure model's ability to capture temporal characteristics. The corresponding calculation formula is:

[0029]

[0030] Where P(t,2j) represents the value of the t-th time step and the 2j-th sensor in the position encoding matrix; d is the dimension of the position encoding, that is, the number of elements in each position encoding vector;

[0031] 3.2) In order to obtain the spatiotemporal feature information of the input data, add a spatiotemporal convolution layer:

[0032] F st =σ(W st ·(X*T s ))

[0033] Among them, F st represents the spatiotemporal characteristics, W st is the weight matrix, X is the sensor data matrix, T s is the temporal feature matrix, σ is the activation function, and * represents the spatiotemporal convolution operation;

[0034] 3.3) The attention mechanism is the core component of the Transformer model, which combines the upper layer input sensor data matrix X with three different weight matrices W q , W k and W vMultiply them together to get the query vector Q, key vector K and content vector V. The corresponding formula is:

[0035] Q=XW Q

[0036] K=XW K

[0037] V=XW V

[0038] 3.4) By calculating the dot product of the query vector Q and the key vector K, the association matrix is ​​obtained, and the weight corresponding to each sensor data in the upper layer is obtained after activation by the Softmax function; the weight obtained by the Softmax function is combined with the spatiotemporal feature information F st , and the content vector V are multiplied to obtain the fused spatiotemporal attention feature output. The calculation formula is:

[0039]

[0040] Among them, F att is the fused spatiotemporal attention feature, d k is the dimension of the key vector K, As a scaling factor, it is used to alleviate the gradient vanishing problem. ⊙ represents element-by-element multiplication.

[0041] 3.5) The multi-head attention layer adopts a multi-head attention mechanism, that is, multiple groups of Q, K, and V are calculated and then the multiple groups of attention outputs are spliced ​​as the final output to balance the deviations that may be generated by the same attention mechanism, thereby improving the effect of the neural network model. The corresponding calculation formula is:

[0042] F fused =Concat(A1,A2,…,A H )W O

[0043] Among them, F fused represents the fused multi-head spatiotemporal attention features; H is the number of heads, that is, the number of parallel self-attention layers in the neural network model; A i is the self-attention output of the i-th head; A i =Attention(Q i ,K i ,V i ) is the self-attention calculation of the i-th head, which is the same as the calculation of a single self-attention layer; W O is the weight matrix used to combine the outputs of all heads;

[0044] The calculation process of multi-head attention is as follows: (1) For each head i, calculate the query Q i , key K i Sum value Vi It is obtained by multiplying the input sensor data matrix X with the corresponding weight matrix; (2) Calculate the self-attention A for each head i ; (3) Output A1, A2, ..., A of all heads H Concatenate them and then multiply them by the weight matrix W O , to obtain the final multi-head attention output;

[0045] 3.6) The final result is the RUL prediction obtained by the output layer. The output layer processes the normalized attention weight information of the attention module, maps the input to the hidden vector space, and predicts the final RUL. The expression is:

[0046]

[0047] in, is the predicted remaining useful life, W out and b out are the weight and bias of the output layer respectively;

[0048] 3.7) The loss function is defined as follows:

[0049]

[0050] Among them, L is the loss function, y i and are the true value and the predicted value respectively, λ and α are regularization parameters, ∥W st ∥ 2 is the weight matrix W st L2 regularization term;

[0051] Step 4: According to the data set obtained in step 1 and step 2 and the neural network model obtained in step 3, training and testing are performed to obtain the predicted RUL value. The root mean square error loss function, aviation fault diagnosis scoring function and early stopping method are used to evaluate the obtained predicted RUL value and the life prediction effect. The calculation formulas are as follows:

[0052]

[0053] in, y i is the true value, is the model's predicted value.

[0054] Beneficial effects of the present invention: The present invention proposes a novel learning framework specifically for RUL prediction of aircraft engines. The framework integrates the diffusion process and the attention-based time module to achieve end-to-end accurate estimation from sensor signals to RUL. In view of the scarcity of aircraft engine data, which often limits the application of deep learning networks in improving RUL prediction accuracy, the present method uses a diffusion process to enrich data diversity and improve data quality. In order to improve the accuracy of RUL prediction, the attention mechanism is used to extract time features within and between samples, and time information is effectively utilized, thereby enhancing the ability to capture the performance degradation process of aircraft engines. Through the application of these technologies, the RUL prediction method of the present invention can provide more reliable support for the health management and maintenance decisions of aircraft engines. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is an overall flow chart of the remaining useful life prediction algorithm of the present invention.

[0056] Figure 2 It is a neural network model framework diagram.

[0057] Figure 3 This is the RUL prediction result diagram. DETAILED DESCRIPTION

[0058] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0059] The experimental verification was carried out using the aviation engine dataset C-MAPSS provided by NASA.

[0060] Step 1: Process the data obtained by the sensor and construct training samples, which include training sets and test sets;

[0061] 1.1) The K-means clustering algorithm is used to conditionally classify the data obtained by the sensor, and the z-score is used to standardize the data of each category. The calculation formula is:

[0062]

[0063] Where i and j represent the i-th sensor and j-th condition classification, respectively, and v j and τ j They represent the mean and standard deviation of the ith sensor under the jth condition classification, respectively, and x ij and α ij Respectively represent the data before and after standardization;

[0064] 1.2) Write a function to change the shape of input data. The input data includes the training set and the test set. Generate input data of the desired shape according to the window length and offset; use the linear degradation model or the piecewise degradation model to predict RUL, set the parameter early_rul; obtain the shape of the specified array according to different requirements; the shape of the initial training data in the experiment is (20631×26). After data preprocessing, the data shape becomes (20631×14).

[0065] Step 2: Perform data enhancement based on the diffusion model. The data includes the training data set and the label data set in the training set.

[0066] 2.1) The forward process of the diffusion model is used to synchronously expand the training dataset and the label dataset; for a given sequence distribution x0~q(x0), the approximate posterior q(x1,…,x0|x0) of the diffusion process is expressed as:

[0067]

[0068] Among them, the transition kernel q(x t ∣x t-1 ) is defined as:

[0069]

[0070] Among them, t and T represent the time step, q(x t ∣x t-1 ) means that at a given x t-1 Under the condition of x t Conditional probability distribution; represents normal distribution; x t and x t-1 is the continuous time step state during the diffusion process; is the scaling factor of the mean; β t is the noise variance at time step t, is the increasing variance schedule used to control the level of added noise; I is the identity matrix;

[0071] 2.2) By sampling the Gaussian vector, we get:

[0072]

[0073] Among them, α t =1-β t , ∈ x is the sampled Gaussian noise of the training dataset;

[0074] 2.3) In order to reduce the uncertainty brought by the generative model, the diffusion process adds the same type of noise to the training dataset and the label dataset at the same time, so that the training dataset sequence and the label dataset sequence remain consistent and intact, and the label formula is generated:

[0075]

[0076] in, β′ t =ωβ t , ω is a fixed scale parameter, ∈ y is the label dataset sampled Gaussian noise;

[0077] Step 3: Based on the training samples constructed in steps 1 and 2, a neural network model is constructed as the engine remaining life prediction model; the neural network model structure includes a time series information encoding layer, a spatiotemporal convolution layer, a multi-head attention layer, and an output layer;

[0078] 3.1) In order to solve the problem that the Transformer self-attention mechanism cannot directly capture the temporal characteristics of the input data, after the input data enters the time series information encoding layer, sine position encoding and cosine position encoding are added through the position encoding layer to provide the relative position information of the input data, thereby enhancing the network structure model's ability to capture temporal characteristics. The corresponding calculation formula is:

[0079]

[0080] Where P(t,2j) represents the value of the t-th time step and the 2j-th sensor in the position encoding matrix; d is the dimension of the position encoding, that is, the number of elements in each position encoding vector;

[0081] 3.2) In order to obtain the spatiotemporal feature information of the input data, add a spatiotemporal convolution layer:

[0082] F st =σ(W st ·(X*T s ))

[0083] Among them, F st represents the spatiotemporal characteristics, W st is the weight matrix, X is the sensor data matrix, T s is the temporal feature matrix, σ is the activation function, and * represents the spatiotemporal convolution operation;

[0084] 3.3) The attention mechanism is the core component of the Transformer model, which combines the upper layer input sensor data matrix X with three different weight matrices W q , W k and W vMultiply them together to get the query vector Q, key vector K and content vector V. The corresponding formula is:

[0085] Q=XW Q

[0086] K=XW K

[0087] V=XW V

[0088] 3.4) By calculating the dot product of the query vector Q and the key vector K, the association matrix is ​​obtained, and the weight corresponding to each sensor data in the upper layer is obtained after activation by the Softmax function; the weight obtained by the Softmax function is combined with the spatiotemporal feature information F st , and the content vector V are multiplied to obtain the fused spatiotemporal attention feature output. The calculation formula is:

[0089]

[0090] Among them, F att is the fused spatiotemporal attention feature, d k is the dimension of the key vector K, As a scaling factor, it is used to alleviate the gradient vanishing problem. ⊙ represents element-by-element multiplication.

[0091] 3.5) The multi-head attention layer adopts a multi-head attention mechanism, that is, multiple groups of Q, K, and V are calculated and then the multiple groups of attention outputs are spliced ​​as the final output to balance the deviations that may be generated by the same attention mechanism, thereby improving the effect of the neural network model. The corresponding calculation formula is:

[0092] F fused =Concat(A1,A2,…,A H )W O

[0093] Among them, F fused represents the fused multi-head spatiotemporal attention features; H is the number of heads, that is, the number of parallel self-attention layers in the neural network model; A i is the self-attention output of the i-th head; A i =Attention(Q i ,K i ,V i ) is the self-attention calculation of the i-th head, which is the same as the calculation of a single self-attention layer; W O is the weight matrix used to combine the outputs of all heads;

[0094] The calculation process of multi-head attention is as follows: (1) For each head i, calculate the query Q i , key K i Sum value Vi It is obtained by multiplying the input sensor data matrix X with the corresponding weight matrix; (2) Calculate the self-attention A for each head i ; (3) Output A1, A2, ..., A of all heads H Concatenate them and then multiply them by the weight matrix W O , to obtain the final multi-head attention output;

[0095] 3.6) The final result is the RUL prediction obtained by the output layer. The output layer processes the normalized attention weight information of the attention module, maps the input to the hidden vector space, and predicts the final RUL. The expression is:

[0096]

[0097] in, is the predicted remaining useful life, W out and b out are the weight and bias of the output layer respectively;

[0098] 3.7) The loss function is defined as follows:

[0099]

[0100] Among them, L is the loss function, y i and are the true value and the predicted value respectively, λ and α are regularization parameters, ∥W st ∥ 2 is the weight matrix W st L2 regularization term;

[0101] Step 4: According to the data set obtained in step 1 and step 2 and the neural network model obtained in step 3, training and testing are performed to obtain the predicted RUL value. The root mean square error loss function, aviation fault diagnosis scoring function and early stopping method are used to evaluate the obtained predicted RUL value and the life prediction effect. The calculation formulas are as follows:

[0102]

[0103] in, y i is the true value, is the model's predicted value.

[0104] Finally, the prediction results are Figure 3 , where True RUL represents the real RUL, and Pred RUL is the predicted result graph of the model.

Claims

1. A method for predicting the remaining useful life of an aircraft engine based on a diffusion model and a spatiotemporal attention mechanism, characterized in that: The following steps are involved: Step 1: Process the data obtained by the sensor and construct training samples, which include training sets and test sets; 1.1) The K-means clustering algorithm is used to conditionally classify the data obtained by the sensor, and the z-score is used to standardize the data of each category. The calculation formula is: Where i and j represent the i-th sensor and j-th condition classification, respectively, and v j and τ j They represent the mean and standard deviation of the ith sensor under the jth condition classification, respectively, and x ij and α ij Respectively represent the data before and after standardization; 1.2) Write a function to change the shape of input data. The input data includes training data and test data used for training. Generate input data of the desired shape according to the window length and offset. Use a linear degradation model or a piecewise degradation model to predict RUL and set the parameter early_rul. Obtain the shape of the specified array according to different requirements. Step 2: Perform data enhancement based on the diffusion model. The data includes the training data set and the label data set in the training set. 2.1) Use the forward process of the diffusion model to synchronously expand the training dataset and the label dataset; for a given sequence distribution x0~q(x0), the approximate posterior q(x1,…,x T |x0) is expressed as: Among them, the transition kernel q(x t ∣x t-1 ) is defined as: Among them, t and T represent the time step, q(x t ∣x t-1 ) means that at a given x t-1 Under the condition of x t Conditional probability distribution; represents normal distribution; x t and x t-1 is the continuous time step state during the diffusion process; is the scaling factor of the mean; β t is the noise variance at time step t, is the increasing variance schedule used to control the level of added noise; I is the identity matrix; 2.2) By sampling the Gaussian vector, we get: Among them, α t =1-β t , ∈ x is the sampled Gaussian noise of the training dataset; 2.3) In order to reduce the uncertainty brought by the generative model, the diffusion process adds the same type of noise to the training dataset and the label dataset at the same time, so that the training dataset sequence and the label dataset sequence remain consistent and intact, and the label formula is generated: in, β′ t =ωβ t , ω is a fixed scale parameter, ∈ y is the label dataset sampled Gaussian noise; Step 3: Based on the training samples constructed in steps 1 and 2, a neural network model is constructed as the engine remaining life prediction model; the neural network model structure includes a time series information encoding layer, a spatiotemporal convolution layer, a multi-head attention layer, and an output layer; 3.1) In order to solve the problem that the Transformer self-attention mechanism cannot directly capture the temporal characteristics of the input data, after the input data enters the time series information encoding layer, sine position encoding and cosine position encoding are added through the position encoding layer to provide the relative position information of the input data, thereby enhancing the network structure model's ability to capture temporal characteristics. The corresponding calculation formula is: Where P(t,2j) represents the value of the t-th time step and the 2j-th sensor in the position encoding matrix; d is the dimension of the position encoding, that is, the number of elements in each position encoding vector; 3.2) In order to obtain the spatiotemporal feature information of the input data, add a spatiotemporal convolution layer: F st =σ(W st ·(X*T s )) Among them, F st represents the spatiotemporal characteristics, W st is the weight matrix, X is the sensor data matrix, T s is the temporal feature matrix, σ is the activation function, and * represents the spatiotemporal convolution operation; 3.3) The attention mechanism is the core component of the Transformer model, which combines the upper layer input sensor data matrix X with three different weight matrices W q , W k and W v Multiply them together to get the query vector Q, key vector K and content vector V. The corresponding formula is: Q=XW Q K=XW K V=XW V 3.4) By calculating the dot product of the query vector Q and the key vector K, the association matrix is ​​obtained, and the weight corresponding to each sensor data in the upper layer is obtained after activation by the Softmax function; the weight obtained by the Softmax function is combined with the spatiotemporal feature information F st , and the content vector V are multiplied to obtain the fused spatiotemporal attention feature output. The calculation formula is: Among them, F att is the fused spatiotemporal attention feature, d k is the dimension of the key vector K, As a scaling factor, it is used to alleviate the gradient vanishing problem. ⊙ represents element-by-element multiplication. 3.5) The multi-head attention layer adopts a multi-head attention mechanism, that is, multiple groups of Q, K, and V are calculated and then the multiple groups of attention outputs are spliced ​​as the final output to balance the deviations that may be generated by the same attention mechanism, thereby improving the effect of the neural network model. The corresponding calculation formula is: F fused =Concat(A1,A2,…,A H )W O Among them, F fused represents the fused multi-head spatiotemporal attention features; H is the number of heads, that is, the number of parallel self-attention layers in the neural network model; A i is the self-attention output of the i-th head; A i =Attention(Q i ,K i ,V i ) is the self-attention calculation of the i-th head, which is the same as the calculation of a single self-attention layer; W O is the weight matrix used to combine the outputs of all heads; The calculation process of multi-head attention is as follows: (1) For each head i, calculate the query Q i , key K i Sum value V i It is obtained by multiplying the input sensor data matrix X with the corresponding weight matrix; (2) Calculate the self-attention A for each head i ; (3) Output A1, A2, ..., A of all heads H Concatenate them and then multiply them by the weight matrix W O , to obtain the final multi-head attention output; 3.6) The final result is the RUL prediction obtained by the output layer. The output layer processes the normalized attention weight information of the attention module, maps the input to the hidden vector space, and predicts the final RUL. The expression is: in, is the predicted remaining useful life, W out and b out are the weight and bias of the output layer respectively; 3.7) The loss function is defined as follows: Among them, L is the loss function, y i and are the true value and the predicted value respectively, λ and α are regularization parameters, ∥W st ∥ 2 is the weight matrix W st L2 regularization term; Step 4: According to the data set obtained in step 1 and step 2 and the neural network model obtained in step 3, training and testing are performed to obtain the predicted RUL value. The root mean square error loss function, aviation fault diagnosis scoring function and early stopping method are used to evaluate the obtained predicted RUL value and the life prediction effect. The calculation formulas are as follows: in, y i is the true value, is the model's predicted value.

Images