Self-supervised representation and life prediction transfer modeling method for aero-engine under variable operating conditions

Abstract

The application discloses a method for self-supervised representation and life prediction migration modeling of an aero-engine under variable working conditions. First, a degradation feature extractor of the aero-engine based on a self-attention mechanism is constructed. The feature extractor can effectively utilize the long short-term dependence of the time sequence context in the aero-engine to perform global modeling and extract degradation features. On this basis, the feature extractor is self-supervised pre-trained by using the method of masking and reconstructing the original data, so that the features extracted by the feature extractor have stronger generalization representation capability. Then, a life prediction network layer is added to the feature extractor under a complex working condition scene, and the feature extractor is fine-tuned, so that the migration modeling and efficient life prediction of the aero-engine under variable working conditions are realized. The method reduces the requirements of the aero-engine life prediction model based on end-to-end deep learning on the amount and distribution of training data, provides a more reliable reference for operation and maintenance decision makers, and can reduce the operation and maintenance cost while improving the operation reliability of the aero-engine.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of aero-engine residual useful life prediction, and particularly relates to a self-supervised representation and life prediction migration modeling method for aero-engines under variable working conditions. BACKGROUND

[0002] An aero-engine is the most core component in an aircraft power system, and its health state is related to the flight safety of the whole aircraft. With the development of aero-engine design technology, the internal structure of the aero-engine tends to be precise and complex. Meanwhile, the aero-engine is often affected by operating conditions such as high temperature, high speed and high pressure, which leads to the performance degradation of the internal sub-components of the aero-engine, and the degradation speed will rapidly increase with the running time, further causing serious failures. If the aero-engine with serious degradation cannot be maintained in time, it may cause serious safety accidents. According to the statistics of the National Aeronautics and Space Administration (NASA), in the civil field, the proportion of engine failures in all aircraft mechanical failures reaches more than 30%, and the daily maintenance and overhaul costs of the engine account for 31% and 27% respectively. In addition, nearly 10% of flights are canceled every year due to unplanned maintenance, causing additional maintenance costs. In the military field, the maintenance of the aero-engine is also the primary factor affecting the combat readiness and mission reliability of the air force. The U.S. Department of Defense spends about 1.3 billion U.S. dollars on purchasing engines every year, and the cost of maintaining existing engines reaches about 3.5 billion U.S. dollars.

[0003] Therefore, it is necessary to use certain technical means to monitor the health state of the aero-engine, discover or predict the abnormality of the aero-engine in time, provide effective operation and maintenance suggestions for operation and maintenance decision makers, and develop a reasonable operation and maintenance plan, so as to improve the operation reliability of the aero-engine while reducing the operation and maintenance cost, that is, to realize the fault prediction and health management (Prognostics and Health Management, PHM) of the aero-engine. Among them, the residual useful life prediction technology is the most critical technology in the field of aero-engine PHM, which aims to use the historical and current sensor data in the operation process of the aero-engine to explore the degradation law of the aero-engine, predict the residual operation time of the engine, and thus form effective operation and maintenance decision suggestions.

[0004] Traditional aero-engine remaining useful life prediction technology adopts the method of constructing physical or statistical model, and suggests aero-engine degradation model. However, due to the complex internal structure and working condition of aero-engine, it is difficult to accurately describe the degradation trend of aero-engine by using such model. In recent years, deep learning technology based on neural network is also applied to aero-engine remaining useful life prediction. Compared with the method of physical or statistical model, the remaining life prediction method based on deep learning can achieve higher prediction accuracy due to its strong nonlinear description ability and the advantage of automatic feature mining.

[0005] However, the aero-engine remaining useful life prediction method based on deep learning has certain limitations. It depends on sufficient data samples containing labels, and assumes that the data distribution in the training data is consistent with that in the actual application scene. However, during the flight of the aircraft, there are multiple actions such as take-off, climbing, cruising and landing, resulting in the changeable operation condition of the aero-engine, i.e. the complex and changeable working condition of the aero-engine. Different working conditions will have different effects on the degradation of the aero-engine. Meanwhile, the cost of collecting sufficient data samples under each working condition and labeling them is very high, so the data samples that can be used to train the deep learning model are often insufficient and unbalanced, and the trained model is also difficult to adapt to the change of the working condition of the aero-engine.

[0006] The present application aims at the problem that the aero-engine remaining useful life prediction model based on deep learning is difficult to adapt to the complex working condition of the aero-engine, and proposes a self-supervised representation and life prediction transfer modeling method for aero-engine under variable working condition. The method first constructs an aero-engine degradation feature extractor based on self-attention mechanism, which can effectively capture the global degradation information in the data samples, and pre-train the feature extractor using the mask self-supervised learning mechanism and a large number of unlabelled samples under a single working condition, so that the trained feature extractor has strong generalization representation ability. Add a life prediction layer at the back end of the feature extractor, and use a small amount of samples under each working condition to fine-tune the model, so as to migrate the model to the variable working condition scene of the aero-engine for application, realize high-precision remaining useful life prediction of the aero-engine under complex working condition, and assist the operation and maintenance personnel to make operation and maintenance decisions. SUMMARY

[0007] The present application aims at the problem that the aero-engine remaining useful life prediction model based on deep learning is difficult to adapt to the complex working condition of the aero-engine, and proposes a self-supervised representation and life prediction transfer modeling method for aero-engine under variable working condition. The method first constructs an aero-engine degradation feature extractor based on self-attention mechanism, which can effectively capture the global degradation information in the data samples, and pre-train the feature extractor using the mask self-supervised learning mechanism and a large number of unlabelled samples under a single working condition, so that the trained feature extractor has strong generalization representation ability. Add a life prediction layer at the back end of the feature extractor, and use a small amount of samples under each working condition to fine-tune the model, so as to migrate the model to the variable working condition scene of the aero-engine for application, realize high-precision remaining useful life prediction of the aero-engine under complex working condition, and assist the operation and maintenance personnel to make operation and maintenance decisions.

[0008] The purpose of the present application is realized by the following technical scheme:

[0009] A self-supervised representation and life prediction transfer modeling method for aero-engine under variable working condition, comprising the following steps:

[0010] Collecting aero-engine operation data under a single common working condition to construct a pre-training data set; each sample of the pre-training data set is aero-engine operation data of multiple operation periods, wherein part of the aero-engine operation data of the operation periods is provided with a mask, and the aero-engine operation data of each operation period is composed of multiple sensor data related to aero-engine degradation and current running time;

[0011] Constructing a pre-training model, the pre-training model comprising an input module, an aero-engine degradation feature encoder and a decoder based on a self-attention mechanism connected in sequence; wherein the input module comprises an embedding layer and a position encoder; the embedding layer is used to embed the input vector into a high-dimensional space to obtain a high-dimensional embedding vector; the position encoder is used to position encode the input high-dimensional embedding vector to obtain an embedded and position encoded vector; the aero-engine degradation feature encoder based on the self-attention mechanism comprises multiple multi-head self-attention feature extraction modules connected in sequence, each multi-head self-attention feature extraction module comprising at least multiple self-attention heads, a full connection layer and a normalization module, wherein each self-attention head is used to obtain an attention feature vector based on the input vector; the full connection layer is used to splice and fuse the attention feature vectors output by each self-attention head to output a multi-head attention feature vector;

[0012] The normalization module is used to normalize the multi-head attention feature vector;

[0013] The decoder is used to decode the feature vector output by the aero-engine degradation feature encoder based on the self-attention mechanism;

[0014] Each sample in the pre-training data set is used as the input of the pre-training model, wherein the high-dimensional embedding vector corresponding to the unmasked data is encoded by the aero-engine degradation feature encoder based on the self-attention mechanism to obtain an output vector with multi-head attention, the output vector with multi-head attention and the high-dimensional embedding vector corresponding to the masked data are decoded by the decoder, and the loss of the decoder output and the original aero-engine operation data with the mask is minimized as the training target; after the training is completed, the decoder is discarded, and the trained input module, the aero-engine degradation feature encoder based on the self-attention mechanism and a life prediction neural network layer constitute a neural network model for efficient life prediction of aero-engines under variable working conditions;

[0015] Collecting aero-engine operation data under complex working conditions to construct an aero-engine remaining life prediction data set under complex working conditions; wherein each sample of the aero-engine remaining life prediction data set under complex working conditions is aero-engine operation data of multiple operation cycles, and each aero-engine operation data of an operation cycle is composed of multiple sensor data and current running time; the label of each sample is the remaining life; each sample in the aero-engine remaining life prediction data set under complex working conditions is taken as the input of a neural network model for efficient life prediction of aero-engine under variable working conditions, and the model is trained to minimize the loss between the model output and the label to obtain a trained neural network model for efficient life prediction of aero-engine under variable working conditions.

[0016] Further, the aero-engine operation data under a single common working condition and the aero-engine operation data under complex working conditions are normalized data.

[0017] Further, each sample of the pre-training data set is obtained based on aero-engine operation data under a single common working condition by using sliding window sampling; each sample of the aero-engine remaining life prediction data set under complex working conditions is obtained based on aero-engine operation data under complex working conditions by using sliding window sampling.

[0018] Further, in each sample of the pre-training data set, the proportion of aero-engine operation data with a mask is 75%.

[0019] Further, the normalization module comprises a first normalization layer, a residual feedforward neural network layer and a second normalization layer connected in sequence.

[0020] Further, a nonlinear activation function GELU is used to activate after the fully connected feedforward neural network layer, and the GELU function is shown in formula (1);

[0021]

[0022] wherein x represents the output of the fully connected feedforward neural network layer, and tanh is the hyperbolic tangent function, and its calculation formula is as follows;

[0023]

[0024] Further, the aero-engine degradation feature encoder based on the self-attention mechanism comprises a 2-layer stacked multi-head self-attention feature extraction module.

[0025] The beneficial effects of the present application are: the present application proposes a self-supervised representation and life prediction migration modeling method of aero-engine under variable working conditions, aiming at the problem that the life prediction model of the aero-engine is difficult to adapt due to the complex and variable working conditions. The method first constructs an aero-engine degradation feature extractor based on a self-attention mechanism, and uses the global information with long and short term dependencies in the input sensor data samples to extract degradation features. On this basis, the mask self-supervised learning method and a large amount of sensor data under single working condition which is easy to collect are used to pre-train the feature extractor, and the generalization representation ability of the feature extractor is further enhanced. Then, the feature extractor is added with a life prediction network layer, and a small amount of samples under complex working conditions are used to fine-tune the model parameters, so as to obtain a high-precision aero-engine remaining life prediction model which can be applied under complex working condition scenes. The life prediction result will also provide more reliable decision reference for operation and maintenance personnel, so that the operation and maintenance personnel can make more timely and reasonable maintenance and repair plan, and further improve the safety and reliability of the aero-engine and reduce the related operation and maintenance cost. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flow chart of the self-supervised representation and life prediction migration modeling method of aero-engine under variable working conditions of the present application.

[0027] Figure 2 is a visual diagram of the sensor full life cycle of a single engine in the example.

[0028] Figure 3 is a structure diagram of the neural network as the feature extractor in the example.

[0029] Figure 4 is the life prediction result of the model for the #44 engine in the FD002 subset test set in the example. DETAILED DESCRIPTION

[0030] The present application will be further described in detail below in combination with the drawings and specific examples.

[0031] The present embodiment uses the C-MAPSS data set opened by the United States National Aeronautics and Space Administration as an example. The data set uses a large commercial turbofan engine simulation tool developed based on the Matlab Simulink environment to simulate the operation of an aero turbine engine system and the failure and performance degradation process of the main rotating parts in a relatively wide operating condition range, and records sensor data. The data set includes four subsets, FD001, FD002, FD003 and FD004. The engine data contained in FD001 is collected under a single operating condition, and the engine data contained in FD002 is collected under varying operating conditions and contains 6 operating conditions. Therefore, the training set of the FD001 data set is used as the pre-training set of the present embodiment, and the FD002 data subset is used as the fine-tuning data set and the test set in the present embodiment. Each data subset contains multiple aeroengine 21 full life cycle sensor data, as shown in the accompanying drawings. Figure 2

[0032] In the present embodiment, the flow of the present method is shown in Figure 1 , and the specific implementation process is as follows:

[0033] (1) Collect aeroengine operating data under a single common operating condition to construct a pre-training data set; in the present embodiment, the FD001 subset in the C-MAPSS data set is directly preprocessed to obtain the pre-training data set, and the specific steps are as follows:

[0034] (1.1) Clean up the 100 aeroengine operating data in the training set of the FD001 subset, retain the sensor data related to aeroengine degradation, and eliminate the sensor data that does not change with the increase of operating time during engine operation, i.e. discard #1, #5, #6, #9, #10, #16, #18, #19 sensor data, retain the data of 14 sensors, and take the operating time as an additional sensor variable, so the number of input sensor variables is 15;

[0035] Further, in order to facilitate subsequent processing, the cleaned sensor data can also be normalized by maximum and minimum, and formula (2) is used for calculation, wherein x s,t represents the value recorded by the s-th sensor at t time (t period), represents the minimum value recorded by the s-th sensor, represents the maximum value recorded by the s-th sensor.

[0036]

[0037] ​(1.2) Sliding window sampling is performed on the sensor data to make a pre-training set without labels, where each sliding window is a sample, and each sample is the aero-engine running data of multiple running periods. In this embodiment, the sliding window step is 1, and the sliding window size is generally selected between 20 and 50 running periods. In this embodiment, the sliding window size is selected to be 40 running periods.

[0038] (1.3) The aero-engine running data of part of the running periods in each sample is masked. In this embodiment, the aero-engine running data of one running period is a vector of length 15, which is divided into 40 vectors of length 15 according to the sliding window size.

[0039] (1.4) A part of the vectors are randomly selected from the 40 vectors. In this embodiment, 30 vectors are selected for masking, that is, the masking rate is 75%. The masked vectors are denoted as V i,masked , and the unmasked vectors are denoted as V j,unmasked , where i = 1, 2, …, 30, and j = 1, 2, …, 10.

[0040] (2) A pre-training model is constructed, which includes an input module, an aero-engine degradation feature encoder and a decoder based on a self-attention mechanism connected in sequence. The input module includes an embedding layer and a position encoder, which respectively embed and position encode the input samples (masked vectors and unmasked vectors). The specific steps are as follows:

[0041] (a1) The vectors are embedded into a high-dimensional space using a fully connected layer to obtain high-dimensional embedding vectors a i,masked and a j,unmasked . The fully connected layer is the embedding layer.

[0042] (a2) The position encoder is used to uniformly position encode all high-dimensional embedding vectors a i,masked and a j,unmasjed . The position encoder is a learnable position encoder, that is, a set of learnable parameters is initialized, the set of parameters is summed with the corresponding embedding vectors of the positions, and the network parameters are trained simultaneously during the network training process.

[0043] (a3) The vectors a i,masked and a j,unmasked after embedding and position encoding are obtained.

[0044] The aero-engine degradation feature encoder based on the self-attention mechanism includes a plurality of multi-head self-attention feature extraction modules connected in sequence. Each multi-head self-attention feature extraction module includes at least a plurality of self-attention heads, a fully connected layer and a normalization module. Each self-attention head is used to extract features based on the input vectors a j,unmaskedj = 1, 2, …, 10, the self-attention mechanism and vector temporal context information extraction with long short-term dependence of global features, the specific steps are as follows:

[0045] (b1) using three fully connected layers to map the input vector a j,unmasked Each is mapped to a query vector, key vector and value vector, respectively denoted as q j ,k j ,v j The parameters in the fully connected layer are W q , W k , W v ;

[0046] (b2) the dot product of the vector itself q and the k of the other vector is calculated, and the dot product result is divided by the scaling factor d k The size of the vector q, k, v is the length of the vector;

[0047] (b3) using the Softmax function to normalize the calculation result described in step (3.3), obtain the attention coefficient, multiply the coefficient with the vector v, obtain the feature vector b j,unmasked With attention, where j = 1, 2, …, 10; as follows:

[0048]

[0049] Multihead(q, k, v) = Concat(head1, head2…, head h )W o

[0050] (b4) the calculation results of each self-attention head are spliced, and the multi-head self-attention is fused by using the fully connected layer, and the feature vector with multi-head self-attention is output.

[0051] Finally, the feature vector with multi-head self-attention is normalized by using the normalization module; in this embodiment, the normalization module includes a first layer normalization layer, a residual feedforward neural network layer and a second layer normalization layer connected in turn. Specifically, as shown in Figure 3 The first layer normalization layer, the feature vector with multi-head self-attention is summed with the input vector and input into the layer normalization layer, and the calculation formula of the layer normalization layer is as follows:

[0052]

[0053] wherein x is the feature vector with multi-head self-attention input into the first layer normalization layer, μ is the mean of all values in a single sample x, σ is the standard deviation of all values in a single sample, γ and β are learnable parameters adjusted during the model training process.

[0054] a residual feedforward neural network layer, inputting the output of the first layer normalization layer into a single-layer feedforward neural network, and then summing the output of the single-layer feedforward neural network and the output of the first layer normalization layer to form a residual connection and output.

[0055] a second layer normalization layer, which has the same calculation process as the first layer normalization layer, and takes the layer normalization result as the output feature of the aviation engine degradation feature encoder based on the self-attention mechanism.

[0056] (3) using the pre-training set in step (1) and gradient descent method to pre-train the pre-trained model for self-supervised representation learning, wherein a j,unmasked inputting the two-layer stacked multi-head self-attention calculation module, i.e., the encoder module, to obtain an output vector b j,unmasked .

[0057] a i,masked and b j,unmasked are input into the decoder for decoding and outputting, in this embodiment, the decoder structure is the same as the encoder structure; the mean square error between the decoder output and the original data of the masked vector is taken as the loss function, the encoder and decoder modules are trained, the training rounds are 200 rounds, the optimizer used is Adam, the learning rate is 0.0001, and the number of samples in each training batch is 256.

[0058] After the training is completed, the decoder is discarded, the aviation engine degradation feature encoder based on the self-attention mechanism is retained as a feature extractor, and the embedding layer and the position encoder are retained as a model input module.

[0059] (4) collecting aviation engine operation data under complex conditions to make a fine-tuning data set for predicting the remaining life of the aviation engine under complex conditions, and the specific steps are as follows:

[0060] (4.1) pre-processing the aviation engine operation data under complex conditions by the same method as step (1.1);

[0061] (4.2) labeling the remaining life of the aviation engine operation data according to the total life and the running time of the aviation engine wherein i represents the i-th engine, represents the total running time of the i-th engine, represents the running time of the i-th engine up to the current time, and a fine-tuning data set for predicting the remaining life of the aviation engine under complex conditions is obtained.

[0062] (5) The feature extractor described in step (3) is added with a life prediction neural network layer, and a neural network model for high-efficiency life prediction of an aero-engine under variable working conditions is constituted.

[0063] The trained model of the application is deployed to a computing platform, and the aero-engine operation data preprocessed through steps (1.1) and (1.2) is input, and the residual life of the aero-engine can be predicted through model calculation.

[0064] The performance of the neural network model for high-efficiency life prediction of an aero-engine under variable working conditions constructed by the method is evaluated by using the RMSE index and the Score index, and the calculation formulas are shown in formulas (3) and (4), wherein represents the residual life prediction value of the i th engine, y i represents the actual value of the life of the i th engine, and m represents the total number of engines. For the life prediction model, the smaller the values of the two indexes, the higher the prediction accuracy of the model. When the predicted life is greater than the actual residual life of the aero-engine, the maintenance decision maker will make a more serious misjudgment, and the Score index imposes a greater penalty on the case where the predicted life is greater than the actual residual life of the aero-engine, so the actual application effect of the method can be better reflected.

[0065]

[0066]

[0067] The residual life of 259 engines in the FD002 test set is predicted and verified, and compared with the prior art.

[0068] Table 1 is the comparison of the present application and the prior art, after comparison, the present application greatly improves the prediction accuracy of the residual life prediction model in the aero-engine fault diagnosis task compared with the prior art. The constructed feature extractor based on the self-attention mechanism utilizes the correlation between the sensor data of different running periods in the input sample for modeling, extracts the global degradation information with long and short term dependencies, can more effectively extract the aero-engine degradation features, and can realize high-precision aero-engine life prediction. On this basis, the mask self-supervised learning mechanism is used for pre-training the model, so that the proposed feature extractor has stronger generalization representation ability, and a small amount of samples under various working conditions are used to fine-tune the model to realize the transfer adaptation of the life prediction model under the complex running conditions of the aero-engine, and the prediction accuracy is further improved, and the Figure 4 The life prediction results of the model for the #44 engine in the FD002 subset are shown, especially when the engine degradation is serious, the prediction accuracy is higher, and these high-precision life prediction results will provide more reliable reference suggestions for operation and maintenance decision makers.

[0069] Table 1 Comparison of the present application and the prior art

[0070] Model RMSE Score DCNN 22.36 10412.00 CNN-LSTM 27.23 9869.00 DATCN 16.95 1842.38 ATS2S 14.65 876.00 AGCNN 19.43 1492.00 DAST 15.25 924.96 Invention (unpretrained) 13.93 1040.62 Invention (pretrained) 12.77 770.19

[0071] Obviously, the above embodiments are only examples for clearly illustrating, not limiting the embodiments. For ordinary skilled in the art, on the basis of the above description, other different forms of changes or variations can also be made. Here, all the embodiments need not and cannot be exhausted. The obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A method for self-supervised representation and life prediction transfer modeling of aero-engine under variable operating conditions, characterized in that, The method comprises the following steps: Collecting aero-engine operation data under a single common working condition to construct a pre-training data set; Each sample of the pre-training data set is aero-engine operation data of multiple operation periods, wherein the aero-engine operation data of part of the operation periods is provided with a mask, and each operation period of aero-engine operation data is composed of multiple sensor data and a current running time; A pre-training model is constructed, which comprises an input module, an aero-engine degradation feature encoder and a decoder based on a self-attention mechanism connected in sequence; wherein the input module comprises an embedding layer and a position encoder; the embedding layer is used to embed the input vector into a high-dimensional space to obtain a high-dimensional embedding vector; the position encoder is used to position encode the input high-dimensional embedding vector to obtain an embedded and position encoded vector; the aero-engine degradation feature encoder based on the self-attention mechanism comprises multiple multi-head self-attention feature extraction modules connected in sequence, each multi-head self-attention feature extraction module comprises at least multiple self-attention heads, a full connection layer and a normalization module, wherein each self-attention head is used to obtain an attention feature vector based on the input vector; the full connection layer is used to splice and fuse the attention feature vectors output by each self-attention head to output a multi-head self-attention feature vector; The normalization module is used to normalize the multi-head self-attention feature vector; The decoder is used to decode the feature vector output by the aero-engine degradation feature encoder based on the self-attention mechanism; Each sample in the pre-training data set is used as the input of the pre-training model, wherein the high-dimensional embedding vector corresponding to the unmasked data is encoded by the aero-engine degradation feature encoder based on the self-attention mechanism to obtain a multi-head self-attention output vector, the multi-head self-attention output vector and the high-dimensional embedding vector corresponding to the masked data are decoded by the decoder, and the training is performed with the objective of minimizing the loss between the output of the decoder and the original aero-engine operation data with the mask, after the training is completed, the decoder is discarded, and the trained input module, the aero-engine degradation feature encoder based on the self-attention mechanism and a life prediction neural network layer constitute a neural network model for efficient aero-engine life prediction under variable working conditions; Collecting aero-engine operation data under complex working conditions to construct an aero-engine remaining life prediction data set under complex working conditions; wherein each sample of the aero-engine remaining life prediction data set under complex working conditions is aero-engine operation data of multiple operation periods, and each operation period of aero-engine operation data is composed of multiple sensor data and a current running time; the label of each sample is the remaining life; Each sample in the aero-engine remaining life prediction data set under complex working conditions is used as the input of the neural network model for efficient aero-engine life prediction under variable working conditions, and the training is performed with the objective of minimizing the loss between the output of the model and the label to obtain the trained neural network model for efficient aero-engine life prediction under variable working conditions.

2. The method of claim 1, wherein, The aero-engine operation data under the single common working condition and the aero-engine operation data under the complex working condition are normalized data.

3. The method of claim 1, wherein, Each sample of the pre-training data set is obtained based on aero-engine operation data under a single common working condition by using sliding window sampling, and each sample of the aero-engine remaining life prediction data set under complex working conditions is obtained based on aero-engine operation data under complex working conditions by using sliding window sampling.

4. The method of claim 1, wherein, In each sample of the pre-training data set, the proportion of aero-engine operation data with a mask is 75%.

5. The method of claim 1, wherein, The normalization module comprises a first normalization layer, a residual feedforward neural network layer and a second normalization layer connected in sequence.

6. The method of claim 5, wherein, A nonlinear activation function GELU is used for activation after the fully connected feedforward neural network layer.

7. The method of claim 1, wherein, The aero-engine degradation feature encoder based on the self-attention mechanism comprises 2 layers of stacked multi-head self-attention feature extraction modules.

Images