Remaining service life prediction method based on multi-scale attention mechanism and graph neural network

Abstract

The invention discloses a residual service life prediction method based on a multi-scale attention mechanism and a graph neural network. Firstly, input sensor signals are screened and preprocessed according to a signal change trend; then constructing a sensor network and a graph data set for the preprocessed signals based on cosine similarity; after the sample is constructed, inputting the sample into the model for training, and setting the maximum number of updating times to update the model so as to obtain an optimal prediction model; and finally, inputting the data into the model to predict the remaining service life. According to the method, a multi-layer graph attention network is constructed in residual service life prediction to capture spatial dependency characteristics among sensor data, and time dependency modeling is performed in combination with a bidirectional long-short-term memory network. In addition, a multi-scale attention mechanism is introduced to focus local details of different scales to capture multi-level information. According to the method, the RUL prediction precision can be effectively improved, and a maintenance decision and equipment maintenance can be made in advance.

Description

Technical Field

[0001] The present invention belongs to the field of aircraft engine remaining service life prediction, and in particular relates to a remaining service life prediction method based on a multi-scale attention mechanism and a graph neural network. Background Art

[0002] Prognostics and Health Management (PHM) uses the remaining useful life (RUL) information of machinery to formulate maintenance strategies, ensure the safe and stable operation of mechanical equipment and reduce maintenance costs. Traditional RUL prediction methods focus on analyzing single sensor signals, but a single sensor can only transmit partial information about the complex mechanical degradation process. Multi-sensor data can provide more degradation information, and using multi-sensor data for RUL prediction can produce more accurate and reliable results. Since multi-sensor data has complex spatiotemporal dependencies, how to establish a RUL prediction model to effectively fuse multi-sensor information and accurately predict the remaining life is crucial for making operation plans and maintenance decisions.

[0003] At present, the remaining useful life prediction methods are mainly divided into two categories: model-driven and data-driven. The model-driven method predicts RUL by establishing a mathematical model or a physical model for the mechanical system. This method is too dependent on the prior knowledge of the equipment degradation process, and in practical applications, due to the complexity of mechanical equipment, it is difficult to establish an accurate mathematical model or a physical model to describe the degradation process. The data-driven method is based on the health status data of mechanical equipment to establish a prediction model that reflects the inherent regularity between health status data and equipment degradation, without requiring specific knowledge of the degradation mechanism.

[0004] With the development and progress of computer science and deep learning technology, the remaining useful life prediction method based on deep learning has gradually become a mainstream research method. However, RUL prediction still faces many challenges: (1) Existing life prediction methods based on traditional neural networks, such as convolutional neural networks and recurrent neural networks, can capture the dynamic dependencies of multi-sensor data in the time dimension, but ignore the spatial dependencies between different sensors. (2) Although methods based on graph neural networks can model spatial dependencies, most existing methods lack consideration of information fusion between different scales. In order to solve the above problems, a remaining useful life prediction method based on multi-scale attention mechanism and graph neural network is proposed. Summary of the invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a method for predicting the remaining useful life based on a multi-scale attention mechanism and a graph neural network. The proposed prediction model is mainly composed of three parts: a multi-layer graph attention network (Multi-GAT), a multi-scale pooling attention mechanism (MSPA) and a bidirectional long short-term memory network (BiLSTM). Among them, the multi-layer graph attention network is used for spatial dependency modeling, and BiLSTM is used for temporal dependency modeling. On this basis, a multi-scale attention mechanism is introduced to focus on local details of different scales to capture multi-level information.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a method for predicting remaining useful life based on a multi-scale attention mechanism and a graph neural network, comprising the following steps:

[0007] Step 1: The Commercial Modular Aero-Propulsion System Simulation (CMAPSS) dataset released by NASA contains multiple sets of sensor data. The sensor data that always shows an upward or downward trend is selected. Different sensor data have different dimensions, so the data is standardized to unify the dimensions and dimensions. The sliding window technology is used to divide the sensor data into training data of the same length. The piecewise linear degradation model is used to mark the label of the engine at each moment, that is, the time from the current moment to the engine failure is taken as the remaining life at that moment. The model marks the remaining life exceeding 130 as 130, and the label below 130 remains unchanged.

[0008] Step 2: Consider each sensor data as a node and the relationship between each sensor data as an edge to build a sensor network. Use cosine similarity to calculate the nonlinear relationship between sensors, and determine the adjacency matrix between different sensor nodes based on its size; generate the corresponding graph data set based on the obtained adjacency matrix.

[0009] Sensor adjacency matrix calculation method based on cosine similarity:

[0010]

[0011] In the formula, x i and x jRepresent different sensor data respectively. The cosine similarity between different sensors is used as the coefficient of its adjacency matrix to represent the relationship between different sensor data, so as to obtain the graph structure of different data sets. The formula is as follows:

[0012]

[0013] The threshold is set to 0 based on prior knowledge. input ,A), where X input Enter data for the sensor.

[0014] Step 3: Input the generated graph dataset into the multi-layer graph attention network to extract the spatial dependency features of the aircraft engine graph dataset. In the multi-layer graph attention network, each layer contains a graph attention layer and a graph pooling layer. By stacking three such layers and introducing jump connections between each layer, a complete multi-layer graph attention network is constructed.

[0015] Step 4: The data processed by the multi-layer graph attention network enters the multi-scale attention layer to focus on local details of different scales to capture multi-level information. The multi-scale attention layer contains three scales, which pool the input data to lengths of 10, 20, and 30 respectively. The calculation process is shown in the following formula:

[0016]

[0017] H i =ReLU(Z i )

[0018] Y i =W2·H i

[0019] A i =σ(Y i )

[0020] Among them, X is the input feature map; s i is the scale; W1 and W2 are the weight matrices of the two convolutional layers, which are used to reduce and restore the feature dimensions respectively; A i is the attention score; σ is the Sigmoid activation function.

[0021] All scales are adjusted to the same length using linear interpolation to integrate the attention scores from different scales, and these features are further fused through the convolutional layer. The following formula describes this process:

[0022] A i '=Interpolate(A i ,T max ,mode='linear')

[0023] A=Concatenate[A1',A'2,…,A' n ]

[0024] O=W final ·A

[0025] Among them, Interpolate represents the linear interpolation operation; Concatenate represents the connection operation; W final is the weight matrix of the last convolutional layer; O is the final attention score. The attention score is weighted and applied to the input to generate the final output.

[0026] Step 5: The degradation information weighted by the multi-scale attention layer is input into the BiLSTM layer for time-dependency modeling, and two fully connected layers are used to map the output information of the BiLSTM layer to output the final RUL prediction result. To prevent overfitting, a dropout layer is inserted between the two fully connected layers, and the dropout value is set to 0.5.

[0027] The advantages of the present invention are as follows: the present invention proposes a method for predicting the remaining useful life based on a multi-scale attention mechanism and a graph neural network, which can simultaneously model the temporal and spatial dependencies of multi-sensor data. The proposed multi-layer graph attention network can effectively capture spatial features and obtain graph representation by stacking graph attention layers and introducing jump connections; the proposed multi-scale attention mechanism can extract multi-level information in the input sequence by focusing on local details of different scales.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Attached is the abstract.

[0030] Figure 2 It is a flowchart of the present invention.

[0031] Figure 3 Sensor network and adjacency matrix diagram for the FD001 subset of the CMAPSS dataset.

[0032] Figure 4 Schematic diagram of the multi-layer graph attention network structure.

[0033] Figure 5 Schematic diagram of the multi-scale attention mechanism structure.

[0034] Figure 6 This is a comparison curve of the true and predicted values ​​of RUL in the FD001 subset of the CMAPSS dataset. DETAILED DESCRIPTION

[0035] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0036] Figure 2 This is the overall flow chart of the present invention, and each step is described in detail below:

[0037] This example uses the NASA public CMAPSS aircraft engine dataset to verify the effectiveness of the proposed method. The CMAPSS dataset contains four sub-datasets: FD001, FD002, FD003, and FD004. This example uses the FD001 dataset for model training and testing. Its training set contains 100 engine data and the test set contains 100 engine data.

[0038] Step 1: Preprocess the original data.

[0039] 101: The FD001 data set contains 21 sensor data. By plotting the change curves of each sensor data over the operation cycle, it can be found that the data of sensors S2, S3, S4, S7, S8, S9, S11, S12, S13, S14, S15, S17, S20, and S21 show a continuous upward or downward trend, so these sensor data are selected to obtain richer degradation information.

[0040] 102: Standardize the sensor data to unify the dimensions and dimensions. The calculation formula is:

[0041]

[0042] Where x' is the normalized sensor data value, x is the raw sensor data value, μ is the mean value of the sensor data, and σ is the standard deviation of the sensor data.

[0043] 103: In the early stages of operation, the aircraft engine is considered to be in a healthy state, and the RUL in this stage is set to a fixed value of 130. Once a failure occurs, the aircraft engine can be considered to have entered a linear degradation stage, and the RUL decreases linearly to 0 in this stage.

[0044] Step 2: Construct the preprocessed data into a graph dataset. Consider each sensor data as a node and the relationship between each sensor data as an edge to construct a sensor network. Use cosine similarity to calculate the nonlinear relationship between sensors, and determine the adjacency matrix between different sensor nodes based on its size; generate the corresponding graph dataset based on the obtained adjacency matrix. The sensor network and adjacency matrix of the FD001 dataset are shown in Figure 2. Figure 3 shown.

[0045] Step 3: Use multi-layer graph attention network, multi-scale attention mechanism and BiLSTM to build a prediction model. In the multi-layer graph attention network, each layer contains a graph attention layer and a graph pooling layer. By stacking three such layers, jump connections are introduced between each layer to build a complete multi-layer graph attention network. Use the multi-layer graph attention network to extract spatial dependency features. Its structure diagram is as follows Figure 4 As shown. The data processed by the multi-layer graph attention network enters the multi-scale attention layer to focus on local details of different scales to capture multi-level information. The multi-scale attention layer contains three scales, which pool the input data to lengths of 10, 20, and 30 respectively. The calculation process is shown in the following formula:

[0046]

[0047] H i =ReLU(Z i )

[0048] Y i =W2·H i

[0049] A i =σ(Y i )

[0050] Among them, X is the input feature map; s i is the scale; W1 and W2 are the weight matrices of the two convolutional layers, which are used to reduce and restore the feature dimensions respectively; A i is the attention score; σ is the Sigmoid activation function.

[0051] All scales are adjusted to the same length using linear interpolation to integrate the attention scores from different scales, and these features are further fused through the convolutional layer. The following formula describes this process:

[0052] A i '=Interpolate(A i ,T max ,mode='linear')

[0053] A=Concatenate[A1',A'2,...,A' n ]

[0054] O=W final ·A

[0055] Among them, Interpolate represents the linear interpolation operation; Concatenate represents the connection operation; W finalis the weight matrix of the last convolutional layer; O is the final attention score. The attention score is weighted and applied to the input to generate the final output. The multi-scale attention mechanism structure is shown in the figure Figure 5 shown.

[0056] The degradation information weighted by the multi-scale attention layer is input into the BiLSTM layer for time dependency modeling, and two fully connected layers are used to map the output information of the BiLSTM layer to output the final RUL prediction result. To prevent overfitting, a dropout layer is inserted between the two fully connected layers, and the dropout value is set to 0.5.

[0057] Step 4: Use the model to predict the remaining useful life.

[0058] 401: The loss function is the mean square error function (MSE), and the calculation formula is:

[0059]

[0060] In the formula, n represents the number of samples, Y i represents the true value of the i-th sample, Y i pre Represents the predicted value of the i-th sample.

[0061] 402: In this example, the optimizer selects the Adam optimizer, uses the learning rate decay strategy, sets the initial learning rate to 0.001, and decays to 0.1 of the initial learning rate every 10 epochs. The batch size is set to 256.

[0062] 403: The evaluation indicators of this example are root mean square error (RMSE) and score function (Score Function), and the calculation formula is:

[0063]

[0064] In the formula, n represents the number of samples, Y i pre represents the predicted value of the i-th sample, Y i Represents the true value of the i-th sample.

[0065] 404: Input the sample to be tested into the trained model and calculate the predicted value of the remaining service life of the aircraft engine at time T+1

[0066] Figure 6The figure shows the comparison curve between the predicted value and the true value. It can be seen from the figure that the model performs well in the test set. The results of this model are compared with the results of other models to verify the effectiveness and accuracy of the proposed model. The comparison results further show that the model can effectively predict the remaining service life and improve the prediction accuracy. The comparison results are shown in Table 1.

[0067] Table 1 Comparison of various methods for FD001

[0068]

[0069]

Claims

1. A method for predicting remaining useful life based on a multi-scale attention mechanism and a graph neural network, comprising the following steps: Step 1: Preprocess the raw data. The FD001 data set contains 21 sensor data. By plotting the change curves of each sensor data over the operation cycle, it can be found that the data of sensors S2, S3, S4, S7, S8, S9, S11, S12, S13, S14, S15, S17, S20, and S21 show a continuous upward or downward trend, so these sensor data are selected to obtain richer degradation information. The sensor data is normalized by Z-SCORE to unify the dimensions and dimensions. The calculation formula is: Where x' is the normalized sensor data value, x is the raw sensor data value, μ is the average value of the sensor data, and σ is the standard deviation of the sensor data. In the early stages of operation, the aircraft engine is considered to be in a healthy state, and the RUL in this stage is set to a fixed value of 130. Once a fault occurs, the aircraft engine can be considered to have entered a linear degradation stage, and the RUL decreases linearly to 0 in this stage. Step 2: Construct the preprocessed data into a graph dataset. Consider each sensor data as a node and the relationship between each sensor data as an edge to construct a sensor network. Use cosine similarity to calculate the nonlinear relationship between sensors, and determine the adjacency matrix between different sensor nodes based on its size; generate the corresponding graph dataset based on the obtained adjacency matrix. The sensor network and adjacency matrix of the FD001 dataset are shown in Figure 3. Step 3: Use multi-layer graph attention network, multi-scale attention mechanism and BiLSTM to build a prediction model. In the multi-layer graph attention network, each layer contains a graph attention layer and a graph pooling layer. By stacking three such layers, jump connections are introduced between each layer to build a complete multi-layer graph attention network. Use the multi-layer graph attention network to extract spatial dependency features. The data processed by the multi-layer graph attention network enters the multi-scale attention layer to focus on local details of different scales to capture multi-level information. The calculation process is shown in the following formula: H i =ReLU(Z i ) AND i =W2·H i A i =σ(Y i ) Among them, X is the input feature map; s i is the scale; W1 and W2 are the weight matrices of the two convolutional layers, which are used to reduce and restore the feature dimensions respectively; A i is the attention score; σ is the Sigmoid activation function. All scales are adjusted to the same length using linear interpolation to integrate the attention scores from different scales, and these features are further fused through the convolutional layer. The following formula describes this process: A i '=Interpolate(A i ,T max ,mode='linear') A=Concatenate[A′1,A'2,…,A' n ] O=W final A Among them, Interpolate represents the linear interpolation operation; Concatenate represents the connection operation; W final is the weight matrix of the last convolutional layer; O is the final attention score. The attention score is weighted and applied to the input to generate the final output. The degradation information weighted by the multi-scale attention layer is input into the BiLSTM layer for time dependency modeling, and two fully connected layers are used to map the output information of the BiLSTM layer to output the final RUL prediction result. To prevent overfitting, a dropout layer is inserted between the two fully connected layers, and the dropout value is set to 0.

5. Step 4: Use the model to predict the remaining useful life. The loss function is the mean square error function (MSE), and the calculation formula is: In the formula, n represents the number of samples, Y i represents the true value of the i-th sample, Y i pre Indicates the predicted value of the i-th sample. In this example, the optimizer selects the Adam optimizer, uses the learning rate decay strategy, sets the initial learning rate to 0.001, decays to 0.1 of the initial learning rate every 10 epochs, and sets the batch size to 256. The evaluation indicators of this example are the root mean square error (RMSE) and the score function, and the calculation formula is: In the formula, n represents the number of samples, Y i pre represents the predicted value of the i-th sample, Y i Indicates the true value of the i-th sample. Input the sample to be tested into the trained model and calculate the predicted value of the remaining service life of the aircraft engine at time T+1.

Images