Equipment Remaining Life Prediction Method Based on Multi-domain Feature Fusion of Double-layer Attention Network

Abstract

The present invention discloses a method for predicting the remaining life of equipment based on the fusion of multi-domain features of a double-layer attention network, including: 1. Integrating domain knowledge to extract the time domain, frequency domain and time-frequency domain statistical features of signal information; 2. Constructing a multi-scale feature extraction module, using a one-dimensional convolutional neural network with different sizes of convolution kernels to extract features from signal information of multiple scales; 3. Based on the scale attention module, weighted combination of the obtained multi-scale comprehensive information is performed; 4. Multi-scale features are input into a long short-term memory network to extract multi-scale deep features; 5. Using a domain attention module, different weights are assigned to multi-scale deep features and three statistical features, and different feature combinations are used in a targeted manner to predict the remaining life of equipment. The present invention can quickly and accurately predict the remaining life of equipment, thereby improving the reliability and safety of high-end equipment of engineering machinery and reducing the risk of failure events.

Description

Technical Field

[0001] The present invention belongs to the technical field of equipment failure prediction and health management, and specifically relates to a method for predicting the remaining service life of equipment based on multi-domain feature fusion of a double-layer attention network. Background Art

[0002] With the development of science and technology, modern equipment has become more precise, automatic and systematic. In the scenario of modern manufacturing, the working environment of equipment is often harsh and changeable. Long-term operation will lead to a decrease in the reliability of equipment, and the equipment's operating capacity will drop to a certain threshold, which may cause the equipment to suddenly stop, and in severe cases, may cause casualties. Therefore, the prediction of the remaining service life of equipment has always been relied on in academia and industry. Recently, since the data-driven remaining service life prediction method does not rely on any failure mechanism, it can directly extract the features of the observed data to identify the degradation trend and predict the future state, which has attracted more and more attention. Data-driven methods can be divided into traditional data-driven methods and deep learning methods. Traditional data-driven methods are usually divided into two steps: feature extraction and model construction, but there are problems such as shallow models and easy loss of connection between the two stages. Deep learning methods have powerful feature extraction capabilities and can automatically extract features from a large number of samples, reducing the dependence on signal processing technology and expert experience. At the same time, its model is relatively deep and can better capture the complex mapping between the original signal and the remaining service life. Therefore, deep learning can cope with the high-dimensional and nonlinear characteristics of equipment in industrial big data scenarios, and provides a new solution for the prediction of the remaining service life of equipment.

[0003] However, existing deep learning methods still have shortcomings in predicting remaining life:

[0004] (1) The advantages of traditional methods based on signal processing technology and expert experience are not considered. In current deep learning research, researchers mainly focus on how to improve the representation ability or prediction performance of deep learning methods. Most studies have developed several deep learning methods, such as improving the network structure to improve the estimation performance, without considering the advantages of utilizing traditional statistical features.

[0005] (2) The importance of information scale in feature extraction is not fully considered. In the current feature extraction step of remaining life prediction, researchers often process data from a single perspective, that is, the length of the sequence calculated each time is single. Using a time scale that is too short or too long for feature extraction may miss important information and fail to fully find the deep correlation between time points in the mechanical vibration signal data to predict the remaining life of the equipment. Summary of the invention

[0006] In order to solve the shortcomings of the above-mentioned prior art, the present invention proposes a method for predicting the remaining life of equipment based on multi-domain feature fusion of a double-layer attention network, in order to integrate statistical features and multi-scale information to construct a prediction model, so as to more effectively and accurately predict the remaining life of the equipment.

[0007] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0008] The invention discloses a method for predicting the remaining life of equipment based on the fusion of multi-domain features of a double-layer attention network, which comprises the following steps:

[0009] Step 1: Build a network training set:

[0010] The sensor installed on the equipment collects vibration signals of N sampling points in the sampling period T and forms a set of samples, so that the network training set is constructed by M sets of samples, which is denoted as T = {X 1 ,X 2 ,…,X m ,…,X M};X m represents the mth group of samples;

[0011] Step 2, statistical feature extraction:

[0012] Extract the mth group of samples X m The time domain characteristics of Represents the mth group of samples X m The nth 1 time domain features; N 1 Indicates the number of extracted time domain features;

[0013] Extract the mth group of samples X m The frequency domain characteristics of Represents the mth group of samples X m The nth 2 frequency domain features; N 2 Indicates the number of extracted frequency domain features;

[0014] Extract the mth group of samples X m The time-frequency domain characteristics of Represents the mth group of samples X m The nth 3 time-frequency domain features; N 3 Indicates the number of extracted time-frequency domain features;

[0015] Step 3: Construct a HAM-RUL network based on double-layer attention multi-domain feature fusion, including: multi-scale feature extraction module, scale attention module, long-term and short-term neural network, domain attention module, and fully connected layer;

[0016] Step 3.1: Construct a multi-scale feature extraction module, including K one-dimensional convolutional neural networks, and respectively 1 ,...,MscaleBlock k ,...,MscaleBlock K , where MscaleBlock k Represents sample X m The kth one-dimensional convolutional neural network module, k = 1, 2, ..., K; the kth one-dimensional convolutional neural network module MscaleBlock k There are L one-dimensional convolutional layers and L layers of downsampling layers in, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth one-dimensional convolutional layer, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth downsampling layer, l = 1, 2, ..., L;

[0017] When l=1, the mth group of samples X m Input the kth one-dimensional convolutional neural network module MscaleBlock k And then pass through the lth one-dimensional convolution layer Convolution processing and the lth downsampling layer After the maximum pooling process, the first layer of multi-scale features is output

[0018] When l>1, the l-1th layer multi-scale features After the lth one-dimensional convolution layer and the lth downsampling layer After processing, the output layer l multi-scale features Thus, the Lth downsampling layer Output the final multi-scale features

[0019] Thus, K one-dimensional convolutional neural network modules MscaleBlock 1 ,...,MscaleBlock k ,...,MscaleBlock K Output multi-scale features And as the final multi-scale feature output by the multi-scale feature extraction module, denoted as in, Represents the mth group of samples X mThe final multi-scale features output after passing through the K-th one-dimensional convolutional neural network module,

[0020] Step 3.2: Construct T scale attention modules, denoted as SawareBlock 1 ,...,SawareBlock t ,...,SawareBlock T , among which, SawareBlock t represents the t-th scale attention module, t = 1, 2, ..., T;

[0021] Step 3.2.1: Calculate the kth final multi-scale feature according to the time step Divide and obtain the multi-scale features after division in, represents the segment of the kth final multi-scale feature corresponding to the tth time step, t = 1, 2, ..., T;

[0022] Step 3.2.2: K one-dimensional convolutional neural network modules MscaleBlock 1 ,...,MscaleBlock k ,...,MscaleBlock K Multi-scale features of output After combining the fragments at the tth time step, the multi-scale combined features are obtained And input the t-th scale attention module SawareBlock t In this way, we can use formula (1) to get the attention scores of K scale features

[0023]

[0024] In formula (1), Indicates that the kth final multi-scale feature corresponds to the tth time step The score obtained, and is the parameter of the scale attention module corresponding to the t-th time step of the k-th final multi-scale feature; s() is a nonlinear activation function;

[0025] Step 3.2.3, attention scores for K scale features Normalize to get the final attention content score Indicates that the kth final multi-scale feature corresponds to the tth time step The final score obtained;

[0026] Step 3.2.4, the t-th scale attention module SawareBlock t Using formula (2), we can get the final output multi-scale features with attention:

[0027]

[0028] Step 3.3: Build T long short-term memory networks, denoted as LSTM cells 1 ,...,LSTMcell t ,...,LSTMcell T , where LSTM cell t represents the t-th LSTM network unit, t = 1, 2, ..., T;

[0029] The tth multi-scale feature with attention Input the tth long short-term memory network unit LSTM cell t Perform deep feature extraction and output the tth time series feature Thus, T long short-term memory network units output T time series features and serve as multi-scale depth features, denoted as in, Represents the mth group of samples X m The nth 4 Multi-scale deep features; N 4 Indicates the number of multi-scale deep features extracted; n 4 =t;N 4 =T;

[0030] Step 3.4: Take the time domain features, frequency domain features, time-frequency domain features and multi-scale depth features as domain features and input them into the domain attention module for processing:

[0031] Step 3.4.1: According to formula (3), the attention score of the d-th domain feature is obtained:

[0032]

[0033] In formula (3), is the d-th domain feature, It is the dth domain feature Corresponding domain attention parameters; d = 1, 2, 3, 4;

[0034] Step 3.4.2: Score the attention After normalization, we get the dth domain feature Final attention score

[0035] Step 3.4.3, according to formula (4), we get the dth domain feature The final feature Thus, the final features of the four domain features are concatenated as the final feature F output by the domain attention module. (m) ;

[0036]

[0037] Step 3.5: The fully connected layer uses formula (5) to calculate the mth group of samples X m The remaining life of the mth group of samples X is predicted m Remaining life

[0038]

[0039] In formula (5), Represents the final feature F (m) The xth final feature in , Represents the xth final feature in the fully connected layer The corresponding parameters are: represents the xth final feature Corresponding to the bias, φ() represents the activation function; X is the number of final features;

[0040] Step 3.6, train the HAM-RUL network and calculate the loss function L(x) as shown in formula (6). When the number of training iterations reaches the set number, the training stops, thereby obtaining the optimal reconstruction model for predicting the remaining life of the equipment:

[0041]

[0042] In formula (6), RUL (m) Represents the mth group of samples X m The corresponding actual remaining life.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention adds statistical features to the end-to-end network, which can help reduce the negative impact caused by the disconnection between feature extraction and prediction in traditional remaining life prediction methods. At the same time, while utilizing deep learning, it adds signal science knowledge and expert experience as a supplement to remaining life prediction, thereby making use of richer information for accurate equipment remaining life prediction.

[0045] 2. This paper proposes an end-to-end attention-based method by using statistical features as a supplement to the dual-layer attention. First, domain-aware attention is used to fuse time domain, frequency domain, time-frequency domain features, and deep features. In addition, a scale attention layer is added to the long short-term memory network to extract deep features from weighted multi-scale information, thereby better characterizing domain knowledge and avoiding information omission caused by a single scale.

[0046] 3. The present invention proposes a multi-scale deep feature extraction method to collect degradation information of different scales and comprehensively utilize them through a multi-scale one-dimensional convolutional neural network layer containing kernel filters of different sizes to obtain more comprehensive signal information for subsequent deep feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the process of the present invention;

[0048] Figure 2 This is a schematic diagram of the scale attention of the present invention; DETAILED DESCRIPTION

[0049] In this embodiment, a method for predicting the remaining life of equipment based on multi-domain feature fusion of a double-layer attention network is described as follows: Figure 1 As shown, including:

[0050] Step 1: Build a network training set:

[0051] The sensor installed on the equipment collects vibration signals of N sampling points in the sampling period T and forms a set of samples, so that the network training set is constructed by M sets of samples, which is denoted as T = {X 1 ,X 2 ,...,X m ,...,X M};X m Represents the mth group of samples; the training set is divided into M groups of samples, denoted by T = {X 1 ,X 2 ,...,X m ,...,X M};X m represents the mth group of samples;

[0052] In this embodiment, bearings are taken as an example, and the bearing accelerated life experimental data provided by the IEEE PHM2012 Challenge is used to verify the method; the experimental data comes from the PRONOSTIA test bench, which is specially designed for the research of bearing fault detection, diagnosis and prediction methods; NSK 6804DD ball bearings are used in this data set; the data set provides real experimental data, describing the degradation process of ball bearings throughout their life, and when the amplitude of the vibration signal exceeds 20g, the life of the bearing is over; the sampling frequency is 25.6kHz, and a total of 2560 samples are recorded in each 10-second monitoring cycle, which takes 0.1s; the monitoring interval is 9.9 seconds (10-0.1 seconds); the working conditions used during the test are 1800RPM and 4000N, and only the horizontal axis vibration signal data is used; a training set is constructed for the vibration signal in step 1; the data of 6 bearings are selected as the training set, and the remaining data are used as the test set; for example, if the data from bearing 1 to bearing 6 are selected as the training set, the data from bearing 7 is used as the test set;

[0053] Step 2, statistical feature extraction:

[0054] Extract the mth group of samples X m The time domain characteristics of Represents the mth group of samples X m The nth 1 time domain features; N 1 Indicates the number of extracted time domain features; the time domain features extracted in this embodiment are shown in Table 1;

[0055] Table 1 Time domain features extracted in this embodiment

[0056]

[0057] Extract the mth group of samples X m The frequency domain characteristics of Represents the mth group of samples X m The nth 2 frequency domain features; N 2 Indicates the number of extracted frequency domain features; the frequency domain features extracted in this embodiment are shown in Table 2;

[0058] Table 2 Frequency domain features extracted in this embodiment

[0059]

[0060] Extract the mth group of samples X m The time-frequency domain characteristics of Represents the mth group of samples X m The nth 3 time-frequency domain features; N 3Indicates the number of extracted time-frequency domain features; in this embodiment, wavelet packet transform and empirical mode decomposition are used to extract time-frequency domain features;

[0061] Step 3: construct a network based on double-layer attention multi-domain feature fusion (HAM-RUL), including: multi-scale feature extraction module, scale attention module, long-term and short-term neural network, domain attention module, and fully connected layer;

[0062] Step 3.1: Construct a multi-scale feature extraction module, including K one-dimensional convolutional neural networks, and respectively 1 ,...,MscaleBlock k ,...,MscaleBlock K , where MscaleBlock k Represents sample X m The kth one-dimensional convolutional neural network module, k = 1, 2, ..., K, in this embodiment, K = 5; the kth one-dimensional convolutional neural network module MscaleBlock k There are L one-dimensional convolutional layers and L layers of downsampling layers in, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth one-dimensional convolutional layer, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth downsampling layer in the middle, l = 1, 2, ..., L, in this embodiment, L = 2; for the kth one-dimensional convolutional neural network module The convolution kernel size is n=1,2,...,K; In this embodiment, The convolution step size is 1, and the padding is zero;

[0063] When l = 1, the mth group of samples X m Input the kth one-dimensional convolutional neural network module MscaleBlock k And then pass through the first one-dimensional convolution layer Convolution processing and the first downsampling layer After the maximum pooling process, the first layer of multi-scale features is output

[0064] When l>1, the l-1 layer multi-scale features After the lth one-dimensional convolution layer and the lth downsampling layer After processing, the output layer l multi-scale features Thus, the Lth downsampling layer Output the final multi-scale features

[0065] Thus, K one-dimensional convolutional neural network modules MscaleBlock 1 ,...,MscaleBlock k ,...,MscaleBlock K Output multi-scale features And as the output of the multi-scale feature extraction module in, Represents sample X m The final multi-scale features output after passing through the K-th one-dimensional convolutional neural network module,

[0066] Step 3.2: Construct T scale attention modules, denoted as SawareBlock 1 ,...,SawareBlock t ,...,SawareBlock T , among which, SawareBlock t represents the t-th scale attention module, t = 1, 2, ..., T, and the specific structure is as follows Figure 2 As shown; in this embodiment, T=10;

[0067] Step 3.2.1: Calculate the kth final multi-scale feature according to the time step Divide it into in, represents the segment of the kth final multi-scale feature corresponding to the tth time step, t = 1, 2, ..., T;

[0068] Step 3.2.2: K one-dimensional convolutional neural network modules MscaleBlock 1 ,...,MscaleBlock k ,...,MscaleBlock K Multi-scale features of output The fragments at the tth time step are combined to obtain the multi-scale combined features And input the t-th scale attention module SawareBlock t In this way, we can use formula (1) to get the attention scores of K scale features

[0069]

[0070] In formula (1), Indicates that the kth final multi-scale feature corresponds to the tth time step The score obtained, and is the parameter of the scale attention module corresponding to the t-th time step of the k-th final multi-scale feature; s() is a nonlinear activation function;

[0071] Step 3.2.3, attention scores for K scale features Normalize to get the final attention content score Indicates that the kth final multi-scale feature corresponds to the tth time step The final score obtained;

[0072] Step 3.2.4, the t-th scale attention module SawareBlock t Using formula (2), we can get the final output multi-scale features with attention:

[0073]

[0074] Step 3.3: Build a long short-term memory network with a total of T long short-term memory network units, denoted as LSTM cells 1 ,...,LSTMcell t ,...,LSTMcell T , where LSTM cell t represents the t-th LSTM network unit, t = 1, 2, ..., T;

[0075] The tth multi-scale feature with attention Input the tth long short-term memory network unit LSTM cell t Perform deep feature extraction and output the tth time series feature Thus, T long short-term memory network units output T time series features and serve as multi-scale depth features, denoted as in, Represents the mth group of samples X m The nth 4 Multi-scale deep features; N 4 Indicates the number of multi-scale deep features extracted; n 4 =t;N 4 =T;

[0076] Step 3.4: Take the time domain features, frequency domain features, time-frequency domain features and multi-scale depth features as domain features and input them into the domain attention module for processing:

[0077] Step 3.4.1: According to formula (3), the attention score of the d-th domain feature is obtained:

[0078]

[0079] In formula (3), is the parameter of the domain attention corresponding to the d-th domain feature; is the d-th domain feature, d = 1, 2, 3, 4;

[0080] Step 3.4.2: Score the attention After normalization, the final attention score of the d-th domain feature is obtained

[0081] Step 3.4.3, according to formula (4), the final feature of the dth domain feature is obtained Thus, the final feature F output by the domain attention module is concatenated from the four domain features. (m) ;

[0082]

[0083] Step 3.5: The fully connected layer uses formula (5) to calculate the mth group of samples X m The remaining life of m Remaining life

[0084]

[0085] In formula (5), Features in the fully connected layer The corresponding parameters are: Represents the final feature F (m) The xth final feature in , Representation characteristics Corresponding to the bias, φ() represents the activation function; X is the number of final features;

[0086] Step 3.6, train the HAM-RUL network and calculate the loss function L(x) shown in formula (6). When the number of training iterations reaches the set number, the training stops, thereby obtaining the optimal reconstruction model for realizing the prediction of the remaining life of the equipment. In this embodiment, the ReLu optimizer is used, and each training iteration is 100 times.

[0087]

[0088] In formula (6), RUL (m) Represents the mth group of samples X m The corresponding actual remaining life.

Claims

1. A method for predicting the remaining life of equipment based on multi-domain feature fusion of a double-layer attention network, characterized by: The steps include: Step 1: Build a network training set: The sensor installed on the equipment collects vibration signals of N sampling points in the sampling period T and forms a group of samples, so that the network training set is constructed by M groups of samples, which is recorded as T = {X1, X2, ..., X m ,...,X M };X m represents the mth group of samples; Step 2, statistical feature extraction: Extract the mth group of samples X m The time domain characteristics of Represents the mth group of samples X m The n1th time domain feature in ; N1 represents the number of extracted time domain features; Extract the mth group of samples X m The frequency domain characteristics of Represents the mth group of samples X m The n2th frequency domain feature in; N2 represents the number of frequency domain features extracted; Extract the mth group of samples X m The time-frequency domain characteristics of Represents the mth group of samples X m The n3th time-frequency domain feature in ; N3 represents the number of extracted time-frequency domain features; Step 3: Construct a HAM-RUL network based on double-layer attention multi-domain feature fusion, including: multi-scale feature extraction module, scale attention module, long-term and short-term neural network, domain attention module, and fully connected layer; Step 3.1, construct a multi-scale feature extraction module, including K one-dimensional convolutional neural networks, and respectively MscaleBlock1,...,MscaleBlock k ,...,MscaleBlock K , where MscaleBlock k Represents sample X m The kth one-dimensional convolutional neural network module, k = 1, 2, ..., K; the kth one-dimensional convolutional neural network module MscaleBlock k There are L one-dimensional convolutional layers and L layers of downsampling layers in, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth one-dimensional convolutional layer, Represents the kth one-dimensional convolutional neural network module MscaleBlock k The lth downsampling layer, l = 1, 2, ..., L; When l=1, the mth group of samples X m Input the kth one-dimensional convolutional neural network module MscaleBlock k And then pass through the lth one-dimensional convolution layer Convolution processing and the lth downsampling layer After the maximum pooling process, the first layer of multi-scale features is output When l>1, the l-1th layer multi-scale features After the lth one-dimensional convolution layer and the lth downsampling layer After processing, the output layer l multi-scale features Thus, the Lth downsampling layer Output the final multi-scale features Thus, K one-dimensional convolutional neural network modules MscaleBlock1,...,MscaleBlock k ,...,MscaleBlock K Output multi-scale features And as the final multi-scale feature output by the multi-scale feature extraction module, denoted as in, Represents the mth group of samples X m The final multi-scale features output after passing through the K-th one-dimensional convolutional neural network module, Step 3.2: Construct T scale attention modules, denoted as SawareBlock1,...,SawareBlock t ,...,SawareBlock T , among which, SawareBlock t represents the t-th scale attention module, t = 1, 2, ..., T; Step 3.2.1: Calculate the kth final multi-scale feature according to the time step Divide and obtain the multi-scale features after division in, represents the segment of the kth final multi-scale feature corresponding to the tth time step, t = 1, 2, ..., T; Step 3.2.2, K one-dimensional convolutional neural network modules MscaleBlock1,...,MscaleBlock k ,...,MscaleBlock K Multi-scale features of output After combining the fragments at the tth time step, the multi-scale combined features are obtained And input the t-th scale attention module SawareBlock t In this way, we can use formula (1) to get the attention scores of K scale features In formula (1), Indicates that the kth final multi-scale feature corresponds to the tth time step The score obtained, is the parameter of the scale attention module corresponding to the t-th time step of the k-th final multi-scale feature; s() is a nonlinear activation function; Step 3.2.3, attention scores for K scale features Normalize to get the final attention content score Indicates that the kth final multi-scale feature corresponds to the tth time step The final score obtained; Step 3.2.4, the t-th scale attention module SawareBlock t Using formula (2), we can get the final output multi-scale features with attention: Step 3.3, construct T long short-term memory networks, denoted as LSTMcell1,...,LSTMcell t ,...,LSTMcell T , where LSTM cell t represents the t-th LSTM network unit, t = 1, 2, ..., T; The tth multi-scale feature with attention Input the tth long short-term memory network unit LSTM cell t Perform deep feature extraction and output the tth time series feature Thus, T long short-term memory network units output T time series features and serve as multi-scale depth features, denoted as in, Represents the mth group of samples X m The n4th multi-scale depth feature in; N4 represents the number of multi-scale depth features extracted; n4 = t; N4 = T; Step 3.4: Take the time domain features, frequency domain features, time-frequency domain features and multi-scale depth features as domain features and input them into the domain attention module for processing: Step 3.4.1: According to formula (3), the attention score of the dth domain feature is obtained: In formula (3), is the d-th domain feature, It is the dth domain feature Corresponding domain attention parameters; d = 1, 2, 3, 4; Step 3.4.2: Score the attention After normalization, we get the dth domain feature Final attention score Step 3.4.3, according to formula (4), we get the dth domain feature The final feature Thus, the final features of the four domain features are concatenated as the final feature F output by the domain attention module. (m) ; Step 3.5: The fully connected layer uses formula (5) to calculate the mth group of samples X m The remaining life of the mth group of samples X is predicted m Remaining life In formula (5), Represents the final feature F (m) The xth final feature in , Represents the xth final feature in the fully connected layer The corresponding parameters are: represents the xth final feature Corresponding to the bias, φ() represents the activation function; X is the number of final features; Step 3.6, train the HAM-RUL network and calculate the loss function L(x) as shown in formula (6). When the number of training iterations reaches the set number, the training stops, thereby obtaining the optimal reconstruction model for predicting the remaining life of the equipment: In formula (6), RUL (m) Represents the mth group of samples X m The corresponding actual remaining life.

Images