Rolling bearing life prediction method based on secondary attention mechanism Transform coding layer and improved time convolution network

Abstract

The invention discloses a rolling bearing life prediction method based on a secondary attention mechanism Transform coding layer and an improved time convolution network, and the method comprises the steps: collecting the full-life vibration signals of a bearing, and constructing a multi-domain and multi-class normalized degradation feature set; then, global information of the service life process of the rolling bearing is obtained through a secondary attention mechanism Transform coding layer; and obtaining key local information of the whole life stage of the rolling bearing by constructing an improved time convolution network with one-dimensional convolution layers connected in series with different scales, and carrying out smoothing processing to obtain a final life prediction result of the rolling bearing. According to the method, the overall change trend of the health state of the bearing can be effectively captured in a self-iteration form of the attention mechanism, and local time features of different scales are extracted by the improved time convolutional network, so that relatively comprehensive rolling bearing health state time feature expression is obtained, and the precision of a rolling bearing life prediction result is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of bearing life prediction, and in particular relates to a rolling bearing life prediction method based on a secondary attention mechanism Transformer encoding layer and an improved temporal convolutional network. Background Art

[0002] As a key component of rotating machinery, rolling bearings are widely used in various industrial equipment. Due to the complex working conditions and harsh working environment of industrial sites, the performance degradation of rolling bearings will affect the production safety of equipment and the economic benefits of the factory, and even cause personnel safety problems. Therefore, accurately predicting the life of bearings and replacing warning bearings in time can not only reduce equipment maintenance downtime, but also improve equipment reliability and factory production efficiency. Research in this direction has become a key challenge and important topic in the engineering field.

[0003] The prediction methods for the remaining useful life (RUL) of rolling bearings are mainly divided into model-driven and data-driven methods. Model-driven methods are not universal due to the complexity of actual working conditions and high randomness of industrial sites, insufficient model generalization, and require certain expert experience. Data-driven methods use machine learning to process the complex nonlinear mapping relationship between bearing status signals and RUL, which is more flexible and has automated processing capabilities. However, the current classic deep learning networks, convolutional neural networks, recurrent neural networks and their variants are not suitable for long-sequence life prediction tasks such as rolling bearing fatigue due to their own limitations.

[0004] Transformer and TCN are popular models in the field of bearing life prediction in recent years. Transformer is good at handling the global performance of complex long-sequence tasks, while TCN can effectively capture the local dependencies of sequence data, and the gradient propagation is more stable. Combining the respective advantages of Transformer and TCN, the global and local features of rolling bearings can be obtained at the same time, thereby improving the prediction accuracy. However, the attention mechanism of the Transformer encoding layer is still lacking in mining the long-term dependencies of data, and the causal convolution and dilated convolution model parameters of TCN are too large, which affects the computational efficiency. In summary, the present invention adopts the secondary attention mechanism Transformer encoding layer and the improved temporal convolutional network, which can not only mine deep-level features, but also simplify the network structure and reduce the requirements for computing resources. Summary of the invention

[0005] In order to address the shortcomings of existing deep learning models in the field of bearing life prediction, the present invention proposes a rolling bearing life prediction method based on a quadratic attention mechanism Transformer encoding layer and an improved temporal convolutional network to achieve more efficient and accurate prediction of the remaining life of the bearing.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a rolling bearing life prediction method based on a secondary attention mechanism Transformer encoding layer and an improved temporal convolutional network, comprising the following steps:

[0007] S1. Collect multiple bearing life cycle vibration signals and divide them into training set and test set;

[0008] S2, extracting the time domain, frequency domain, time-frequency domain, and entropy features of the full life cycle vibration signal to form a multi-domain and multi-class feature set;

[0009] S3, normalizing the multi-domain and multi-class feature sets to obtain the rolling bearing degradation feature set;

[0010] S4. Build a Transformer encoding layer with a secondary attention mechanism, where the first layer of residual attention captures the basic characteristics of the rolling bearing state, and the second layer also uses the serial structure of residual attention to form a self-iteration of the attention mechanism to further extract deep state features;

[0011] S5, constructing an improved time convolution network by connecting two convolution layers of different scales in series, extracting local features of the vibration signal at different time scales, and fusing the local features of different scales, and then using the output of step S4 as the input of the improved time convolution network to obtain the initial prediction result of the rolling bearing life;

[0012] S6. Use the divided training set to train the above-built model, then use the test set to predict the life of the rolling bearing, and finally smooth the output results of the proposed model to obtain the final life prediction result of the rolling bearing.

[0013] The secondary attention mechanism Transformer encoding layer of S4 is specifically: using a two-layer serially connected residual self-attention mechanism, and adding a layer of Batchnorm before each self-attention layer to batch normalize the data to accelerate training and improve model stability. Each layer of self-attention layer is followed by a layer of activation function ELU for nonlinear activation.

[0014] The improved temporal convolutional network of S5 has two one-dimensional Same convolutional layers with convolution kernels of different scales, the Same convolutional layer is followed by a Layernorm layer, and the last Layernorm layer is followed by a layer of activation function ReLU for nonlinear activation.

[0015] The present invention has the following beneficial effects:

[0016] The bearing life prediction method provided by the present invention uses a two-layer serially connected residual self-attention mechanism to extract the global dependency of bearing vibration data, which can make the self-attention mechanism self-iterate to a certain extent, effectively improving the model's feature extraction ability under long time series; and innovatively combined with an improved time convolutional network, by stacking convolutional layers with different convolution kernel sizes to extract time features of different scales, thereby capturing features of different time ranges in sequence data, which makes the network more sensitive to local dependencies in sequence data. Combining the advantages of both, the model has powerful local feature extraction capabilities and global perception capabilities, and can predict bearing life more efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The flowchart for rolling bearing life prediction is as follows;

[0018] Figure 2 A schematic diagram of the platform for the data set used in the present invention;

[0019] Figure 3 The prediction result of Experiment 1 Bearing1_4 in the embodiment;

[0020] Figure 4 This is the prediction result of Experiment 2 Bearing2_4 in the embodiment. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example 1

[0023] refer to Figure 1 , a rolling bearing life prediction method based on the secondary attention mechanism Transformer encoding layer and the improved temporal convolutional network includes the following steps:

[0024] Step 1: Use the IEEE PHM 2012 public data set for testing. Two DYTRAN3035B micro accelerometers are installed on the PRONOSTIA experimental platform. They are radially installed on the outer ring of the bearing at 90° to each other. Figure 2As shown. The two accelerometers collect bearing vibration signals at a sampling frequency of 25.6kHz, sampling once every 10 seconds, and collecting 0.1 seconds of data each time. The test bench and sensor layout are shown in Figure 2 Each bearing subset contains vibration signals of two channels, horizontal and vertical. In this example, the horizontal vibration signal is selected for analysis.

[0025] The data of working condition 1 and working condition 2 are selected to verify the model. In working condition 1, Bearing1_3 and Bearing1_7 are selected as the training set, and Bearing1_4 is selected as the test set; in working condition 2, Bearing2_7 is selected as the training set, and Bearing2_4 is selected as the test set.

[0026] The specific division of training and testing bearings is shown in the following table:

[0027]

[0028] Step 2, extract 19 commonly used time domain features and 7 frequency domain features. The time-frequency domain features are the energy values ​​of 8 frequency bands generated by three-layer wavelet packet decomposition of the signal, and 8 entropy features (Shannon entropy, sample entropy, approximate entropy, permutation entropy, fuzzy entropy, Renyi entropy, Tsallis entropy, LogEnergy entropy). After feature extraction of the samples obtained from each sampling in the whole life stage, a multi-domain and multi-class feature set is obtained.

[0029] The time domain characteristics are as follows, where n is the total number of sampling points, x i is the signal sequence obtained by the i-th sampling.

[0030]

[0031] The frequency domain characteristics are as follows, where n is the total number of sampling points, f i is the signal frequency, X i is the spectrum amplitude.

[0032]

[0033] Step 3: Normalize the feature set and map the features to the interval [0,1] to eliminate the influence of dimension, so as to facilitate data analysis and processing, thereby improving the convergence speed of the deep learning model and avoiding the problem of gradient explosion in the prediction process;

[0034] The normalization formula is as follows:

[0035]

[0036] Where: x max is the maximum value of the feature, x min is the minimum value of the feature, x* is the normalized eigenvalue;

[0037] The remaining service life of the bearing is expressed as a percentage and is calculated as follows:

[0038]

[0039] Where: t i is the current moment, y i is the sample label at the current moment, T e The expiration time.

[0040] Step 4. Connect the secondary attention mechanism Transformer encoding layer, improved temporal convolutional network and regression layer in sequence to build a bearing life prediction model. Train the bearing life prediction model according to the training set to obtain the trained bearing life prediction model. Then input the test set to predict the bearing life. Finally, smooth the model output to obtain the bearing life prediction result.

[0041] Among them, the secondary attention mechanism Transformer encoding layer is specifically: using a two-layer serially connected residual self-attention mechanism, and adding a layer of Batchnorm before each self-attention layer to batch normalize the data to accelerate training and improve model stability. Each layer of self-attention layer is followed by a layer of activation function ELU for nonlinear activation.

[0042] It should be noted that the improved temporal convolutional network has two one-dimensional Same convolutional layers with convolution kernels of different scales. The Same convolutional layer is followed by a layer of Layernorm, and the last layer of Layernorm is followed by a layer of activation function ReLU for nonlinear activation.

[0043] The prediction results are as follows Figure 3 and Figure 4 As shown in the figure, it can be seen that the present invention can well predict the bearing life, the life prediction result is basically consistent with the actual life trend, the life fitting effect is very appropriate, and it well reflects the degradation trend of the bearing, and is basically consistent at the end of the bearing life. It accurately predicts the bearing life, ensures the bearing utilization rate, reduces the cost of use, and also avoids the risk caused by the failure of bearing failure to be warned in time. It can be seen that the present invention has the advantages of high prediction accuracy and strong generalization, and has significant practical application value.

[0044] The details of the model structure proposed by the present invention are as follows:

[0045] (1) Input layer

[0046] This layer is responsible for inputting the multi-domain feature set of vibration data extracted previously into the model. The shape of the input data is (BatchSize, feature length, feature dimension).

[0047] (2) Position encoding layer Positionembed

[0048] The multi-domain features of vibration signals are temporal, but the model tends to ignore the temporal order of data when processing. To solve this problem, the position encoding layer adds temporal position information to each time step to help the model better capture the temporal features. The position encoding weight matrix size is 42×512.

[0049] The purpose of position encoding is to add position information to each time step to preserve the time order. The calculation formula is as follows:

[0050]

[0051] Where: pos represents the position of the feature vector, i represents the dimension of the position encoding, d model Indicates the dimension size of the feature.

[0052] (3) Addition layer

[0053] The main function of the addition layer is to superimpose the input data with the position encoding information to incorporate the position information into the sequence modeling. One of the core features of the Transformer model is to remove the loop structure of the traditional RNN and instead process the sequence data through a global self-attention mechanism. However, the self-attention mechanism essentially performs a global calculation on the context information of the entire sequence and cannot directly identify the time or position information of each element in the sequence. Therefore, in order to ensure that the model can effectively capture the position features of each time step in the sequence, explicit position information must be injected to supplement it.

[0054] (4) Batch Normalization

[0055] This layer performs batch normalization on the input data to reduce the change in data distribution, thereby accelerating model convergence and improving generalization ability. Each batch of data is normalized through this layer. Batch Normalization (BN) standardizes the input data to reduce the offset of feature distribution and improve training stability. The calculation formula of BN is:

[0056]

[0057]

[0058] In the formula, x i is the input feature vector, μ is xi The mean of x i The standard deviation of , B refers to the size of the mini-batch, that is, the number of samples processed simultaneously in each training. i is a loop variable used to iterate each sample in the mini-batch in the formula. ε is a small value to prevent division by zero, usually with a default value of 0.001. γ and β are trainable parameters, representing the scaling factor and offset factor, respectively.

[0059] (5) Self-Attention Mechanism

[0060] The self-attention mechanism is the core module of this method, which is used to capture the global dependencies of time series data. In the Transformer architecture, the self-attention mechanism allows the model to establish direct connections between any two positions in the sequence, thereby capturing long-distance dependencies. A second layer of self-attention is added after the original self-attention layer. This allows the model to deepen the extraction of features. The first layer of self-attention captures basic features, and the second layer further extracts deeper features on this basis. The second layer of self-attention can refine the output of the first layer, strengthen key features, and suppress features of less importance. This can more effectively capture and model the complex dynamic behavior of bearings under different working conditions. The two-layer self-attention mechanism enables the model to better adapt to different data distributions and task requirements, thereby improving the generalization of the model so that it can cope with complex working conditions in actual industrial production.

[0061] The self-attention mechanism is a core module used to capture the global dependencies of time series data. Query, key, and value are the three core concepts of self-attention, which together form the basis of the self-attention mechanism. The weight matrices of the self-attention mechanism module are Query, Key, and Value, and the matrix size is 256×42. Query represents the query vector of the current time step. Key represents the feature representation of all time steps in the time series. Value represents the feature value of the corresponding time step. Self-attention obtains the attention weight by calculating the inner product of Query and Key, and then uses these weights to weighted sum the Value to generate a feature representation containing global information. The ability of the model to learn different features is further enhanced through the multi-head attention mechanism. The self-attention mechanism is a core module used to capture the global dependencies between time steps.

[0062] The input feature matrix is ​​X, and the Query, Key, and Value matrices are generated through three sets of linear transformations:

[0063] Q=XW Q

[0064] K=XWK

[0065] V=XW V

[0066] Where: W Q , W K , W V is the trainable parameter matrix.

[0067] The attention weight is calculated by the dot product of Query and Key:

[0068]

[0069] Where: d k is the dimension of Q and K matrices, preventing QK T The value is too large, which leads to a very small Softmax value and causes the gradient vanishing problem. T For similarity scores, Softmax converts similarity scores into weights.

[0070] In order to capture the attention of different features, a multi-head mechanism is used for parallel calculation:

[0071] Multihcad(Q,K,V)=Concat(head1,head2,...,head h )W O

[0072] head i =Attention(Q i , K i , V i )

[0073] Where: h is the number of attention heads, W O is the output transformation matrix, head i Represents the i-th head in the multi-head attention mechanism.

[0074] (6)ELU activation function ExponentialLinearUnit

[0075] The advantage of ELU is that it handles negative values ​​more smoothly, thereby reducing the gradient vanishing problem and enhancing the nonlinear fitting ability of the model. The calculation formula is as follows:

[0076]

[0077] Where x is the value that needs to be processed by the activation function, and α is a hyperparameter used to control the shape of the function in the negative interval. In practical applications, α is usually set to 1.

[0078] (7) 1D-Convolution

[0079] One of the best techniques for extracting local information is convolution, which extracts local abstract features within a small receptive field. This method uses two one-dimensional Same convolution layers with different convolution kernels to extract and fuse the features of bearing vibration signals at different scales. One-dimensional convolution is a common convolution operation. In this study, it is used to process the features extracted by the self-attention module, extract features of different time scales and fuse them, which enables the model to more effectively learn the complex nonlinear relationship between bearing vibration signals and their lifespan.

[0080] One-dimensional Same convolution can be calculated more efficiently when processing sequence data. The Same convolution layer can flexibly control the receptive field by adjusting the size and step size of the convolution kernel while maintaining a low amount of calculation. This design allows the network to be deepened without increasing the difficulty of calculation, thereby introducing more nonlinearities and improving the expressiveness of the model. Using two Same convolution layers can simplify the model structure and provide greater flexibility. The convolution kernel size and step size can be adjusted independently to adapt to different sequence lengths and feature extraction requirements. This simplified structure helps to reduce the complexity of the model, reduce the risk of overfitting, and improve the generalization ability of the model. By stacking two layers of Same convolution layers, the model can extract higher-level features layer by layer and fuse them, thereby improving the richness of feature expression. This makes the model more valuable in bearing life prediction tasks. Especially when processing complex vibration signal data, the model can more effectively capture and utilize patterns in the data, thereby improving the accuracy of prediction.

[0081] The core of one-dimensional convolution is to extract features on the sequence through a one-dimensional sliding window. The calculation formula is as follows:

[0082]

[0083] In the formula: f(τ) is the input signal, g(t) is the convolution kernel, * represents the convolution operation, τ is the integral variable used for displacement in the convolution operation, and (f*g)(t) is the result of the convolution operation, that is, the feature extracted by the convolution operation.

[0084] (8) Layer Normalization

[0085] As the number of neural network layers increases, the update of each layer's parameters may cause changes in the input data distribution of subsequent layers. This phenomenon is called internal covariate shift (ICS). ICS may make network training more difficult because each layer needs to continuously adapt to the new input distribution. Layer Normalization (LN) normalizes the input of each layer, reducing the impact of changes in input distribution between layers and effectively alleviating the impact of ICS. The calculation formula of LN is as follows:

[0086]

[0087] In the formula, x i is the input feature vector, μ is x i The mean of x i The standard deviation of , B refers to the size of the mini-batch, that is, the number of samples processed simultaneously in each training, i is a loop variable used to iterate each sample in the mini-batch in the formula, ε is a small value to prevent division by zero, usually the default value is 0.001. γ and β are trainable parameters, representing the scaling factor and offset factor respectively.

[0088] (9) ReLU activation function Rectified Linear Unit

[0089] ReLU is one of the most commonly used activation functions. ReLU introduces nonlinear features by setting the negative part of the input value to 0 and keeping the positive part unchanged. Therefore, the calculation of ReLU is simpler and can effectively alleviate the problem of gradient vanishing. Its calculation formula is as follows:

[0090] ReLU(x)=max(0,x)

[0091] In the formula, x is the value that needs to be processed by the activation function.

[0092] (10) Fully Connected

[0093] The fully connected layer integrates the features extracted previously, and then performs a linear transformation through the weight matrix. Finally, the regression layer directly maps the output of the fully connected layer to the predicted value of the bearing life.

[0094] This paper proposes a rolling bearing life prediction method based on the Transformer coding layer of the quadratic attention mechanism and the improved temporal convolutional network. By integrating the Transformer coding layer of the quadratic attention mechanism and the improved temporal convolutional network, the accuracy of bearing life prediction is improved. Finally, it is tested on the PHM2012 public dataset to further verify its reliability. The following conclusions are drawn:

[0095] (1) The underlying network extracts global features from the time series through the self-attention mechanism. Through the self-iteration of the self-attention mechanism, feature extraction continues based on the previous feature extraction result, which can effectively capture the long-term trend of the time series and provide a comprehensive temporal feature expression for subsequent models.

[0096] (2) The top-level network uses an improved temporal convolutional network to capture local temporal features, extract features of different temporal scales and fuse them, thus enhancing the feature extraction capability of the model.

[0097] (3) The serial structure of the upper and lower networks can further enhance the prediction ability of the model. The bottom network is responsible for extracting multi-level time series features, while the top network uses these features to model complex global dependencies and dynamic patterns. The two realize the division of labor and cooperation of time series data, retaining local details while capturing the overall change trend, thereby significantly improving the accuracy of bearing life prediction.

[0098] (4) Through experiments on the PHM2012 public data set, the life prediction results of the method proposed in the present invention are consistent with the actual life curve and are well matched at the end of the bearing life, ensuring the utilization rate of the bearing and reducing the cost of use, while also avoiding the risks caused by the failure to warn of bearing failure in a timely manner.

Claims

1. A rolling bearing life prediction method based on a secondary attention mechanism Transformer encoding layer and an improved temporal convolutional network, characterized in that The following steps are involved: S1. Collect multiple bearing life cycle vibration signals and divide them into training set and test set; S2, extracting the time domain, frequency domain, time-frequency domain, and entropy features of the full life cycle vibration signal to form a multi-domain and multi-class feature set; S3, normalizing the multi-domain and multi-class feature sets to obtain the rolling bearing degradation feature set; S4. Build a Transformer encoding layer with a secondary attention mechanism, where the first layer of residual attention captures the basic characteristics of the rolling bearing state, and the second layer also uses the serial structure of residual attention to form a self-iteration of the attention mechanism to further extract deep state features; S5, constructing an improved time convolution network by connecting two convolution layers of different scales in series, extracting local features of the vibration signal at different time scales, and fusing the local features of different scales, and then using the output of step S4 as the input of the improved time convolution network to obtain the initial prediction result of the rolling bearing life; S6. Use the divided training set to train the above-built model, then use the test set to predict the life of the rolling bearing, and finally smooth the output results of the proposed model to obtain the final life prediction result of the rolling bearing.

2. The rolling bearing life prediction method based on the secondary attention mechanism Transformer encoding layer and the improved temporal convolutional network according to claim 1 is characterized in that: The secondary attention mechanism Transformer encoding layer of S4 is specifically: using a two-layer serially connected residual self-attention mechanism, adding a layer of Batchnorm before each self-attention layer to batch normalize the data, and each self-attention layer is followed by a layer of activation function ELU for nonlinear activation.

3. The rolling bearing life prediction method based on the secondary attention mechanism Transformer encoding layer and the improved temporal convolutional network according to claim 1 is characterized in that: The improved temporal convolutional network of S5 has two one-dimensional Same convolutional layers with convolution kernels of different scales, the Same convolutional layer is followed by a Layernorm layer, and the last Layernorm layer is followed by a layer of activation function ReLU for nonlinear activation.

Images