Bearing life prediction method based on SE attention and improved Transform coding layer

Abstract

The invention discloses a bearing life prediction method based on SE attention and an improved Transform coding layer, and the method comprises the steps: 1, carrying out the trigonometric function transformation of the whole life data of a plurality of bearings, obtaining a first type of features, and carrying out the multi-domain feature extraction, and obtaining a second type of features; forming a feature set according to the first type of features and the second type of features; 2, carrying out absolute value standardization on the feature set; 3, dividing the absolute value standardized feature data into a training set and a test set, and labeling the training set and the test set; and 4, training the bearing life prediction model according to the training set to obtain a trained bearing life prediction model. According to the method, the problems of key state information loss, insufficient model prediction accuracy and insufficient model generalization ability caused by insufficient bearing degradation information mining, insufficient full-life data time dependency relationship, insufficient attention mechanism application research and the like of an existing service life prediction method can be well solved.

Description

Technical Field

[0001] The invention relates to a bearing life prediction method based on SE attention and improved Transformer encoding layer, and belongs to the field of mechanical fault diagnosis. Background Art

[0002] Rolling bearings are known as "the joints of industry" and are an important part of rotating mechanical equipment. They are widely used in various industrial fields. Since bearings often work in harsh environments, once a failure occurs, it may cause equipment to stop running, economic losses, or even cause safety accidents and casualties. Therefore, it is crucial to ensure the normal operation of bearings. In order to improve the reliability and safety of bearings, it is of great research significance to accurately and timely predict the remaining useful life (RUL) of bearings. Remaining useful life prediction methods can be mainly divided into two categories: model-based prediction methods and data-driven prediction methods. Model-based prediction methods rely on the establishment of mathematical or physical models to understand the failure mechanism. However, due to the complexity and randomness of the system, it is difficult to understand all fault models and degradation processes, which limits the applicability of this method. In contrast, data-driven prediction methods use data collected by sensors and use techniques such as machine learning and probability statistics to establish a mapping relationship between sample data and remaining useful life. The advantage of this method is that remaining useful life prediction can be performed in the absence of prior knowledge. In recent years, with the rapid development of deep learning and industrial big data, data-driven prediction methods have gradually become mainstream, providing a more effective and reliable solution for bearing health management.

[0003] At present, deep learning has become the mainstream of life expectancy prediction based on data-driven methods. In the field of sequence modeling, recurrent neural networks and their variants, long short-term memory networks, are widely used in prediction tasks due to their excellence in time series processing. However, the serial operation of recurrent neural networks limits their ability to capture dependencies in long sequence data and seriously affects the running speed of the model.

[0004] In order to make up for the shortcomings of existing deep learning models in the field of bearing life prediction, the present invention is specially proposed. Summary of the invention

[0005] The present invention provides a bearing life prediction method based on SE attention and an improved Transformer coding layer. A prediction model that is more suitable for bearing life prediction is constructed by combining SE attention with the improved Transformer coding layer.

[0006] The technical solution of the present invention is: according to the first aspect of the present invention, a bearing life prediction method based on SE attention and improved Transformer encoding layer is provided, including: step one: performing trigonometric function transformation on the full life data of multiple bearings to obtain first-class features, and performing multi-domain feature extraction to obtain second-class features; forming a feature set based on the first-class features and the second-class features; step two: normalizing the feature set by absolute values; step three: dividing the feature data normalized by absolute values ​​into a training set and a test set and labeling them; step four: training the bearing life prediction model based on the training set to obtain a trained bearing life prediction model.

[0007] The method of performing trigonometric transformation on the bearing life data to obtain the first type of features is specifically: performing trigonometric transformation on the bearing life data using a monotonic trigonometric function and then extracting the standard deviation to obtain the first type of features.

[0008] The bearing life prediction model is constructed by sequentially connecting the SE attention mechanism, the improved Transformer encoding layer network, and the regression layer.

[0009] The SE attention mechanism is specifically as follows: the feature data with absolute value normalization is input, and the feature dimension of each time step is compressed and reduced in dimension through the adaptive average pooling layer, i.e., the Squeeze operation; then, the Excitation operation is performed through two layers of full connection to perform nonlinear interactive learning between time steps. Specifically, the first layer of full connection mainly expands and maps the dimension on the time step, and then performs nonlinear activation through the activation function ReLU; the second layer of full connection mainly maps the time step dimension after the first layer mapping back to restore the dimension after the previous Squeeze operation, and then applies the Sigmoid activation function to obtain the attention weight, and multiplies the original input feature data by the attention weight.

[0010] The improved Transformer coding layer network specifically comprises: adding a multi-scale feature extraction fusion block to the traditional Transformer coding layer network and adding relative position encoding to the multi-head attention mechanism; the multi-scale feature extraction fusion block is located between the position encoding layer and the multi-head attention mechanism of the traditional Transformer coding layer network.

[0011] The multi-scale feature extraction and fusion module consists of three parallel branches: a trunk branch, a residual attention mask branch and a residual structure branch; wherein: the trunk branch adopts 1-3 layers of multi-scale residual pyramid attention layers connected in series to deeply extract multi-scale features; three pyramid attention blocks are set in each multi-scale residual pyramid attention layer to extract corresponding features of different scales respectively, and realize feature integration through step-by-step fusion; the residual attention mask branch generates a feature mask matching the size of the trunk branch by combining upsampling and downsampling attention mechanisms for correcting feature information; the residual attention mask branch and the residual structure branch together constitute a gradient protection mechanism.

[0012] The relative position encoding is mainly used in the softmax of the multi-head attention mechanism.

[0013] According to the second aspect of the present invention, there is provided a bearing life prediction system based on SE attention and improved Transformer coding layer, comprising: a first acquisition module, used to perform trigonometric function transformation on the full life data of multiple bearings to obtain first-class features, and perform multi-domain feature extraction to obtain second-class features; a feature set is composed according to the first-class features and the second-class features; a processing module, used to perform absolute value standardization on the feature set; a division module, used to divide the absolute value standardized feature data into a training set and a test set and label them; a second acquisition module, used to train the bearing life prediction model according to the training set to obtain a trained bearing life prediction model.

[0014] According to a third aspect of the present invention, a terminal is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute any one of the above methods.

[0015] The beneficial effects of the present invention are:

[0016] (1) Through the calibration of the bearing time step information by SE attention, the model can dynamically capture the information of the bearing state changes at different time points, so as to make up for the fact that the Transformer encoding layer may not pay enough attention to the importance of each time step information in the data, thereby improving the model's ability to capture the dynamic characteristics of bearing life and the prediction accuracy.

[0017] (2) The Transformer model not only shows obvious advantages in processing long time series data, but also has the ability of parallel computing, which can effectively solve the problems encountered by recurrent neural networks; and further utilizes the residual attention module to correct local features, alleviate the gradient vanishing and explosion caused by the network depth, and combine the feature pyramid to extract deep features. At the same time, considering the influence of multi-scale feature factors on the bearing life accuracy, a multi-scale feature fusion block is established in the Transformer encoding layer, which can effectively extract and fuse multi-scale features in depth, alleviate the problem of incomplete or poor information extraction caused by a single scale problem, and improve the prediction performance of the model.

[0018] (3) The relative position encoding is placed after the softmax in the attention layer, thereby improving the model's generalization ability and data perception for time series data, while enhancing the performance of the multi-head attention mechanism and the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the bearing life prediction method based on SE attention and improved Transformer encoding layer of the present invention;

[0020] Figure 2 It is the SE attention module diagram in the model of the present invention;

[0021] Figure 3 This is a diagram of the multi-scale residual pyramid attention layer structure in the model of the present invention;

[0022] Figure 4 Add a relative position encoding map to the multi-head attention mechanism in the model of the present invention;

[0023] Figure 5 The global graph is connected to the SE attention, improved Transformer encoding layer, and regression layer of the present invention;

[0024] Figure 6 Graph showing the life prediction results of the model of the present invention; (a) corresponds to Bearing2_2, and (b) corresponds to Bearing2_5. DETAILED DESCRIPTION

[0025] The invention will be further described below in conjunction with the accompanying drawings and embodiments, but the content of the invention is not limited to the scope of the embodiments.

[0026] Example 1: Figure 1-6As shown, according to the first aspect of an embodiment of the present invention, a bearing life prediction method based on SE attention and an improved Transformer encoding layer is provided, including: step one: performing trigonometric function transformation on the full life data of multiple bearings to obtain first-class features, and performing multi-domain feature extraction to obtain second-class features; forming a feature set based on the first-class features and the second-class features; step two: performing absolute value standardization on the feature set; step three: dividing the absolutely standardized feature data into a training set and a test set and labeling them; step four: training a bearing life prediction model based on the training set to obtain a trained bearing life prediction model; step five: importing the test set into the trained bearing life prediction model to obtain the result of bearing life prediction.

[0027] Furthermore, the first type of features are obtained by performing trigonometric transformation on the bearing life data, specifically: performing trigonometric transformation on the bearing life data using a monotonic trigonometric function and then extracting the standard deviation to obtain the first type of features.

[0028] Furthermore, the bearing life prediction model is constructed by using a sequentially connected SE attention mechanism, an improved Transformer encoding layer network, and a regression layer.

[0029] Furthermore, the SE attention mechanism is specifically as follows: the feature data with absolute value normalization is input, and the feature dimension of each time step is compressed and reduced in dimension, i.e., the Squeeze operation, is performed through the adaptive average pooling layer; then the Excitation operation is performed through two layers of full connection to perform nonlinear interactive learning between time steps, specifically: the first layer of full connection mainly expands and maps the dimension on the time step, and then performs nonlinear activation through the activation function ReLU, and the second layer of full connection mainly maps the time step dimension after the first layer mapping back to restore the dimension after the previous Squeeze operation, and then applies the Sigmoid activation function to obtain the attention weight, and multiplies the original input feature data by the attention weight.

[0030] Furthermore, the improved Transformer coding layer network specifically comprises: adding a multi-scale feature extraction fusion block to the traditional Transformer coding layer network and adding relative position encoding to the multi-head attention mechanism; the multi-scale feature extraction fusion block is located between the position encoding layer and the multi-head attention mechanism of the traditional Transformer coding layer network.

[0031] Furthermore, the multi-scale feature extraction and fusion module consists of three parallel branches: the trunk branch, the residual attention mask branch and the residual structure branch. Among them: the trunk branch adopts 1-3 layers of multi-scale residual pyramid attention layers connected in series to deeply extract multi-scale features; 3 pyramid attention blocks are set in each multi-scale residual pyramid attention layer to extract corresponding features of different scales respectively, and realize feature integration through step-by-step fusion. The residual attention mask branch generates a feature mask that matches the size of the trunk branch by combining the upsampling and downsampling attention mechanisms for the correction of feature information. This branch and the residual structure branch together constitute a gradient protection mechanism, which effectively alleviates the problem of gradient explosion or disappearance caused by network deepening. The number of layers of the multi-scale residual pyramid attention layer described above is 1-3, and the structure of each layer is the same.

[0032] Furthermore, the relative position encoding is mainly used in the softmax of the multi-head attention mechanism.

[0033] By applying the above technical solution, it can be known that the invention proposes to obtain key time step information by calibrating the time step information of the feature set by adopting the SE (Squeeze-and-Excitation) attention mechanism. Then, a deep network multi-scale feature extraction fusion module composed of multiple identical multi-scale residual pyramid attention layers in series is constructed to further explore the rolling bearing state information; and relative position encoding suitable for time series prediction is introduced into the multi-head attention mechanism to establish a Transformer encoding layer network architecture based on secondary attention enhancement. Based on the above, the present invention can better solve the problems of insufficient mining of bearing degradation information, insufficient time dependency of full life data, insufficient in-depth research on the application of attention mechanism, etc., which lead to the loss of key state information, insufficient model prediction accuracy and model generalization ability.

[0034] According to the second aspect of an embodiment of the present invention, a bearing life prediction system based on SE attention and improved Transformer encoding layer is provided, including: a first acquisition module, used to perform trigonometric function transformation on the full life data of multiple bearings to obtain first-category features and perform multi-domain feature extraction to obtain second-category features; a feature set is composed according to the first-category features and the second-category features; a processing module, used to perform absolute value standardization on the feature set; a division module, used to divide the absolutely standardized feature data into a training set and a test set and label them; a second acquisition module, used to train a bearing life prediction model according to the training set to obtain a trained bearing life prediction model; a third acquisition module, used to import the test set into the trained bearing life prediction model to obtain a bearing life prediction result.

[0035] According to a third aspect of an embodiment of the present invention, a terminal is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute any one of the methods described above.

[0036] Example 2: The experiment of this example uses the public data set of accelerated bearing degradation in working condition 2 of Xi'an Jiaotong University. The sampling frequency of the data is 25.6kHz, the sampling interval is 1min, and each sampling is 1.28s.

[0037] The overall flow chart of a bearing life prediction method based on SE attention and improved Transformer encoding layer according to an embodiment of the present invention is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0038] Step 1: Perform trigonometric function changes and multi-domain feature extraction on the full life data of rolling bearings.

[0039] The trigonometric function transformation mainly uses the monotonically increasing inverse sine (asinh) and inverse tangent (atan) functions to process the original rolling bearing life data samples sampled at intervals one by one, and then extract the standard deviation respectively (that is, obtain the two features of inverse sine standard deviation and inverse tangent standard deviation as the first type of features). The purpose of doing this is that the trigonometric features combined with trigonometric functions and standard deviations have higher monotonicity, trend and lower scale, which is conducive to the training process of the prediction model. At the same time, in order to ensure the richness of features, 14 features representing the bearing status information are extracted from the original rolling bearing life data as the second type of features (that is, variance, mean, energy, root mean square, peak factor, kurtosis, sum of absolute difference values, information entropy, waveform factor, pulse factor, root mean square frequency, root amplitude, center of gravity frequency, standard deviation in Table 1). The specific feature information is shown in Table 1 below. Where: n represents the number of data points contained in a single sample, x represents the number of data points contained in a single sample, and x represents the number of data points contained in a single sample. i represents the i-th data point in the sample, P(x i ) represents the calculation for the current category x i The probability is the number of occurrences divided by the total number of records; D is the total number of spectral lines, f j is the frequency of the jth spectrum line, and u(j) is the jth spectrum amplitude.

[0040] Table 1 Characteristics and their corresponding calculation formulas

[0041]

[0042]

[0043] Step 2: Normalize the feature set by absolute value. The absolute value normalization formula is:

[0044]

[0045] Where: x ij is the i-th eigenvalue of the j-th sample, |x imax | is the absolute maximum value of the i-th feature column, x i ' j is the normalized value of the absolute value of the ith feature of the jth sample, K is the number of features, i.e. 16, and N is the total number of full life data samples of a single rolling bearing.

[0046] Step 3: Divide the absolute value standardized data into training set and test set.

[0047] Specifically, the experiment in this embodiment uses the public data set of accelerated degradation of bearings under working condition 2 of Xi'an Jiaotong University. The specific division of training and test bearings is shown in Table 2 below.

[0048] Table 2 Division of test training set for working condition 2 experimental data of Xi'an Jiaotong University

[0049]

[0050] The training set and the test set are divided according to the data standardized by the absolute values ​​obtained for the bearings corresponding to Table 2 above.

[0051] The full life cycle of the bearings in the divided training and test sets is normalized to reflect the health status label. The label is marked in the range of 0 to 1 to facilitate observation of the predicted life results. The specific formula is as follows:

[0052]

[0053] Where: RUL t and RUL N They represent the sample ranking t and the total number of samples N in the entire life cycle respectively.

[0054] Step 4: According to the training set, the bearing life prediction model constructed by the SE attention mechanism, the improved Transformer encoding layer network, and the regression layer connected in sequence is trained to obtain a trained bearing life prediction model.

[0055] 1. The SE attention mechanism (SEAttention) is specifically as follows: input the feature data with normalized absolute values, and compress and reduce the feature dimension of each time step through the adaptive average pooling layer AdaptiveAvgpool1d, i.e., the Squeeze operation, to alleviate the dependence on the time step channel information. Then, the Excitation operation is performed through two layers of fully connected linear networks to perform nonlinear interactive learning between time steps. Specifically, the first layer of full connection mainly expands and maps the dimensions on the time step to improve the feature richness, and then the activation function ReLU is used for nonlinear activation. The second layer of full connection mainly maps the time step dimension after the first layer mapping back to restore the dimension after the previous Squeeze operation, and then the Sigmoid activation function is applied to obtain the attention weight, and the original input data is multiplied by the attention weight. Through these operations, the SE attention module can calibrate the time step information in the input data. The time step information calibration mainly helps the model dynamically capture the information of the state changes of the bearing at different time points, and improves the model's ability to capture the dynamic characteristics of the bearing life and the prediction accuracy.

[0056] The weight calculation formula and activation function used in SE attention are as follows. The specific structure is shown in the attached Figure 2 shown.

[0057]

[0058] Squeeze operation:

[0059] Excitation operation: s = F ex (z c ,W)=y2(g(z c ,W))=y2(W2y1(W1z c ));

[0060] Where: x represents the input of the activation function. c Represents the Squeeze operation, i.e. F sq (·) Input scale batch data B, time step C and feature dimension W of each time step, that is, feature u of B×C×W c Transformed into an output weight vector of dimension B×C×1. s represents the Excitation operation, i.e., F ex (·) The obtained z c Perform dimension mapping of the fully connected layer and restore it to the original feature dimension B×C×1. The first layer of fully connected trainable weight is W1, and the activation function is ReLU. The second layer of fully connected trainable weight is W2, and the activation function is sigmoid.

[0061] 2. Improved Transformer encoding layer network

[0062] 1) Embedding layer embedLayer and positional encoding layer (Positional Encoding)

[0063] The embedding layer mainly uses linear mapping to increase the dimension of data features to adapt to the dimension of the encoding layer. The dimension of the encoding layer is limited by the number of attention heads, and needs to satisfy the requirement that the dimension divided by the number of attention heads can be divided evenly. The linear mapping calculation formula is as follows:

[0064] y=LayerNorm(w l x+b l );

[0065] Where: x represents the input feature data, w l 、b l They represent the weight and bias of the mapping layer respectively, and y represents the feature data after layer normalization after the output of the mapping layer.

[0066] The position encoding layer uses sine and cosine functions of different frequencies to encode the feature data, and obtains the feature data position embedding matrix. By directly adding it to the feature data, the position information of each feature data can be obtained. The specific calculation formula for position encoding is as follows:

[0067]

[0068] Where: pos represents the position of the feature vector, d model Represents that the embedding dimension of the encoding layer is also the feature vector dimension, i∈[0,d model / 2), PE represents the position embedding matrix of feature data.

[0069] 2) Multi-scale feature extraction and fusion module

[0070] The multi-scale feature extraction fusion module is mainly composed of two parts: the first part is the trunk branch, which uses three layers of multi-scale residual pyramid attention layers in series to achieve deep extraction of multi-scale features. Each layer of multi-scale residual pyramid attention layer integrates three pyramid attention blocks. This structure can extract corresponding features of different scales and realize feature integration through step-by-step fusion, such as Figure 3 shown.

[0071] The second part is the residual attention mask branch and the residual structure branch. The residual attention mask branch generates a feature mask that matches the size of the trunk branch by integrating the upsampling and downsampling attention (Pooling) mechanisms. This feature mask is used as the weight matrix of the trunk output feature to achieve precise adjustment of the output feature information. In addition, the residual attention mask branch and the residual structure branch jointly construct a protection mechanism to effectively suppress the gradient explosion or disappearance phenomenon during the network deepening process.

[0072] Multiscale Residual PyramidAttention Layer:

[0073] The multi-scale residual pyramid attention layer is mainly designed by combining the advantages of residual attention and pyramid attention. It can extract more data information than a single residual attention or pyramid attention mechanism. And considering that the convolution layer with a large convolution kernel in the original pyramid network can obtain a large receptive field, it will generate a lot of calculations and parameters. Therefore, in the multi-scale residual pyramid layer, two convolution layers Conv1×3 with a convolution kernel of 3 are used to replace a convolution layer with a convolution kernel of 5, and three convolution layers with a convolution kernel of 3 are used to replace a convolution layer with a convolution kernel of 7. While achieving the same effect, the number of parameters can be greatly reduced. The multi-scale operation in the multi-scale residual pyramid attention layer is mainly achieved through upsampling and average pooling operations AvgPooling. The present invention adopts 1 / 2, 1 / 4, and 1 / 8 scale information in the data feature dimension, and then the depth information obtained by the different scale feature data flowing through the same residual attention and pyramid attention combined network block is fused by addition. The specific structure is shown in the attached Figure 3 The present invention establishes a hyperparameter for the number of attention layers of the multi-scale residual pyramid, with a value range of 1-3. Figure 6 The best prediction result when 3 layers are connected in series (such as Figure 5 The structure diagram shows a series of three layers. Figure 6 (a) is the Bearing2_2 bearing, (b) is the Bearing2_5 bearing, True RUL represents the true life, The Paper Prediction represents the life predicted by the method of the present invention, and The Paper Error represents the absolute deviation between the true life and the predicted life. Figure 6It can be seen that the model of the present invention performs well in prediction, especially in the later stage, the predicted life is closer to the real life, which shows that the model of the present invention has good prediction performance. ). The present invention mainly takes into account that the traditional bearing life prediction method usually only considers data information of a single scale, ignoring the relationship between multiple scales and multiple features within the model. This is done to more comprehensively capture the multi-scale feature information in the bearing life prediction problem and improve the accuracy of life prediction. The calculation formula of the multi-scale residual pyramid attention layer is as follows:

[0074] H i (x) = x i +S3(S2(S1(x i )))×M(x i );

[0075] Where: x i represents the data features of the previous network output, i represents the serial level number of the multi-scale residual pyramid attention layer, S1, S2, and S3 are the network parameters of the three-layer pyramid structure, M(x i ) is the residual attention mask branch, S3(S2(S1(x i The combination of ))) is mainly to show the fusion process between the three adjacent levels. i (x) represents the output features after the multi-scale residual pyramid attention layer.

[0076] 3) Adding Multi-Head Attention Mechanism for Relative Position Encoding

[0077] After the data is extracted and fused through the multi-scale residual pyramid attention layer, the multi-head attention mechanism further extracts and integrates feature information. It can focus on the features of multiple subspaces at the same time, then concatenate the learned information of different subspaces and obtain the final attention value through linear transformation. The specific calculation formula is as follows:

[0078]

[0079] Where: X represents the local input data of the model, n represents the total number of attention heads, W i q , W i k , W i v Represents Q under the i-th attention i , K i 、V i The mapping layer can train the weight matrix, d h represents the dimension obtained by dividing the embedding dimension of the encoding layer by the number of attention heads, h iRepresents the attention value under the i-th attention head, W represents the trainable weight matrix of the mapping layer after merging multiple attention heads, Q represents the query matrix, and K represents the key matrix.

[0080] In the process of multi-head attention mechanism integration and extraction, the model has certain limitations in data generalization and perception. Therefore, at the softmax of the multi-head attention mechanism layer, a relative position encoding suitable for time series prediction is introduced, which strengthens the position information of feature data, improves the model's perception of data, and enhances the generalization of the model. Its basic principle is: establish a time step coordinate (x, y) in a two-dimensional coordinate system. For convenience, the coordinate in the y direction is set to 0, and the x direction is set from 0 to the total length of each training time step L in the order of each time step. After that, each time step coordinate is subtracted from all training time step coordinates including itself to obtain a relative position coordinate index matrix with a shape of (2, L, L). In order to ensure that the position index is within the expected range, a boundary condition is set, and the index in the x direction in the relative position index matrix is ​​uniformly added to L-1 to ensure that the relative position index of each time step is within the range of 0 to 2L-1. At the same time, the x-direction increment and y-direction increment in the relative position index matrix of shape (2,L,L) are added to obtain the final relative position index matrix, which is actually the result after taking the boundary condition in the x direction, and its shape is (L,L). The subsequent attention value shape (L,L) is added to the corresponding relative displacement found according to the relative position index matrix, and the data contains the position information between time steps. Specifically, a trainable static parameter W of size 2L-1 is created for each attention head, such as Figure 4 The relative position relationship part shown. The size of the trainable parameters is mainly because the maximum distance between two time steps is 2L-1, and the position relationship scalar before establishing the boundary conditions for the two time steps j and s is W j-s , the position relationship scalar after establishing the boundary conditions is expressed as W j-s+L-1 The position weight in this relative position encoding considers more the relative displacement between different time steps, which is equivalent to the equivalent property of convolution translation, which can enhance the performance of the attention mechanism and enhance the generalization of the model. The specific structure of relative position encoding and multi-head attention is shown in Figure 4 , the relative position weight and attention weight are combined as follows:

[0081] a i =softmax(X i )+W j-s+L-1 ;

[0082] Where: a i represents the attention weight of the i-th attention head plus the position encoding weight, W j-s+L-1represents the relative displacement weight after the boundary condition is established between the jth time step and the sth time step, X i represents the input features of Q and K aggregation under the i-th attention head, Q represents the query matrix, and K represents the key matrix.

[0083] 4) Feedforward network layer (FeedForward)

[0084] After the data with different weights are normalized by the layer, the data will be sent to the feedforward network layer to capture the long-distance dependencies. This network usually consists of two linear layers, and the two layers are connected by the nonlinear activation function ReLU. The feedforward network layer formula is as follows:

[0085] FFN(x)=W2·max(0,(W1x+b1))+b2;

[0086] Where: W1 and W2 represent the trainable weight parameters of the first two linear layers, and b1 and b2 are trainable biases.

[0087] 3. Regression Layer

[0088] The high-level features obtained from the feedforward network layer are expanded after the normalization layer and then enter the regression layer. In the regression layer, three layers of mapping and two layers of nonlinear activation are performed to obtain the bearing life prediction results. The regression layer formula is as follows:

[0089]

[0090] Where: W1, W2, W3 are the trainable parameters of the three mapping layers, b1, b2, b3 are the corresponding trainable biases, x1 is the high-level feature of the local input, is the predicted value output by the model.

[0091] In step 5, the life prediction accuracy of the currently measured rolling bearing is calculated by the root mean square error RMSE and the mean absolute error MAE. The smaller the RMSE value, the smaller the deviation between the predicted life and the actual life, the more stable the prediction performance, and the smaller the MAE value, the higher the model prediction accuracy.

[0092]

[0093]

[0094] Where: N is the number of samples, y i is the actual remaining life at time i, is the predicted remaining life at time i.

[0095] In order to further illustrate the superiority of the inventive method, a life prediction model with strong correlation with the present invention is selected for comparison. Specifically, the present invention is compared with a bearing life prediction method based on a Transformer coding layer and a bearing life prediction method based on SE attention and a Transformer coding layer, wherein the bearing life prediction method based on a Transformer coding layer indicates that the prediction model is constructed using only a traditional Transformer coding layer and a regression layer connected in sequence, and the bearing life prediction method based on SE attention and a Transformer coding layer indicates that the prediction model is constructed using a SE attention mechanism, a traditional Transformer coding layer, and a regression layer connected in sequence. The specific results are shown in Table 3, which shows a comparison of the bearing life prediction performance of different methods (the bold data in the table represents good life prediction performance. A single cell in the prediction index column represents the prediction index of two bearings under the corresponding prediction method, wherein the first row in the prediction index cell corresponding to the prediction method represents the prediction index of bearing bearing2_2, and the second row represents the prediction index of bearing bearing2_5. The average prediction performance is the result obtained by adding the same index of the two bearings and dividing it by 2). In Table 3, the method proposed by the present invention achieves the best bearing life prediction performance in terms of RMSE and MAE indicators, and also ranks first in terms of average prediction performance. It can be seen that the method of the present invention has a high prediction performance.

[0096] Table 3 Comparison of rolling bearing life prediction methods

[0097]

[0098] Through the above detailed introduction to the key parts of each stage of the present invention, it can be seen that the present invention can fully solve the problems of incomplete and poor extraction of degradation information caused by the relatively single characteristic scale within the traditional model, and more comprehensively and dialectically view the influence of multi-scale factors on the accuracy of bearing life. It also fully solves the problem of poor generalization and perception of the model for time series data, and improves the performance of model prediction.

[0099] The specific implementation modes of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A bearing life prediction method based on SE attention and improved Transformer encoding layer, characterized in that: include: Step 1: Perform trigonometric function transformation on the full life data of multiple bearings to obtain the first type of features, and perform multi-domain feature extraction to obtain the second type of features; and form a feature set based on the first type of features and the second type of features; Step 2: Normalize the absolute value of the feature set; Step 3: Divide the absolute value standardized feature data into training set and test set and label them; Step 4: Train the bearing life prediction model based on the training set to obtain a trained bearing life prediction model.

2. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 1 is characterized in that: The method of performing trigonometric transformation on the bearing life data to obtain the first type of features is specifically: performing trigonometric transformation on the bearing life data using a monotonic trigonometric function and then extracting the standard deviation to obtain the first type of features.

3. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 1 is characterized in that: The bearing life prediction model is constructed by sequentially connecting the SE attention mechanism, the improved Transformer encoding layer network, and the regression layer.

4. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 3 is characterized in that: The SE attention mechanism is specifically as follows: the feature data with absolute value normalization is input, and the feature dimension of each time step is compressed and reduced in dimension through the adaptive average pooling layer, i.e., the Squeeze operation; then, the Excitation operation is performed through two layers of full connection to perform nonlinear interactive learning between time steps. Specifically, the first layer of full connection mainly expands and maps the dimension on the time step, and then performs nonlinear activation through the activation function ReLU; the second layer of full connection mainly maps the time step dimension after the first layer mapping back to restore the dimension after the previous Squeeze operation, and then applies the Sigmoid activation function to obtain the attention weight, and multiplies the original input feature data by the attention weight.

5. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 3 is characterized in that: The improved Transformer coding layer network specifically comprises: adding a multi-scale feature extraction fusion block to the traditional Transformer coding layer network and adding relative position encoding to the multi-head attention mechanism; the multi-scale feature extraction fusion block is located between the position encoding layer and the multi-head attention mechanism of the traditional Transformer coding layer network.

6. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 5 is characterized in that: The multi-scale feature extraction and fusion module consists of three parallel branches: a trunk branch, a residual attention mask branch and a residual structure branch; wherein: the trunk branch adopts 1-3 layers of multi-scale residual pyramid attention layers connected in series to deeply extract multi-scale features; three pyramid attention blocks are set in each multi-scale residual pyramid attention layer to extract corresponding features of different scales respectively, and realize feature integration through step-by-step fusion; the residual attention mask branch generates a feature mask matching the size of the trunk branch by combining upsampling and downsampling attention mechanisms for correcting feature information; the residual attention mask branch and the residual structure branch together constitute a gradient protection mechanism.

7. The bearing life prediction method based on SE attention and improved Transformer encoding layer according to claim 5 is characterized in that: The relative position encoding is mainly used in the softmax of the multi-head attention mechanism.

8. A bearing life prediction system based on SE attention and improved Transformer encoding layer, characterized in that: include: The first acquisition module is used to perform trigonometric function transformation on the full life data of multiple bearings to obtain the first type of features, and perform multi-domain feature extraction to obtain the second type of features; and form a feature set based on the first type of features and the second type of features; A processing module, used for normalizing the absolute value of the feature set; The partitioning module is used to divide the feature data with absolute value normalization into training set and test set and label them; The second acquisition module is used to train the bearing life prediction model according to the training set to obtain the trained bearing life prediction model.

9. A terminal comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: The processor is configured to execute the method of any one of claims 1-7.

Images