A bearing life prediction method based on attention mechanism and residual neural network

Abstract

The application discloses a bearing life prediction method based on an attention mechanism and a residual neural network, comprising the following steps: firstly, using time domain features and a k-means clustering algorithm to adaptively identify the degradation state of the bearing; using an empirical mode method to decompose the bearing data signal and remove noise; and training and using a residual neural network with an introduced attention mechanism to complete the life prediction of the bearing. The application has the following advantages: the degradation state of the bearing is adaptively identified, avoiding the interference of data generated in the normal state on the life prediction; the empirical mode method is used to decompose the bearing data signal and remove noise, preventing the mutual interference and cancellation of degradation and fault features at different frequencies, and thus making the features learned by the model more effective; and the residual neural network with the introduced attention mechanism is used, so that the neural network model is always affected by the original signal and the attention is focused on the channel where the fault features are located, preventing the model from overfitting.

Description

Technical Field

[0001] This invention relates to a bearing life prediction method that incorporates an attention mechanism and a residual neural network. It is a method for predicting the remaining life of bearings and belongs to the interdisciplinary field of engineering applications and information science. Background Technology

[0002] Bearings are vulnerable components among precision parts, accounting for 45% to 55% of mechanical equipment failures. When a bearing fails, it directly affects related rotating shafts, gears, and other parts, potentially causing the entire machine or production line to malfunction and halt production, resulting in economic losses. In severe cases, it can even lead to equipment damage and threaten the safety of operators. If a reliable method for predicting remaining life were available, replacing bearings before they fail could effectively extend the lifespan of mechanical equipment and prevent system failure. Therefore, research into early fault diagnosis technology for rolling bearings is of paramount importance.

[0003] Current research on bearing life prediction methods can be categorized into three types: physical model-based, data-driven, and fusion methods. Since establishing physical models becomes increasingly difficult with increasing time series complexity, data-driven remaining life prediction methods have become the mainstream approach. Bearings often operate in harsh environments such as high temperatures, alternating loads, and heavy loads, so fault and degradation information in bearing signals is hidden within complex background noise. Traditional deep learning methods struggle to avoid noise interference. For traditional convolutional neural networks, the vector dimension and original feature information are significantly reduced after each convolution and pooling. This operation of reducing the vector dimension is called downsampling. Downsampling can extract the abstract features needed by the classifier while reducing the vector dimension. However, as the number of neural network layers and downsampling times increase, problems such as gradient explosion and gradient vanishing occur, leading to a significant decline in network performance.

[0004] To address the aforementioned problems, this invention proposes a bearing life prediction method incorporating an attention mechanism and a residual neural network. To solve the gradient decay problem, a "short-circuit" is added to the residual convolutional neural network, which intelligently skips unnecessary network layers, thus preserving more feature information while downsampling. Simultaneously, the residual neural network is an ensemble model assembled from a series of path sets. This unique path set structure facilitates the introduction of the attention mechanism. Traditional residual neural networks have no connection and are equivalent between channels up to the fully connected layer. However, fault and degradation signals in bearing signals are concentrated only within a subset of frequencies. Therefore, this invention compresses the information of each channel into a global feature, then adds these global features to the fully connected neural network, using the activation value of the sigmoid activation function in the fully connected neural network as the channel weights, thereby making the model more discriminative of the features of each channel. Summary of the Invention

[0005] Objective of this invention: The objective of this invention is to provide an effective method for predicting the remaining life of bearings based on a residual neural network with an attention mechanism. This method uses a short-circuit approach to address the gradient decay problem that traditional neural networks easily encounter when predicting bearing life. Secondly, considering that the effective features in the bearing signal are concentrated in a few specific frequencies, this invention uses an attention mechanism to allow the model to adaptively focus on the channel containing the bearing signal.

[0006] Technical Solution: To achieve the above objectives, this invention proposes a k-means algorithm for clustering bearing degradation patterns, dividing the bearing degradation process into normal operating state, degradation state, and failure state. Since bearings in the normal operating state do not exhibit degradation signals, this invention removes bearing signals from the dataset in the normal operating state. Secondly, considering that degradation signals are concentrated in specific frequencies, this paper uses EMD (Empirical Mode Decomposition) to divide the original signal into eight IMFs (Initial Mode Components). Finally, an attention mechanism is introduced into the neural network model, assigning appropriate weights to each channel, thereby enabling the model to better learn bearing degradation characteristics, thus more effectively predicting the remaining life of the bearing and preventing overfitting.

[0007] Step 1: Cluster the degradation patterns of the bearings using the k-means algorithm:

[0008] (1) Clean and normalize the data to remove outliers that do not conform to physical characteristics.

[0009] (2) Before using the clustering algorithm, it is necessary to extract the features of the vibration signal. The vibration signal features used are divided into two types: dimensional features and dimensionless features. The former includes peak-to-peak value and root mean square value; the latter includes skewness index, kurtosis index, peak value index, impulse index, margin index, and waveform index. The specific calculation methods are as follows:

[0010]

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017] X P-P =max{x(n)}-min{x(n)}

[0018] Where N represents the signal length, μ represents the signal average, σ represents the signal standard deviation, and x(n) refers to the signal value of the vibration signal at time n. RMS represents the root mean square value, SK represents the skewness factor, CF represents the peak factor, IF represents the impulse factor, SK represents the kurtosis factor, CLF represents the margin factor, and IF represents the waveform factor.

[0019] (3) Substitute the input data {x(1), x(2)…x(m)} into the above formula to find the corresponding eigenvalues ​​{x′(1), x′(2)…x′(m)}, x′(i)∈R d Then, randomly select 3 values ​​{x′(1), x′(2)…x′(m)} from all eigenvalues, where x′(i)∈R. d As cluster centers, find the clusters from each sample point to the cluster centers, and assign the sample points to the cluster centers with the smallest distances.

[0020] (4) Find the cluster of each sample point to the cluster center, and assign the sample point to the cluster center with the smallest distance. Take the average value of all sample points in each cluster as the new cluster center.

[0021] (5) Repeat step (4) until the cluster centers no longer change and the model converges.

[0022] Step 2: Bearing data denoising and decomposition

[0023] (1) Remove the data from the normal operation status part of the original dataset.

[0024] (2) The original bearing data is separated using EMD (Empirical Mode Decomposition). Assume the original signal is x(t), and the sequence of its upper and lower envelope means is m(t). Let h1(t) be the original signal minus the upper and lower envelope means. Repeat the above steps k times, treating h(t) as the original signal, until the processed signal satisfies the characteristics of IMF (Initial Mode Components). The specific calculation method is as follows:

[0025] h k (t)=h k-1 (t)-m k (t)

[0026] r(t) = h k (t)

[0027] Where h k (t) represents the value of the signal at time t after k processing iterations, m k (t) refers to h k-1 The sequence consisting of the upper and lower envelope mean values ​​of (t).

[0028] (3) Record the final result from the previous step as an IMF signal, and separate this IMF signal from the original signal to obtain the remaining signal c(t). Substitute the remaining signal as the initial signal into (2) to obtain a new IMF signal. Repeat the above steps 8 times to obtain 8 IMF signals. The specific calculation method is as follows:

[0029] r i (t)=r i-1 (t)-c i (t)

[0030] Its r i (t) represents the value of the i-th IMF signal at time t, and its c i (t) represents the value of the remaining signal at time t after the i-th IMF signal is extracted.

[0031] (3) For the eight IMFs (this feature function) generated after decomposition, calculate the kurtosis of each IMF and remove IMFs with a kurtosis less than 3. The kurtosis is calculated as follows:

[0032]

[0033] Where N represents the length of the signal, μ represents the average value of the signal, and σ represents the standard deviation of the signal.

[0034] Step 3: Train the residual neural network model that incorporates the attention mechanism.

[0035] (1) Input the processed data into the residual neural network prediction model. Each IMF signal is used as an input channel. The overall input format is (N, C, T), where N represents N data points extracted from the dataset, C represents C channels in each data point, and T represents the time step of each channel.

[0036] (2) The vibration characteristics of the vibration signal of each channel are extracted by the one-dimensional convolutional layer.

[0037] (3) Use the SE (attention mechanism extraction) layer to calculate the influence weight of vibration features in different channels on the final result, so that the model pays more attention to the channels that have a greater impact on the result.

[0038] (4) Save the model with the smallest error during the training process.

[0039] (5) Use the model with the smallest process error to predict the remaining life of the bearing.

[0040] Beneficial effects:

[0041] This invention proposes a novel and effective method for predicting the remaining life of bearings based on a residual neural network incorporating an attention mechanism, targeting multidimensional telemetry time-series data. This method effectively addresses a series of issues, such as potential dependencies, that traditional deep learning methods fail to consider. By providing bearing remaining life prediction methods, this invention offers crucial information for early warning of bearing failure, preventing major malfunctions such as damage to mechanical systems. Attached Figure Description

[0042] Figure 1 This is the overall flowchart of the method of the present invention.

[0043] Figure 2 This is a flowchart of the clustering algorithm in this invention.

[0044] Figure 3 This is a flowchart of the process of separating the original signal using EMD in this invention.

[0045] Figure 4 This invention introduces an attention mechanism into the residual neural network structure. Detailed Implementation

[0046] The invention will now be further described with reference to the accompanying drawings.

[0047] The overall process of this invention is as follows: Figure 1 As shown. Its sub-module processes are respectively as follows: Figure 2 , Figure 3 , Figure 4 As shown in the figures, the following is a detailed explanation.

[0048] This invention first adaptively identifies the degradation state of bearings; secondly, it decomposes the bearing data signal and removes bearing noise; and finally, it trains and uses a residual neural network with an attention mechanism. The network model unit used in this invention includes a feature module and an attention mechanism unit. Each network unit's feature learning module consists of three layers of convolutional neural networks. The first and second layers of these three neural networks are convolutional layers with a fixed kernel size of 1, while the kernel size of the middle convolutional layers depends on the network structure. The convolutional layers are connected using the ReLU activation function. Each network unit's attention mechanism module consists of a global pooling layer, two fully connected neural network layers, and a Sigmoid activation function. The global pooling layer encodes all features of each channel extracted in (4-2) into a global feature. The extraction method is global average pooling. Finally, the Sigmoid function is used to calculate the weight of a certain channel. The data obtained from the feature learning module is multiplied by the channel weights to obtain the features of the neural network. Finally, the calculated feature value is added to the sum of the original data input to the network unit as the final output of the module. The overall process is shown in the appendix. Figure 1 .

[0049] 1. Clustering of bearing degradation patterns using the k-means algorithm:

[0050] Bearing degradation involves three stages: normal operation, degradation, and failure. Data generated during normal operation does not contain any degradation or failure information. Furthermore, the operating time of bearings in normal operation typically accounts for more than 60% of the total bearing lifespan dataset. If this distinction is not made and all data is used directly to train the model, the convergence speed of the neural network model will be significantly slowed down, and it will be more prone to overfitting. Therefore, it is necessary to identify the degradation state of the bearing and extract the data generated during normal operation. The specific process is as follows... Figure 2 As shown.

[0051] (1) Clean and normalize the data to remove outliers that do not conform to physical characteristics.

[0052] (2) Before using clustering algorithms, it is necessary to extract the features of the vibration signal. Commonly used vibration signal features are divided into two types: dimensional features and dimensionless features. The former includes peak-to-peak value and root mean square value; the latter includes skewness index, kurtosis index, peak value index, impulse index, margin index, and waveform index. The specific calculation methods are as follows:

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060] X P-P =max{x(n)}-min{x(n)}

[0061] Where N represents the signal length, μ represents the signal average, σ represents the signal standard deviation, and x(n) refers to the signal value of the vibration signal at time n. RMS represents the root mean square value, SK represents the skewness factor, CF represents the peak factor, IF represents the impulse factor, SK represents the kurtosis factor, CLF represents the margin factor, and IF represents the waveform factor.

[0062] (3) Substitute the input data {x(1), x(2)…x(m)} into the above formula to find the corresponding eigenvalues ​​{x′(1), x′(2)…x′(m)}, x′(i)∈R d Then, randomly select 3 values ​​{x′(1), x′(2)…x′(m)} from all eigenvalues, where x′(i)∈R. d As cluster centers, find the clusters from each sample point to the cluster centers, and assign the sample points to the cluster centers with the smallest distances.

[0063] (4) Find the cluster of each sample point to the cluster center, and assign the sample point to the cluster center with the smallest distance. Take the average value of all sample points in each cluster as the new cluster center.

[0064] (5) Repeat step (4) until the cluster centers no longer change and the model converges.

[0065] 2. Noise reduction and decomposition of bearing vibration data

[0066] Traditional deep learning methods for predicting bearing life directly train the model using raw bearing data. Because the features the neural network model needs to learn are hidden in complex background noise, directly using raw data makes the model more prone to overfitting. Furthermore, degradation and fault features at different frequencies are mixed together, interfering with and canceling each other out, further increasing the training difficulty of traditional neural networks. Therefore, it is necessary to separate bearing signals at different frequencies and remove frequencies that do not contain vibration signals. This improves the model's accuracy. The specific process is as follows: Figure 3 As shown.

[0067] (1) Remove the data from the normal operation status part of the original dataset.

[0068] (2) The original bearing data is separated using EMD (Empirical Mode Decomposition). Assume the original signal is x(t), and the sequence of its upper and lower envelope means is m(t). Let h1(t) be the original signal minus the upper and lower envelope means. Repeat the above steps k times, treating h(t) as the original signal, until the processed signal satisfies the characteristics of IMF (Initial Mode Components). The specific calculation method is as follows:

[0069] h k (t)=h k-1 (t)-m k (t)

[0070] r(t) = h k (t)

[0071] Where h k (t) represents the value of the signal at time t after k processing iterations, m k (t) refers to h k-1 The sequence consisting of the upper and lower envelope mean values ​​of (t).

[0072] (3) Record the final result from the previous step as an IMF signal, and separate this IMF signal from the original signal to obtain the remaining signal c(t). Substitute the remaining signal as the initial signal into (2) to obtain a new IMF signal. Repeat the above steps 8 times to obtain 8 IMF signals. The specific calculation method is as follows:

[0073] r i (t)=r i-1 (t)-c i (t)

[0074] Its r i (t) represents the value of the i-th IMF signal at time t, and its c i (t) represents the value of the remaining signal at time t after the i-th IMF signal is extracted.

[0075] (3) For the eight IMFs (this feature function) generated after decomposition, calculate the kurtosis of each IMF and remove IMFs with a kurtosis less than 3. The kurtosis is calculated as follows:

[0076]

[0077] Where N represents the length of the signal, μ represents the average value of the signal, and σ represents the standard deviation of the signal.

[0078] 3. Training a residual neural network model incorporating an attention mechanism

[0079] For convolutional neural networks (CNNs), the vector dimension and original feature information are significantly reduced after each convolution and pooling. This operation of reducing the vector dimension is called downsampling. Downsampling can extract the abstract features needed by the classifier while reducing the vector dimension. However, as the number of neural network layers and the number of downsampling operations increase, problems such as gradient explosion and gradient vanishing occur, causing a significant decline in network performance. When updating the gradient of a traditional neural network, it is necessary to calculate the error term between the current layer and the next layer. To completely solve the gradient decay problem, a "short-circuit" is added to residual convolutional neural networks. This "short-circuit" can intelligently skip some unnecessary network layers, thereby retaining more feature information while downsampling. This invention introduces an attention mechanism along with the "short-circuit," allowing the model to focus on the channels containing effective features, further reducing the possibility of model overfitting. The specific process is as follows: Figure 4 As shown.

[0080] (1) Input the processed data into the residual neural network prediction model. Each IMF signal is used as an input channel. The overall input format is (N, C, T), where N represents N data points extracted from the dataset, C represents C channels in each data point, and T represents the time step of each channel.

[0081] (2) A three-layer convolutional neural network is used to extract the vibration features of the vibration signal in each channel. The first and second layers of this neural network are convolutional layers with a fixed kernel size of 1, and the kernel size of the middle convolutional layers depends on the network structure. The convolutional layers are connected using the ReLU activation function.

[0082] (3) The SE (Attention Mechanism Extraction) layer is used to calculate the influence weights of vibration features in different channels on the final result, so that the model pays more attention to the channels that have a greater impact on the result. It consists of a global pooling layer, two fully connected neural network layers and a Sigmoid activation function. The global pooling layer encodes all features of each channel extracted in (2) into a global feature. The extraction method is global average pooling. Finally, the Sigmoid function is used to calculate the weight of a certain channel.

[0083] (4) Multiply the data obtained by the feature learning module by the channel weights to obtain the features of the neural network. Finally, add the calculated feature values ​​to the sum of the original data input to the network unit as the final output of the module.

[0084] (5) Save the model with the smallest error during training and use this model to predict the remaining life of the bearing.

Claims

1. A bearing life prediction method based on attention mechanism and residual neural network, the main features of which include the following steps: (1) Adaptive degradation pattern recognition: First, the bearing data is cleaned and normalized, and the bearing state is divided into normal operation, degradation and fault state according to features and clustering algorithms; (2) Bearing data denoising and decomposition: The processed degradation and fault state data are divided into 8 basic mode components using the empirical mode decomposition method; (3) Training and use of residual neural network model: Input the processed test data into the residual neural network model to train the residual neural network model; the residual neural network model uses multiple network modules stacked together. Each module contains a feature extraction structure consisting of three convolutional layers and attention mechanism weights consisting of global pooling layers and fully connected layers. Finally, the trained residual neural network model is used to predict the bearing life. The implementation method is as follows: (3-1) After the basic pattern components are separated, the test data is introduced into the residual neural network model based on the attention mechanism. Each basic pattern component is used as a channel of input data. The residual neural network model includes a feature module and an attention mechanism module. (3-2) The feature module of the residual neural network model consists of three convolutional layers. All three convolutional layers have a fixed kernel size of 1. The kernel size of the middle convolutional layer depends on the network structure. The convolutional layers are connected by the ReLU activation function. The attention mechanism module of the residual neural network model described in (3-3) consists of a global pooling layer, two fully connected neural network layers, and a sigmoid activation function. The global pooling layer encodes all features of each channel extracted in step (3-2) into a global feature by global average pooling. Finally, the sigmoid function is used to calculate the weight of each channel. (3-4) Multiply the data obtained by the feature module by the channel weights to obtain the features of the residual neural network model. Finally, add the calculated feature values ​​to the sum of the original data input to the residual neural network model to obtain the final output of the residual neural network model. (3-5) Save the residual neural network model with the smallest error during the training process, and use this residual neural network model to predict the remaining life of the bearing.

2. The bearing life prediction method based on attention mechanism and residual neural network according to claim 1, characterized in that, Step (1) is adaptive degradation pattern recognition, and its implementation method includes: (1-1) The bearing data is cleaned and normalized to obtain dataset 1. The bearing signal features are extracted from dataset 1: peak-to-peak value, root mean square value, skewness index, kurtosis index, peak value index, impulse index, margin index and waveform index. (1-2) The k-means clustering algorithm is used to cluster the bearing operating status into normal operating status, degradation status and fault status according to the bearing signal characteristics.

3. The bearing life prediction method based on attention mechanism and residual neural network according to claim 2, characterized in that, Step (2) involves using the empirical pattern decomposition method to divide the bearing data into multiple pattern components: (2-1) Select data from dataset 1 that includes degradation and fault states; (2-2) The empirical mode decomposition method is used to divide the processed degradation and fault state data into 8 basic mode components, and the components with kurtosis less than 3 are removed.

Images