A bearing life prediction method of improved TCN under a hybrid attention mechanism

Abstract

The application discloses a bearing life prediction method based on an improved TCN under a mixed attention mechanism, and the steps are as follows: step 1, standardizing a data set; step 2, dividing the data set into a training set and a test set; step 3, extracting features from collected full-life vibration signals of a rolling bearing, so as to facilitate subsequent establishment of an accurate life prediction model; step 4, establishing a feature selection index, and screening out statistical features from a feature pool; step 5, building a multi-scale deep convolutional neural network module, and extracting multi-scale deep degradation features from original samples; and step 6, constructing a life prediction module, completing model training, and obtaining a life prediction model used for estimating the remaining service life of the bearing. The application has multi-feature fusion capability, fuses deep features and statistical features, has stronger representation capability, can enhance the accuracy of the life prediction model, and can also improve the prediction accuracy and stability of the model.

Description

Technical Field

[0001] This invention relates to a bearing life prediction method, and more particularly to a bearing life prediction method based on an improved TCN under a hybrid attention mechanism. Background Technology

[0002] Bearings, as common components of mechanical equipment, play a crucial role in transportation, clinical medicine, aerospace, and military defense. Due to the complex and variable working environment of mechanical equipment, and the inevitable wear and performance degradation after prolonged high-intensity operation, bearings are prone to failure. Therefore, they are among the most easily damaged components. Once component degradation spreads throughout the system over time, it leads to frequent equipment failures, gradually causing the equipment to evolve from its normal operating mode to its final failure mode, reducing reliability and drastically increasing maintenance costs. Remaining Useful Life (RUL) prediction involves monitoring the status data of key components within machinery, extracting information, assessing the stage of health of the parts, and predicting their remaining useful life. Currently, bearing RUL prediction has achieved some success, and methods can be categorized into model-based methods, data-driven methods, and hybrid methods that combine both. Data-driven methods are currently the mainstream approach for remaining lifetime prediction. These methods do not require precise analytical models of the degradation process; instead, they directly process and analyze observed data to uncover hidden lifetime information, providing reliable information for fault prediction and maintenance decisions. Among the various data-driven methods, deep learning stands out for its superior ability to process large amounts of data and its intelligent feature extraction capabilities.

[0003] Currently, sequence modeling in the context of deep learning is mainly related to recurrent neural network architectures (such as Long Short-Term Memory networks LSTM and Gated Recurrent Units GRU). Recurrent Neural Networks (RNNs) lack parallel processing capabilities; predictions for subsequent time steps must wait for the completion of the previous step, reducing model flexibility and causing errors to accumulate step by step. Transient Convolutional Networks (TCNs) can break free from the RNN framework. When modeling sequence data, they consider convolutional networks as well, using a combination of residual networks and dilated convolutions to enhance network memory, making them more effective in long-term sequence prediction tasks. Experiments have shown that TCNs can achieve better performance than RNNs in some tasks while avoiding common defects of recurrent models, such as gradient explosion / vanishing problems or lack of memory retention. Furthermore, using TCNs instead of recurrent networks can improve network computational performance through parallel computation of outputs.

[0004] Therefore, given the advantages of TCN in processing long-term sequences and its good parallel processing capabilities, research on TCN networks is conducted. Summary of the Invention

[0005] The purpose of this invention is to propose a bearing RUL prediction method based on an improved TCN (Tracking Network Network) with a hybrid attention mechanism. By improving the TCN network to integrate a wider range of feature inputs, and utilizing the attention mechanism to enhance the TCN network model's utilization of effective feature information, this addresses the problem of poor prediction accuracy and generalization caused by redundant information from multi-dimensional inputs. Furthermore, it enhances the model's perception of multi-dimensional degradation information, forming a degradation perception with temporal and spatial dimensions.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: an improved bearing life prediction method under a hybrid attention mechanism, which includes the following steps:

[0007] Step 1: Data Processing: Standardize the experimental dataset for the entire life cycle of rolling bearings;

[0008] Step 2: Dataset Splitting: Divide the dataset into training and test sets;

[0009] Step 3: Establish a statistical feature pool: Extract features from the collected rolling bearing vibration signals throughout their entire life cycle to facilitate the establishment of an accurate life prediction model. Specifically, the original samples are divided into several non-overlapping segments of length N according to the sampling time to obtain time-domain features. Then, the frequency-domain features that are from the same source as the original time-domain signal are obtained through Fourier transform to establish a feature pool.

[0010] Step 4: Feature selection: Establish feature selection criteria, sort the features by calculating the values ​​of composite feature criteria, and select statistical features from the feature pool;

[0011] Step 5: Multi-scale deep features: Build a multi-scale deep convolutional neural network module to extract multi-scale deep degradation features from the original samples;

[0012] Step 6: Life Prediction Model: Construct a life prediction module. Use statistical features and deep degradation features as inputs to train the life prediction module. Then, update the parameters of the life prediction module through the Adam optimizer to complete the model training and obtain a life prediction model for estimating the remaining service life of the bearing.

[0013] Furthermore, in step 1, the standardized formula is:

[0014]

[0015] Where x i (t) and Let be the values ​​of the i-th original and standardized feature sequences at time t, respectively. Let be the average, maximum, and minimum values ​​of the i-th feature sequence value at all times;

[0016] Furthermore, the entire lifecycle of the bearing is normalized, and a health status label ranging from 0 to 1 is generated. The formula is as follows:

[0017]

[0018] Among them, RUL t With RUL T These represent the sorting order t of the sampling points and the total number of sampling points throughout the entire lifecycle, respectively.

[0019] Furthermore, in step 2, the first 70% of the vibration samples in the dataset are used as the training set, and the last 30% of the vibration samples are used as the test set.

[0020] Furthermore, the feature selection process in step 4 is as follows:

[0021] (1) Establish a composite index J(X), which consists of the correlation index Corr(X), the monotonicity index Mon(X), and the robustness index Rob(X). Among them, the correlation index Corr(X) is the Spearman correlation coefficient between the characteristic sequence and its time series, which describes the degree of correlation between the characteristic sequence and the bearing performance change trend. The monotonicity index Mon(X) refers to the consistency of the bearing performance degradation trend. The performance degradation of the bearing during use is irreversible, so the characteristic index should have a monotonic change trend. The robustness index Rob(X) refers to the tolerance of the characteristic index to abnormal conditions.

[0022] (2) Use the correlation index Corr(X), monotonicity index Mon(X) and robustness index Rob(X) to comprehensively evaluate the degenerate features in the feature pool. Rank the features by calculating the values ​​of the composite feature indexes and select the top four statistical features from the feature pool.

[0023] Furthermore, the correlation metric Corr(X) is expressed as follows:

[0024]

[0025] The expression for the monotonicity index Mon(X) is as follows:

[0026]

[0027] The robustness index Rob(X) is expressed as follows:

[0028]

[0029] Where K represents the number of degenerate features, X = (x1, x2, ..., x...) k ) represents the acquired degenerate feature sequence, x i The trend sequence of the acquired status monitoring indicators is represented by ε(·), where ε(·) represents the step function. Representing the degenerate characteristics after smoothing, T = (t1, t2, t3, ..., t... k ) represents the corresponding time sequence.

[0030] Furthermore, in step 5, the process of building the multi-scale deep convolutional neural network module is as follows:

[0031] (1) Input the original sample data in parallel into four groups of deep convolutional neural network modules with different kernel sizes, and design the kernel of each deep convolutional neural network module to slide along the time axis.

[0032] (2) Set the number of layers for each deep convolutional neural network module. Each module uses a combination of convolutional and pooling layers, with max pooling used for the pooling layers. The pooling size is the same for each module. The activation function for the convolutional layers is ReLU, and its calculation formula is as follows:

[0033] f(x) = max(0,x)

[0034] Where x represents the input to the activation function;

[0035] (3) The degradation feature sequences learned by each deep convolutional neural network module are concatenated and spliced ​​along the feature sequence dimension to obtain multi-scale bearing degradation features.

[0036] Furthermore, in step (6), the remaining useful life prediction module is trained using the training set to obtain the AM-MTCN model for estimating the remaining useful life of the component. The process is as follows:

[0037] (1) Improved TCN model: Uses a pyramid-shaped TCN spatial structure. Its spatial topology replaces the original single input by using the channel attention mechanism to fuse multiple inputs in the original dilated convolutional layer. The improved TCN structure is composed of stacked residual blocks. Each residual block contains a multi-input dilated causal convolutional layer, a dropout layer, and a batch normalization layer. The residual method is used to connect the inputs and outputs of residual blocks in different layers. The outputs of two residual blocks in each layer are the inputs of the residual blocks in the next layer. The final fused features are obtained by continuously stacking them.

[0038] (2) During the fusion process, the channel attention layer is used to assign weights of different spatial layers to the output of the fused residual block;

[0039] (3) Construct a time attention layer. The input of the time attention layer is the output of the TCN. The time attention layer adaptively assigns different weights on the time sequence for different time steps.

[0040] (4) The output values ​​with different weights are input into the fully connected layer, and then the outputs of the fully connected layer are summed to produce the final prediction result.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] The model structure of TCN has been improved to enable it to fuse multiple features, including deep and statistical features, resulting in stronger representation capabilities and enhanced accuracy of lifetime prediction models.

[0043] By extracting temporal and spatial features of the input sequence at different scales through multi-scale convolution and channel attention, and assigning different weights to the output of the TCN by the input of the temporal attention layer, the prediction accuracy and stability of the model are improved. Attached Figure Description

[0044] Figure 1 This is the overall flowchart of this method;

[0045] Figure 2 This is a schematic diagram of the vibration signals of the X and Y axes of bearings 1-4 throughout their entire life cycle in the training dataset.

[0046] Figure 3 It is a multi-scale deep feature extraction module;

[0047] Figure 4 It is an extended causal convolutional model with dual-channel input and a channel attention mechanism;

[0048] Figure 5 It is an improved stacked residual block model used in the TCN model;

[0049] Figure 6 It is an improved TCN model;

[0050] Figure 7 It is a bearing remaining life prediction module that combines an improved temporal convolutional network model and an attention mechanism;

[0051] Figure 8 This is a diagram of the training process for bearing 1_4. Detailed Implementation

[0052] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0053] The experiment in this embodiment uses the accelerated degradation test bearing dataset from the IEEE PHM 2012 challenge for training and testing. The sampling frequency in the experiment is 25.6kHz, the single sampling time is 0.1s, and the sampling interval is 10s. Therefore, each sampling can obtain 2560 sampling points. Figure 2 The vibration signals of bearings 1-4 throughout their entire life cycle, extracted along the horizontal and vertical directions, are displayed.

[0054] The overall process of the improved TCN bearing life prediction method under the hybrid attention mechanism in this embodiment is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0055] Step 1: Data Processing: Standardize the experimental dataset for the entire life cycle of rolling bearings. The standardization formula is as follows:

[0056]

[0057] Where x i (t) and Let be the values ​​of the i-th original and standardized feature sequences at time t, respectively. Let be the average, maximum, and minimum values ​​of the i-th feature sequence value at all times.

[0058] Since the lengths of the full life cycle signal data for different bearings are not consistent, their life cycle labels are normalized to achieve better training results, and health status labels ranging from 0 to 1 are assigned. The formula is as follows:

[0059]

[0060] Among them, RUL t With RUL T These represent the sorting order t of the sampling points and the total number of sampling points throughout the entire lifecycle, respectively.

[0061] Step 2: Dataset splitting: Use the first 70% of vibration samples in the dataset as the training set and the last 30% of vibration samples as the test set;

[0062] Step 3: Establish a feature pool: Extract features from the collected rolling bearing vibration signals throughout their entire lifecycle to reduce the dimensionality of the vibration signal data, facilitating the subsequent establishment of an accurate degradation model. Specifically, the collected original vibration signal is divided into several non-overlapping segments of length N according to the sampling time using a sliding window technique to obtain time-domain features. The Fourier transform method is then used to obtain frequency-domain features that are homologous to the original time-domain signal, thus establishing a feature pool.

[0063] Table 1 Temporal Characteristics

[0064]

[0065]

[0066] Table 2 Frequency Domain Performance Indicators

[0067]

[0068] Step 4: Feature Selection: A comprehensive composite evaluation index J(X) is constructed by linearly combining the correlation coefficient (Corr(X)), monotonicity (Mon(X)), and robustness (Rob(X)).

[0069] J(X)=ω1Corr 1:k +ω2Mon 1:k +ω3Rob 1:k

[0070]

[0071] Where: ω1, ω2 and ω3 are the weight coefficients of the correlation coefficient index Corr(X), the monotonicity index Mon(X) and the robustness index Rob(X), respectively;

[0072] Correlation (Corr(X)) is the feature sequence X = (x1, x2, ..., x...). k ) and its time series T = (t1, t2, t3, ..., t k Spearman correlation coefficient describes the degree of correlation between a feature sequence and the performance change trend of the device;

[0073]

[0074] Monotonicity Mon(X) refers to the consistency of the bearing performance degradation trend, which is unavoidable. Therefore, the obtained characteristic index should have a monotonic change trend.

[0075]

[0076] Robustness (Rob(X)) refers to the tolerance of a characteristic index under abnormal conditions;

[0077]

[0078] Where K represents the number of degenerate features, X = (x1, x2, ..., x...) k ) represents the acquired degenerate feature sequence, x i The trend sequence of the acquired status monitoring indicators is represented by ε(·), where ε(·) represents the step function. Representing the degenerate characteristics after smoothing, T = (t1, t2, t3, ..., t... k ) represents the corresponding time sequence.

[0079] Effective features are selected from the extracted features in the feature pool based on the constructed comprehensive evaluation index J(X). Generally, features with lower similarity measures, higher temporal correlation, and better monotonicity are more effective and suitable as features for RUL prediction. Therefore, the coefficients of J(X) are selected as follows: ω1 = 0.5, ω2 = 0.3, ω3 = -0.2. Only the top 4 statistical features are retained from the feature pool using the composite index.

[0080] Step 5: Design a multi-scale deep convolutional neural network module, such as... Figure 3 As shown, the original samples will be input into the multi-scale deep convolutional network to learn multi-scale degradation features;

[0081] (1) Input the original samples into four groups of deep convolutional neural network modules with different kernel sizes, and design the kernel of each deep convolutional neural network module to slide along the time axis to extract features.

[0082] (2) Set the number of layers for each deep convolutional neural network module. Each module uses a combination of convolutional and pooling layers, with max pooling used for the pooling layers. The pooling size is the same for each module. The activation function for the convolutional layers is ReLU. The convolutional pooling calculation formula is as follows:

[0083] convolution:

[0084] Where: Q = g(x,y) is the input data, M×N is its dimension, the x-coordinate of g(x,y) in the input data Q is x, and its y-coordinate is y, and l(x,y) is the convolution kernel with a×b size. The feature signal obtained after processing the original input data by the convolution kernel is A(s,t).

[0085] Pooling: f max (x) = max(0,x)

[0086] Where x represents the input to the activation function;

[0087] (3) The degradation feature sequences learned by each deep convolutional neural network module are concatenated and spliced ​​along the feature sequence dimension to obtain the bearing degradation features of multi-scale bearings.

[0088] Step 6: Construct the AM-MTCN remaining lifetime prediction model, such as... Figure 7 As shown, the remaining useful life prediction module is trained using the training set to obtain the AM-MTCN model for estimating the remaining useful life of components.

[0089] (1) Improved TCN model: such as Figure 6As shown, a pyramid-shaped TCN spatial structure is used. Its spatial topology primarily involves replacing the original single input with a dual-channel input dilated convolutional layer by fusing multiple inputs using a channel attention mechanism. This results in a dual-channel input dilated causal convolutional model incorporating a channel attention mechanism. Figure 4 As shown, the improved TCN structure consists of stacked residual blocks. Each residual block contains a multi-input dilated causal convolutional layer, a Dropout layer, and a batch normalization layer. A residual method is used to connect the inputs and outputs of residual blocks from different layers, as shown below. Figure 4 As shown; the outputs of the two residual blocks in each layer serve as the inputs to the residual blocks in the next layer, and these are continuously stacked to obtain the final fused features; the calculation formula is as follows:

[0090] Residual block calculation formula: o = Activation(x) A +x B +F(x A +x B ))

[0091] Where o is the output, x A +x B For input, F(·) is an input-output mapping function consisting of a multi-input dilated causal convolutional layer, a Dropout layer, and a batch normalization layer.

[0092] (2) The fusion process uses a channel attention mechanism. Residual blocks at the same level share the same channel attention layer. The channel attention layer at the same level assigns weights of different spatial layers to the inputs being fused at this level. The specific calculation formula is as follows:

[0093]

[0094] Among them, F sq (·) represents the Squeeze function, which extracts a feature u with C channels and H×W dimensions. c The weight vector z is compressed to 1×1×C. C ,

[0095] Excitation: s = F ex (z c ,W)=σ(g(z c ,W))

[0096] Where F ex (·) represents the C-dimensional vector z obtained by the Excitation function after global pooling of squeeze. CPerform a fully connected operation to compress the dimension to obtain a C / r dimensional vector. Then, perform ReLU activation to add nonlinearity. Perform another fully connected operation to transform the C / r dimensional vector back into a C dimensional vector. Finally, perform sigmoid activation (so that the values ​​are between 0 and 1). This gives us the one-dimensional weight vector.

[0097] (3) Construct a temporal attention layer. The input of the temporal attention layer is the output of the TCN. The temporal attention layer adaptively assigns different weights to different time steps. The formula is as follows:

[0098]

[0099]

[0100] Where W represents the trainable parameters, softmax and ReLU are activation functions, and T is the transpose. The current output vector o of the TCN... t and the output vectors M = [o] of the L TCNs before time t t-L ,...,o t-2 ,o t-1 ], by calculating the current output vector o t The relationship between M and the attention score vector a at time t is obtained. t And according to a t Obtain the context vector c t , will c t and o t Combined with the remaining life of rolling bearings The prediction.

[0101] (4) The output values ​​with different weights are input into the fully connected layer, and then the outputs of the fully connected layer are summed to produce the final prediction result, the formula of which is:

[0102]

[0103] In this embodiment of the invention, the mean squared error (RMSE) is used as the loss function for training, and the calculation formula is as follows:

[0104]

[0105] Where N is the number of training samples. For the predicted value, y i This is the tag value.

[0106] The following is a network with an improved TCN structure, with specific parameters shown in the table below:

[0107] Table 3 Structural parameters of the first-layer residual block

[0108]

[0109] Table 4. Structural parameters of the two-layer residual block

[0110]

[0111] Table 5. Structural parameters of the three-layer residual block

[0112]

[0113] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the scope of protection of the present invention in any way, and all technical solutions obtained by equivalent substitution or other means fall within the scope of protection of the present invention. Parts not covered in this invention are the same as or can be implemented using existing technology.

Claims

1. A bearing life prediction method based on an improved TCN (Transmission-Concentration Network) mechanism, characterized in that, Includes the following steps: Step 1: Data Processing: Standardize the experimental dataset for the entire life cycle of rolling bearings; Step 2: Dataset Splitting: Divide the dataset into training and test sets; Step 3: Establish a statistical feature pool: Extract features from the collected rolling bearing vibration signals throughout their entire life cycle to facilitate the establishment of an accurate life prediction model. Specifically, the original samples are divided into several non-overlapping segments of length N according to the sampling time to obtain time-domain features. Then, the frequency-domain features that are from the same source as the original time-domain signal are obtained through Fourier transform to establish a feature pool. Step 4: Feature selection: Establish a composite index consisting of correlation index, monotonicity index and robustness index. Calculate and sort the values ​​of the composite index to select the top four statistical features from the feature pool. Step 5: Multi-scale depth features: Construct a multi-scale depth convolutional neural network module to extract multi-scale depth degradation features from the original samples; the process of constructing the multi-scale depth convolutional neural network module is as follows: (1) Input the original sample data in parallel into four groups of deep convolutional neural network modules with different kernel sizes, and design the kernel of each deep convolutional neural network module to slide along the time axis; (2) Set the number of layers for each deep convolutional neural network module. Each module uses a combination of convolutional and pooling layers, with max pooling used for the pooling layers. The pooling size is the same for each module. The activation function for the convolutional layers is ReLU, and its calculation formula is as follows: Where x represents the input to the activation function; (3) Concatenate and splice the degradation feature sequences learned by each deep convolutional neural network module along the feature sequence dimension to obtain multi-scale bearing degradation features; Step 6: Life Prediction Model: Construct a life prediction module. Statistical features and deep degradation features are used as inputs to train the life prediction module. Then, the Adam optimizer is used to update the parameters of the life prediction module, completing the model training and obtaining a life prediction model for estimating the remaining service life of the bearing. The specific process is as follows: (1) Improved TCN model: A pyramid-shaped TCN spatial structure is used. Its spatial topology replaces the original single input by using the channel attention mechanism to fuse multiple inputs in the original dilated convolutional layer. The improved TCN structure is composed of stacked residual blocks. Each residual block contains a multi-input dilated causal convolutional layer, a Dropout layer, and a batch normalization layer. The residual method is used to connect the inputs and outputs of residual blocks in different layers. The outputs of two residual blocks in each layer are the inputs of the residual blocks in the next layer. The final fused features are obtained by continuously stacking the residual blocks. The formula for calculating the residual block is as follows: Where o represents the output. For input, An input-output mapping function consisting of a multi-input dilated causal convolutional layer, a Dropout layer, and a batch normalization layer; (2) Channel attention layers are used during the fusion process. The same channel attention layer is shared in residual blocks of the same level. The channel attention layer of the same level assigns weights of different spatial layers to the input features being fused in this level. The specific calculation formula is as follows: Squeeze: ; in, The Squeeze function has C channels and a size of [value missing]. Features Compress to weight vector , Excitation: in The C-dimensional vector obtained by the Excitation function after global pooling of squeeze. Perform a fully connected layer to compress the dimension to obtain a C / r dimensional vector, then perform ReLU activation to add nonlinearity, perform another fully connected layer to transform the C / r dimensional vector back into a C dimensional vector, and then perform sigmoid activation to obtain a one-dimensional weight vector; (3) Construct a temporal attention layer. The input of the temporal attention layer is the output of the TCN. The temporal attention layer adaptively assigns different weights on the time sequence for different time steps. The formula is as follows: Where W represents the trainable parameters, softmax and ReLU are activation functions, and T is the transpose; the current output vector of the TCN. and the output vectors of the L TCNs before time t By calculating the current output vector The relationship between M and the attention score vector at time t is obtained. And according to Get the context vector ,Will and Combined with the remaining life of rolling bearings The prediction; (4) The output values ​​with different weights are input into the fully connected layer, and then the outputs of the fully connected layer are summed to produce the final prediction result.

2. The bearing life prediction method based on a hybrid attention mechanism according to claim 1, characterized in that, In step 1, the standardized formula is: ， in and Let be the values ​​of the i-th original and standardized feature sequences at time t, respectively. , , Let be the average, maximum, and minimum values ​​of the i-th feature sequence value at all times.

3. A bearing life prediction method based on a hybrid attention mechanism according to claim 1 or 2, characterized in that, The bearing's entire lifecycle is normalized, and a health status label ranging from 0 to 1 is generated. The formula is as follows: in, and These represent the sorting order t of the sampling points and the total number of sampling points throughout the entire lifecycle, respectively.

4. The bearing life prediction method based on a hybrid attention mechanism according to claim 1, characterized in that, In step 2, the first 70% of the vibration samples in the dataset are used as the training set, and the last 30% of the vibration samples are used as the test set.

5. The bearing life prediction method based on a hybrid attention mechanism according to claim 1, characterized in that, Correlation indicators The expression is as follows: Monotonicity index The expression is as follows: Robustness index The expression is as follows: Where K represents the number of degenerate features, This represents the acquired degenerate feature sequence. This represents the trend sequence of the acquired status monitoring indicators. Represents the step function. This represents the degeneracy characteristics after smoothing. This represents the corresponding time sequence.

Images