Bearing residual life prediction method based on MA and MSTCN

Through the multi-head attention mechanism and multi-scale time convolution network, the limitations of single-scale networks in capturing different scale characteristics are solved, and more accurate prediction of bearing residual service life is achieved.

CN118936883APending Publication Date: 2024-11-12SHENYANG INST OF TECH
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411262483.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing bearing residual service life prediction method is inaccurate, and the single-scale time convolution network is limited in capturing time characteristics at different scales, which affects the prediction performance.

Method used

Using a multi-head attention mechanism and a multi-scale time convolution network, diversified time features are extracted through multi-head attention dynamic allocation weights and multi-scale convolution kernels, combining one-dimensional extended causal convolution, maximum pooling, regularization and global maximum pooling to improve feature extraction efficiency and accuracy.

Benefits of technology

More accurate prediction of the remaining service life of bearings is achieved, and the multi-head attention mechanism is used to capture key features and multi-scale time convolution networks to extract multi-scale features, improving prediction accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118936883A_ABST
    Figure CN118936883A_ABST
Patent Text Reader

Abstract

The invention relates to a bearing residual life prediction method based on MA and MSTCN, and belongs to the technical field of bearing life prediction methods, and the method comprises the steps: firstly capturing important key features through employing a multi-head attention mechanism, then extracting multi-scale features from data reconstructed by the multi-head attention mechanism through employing a multi-scale time convolution network, and carrying out the extraction of the multi-scale features through employing a multi-scale time convolution network; more accurate prediction is realized in the RUL task; and then, a prediction value is used for performing a comparison and ablation experiment on a prediction and health management 2012 data set, and a comparison experiment result shows that a proposed prediction framework improves the prediction precision, has relatively good robustness and is suitable for prediction of the remaining service life of the rolling bearing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of bearing life prediction methods, and in particular to a bearing life prediction method based on multi-head attention and multi-scale time convolutional network. Background Art

[0002] Prognostics and health management (PHM) is a method that combines data analysis, predictive modeling and health management technology. It aims to detect potential problems in advance and manage them effectively by monitoring, diagnosing and predicting the status of equipment or systems. Among them, remaining useful life (RUL) prediction is one of the core goals of prognostics and health management research, which aims to estimate the remaining operating time of equipment before failure.

[0003] Rolling bearings are widely used in rotating machinery, and the prediction of their remaining service life plays an increasingly important role in ensuring the safe operation of the system and reducing maintenance costs. Accurately predicting the remaining life of bearings can not only significantly reduce equipment maintenance costs, but also effectively prevent serious consequences caused by sudden failures. Since bearings generate a large number of complex and highly nonlinear vibration signals during operation, how to extract effective features from them and make accurate predictions has become an important challenge in current research.

[0004] Remaining useful life prediction plays an important role in the field of prediction and health management, involving complex multivariate time series regression problems. In remaining useful life prediction, a key issue is how to attach importance to important features that contain more degradation information. Attention mechanism is an effective method to learn feature importance. Some researchers have proposed a temporal convolutional network with distributed attention, which has an attention mechanism for weighting the representation of sequence data. Through experimental verification, this method has high accuracy. Some researchers have also designed an improved temporal convolutional neural network with attention mechanism to solve the problem of remaining useful life prediction. The attention mechanism is introduced to help the calculation of dynamic thresholds to achieve the purpose of extracting features from different degrees of noise. However, these methods have a major disadvantage. Although a single-scale temporal convolutional network (TCN) can obtain features of different scales by adjusting parameters, it can only select one scale type at a single scale. The single-scale temporal convolutional network still has limited ability to capture temporal features of different scales, which affects the performance of remaining useful life prediction.

[0005] Therefore, how to provide a bearing life prediction method based on multi-head attention and multi-scale time convolutional network to improve the accuracy of bearing remaining service life prediction is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] The present invention provides a bearing remaining service life prediction method based on MA and MSTCN, which solves the technical problem that the existing bearing remaining service life prediction method is inaccurate.

[0007] The technical solution of the present invention to solve the above technical problem is as follows: a bearing remaining life prediction method based on MA and MSTCN, comprising:

[0008] The multi-head attention mechanism dynamically assigns weights to capture important information at different locations in the data;

[0009] Multi-scale temporal convolutional networks use convolution kernels of different scales to extract more diverse temporal features;

[0010] The specific steps are as follows:

[0011] S1, input data, i.e., vibration signals in horizontal and vertical directions;

[0012] S2, perform a one-dimensional extended causal convolution operation on the input data;

[0013] S3, perform a one-dimensional maximum pooling operation on the convolved data to reduce the data dimension and retain important features;

[0014] S4, regularize the pooled data to reduce overfitting;

[0015] S5. Perform multi-head attention operations on the regularized data to capture key features to improve the extraction efficiency of the multi-scale temporal convolutional network;

[0016] S6. Use three parallel temporal convolutional networks to extract features from the data processed by multi-head attention. Each temporal convolutional network uses different convolution kernels to extract features of different scales. Each temporal convolutional network uses sufficient pre-activation connections to output the features extracted by each temporal convolutional network respectively.

[0017] S7, perform one-dimensional global maximum pooling operation on the extracted features to reduce the time cost of model training;

[0018] S8, perform full connection operation on the pooled data;

[0019] S9. Predicted remaining service life of output bearing.

[0020] The beneficial effect of adopting the above technical solution is: first use the multi-head attention mechanism to capture important key features, and then use the multi-scale temporal convolutional network to extract multi-scale features from the data reconstructed by the multi-head attention mechanism, so as to achieve more accurate prediction in the RUL task.

[0021] Introduction to multi-head attention mechanism:

[0022] The attention mechanism is an important component of the neural network architecture. It assigns different weights to the input information for the process of feature extraction and focusing on important information. This helps the model focus on key information and reduce excessive attention to non-critical parts. Based on the attention mechanism, multi-head self-attention is designed to capture information from different subspaces, further enhancing the model's ability to model complex associations. The multi-head attention (MA) mechanism is a key technology for processing sequence data. It calculates the attention weight by treating each element in the input sequence as a query, key, and value, so that the model can capture the complex dependencies between the elements within the input sequence. This mechanism uses multiple attention heads to calculate multiple different focus points in parallel, thereby improving the model's ability to model and express input information.

[0023] Some workers have made many studies on RUL prediction using the MA mechanism. SHEN et al. proposed a network attention mechanism based on multi-head attention bidirectional long short-term memory, which improves the accuracy of bearing RUL prediction by adaptively weakening the influence of redundant information. QIN et al. proposed to use supervised multi-head self-attention autoencoder to extract health indicators to ensure the accuracy and reliability of RUL prediction. GUO et al. proposed temporal convolution with residual multi-head attention mechanism to improve the prediction performance of RUL.

[0024] Introduction to Temporal Convolutional Networks:

[0025] The single-scale temporal convolutional network (TCN) is a variant of the convolutional neural network (CNN) that aims to overcome the limitations associated with processing time series data and provides a promising alternative to traditional CNNs and recurrent neural networks (RNNs). The concept of TCN was first introduced by Bai et al., combining dilated causal convolutions and residual connections to effectively extract relevant temporal features. Traditional convolutional neural networks usually use regular convolution operations, in which the dilation factor is fixed. This means that during the convolution process, there is no gap between the convolution kernel and the mapped elements in the feature map. Causal convolution is a variant of regular convolution. The problem with causal convolution is that when processing long sequence data, large filters or deep layers are required to obtain sufficient receptive fields. To overcome this limitation, the properties of dilated convolution are introduced on the basis of causal convolution, forming dilated causal convolution, which can achieve a larger receptive field. Dilated causal convolution can increase the receptive field size of the convolution kernel without increasing the number of parameters, thereby improving the network's ability to perceive information. To alleviate the gradient vanishing and explosion problems in deep network training, the network can be trained deeper and more stably. TCN adopts residual learning, by introducing residual blocks and jump connections. The residual connection simplifies gradient propagation, allowing the construction of deep networks, effectively capturing long-term dependencies, and improving stability and performance. In response to the problem of inaccurate life prediction of rolling bearings, Qi Chenman et al. proposed a bearing RUL prediction method based on a wide temporal convolutional network with less resource consumption. Jiang Guangjun et al. proposed a bearing RUL prediction model based on a temporal convolutional network to alleviate the problem that deep neural networks cannot meet the medium and long-term prediction tasks well. In order to solve the problem of low efficiency of LSTM when processing long historical sequences, CHEN et al. proposed life prediction based on a temporal convolutional network.

[0026] Introduction to Multi-Scale Temporal Convolutional Networks:

[0027] The multi-scale temporal convolutional network (MsTCN) is a variant structure based on TCN, which introduces multi-scale convolution operations to capture features in sequences of different time scales. MsTCN consists of three parallel TCN branches. Each TCN adopts full pre-activation

[17] , placing the BN layer and ReLU activation function before the dilated causal convolution. Before the first dilated convolution, LeakyReLU is used as the activation function, which sacrifices the hard zero sparsity of the gradient and is more robust during parameter optimization

[18] . The LeakyReLU function is shown in formula (1), where a is the leakage rate, a constant in the interval [0,1].

[0028]

[0029] Each branch of TCN uses basic convolution kernels of different lengths. The lengths of the convolution kernels of the three branches are set to 1, 5, and 13 respectively to extract small-scale, medium-scale, and large-scale features from time series data. This multi-branch structure enables MsTCN to fully mine the information in the input sequence at different time scales, thereby improving the model's ability to represent time series features and its prediction performance.

[0030] Introduction to RUL prediction model:

[0031] The network input is the preprocessed data of the original vibration signals from the horizontal and vertical vibration sensors, including data normalization and label reconstruction, to ensure the quality and accuracy of the input model data. The data flows through the extended one-dimensional convolution layer, the one-dimensional maximum pooling layer, the MA layer, the MsTCN layer, the one-dimensional global maximum pooling layer and the fully connected layer, and finally outputs the predicted value of RUL. Dropout is regularization, which is used to reduce overfitting, and ReLU is the activation function.

[0032] Furthermore, in order to verify the prediction effect of the model, three indicators including mean absolute error (MAE), root mean square error (RMSE) and designed score function (Score) were used for performance evaluation.

[0033] The calculation of MAE and RMSE is shown in formula (3) and formula (4). t is the true RUL value of the rolling bearing at the tth moment; is the predicted RUL value of the rolling bearing at the tth moment; n is the number of samples in the test set.

[0034]

[0035] In actual work scenarios, underestimating the early prediction value of RUL or overestimating the later prediction value of RUL will have different degrees of impact on the operation of mechanical equipment. Underestimation may cause unnecessary downtime of mechanical equipment, while overestimation may cause equipment damage or even endanger the safety of production operators. Therefore, compared with underestimation, overestimation may have more serious consequences. In order to objectively and comprehensively evaluate the prediction model, it is necessary to consider the impact of underestimation, overestimation and different stages in the life cycle of mechanical equipment when designing the scoring function.

[0036] The scoring function in PHM2012 is improved as shown in formula (5). t is the weighted error between the predicted and actual RUL values ​​at time step t, E t ≤0 indicates overestimation (i.e., late prediction of RUL), E t ≥0 indicates an underestimate (i.e., an early prediction of RUL).

[0037]

[0038] The final model score function is shown in formula (6). Where N is the total time step, M is the percentage of the early stage in the entire stage, and α 1 and α 2 are the weights of the early and late stages respectively. In this paper, α 1 =0.3,α 2 =0.7, M=N / 2, that is, the RUL prediction value in the second half is more convincing and meaningful than that in the early period.

[0039]

[0040] The value of Score is in the range of (0,1), and the higher the value, the better the performance of the model.

[0041] Hyperparameters Introduction:

[0042] During the training of the model, the optimizer used is Adam. The dilated causal convolution layer uses L2 regularization to reduce the risk of overfitting. The global learning rate, batch size, and number of training iterations are 0.001, 128, and 500, respectively. The number of parallel heads of the multi-head attention is 8.

[0043] Introduction to the 2012 Prognostic and Health Management Dataset:

[0044] The prognostics and health management (PHM) 2012 dataset is a dataset of bearings from initial normal state to failure released by the FEMTO-ST Institute. The dataset was collected on the PRONOSTIA accelerated aging platform and contains two working conditions: working condition 1, with a speed of 1800rpm and a load of 4000N; working condition 2, with a speed of 1650rpm and a load of 4200N.

[0045] Data processing introduction:

[0046] In order to use the vibration signals in both horizontal and vertical directions in RUL prediction, the data collected every 10 seconds is classified as one sample, that is, the input shape is x∈[] 2560×2 During data preprocessing, the life percentage of each bearing is used as the output label of RUL, that is, the actual RUL of each bearing is normalized within the range of 0ˉ100%, and the output RUL label R t See formula (7) for definition. t is the true RUL value of time step t, and X is the true RUL value of the total time step.

[0047]

[0048] Effect evaluation and comparison introduction:

[0049] Experiments were conducted in two working conditions of the PHM2012 dataset. In each working condition, only one bearing was reserved for model testing, and the remaining bearings were used for model training. Similarly, 7 experiments were conducted for each working condition, and a total of 14 experiments were conducted. The values ​​of the three evaluation indicators, MAE, RMSE, and Score, were recorded for each experiment, and one prediction curve was given for each working condition.

[0050] The MAE, RMSE and Score values ​​of the seven network models, LSTM, GRU, CNN-Bi-LSTM, CNN-Bi-GRU, TCN, MsTCN and MA-MsTCN, are shown in Tables 1 and 2. According to the comparative experimental results, it can be seen that most of the performance indicators of the proposed method are better than other algorithms. Tables 1 and 2 list the performance indicators of TCN and MsTCN in the ablation experiment, the performance indicators of other mainstream prediction networks LSTM, GRU, and hybrid prediction networks CNN-Bi-LSTM, CNN-Bi-GRU, and the performance indicators of the proposed prediction model MA-MsTCN.

[0051] From the comparison of MAE, RMSE and Score results of TCN and MsTCN in Table 1 and Table 2, it can be seen that MsTCN has 9 groups of values ​​​​better than TCN in these three indicators. It can be seen that MsTCN's performance in life prediction is stronger than that of a single TCN, which also shows that MsTCN is helpful for prediction by extracting features at multiple scales. From the comparison of MAE, RMSE and Score results of MsTCN and MA-MsTCN in Table 2, it can be seen that the values ​​of MA-MsTCN in these three indicators are better than MsTCN in most cases, which shows that the decision to use MA to assist MsTCN is correct. From the comparison of MAE, RMSE and Score results of LSTM, GRU, CNN-Bi-LSTM, CNN-Bi-GRU and MA-MsTCN in Table 1 and Table 2, it can be seen that at least 10 groups of MA-MsTCN values ​​​​in these three indicators are completely better than them, which shows the powerful ability of MA-MsTCN in life prediction. In addition, by comparing with the results of existing literature, the performance indicators of the MA-MsTCN model are better than those of existing literature in most cases, which further verifies the superiority of the model.

[0052] In summary, in the 14 experiments of bearing RUL prediction, the performance of MA-MsTCN on at least 10 of these three indicators is better than that of other models, and it has good prediction performance.

[0053] Table 1 Performance comparison of different methods on the PHM2012 dataset

[0054] Tab.2Performance comparison of different methods on the PHM2012

[0055] dataset

[0056]

[0057] Table 22 Performance comparison of different methods on the PHM2012 dataset

[0058] Tab.3Performance comparison of different methods on the PHM2012

[0059] dataset

[0060] BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1A flowchart of a bearing remaining life prediction method based on MA and MSTCN according to the present invention;

[0062] Figure 2 It is a flow chart of a multi-scale time convolution network in a bearing remaining life prediction method based on MA and MSTCN in the present invention;

[0063] Figure 3 It is a first bearing remaining service life prediction curve diagram in a bearing remaining service life prediction method based on MA and MSTCN of the present invention;

[0064] Figure 4 This is a second bearing remaining service life prediction curve diagram in a bearing remaining service life prediction method based on MA and MSTCN of the present invention. DETAILED DESCRIPTION

[0065] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0066] like Figure 1 and Figure 2 As shown, a bearing remaining life prediction method based on MA and MSTCN includes:

[0067] The multi-head attention mechanism dynamically assigns weights to capture important information at different locations in the data;

[0068] Multi-scale temporal convolutional networks use convolution kernels of different scales to extract more diverse temporal features;

[0069] The specific steps are as follows:

[0070] S1. Input data;

[0071] S2, perform a one-dimensional extended causal convolution operation on the input data;

[0072] S3, perform a one-dimensional maximum pooling operation on the convolved data to reduce the data dimension and retain important features;

[0073] S4, regularize the pooled data to reduce overfitting;

[0074] S5. Perform multi-head attention operations on the regularized data to capture key features to improve the extraction efficiency of the multi-scale temporal convolutional network;

[0075] S6. Use three parallel temporal convolutional networks to extract features from the data processed by multi-head attention. Each temporal convolutional network uses different convolution kernels to extract features of different scales. Each temporal convolutional network uses sufficient pre-activation connections to output the features extracted by each temporal convolutional network respectively.

[0076] S7, perform one-dimensional global maximum pooling operation on the extracted features to reduce the time cost of model training;

[0077] S8, perform full connection operation on the pooled data;

[0078] S9, predicted remaining service life of the output bearing;

[0079] The above technical solution first uses the multi-head attention mechanism to capture important key features, and then uses a multi-scale temporal convolutional network to extract multi-scale features from the data reconstructed by the multi-head attention mechanism, thereby achieving more accurate predictions in the RUL task.

[0080] like Figure 3 and Figure 4 As shown, in some specific embodiments, the accuracy of the prediction results is verified, and the mean absolute error, root mean square error and designed score function of the predicted values ​​are used to perform comparison and ablation experiments on the Prediction and Health Management 2012 dataset; the comparative experimental results show that the proposed method is superior to the current deep learning model in RUL prediction, and the ablation experiment further verifies the powerful ability of the MA-MsTCN model in feature extraction and prediction.

[0081] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A bearing remaining life prediction method based on MA and MSTCN, characterized in that: include: The multi-head attention mechanism dynamically assigns weights to capture important information at different locations in the data; Multi-scale temporal convolutional networks use convolution kernels of different scales to extract more diverse temporal features; The specific steps are as follows: S1. Input data; S2, perform a one-dimensional extended causal convolution operation on the input data; S3, perform a one-dimensional maximum pooling operation on the convolved data to reduce the data dimension and retain important features; S4, regularize the pooled data to reduce overfitting; S5. Perform multi-head attention operations on the regularized data to capture key features to improve the extraction efficiency of the multi-scale temporal convolutional network; S6. Use three parallel temporal convolutional networks to extract features from the data processed by multi-head attention. Each temporal convolutional network uses different convolution kernels to extract features of different scales. Each temporal convolutional network uses sufficient pre-activation connections to output the features extracted by each temporal convolutional network respectively. S7, perform one-dimensional global maximum pooling operation on the extracted features to reduce the time cost of model training; S8, perform full connection operation on the pooled data; S9. Predicted remaining service life of the output bearing.

2. A bearing remaining life prediction method based on MA and MSTCN according to claim 1, characterized in that: It also includes S10. S10 verifies the accuracy of the prediction results and uses the mean absolute error, root mean square error and designed score function of the predicted values ​​to conduct comparison and ablation experiments on the Prediction and Health Management 2012 dataset.

Citation Information

Cited By

  • Method for predicting residual service life of aero-engine based on fine-grained space-time interaction graph neural network

    CN119719744A

  • Tool grinding wheel dressing device and method based on annular array and dynamic multi-scale convolution

    CN120395583A

  • Residual life monitoring method and system for vehicle hub bearing

    CN121090090A

  • Residual life prediction method based on deep residual shrinkage TCN

    CN121980376A