Bearing Remaining Life Prediction Method Based on TESAformer

Abstract

The present invention discloses a method for predicting the remaining service life of a bearing based on TESAformer. The method extracts time-frequency features from the original vibration signal and uses the Gram angle field feature structure similarity method for feature screening. The filtered features are then input into a time series trend segmentation algorithm based on information gain to divide the degradation stages of the bearing. Different penalty factors are used to filter out outliers in different degradation stages. A second feature engineering is performed on the cleaned data to obtain new time domain features and frequency domain features and corresponding labels. A bearing remaining service life prediction network is constructed, the extracted features are input into the network, and the corresponding remaining service life is output. The present invention combines the information gain time series trend segmentation method to divide the degradation stages of the bearing, and combines multi-segment data cleaning to extract high-quality features, reduce the impact of outliers on the remaining service life prediction, and improve the accuracy of the prediction task.

Description

Technical Field

[0001] The present invention relates to a life prediction method, and in particular to a bearing remaining life prediction method based on TESAformer. Background Art

[0002] With the rapid development of intelligent industry, mechanical equipment is becoming increasingly complex, integrated, and intelligent. This has significantly increased the difficulty of maintaining these machines, significantly impacting their safety and efficiency. Rolling bearings, as key components of rotating machinery, are widely used in core equipment in aerospace, transportation, wind power generation, and marine vessels. Due to the complex and harsh operating environments, bearings must maintain high precision, operate under high loads, and withstand various shocks. Therefore, bearings are one of the most susceptible components to failure in mechanical equipment. Once a bearing fails, over time, it often damages the entire mechanical system, impacting equipment efficiency and increasing repair costs. Remaining useful life prediction technology monitors and collects equipment operating data, analyzes the collected signals, and determines the condition of relevant components, simultaneously predicting their remaining useful life. This technology can transform traditional passive or planned equipment maintenance plans into proactive and predictive maintenance, significantly reducing equipment maintenance costs and improving maintenance efficiency.

[0003] At present, the remaining service life prediction methods for bearings are mainly based on two types of methods: model-based and data-driven. Among them, the model-based method requires a lot of expert knowledge when establishing the prediction model, and the model is only applicable to specific working environments, making this method difficult to be widely used; the data-driven remaining service life prediction technology for bearings is mainly based on deep learning methods. This method only requires simple preprocessing of the collected historical monitoring data, and the processed data is sent to the deep learning model for training. By mining the deep features hidden in the historical data through the model, a mapping relationship between the detection data and the remaining service life is constructed, and an efficient bearing remaining service life prediction model can be trained.

[0004] Recurrent neural networks (RNNs) and convolutional neural networks (CNNs) excel at processing time series data and have been widely used in RUL prediction. RNN variants, including long short-term memory (LSTM) and gated recurrent units (GRU), are known for capturing temporal dependencies, while CNNs excel at extracting local features through convolution operations. Recently, the Transformer model, with its self-attention (SA) mechanism, has been introduced to RUL prediction. This mechanism allows the Transformer to more effectively capture long-term dependencies by assigning weights based on the relevance of specific time steps, providing an improvement over traditional RNN and CNN methods.

[0005] Currently, the primary method for monitoring the operating condition of bearings is to use accelerometers to collect vertical and horizontal vibration data during operation. This vibration data is then analyzed to determine the bearing's operating condition and remaining useful life. However, in practice, the operating environment of bearings is often complex and variable, and the collected vibration information contains a significant amount of noise, making it difficult to accurately predict the remaining useful life of the bearing. Furthermore, the staged nature of the bearing degradation process and the complex and variable vibration signals collected during this process make it difficult for models to accurately predict the remaining useful life at different degradation stages.

[0006] Patent CN2024115401982 discloses a device fault diagnosis method and system based on Gram angular field and VAEGAN. This technical solution converts data into Gram angular field images, uses the Gram angular field images as initial samples for sample augmentation and training, that is, the Gram angular field images are input data, and the one-dimensional data is converted into two-dimensional data. For example, patent CN112016041A discloses a real-time classification method for time series data based on Gram sum angular field visualization and Shortcut-CNN, which also processes the data as the input of the model and converts the one-dimensional data into two-dimensional data. However, the above-mentioned prior art simply converts the one-dimensional data into two-dimensional image data, resulting in low quality of the extracted features, which in turn leads to low prediction accuracy. Summary of the Invention

[0007] Purpose of the invention: The purpose of the present invention is to solve the deficiencies in the prior art and provide a bearing remaining life prediction method based on TESAformer.

[0008] Technical solution: A bearing remaining life prediction method based on TESAformer of the present invention comprises the following steps:

[0009] Step 1: Collect the original vibration data, extract the time domain features and frequency domain features (the time domain features are shown in Table 1, and the frequency domain features are shown in Table 2), and normalize the time domain features and frequency domain features respectively;

[0010] Step 2: For the processed time domain features and frequency domain features, use the Gram angular field structure similarity method to perform feature screening to obtain the filtered features. The specific method is as follows:

[0011] First, a feature similar to the bearing degradation process is selected as a reference feature and normalized. After obtaining the scaled new time series X, the value is encoded as the angular cosine and the timestamp is encoded as the radius, thereby representing the new time series in polar coordinates. Next, the SSIM image structural similarity metric is calculated for the two sets of time series. The structural similarity between the Gram angle field images generated by the two features is then calculated based on the structural similarity metric. Features whose structural similarity exceeds a preset threshold are considered redundant features, and the filtered features are obtained.

[0012] Step 3: For the filtered features obtained in step 2, the bearing degradation stage is divided using the bearing degradation stage method based on information gain to obtain the degradation point index on the original data;

[0013] Step 4: For the original data degradation point index obtained in step 3, use the segmented data cleaning method to clean abnormal outliers, and perform a second feature extraction on the cleaned data. Calculate the corresponding label value for the secondary filtered features obtained in this extraction, and divide the data into training set, validation set, and test set;

[0014] Step 5: Build a bearing remaining service life prediction network based on TESAformer. Input the secondary screening features obtained in step 4 into the bearing remaining service life prediction network. The output is the remaining service life of the bearing.

[0015] The bearing remaining service life prediction network includes dimension embedding position encoding, TESA attention mechanism and predictor MLP. The dimension embedding position encoding is used to project the secondary filtered features into a high-dimensional space. The TESA attention mechanism is used to extract features, capture the correlation in the time series, and automatically capture important channel information. The predictor MLP includes multiple fully connected layers to obtain the predicted value of the remaining service life.

[0016] Step 6: Use the training set data to train the bearing remaining service life prediction network constructed in step 5, and use the loss function Update the bearing remaining service life prediction network until both the training set data and the test set data achieve good prediction results, and save the training model;

[0017] After following steps 1 to 4 for the newly collected bearing data, read the trained model to predict the life of the new bearing data.

[0018] Furthermore, the detailed method of step 2 is:

[0019] Step 2.1. Select a feature similar to the bearing degradation process as a reference feature , and normalize the reference features: During the bearing degradation process, time domain features are key indicators for monitoring degradation trends due to their simple calculation, clear physical meaning, and ability to effectively reflect the overall energy, impact characteristics, and distribution changes of the vibration signal. With degradation processes such as increased bearing wear, expanded pitting, and crack propagation, the overall energy of the vibration signal typically increases significantly. The RMS value can stably reflect this upward trend in overall energy, especially in the middle and late stages of degradation. It is one of the most commonly used and stable degradation indicators.

[0020] Here you can select the RMS value as the reference feature ;

[0021] ;

[0022] Reduce the impact of the distance between two sets of time series on subsequent similarity evaluation indicators.

[0023] Step 2.2: After obtaining the scaled new time series X, encode the value as the cosine of the angle and the timestamp as the radius to represent the new time series in polar coordinates. ;

[0024] ;

[0025] in, is the radius, Timestamp, is a constant used to normalize the span of polar coordinates;

[0026] As time goes by, the corresponding values ​​may be distorted at different corners on the span circle. To solve this problem, the sine of the angle difference in the Gram angle difference field is calculated;

[0027] ;

[0028] Where I is the unit row vector, and are different row vectors; and Over time, the same value will produce the arccosine value of the offset at different times;

[0029] Step 2.3, based on the SSIM image structural similarity metric, these changes are intuitively represented by changes in brightness, contrast, and overall image structure. Based on the above theory, quantifying the variability of two different time series according to trends is just a matter of quantifying the variability of two GADF images according to factors such as brightness, contrast, and structure. A structural similarity index measure (SSIM) for two images is proposed; the structural similarity metric defines structural information as a combination of three different factors independent of brightness, contrast, and structure. The mean is used as an estimate of brightness, the standard deviation is used as an estimate of contrast, and the covariance is used as an estimate of structural similarity. The mean of the two sets of data is calculated. , standard deviation and covariance Three sets of values;

[0030] ;

[0031] n is the sample data size, is the data value of the i-th data point;

[0032] ;

[0033] ;

[0034] Step 2.4: Calculate the structural similarity between the Gram angle field images generated by the two features. The formula is as follows:

[0035] ;

[0036] in, Represent the mean of image A and image B respectively; Represent the standard deviation of image A and image B respectively; Denote the variance of image A and image B respectively, Represent the covariance of image A and image B respectively; C1 and C2 are constants;

[0037] Features with a structural similarity index of 0.85 or above are considered redundant features. The filtered features can be obtained by deleting the redundant features.

[0038] Furthermore, the detailed method of step 3 is:

[0039] Step 3.1. A multivariate time series trend segmentation algorithm - time series segmentation based on information gain. Information gain is the expected reduction in entropy resulting from segmenting a given time series. The cost function is defined as follows:

[0040] ;

[0041] Among them, L represents information gain, k represents the number of segments, which is divided into k+1 segments. The above formula calculates the sum of the total information entropy and the weighted entropy of each segment under the current segmentation scheme, and H(S) represents the information entropy of the entire sequence, S is the total data, The data is a certain time series window;

[0042] Information entropy is defined as follows:

[0043] ;

[0044] in, represents the entropy of the jth segment, represents the probability of the feature of the jth segment, Represents the value of the qth vibration signal of the i-th feature sequence, and m represents the number of features. Traverse all data points and find the point with the largest information gain among all data points.

[0045] Step 3.2: Dynamic programming is used, but information gain and dynamic programming are incompatible. Therefore, the cost function is transformed to fit the dynamic programming framework:

[0046] .

[0047] Furthermore, the detailed method of step 4 is as follows:

[0048] Step 4.1: Use the multivariate time series trend segmentation method IGTS to divide the original vibration signal sequence X into three subsequences ,Different penalty factors are applied to each stage to filter out outliers;

[0049] Take the first subsequences No. points ;

[0050] Calculate first The local reachable density of :

[0051] ;

[0052] ;

[0053] ;

[0054] in express and k-distance (Euclidean distance), ,express and The reachable distance, express The k-proximity distance, express The local reachable density of ;

[0055] Step 4.2, calculation The local outlier factor of :

[0056] ;

[0057] in, express The local factors in the k-neighborhood. Again, the local anomaly factors are sorted in descending order and accumulated in ascending order;

[0058] Finally, find the maximum number of anomalies, ;

[0059] Where LOFthreshold represents the threshold of the local anomaly factor;

[0060] The proportion of anomalies is calculated as follows: ;

[0061] According to the given contamination value, the original data will be cleaned to varying degrees, outliers will be removed, and the deleted outliers will be replaced by the average of the surrounding 20 data points;

[0062] Obtain vibration signal data after cleaning;

[0063] Step 4.3: Extract the filtered high-quality time domain and frequency domain signals from the cleaned vibration signal. Since there are many data signal points, extract the features according to the sliding window operation and use the time window method to perform time stepping. The calculation formula is: ;

[0064] Since there are many bearing vibration data points collected, sliding window processing is performed on the collected bearing vibration data. The data points in a sliding window are a segment of data, and a feature point is extracted from the segment of data.

[0065] For example, the sampling frequency of bearing a is 25600, the sampling time is 0.1, and a total of 2803 samples are collected. That is, the 2560 data points in the first sample are regarded as a signal segment, and the time domain features are extracted.

[0066] rms = np.sqrt(np.mean(signal ** 2)), where signal has 2560 data points, and the root mean square of the 2560 data points is extracted as the first data point of the time domain feature RMS;

[0067] Normalize the extracted signal to improve data quality. The calculation formula for the RUL label corresponding to each time step is: ;

[0068] In the formula is the current time step, is the total number of time steps in the entire life cycle of the bearing, The remaining life of the current time step is the percentage of the total life cycle;

[0069] The above data is divided into training set, validation set, and test set.

[0070] Furthermore, the remaining useful life prediction network of the bearing in step 5 includes dimension embedding position encoding, TESA attention mechanism and predictor MLP:

[0071] Step 5.1: The network data is dimensionally embedded. The input data (secondary extracted features) is projected into a higher-dimensional space through dimension embedding and position encoding to capture more complex feature representations: . in, is a learnable weight matrix for embedding, and is the projection input; k is the feature dimension of the original input data. For example, if 18 features are extracted, the original shape is (n, 18). The feature dimension (n, dk) after dimension embedding is projected to a high dimension through embedding and then position encoding is performed;

[0072] Since the Transformer architecture lacks the sequential processing inherent in RNN, positional encoding is applied as follows:

[0073] ;

[0074] ;

[0075] in is the position index of the time step (ranging from 0 to n-1), and i is the dimension index (ranging from 0 to );

[0076] Use these equations to generate the position encoding matrix , each position in the sequence Each element has a unique encoding, which provides the model with information about the relative and absolute positions of the elements. The final input to the encoding layer is obtained by adding the position encoding to the projection input:

[0077] ;

[0078] Step 5.2: Linear projection is combined with convolution operations to effectively capture long-term dependencies while enhancing the ability to learn local fine-level features.

[0079] First, Q, K, V are generated from the embedding by combining linear projection and convolution;

[0080] ;

[0081] ; ;

[0082] in, represents the trainable parameters of the linear projection, Represent the weight matrix and bias of the convolution kernel respectively , c represents the number of convolution kernels, z represents the size of the convolution kernel;

[0083] Through this method, Processing information at each point in the sequence , contains the local feature change signal during the operation. In order to ensure that the input and output dimensions are consistent, zero padding is performed on the Conv2d operation; there is a complex spatial correlation between the time series of different features.

[0084] Step 5.3, the SKN unit (Selective Kernel Convolution) consists of three parts: segmentation, fusion, and selection.

[0085] First, convolution kernels of different scales are used to extract features from the input information, generating features that contain information of different scales and correspond to neurons with different receptive fields. ;

[0086] Then, the feature information generated by all neurons is fused to generate composite information U: Then compress the composite information U to generate channel statistics , ;

[0087] In addition, a compact feature z ∈ Rd×1 is created to enable accurate and adaptive selection guidance. This is achieved through a simple fully connected (fc) layer that reduces the dimensionality for improved efficiency:

[0088] ;

[0089] Where δ is the ReLU function, B represents batch normalization, , cross-channel soft attention is used to adaptively select information at different spatial scales, which is guided by the compact feature descriptor z; specifically, the softmax operator is applied to the channel number:

[0090] ,

[0091] ;

[0092] in , a, b represent the and Soft attention vector of ;

[0093] Notice, is the c-th row of A, ac is the c-th element of a. In the case of two branches, the matrix B is redundant because ac + bc = 1, and the final feature map V is obtained by the attention weights on various kernels:

[0094] ;

[0095] ;

[0096] final, , ;

[0097] in, ,The model includes the local variation features of bearing degradation and the spatial correlation features between different features, which effectively enhances the feature expression ability of the model. Then, for each element in the Q matrix, the matrix Q and the matrix The dot product of is used to calculate the attention score of the current position and other local fields to determine the relevance with the local field; the softmax function is used to obtain the weights of different local fields relative to the current position along the time dimension, as shown below:

[0098] ;

[0099] in, Represents the learned attention weights for the current position with local fields.

[0100] Finally, the output of TESA is calculated as and The product is as follows:

[0101] ;

[0102] in, is the output of TESA, which is connected to the input , the dimensions of TESA are the same. In addition, TESAformer integrates a multi-head mechanism, which can perform more comprehensive analysis of data and greatly enhance its feature extraction capabilities; the process of multi-head TESA is as follows:

[0103] ;

[0104] in, It is the output of multi-head TESA, represents the linear projection matrix parameters, h is the number of heads, and the output of each head is:

[0105] ;

[0106] The encoding layers are evenly stacked into layers, enabling the model to obtain deep features;

[0107] These extracted features will be used as input to the output layer, whose task is to predict RUL;

[0108] Step 5.4: The structure of the output layer includes a flattening layer before the two FCN layers. The calculations within these layers are performed as follows: ;

[0109] in, is the RUL prediction result, is the activation function, and is a learnable parameter matrix, and the predicted remaining useful life is obtained from the above formula.

[0110] Furthermore, the detailed method of step 6 is:

[0111] Step 6.1: Input the training bearing data samples into the remaining service life prediction model in a certain batch size and perform forward reasoning. The final fully connected output is the predicted value of the sample. ;

[0112] Step 6.2: MSE loss function calculates the model prediction output Lifespan label corresponding to the sample The loss between them, the MSE calculation formula is as follows:

[0113] ;

[0114] Where, is the batch size, is the model’s predicted output value of the remaining useful life of the i-th sample, is the actual value of the remaining useful life of the i-th sample;

[0115] Through the loss function Update the model until it achieves good prediction results on both the training set data and the test set data in the source domain, and save the trained model.

[0116] Step 6.3: After performing the above steps S1-S4 on the newly collected bearing vibration data, the trained model can be input and the optimal model data can be read to predict the remaining service life of the bearing.

[0117] Beneficial effects: Compared with the prior art, the present invention has the following technical effects:

[0118] (1) This paper proposes a multi-domain feature selection method based on time series shape similarity. Unlike existing distance and correlation-based metrics, this method converts multi-domain features into two-dimensional graphs and establishes a structural similarity index between images to select effective features, effectively achieving large-scale time series similarity selection tasks.

[0119] (2) This paper proposes a segmented data cleaning method for signals. Unlike existing preprocessing methods, this method achieves segmented data cleaning based on information entropy gain and local outlier factors. A global outlier factor is introduced for unsupervised cleaning, effectively reducing the imbalance in the distribution density of outliers at different degradation stages.

[0120] (3) The proposed bearing remaining service life prediction network TESAformer combines the advantages of linear projection and convolution operations, replacing the traditional SA mechanism. This allows the model to effectively capture long-term dependencies and local context information. TESA enhances the model's ability to extract local degradation features while maintaining the ability to model global temporal relationships. ECSA dynamically captures the spatial correlation between different sensor channels. This enables the model to adaptively weight the importance of different channels, thereby improving multi-feature data integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 The overall flow chart predicted for the present invention;

[0122] Figure 2 Schematic diagram of the structure of the bearing remaining service life prediction network TESAformer of the present invention;

[0123] Figure 3 It is a prediction effect diagram of an embodiment;

[0124] Figure 4 A comparison chart of the prediction effects of the present invention and the existing model;

[0125] Figure 5 This is a comparison chart of the RMSE indicators of the present invention and the existing model;

[0126] Figure 6Comparison chart of MAE indicators between the present invention and the existing model

[0127] Figure 7 The present invention and the existing model Indicator comparison chart

[0128] Figure 8 This is a comparison chart of the MSE indicators of the present invention and the existing model. DETAILED DESCRIPTION

[0129] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments.

[0130] This paper uses the Gram angle field as a tool to compare Gram angle field images generated by different feature conversions, filtering out identical features with high similarity and avoiding redundant features from entering the model, which can lead to overfitting. Through feature selection, redundant features are eliminated, improving the model's generalization ability. The shape-based time series similarity screening algorithm proposed in this paper helps effectively filter out the original feature set in both the time and frequency domains.

[0131] Table 1 Time domain characteristics

[0132]

[0133] Table 2 Frequency domain characteristics

[0134]

[0135] like Figure 1 As shown, the bearing remaining life prediction method based on TESAformer of the present invention includes the following steps:

[0136] Step 1: Collect the original vibration data, extract the time domain features and frequency domain features (the time domain features are shown in Table 1, and the frequency domain features are shown in Table 2), and normalize the time domain features and frequency domain features respectively;

[0137] Step 2: For the processed time domain features and frequency domain features, use the Gram angular field structure similarity method to perform feature screening to obtain the filtered features. The specific method is as follows:

[0138] First, a feature similar to the bearing degradation process is selected as a reference feature and normalized. After obtaining the scaled new time series X, the value is encoded as the angular cosine and the timestamp is encoded as the radius, thereby representing the new time series in polar coordinates. Next, the SSIM image structural similarity metric is calculated for the two sets of time series. The structural similarity between the Gram angle field images generated by the two features is then calculated based on the structural similarity metric. Features whose structural similarity exceeds a preset threshold are considered redundant features, and the filtered features are obtained.

[0139] Step 3: For the filtered features obtained in step 2, the bearing degradation stage is divided using the bearing degradation stage method based on information gain to obtain the degradation point index on the original data;

[0140] Step 4: For the original data degradation point index obtained in step 3, use the segmented data cleaning method to clean abnormal outliers, and perform a second feature extraction on the cleaned data. Calculate the corresponding label value for the secondary filtered features obtained in this extraction, and divide the data into training set, validation set, and test set;

[0141] Step 5: Build a bearing remaining service life prediction network based on TESAformer. Input the secondary screening features obtained in step 4 into the bearing remaining service life prediction network. The output is the remaining service life of the bearing.

[0142] The bearing remaining service life prediction network includes dimension embedding position encoding, TESA attention mechanism and predictor MLP. The dimension embedding position encoding is used to project the secondary filtered features into a high-dimensional space. The TESA attention mechanism is used to extract features, capture the correlation in the time series, and automatically capture important information in the channel. The predictor MLP includes multiple fully connected layers to obtain the predicted value of the remaining service life.

[0143] Step 6: Use the training set data to train the bearing remaining service life prediction network constructed in step 5, and use the loss function Update the remaining useful life prediction network for the bearing until both the training set data and the test set data achieve good prediction results, and save the training model. Follow steps 1 to 4 for the newly collected bearing data, then read the trained model to predict the life of the new bearing data.

[0144] The detailed method of step 2 of this embodiment is:

[0145] Step 2.1. Select a feature similar to the bearing degradation process as a reference feature , and normalize the reference features: During the bearing degradation process, time domain features are key indicators for monitoring degradation trends due to their simple calculation, clear physical meaning, and ability to effectively reflect the overall energy, impact characteristics, and distribution changes of the vibration signal. With degradation processes such as increased bearing wear, expanded pitting, and crack propagation, the overall energy of the vibration signal typically increases significantly. The RMS value can stably reflect this upward trend in overall energy, especially in the middle and late stages of degradation. It is one of the most commonly used and stable degradation indicators.

[0146] Here you can select the RMS value as the reference feature ;

[0147] ;

[0148] Reduce the impact of the distance between two sets of time series on subsequent similarity evaluation indicators.

[0149] Step 2.2: After obtaining the scaled new time series X, encode the value as the cosine of the angle and the timestamp as the radius to represent the new time series in polar coordinates. ;

[0150] ;

[0151] in, is the radius, Timestamp, is a constant used to normalize the span of polar coordinates;

[0152] As time goes by, the corresponding values ​​may be distorted at different corners on the span circle. To solve this problem, the sine of the angle difference in the Gram angle difference field is calculated;

[0153] ;

[0154] Where I is the unit row vector, and are different row vectors; and Over time, the same value will produce the arccosine value of the offset at different times;

[0155] Step 2.3, based on the SSIM image structural similarity metric, these changes are intuitively represented by changes in brightness, contrast, and overall image structure. Based on the above theory, quantifying the variability of two different time series according to trends is just a matter of quantifying the variability of two GADF images according to factors such as brightness, contrast, and structure. A structural similarity index measure (SSIM) for two images is proposed; the structural similarity metric defines structural information as a combination of three different factors independent of brightness, contrast, and structure. The mean is used as an estimate of brightness, the standard deviation is used as an estimate of contrast, and the covariance is used as an estimate of structural similarity. The mean of the two sets of data is calculated. , standard deviation and covariance Three sets of values;

[0156] ;

[0157] n is the sample data size, is the data value of the i-th data point;

[0158] ;

[0159] ;

[0160] Step 2.4: Calculate the structural similarity between the Gram angle field images generated by the two features. The formula is as follows:

[0161] ;

[0162] in, Represent the mean of image A and image B respectively; Represent the standard deviation of image A and image B respectively; Denote the variance of image A and image B respectively, Represent the covariance of image A and image B respectively; C1 and C2 are constants;

[0163] Features with a structural similarity index of 0.85 or above are considered redundant features. The filtered features can be obtained by deleting the redundant features.

[0164] The detailed method of step 3 of this embodiment is:

[0165] Step 3.1. A multivariate time series trend segmentation algorithm - time series segmentation based on information gain. Information gain is the expected reduction in entropy resulting from segmenting a given time series. The cost function is defined as follows:

[0166] ;

[0167] Among them, L represents information gain, k represents the number of segments, which is divided into k+1 segments. The above formula calculates the sum of the total information entropy and the weighted entropy of each segment under the current segmentation scheme, and H(S) represents the information entropy of the entire sequence, S is the total data, The data is a certain time series window;

[0168] Information entropy is defined as follows:

[0169] ;

[0170] ;

[0171] in, represents the entropy of the jth segment, represents the probability of the feature of the jth segment, Represents the value of the qth vibration signal of the i-th feature sequence, and m represents the number of features. Traverse all data points and find the point with the largest information gain among all data points.

[0172] Step 3.2: Dynamic programming is used, but information gain and dynamic programming are incompatible. Therefore, the cost function is transformed to fit the dynamic programming framework:

[0173] .

[0174] The detailed method of step 4 of this embodiment is:

[0175] Step 4.1: Use the multivariate time series trend segmentation method IGTS to divide the original vibration signal sequence X into three subsequences ,Different penalty factors are applied in each stage to filter out outliers;

[0176] Take the first subsequences No. points ;

[0177] Calculate first The local reachable density of :

[0178] ;

[0179] ;

[0180] ;

[0181] in express and k-distance (Euclidean distance), ,express and The reachable distance, express The k-proximity distance, express The local reachable density of ;

[0182] Step 4.2, calculation The local outlier factor of :

[0183] ;

[0184] in, express The local factors in the k-neighborhood. Again, the local anomaly factors are sorted in descending order and accumulated in ascending order;

[0185] Finally, find the maximum number of anomalies, ;

[0186] Where LOFthreshold represents the threshold of the local anomaly factor;

[0187] The proportion of anomalies is calculated as follows: ;

[0188] According to the given contamination value, the original data will be cleaned to varying degrees, outliers will be removed, and the deleted outliers will be replaced by the average of the surrounding 20 data points;

[0189] Obtain vibration signal data after cleaning;

[0190] Step 4.3: Extract the filtered high-quality time domain and frequency domain signals from the cleaned vibration signal. Since there are many data signal points, extract the features according to the sliding window operation and use the time window method to perform time stepping. The calculation formula is: ;

[0191] Since there are many bearing vibration data points collected, sliding window processing is performed on the collected bearing vibration data. The data points in a sliding window are a segment of data, and a feature point is extracted from the segment of data.

[0192] For example, the sampling frequency of bearing a is 25600, the sampling time is 0.1, and a total of 2803 samples are collected. That is, the 2560 data points in the first sample are regarded as a signal segment, and the time domain features are extracted.

[0193] rms = np.sqrt(np.mean(signal ** 2)), where signal has 2560 data points, and the root mean square of the 2560 data points is extracted as the first data point of the time domain feature RMS;

[0194] Normalize the extracted signal to improve data quality. The calculation formula for the RUL label corresponding to each time step is: ;

[0195] In the formula is the current time step, is the total number of time steps in the entire life cycle of the bearing, The remaining life of the current time step is the percentage of the total life cycle;

[0196] The above data is divided into training set, validation set, and test set.

[0197] like Figure 2 As shown in the figure, the left side is the data preprocessing module, the feature extraction module is the transformer structure composed of the encoder in the figure, and the predictor is equivalent to the RUL prediction module, that is, the output_layer in the figure. The detailed method of the bearing remaining service life prediction network based on TESAformer in this embodiment is as follows:

[0198] Step 5.1: The network's data is dimensionally embedded, projecting the input data (secondary extracted features) into a higher-dimensional space to capture more complex feature representations: . in, is a learnable weight matrix for embedding, and is the projection input; k is the feature dimension of the original input data. For example, if 18 features are extracted, the original shape is (n, 18). The feature dimension (n, dk) after dimension embedding is projected to a high dimension through embedding and then position encoding is performed;

[0199] Since the Transformer architecture lacks the sequential processing inherent in RNN, positional encoding is applied as follows:

[0200] ;

[0201] ;

[0202] in is the position index of the time step (ranging from 0 to n-1), and i is the dimension index (ranging from 0 to );

[0203] Use these equations to generate the position encoding matrix , each position in the sequence Each element has a unique encoding, which provides the model with information about the relative and absolute positions of the elements. The final input to the encoding layer is obtained by adding the position encoding to the projection input:

[0204] ;

[0205] Step 5.2: Linear projection is combined with convolution operations to effectively capture long-term dependencies while enhancing the ability to learn local fine-level features.

[0206] First, Q, K, V are generated from the embedding by combining linear projection and convolution;

[0207]

[0208] in, represents the trainable parameters of the linear projection, Represent the weight matrix and bias of the convolution kernel respectively , c represents the number of convolution kernels, z represents the size of the convolution kernel;

[0209] Through this method, Processing information at each point in the sequence , contains the local feature change signal during the operation. In order to ensure that the input and output dimensions are consistent, zero padding is performed on the Conv2d operation; there is a complex spatial correlation between the time series of different features.

[0210] Step 5.3, the SKN unit (Selective Kernel Convolution) consists of three parts: segmentation, fusion, and selection.

[0211] First, convolution kernels of different scales are used to extract features from the input information, generating features that contain information of different scales and correspond to neurons with different receptive fields. ;

[0212] Then, the feature information generated by all neurons is fused to generate composite information U: Then compress the composite information U to generate channel statistics , ;

[0213] In addition, a compact feature z ∈ Rd×1 is created to enable accurate and adaptive selection guidance. This is achieved through a simple fully connected (fc) layer that reduces the dimensionality for improved efficiency:

[0214] ;

[0215] Where δ is the ReLU function, B represents batch normalization, , cross-channel soft attention is used to adaptively select information at different spatial scales, which is guided by the compact feature descriptor z; specifically, the softmax operator is applied to the channel number: ; ;

[0216] in , a, b represent the and Soft attention vector of ;

[0217] Notice, is the c-th row of A, ac is the c-th element of a. In the case of two branches, the matrix B is redundant because ac + bc = 1, and the final feature map V is obtained by the attention weights on various kernels:

[0218] ;

[0219] ;

[0220] final, , ;

[0221] in, ,The model includes the local variation features of bearing degradation and the spatial correlation features between different features, which effectively enhances the feature expression ability of the model. Then, for each element in the Q matrix, the matrix Q and the matrix The dot product of is used to calculate the attention score of the current position and other local fields to determine the relevance with the local field; the softmax function is used to obtain the weights of different local fields relative to the current position along the time dimension, as shown below:

[0222] ;

[0223] in, represents the learned attention weight of the current position with the local field. Finally, the output of TESA is calculated as and The product is as follows:

[0224] ;

[0225] in, is the output of TESA, which is connected to the input , the dimensions of TESA are the same. In addition, TESAformer integrates a multi-head mechanism, which can perform more comprehensive analysis of data and greatly enhance its feature extraction capabilities; the process of multi-head TESA is as follows:

[0226] ;

[0227] in, It is the output of multi-head TESA, represents the linear projection matrix parameters, h is the number of heads, and the output of each head is:

[0228] ;

[0229] The encoding layers are evenly stacked into layers, enabling the model to obtain deep features;

[0230] These extracted features will be used as input to the output layer, whose task is to predict RUL;

[0231] Step 5.4: The structure of the output layer includes a flattening layer before the two FCN layers. The calculations within these layers are performed as follows: ;

[0232] in, is the RUL prediction result, is the activation function, and is a learnable parameter matrix, and the predicted remaining useful life is obtained from the above formula.

[0233] The detailed method of step 6 of this embodiment is:

[0234] Step 6.1: Input the training bearing data samples into the remaining service life prediction model in a certain batch size and perform forward reasoning. The final fully connected output is the predicted value of the sample. ;

[0235] Step 6.2: MSE loss function calculates the model prediction output Lifespan label corresponding to the sample The loss between them, the MSE calculation formula is as follows:

[0236] ;

[0237] Where, is the batch size, is the model’s predicted output value of the remaining useful life of the i-th sample, is the actual value of the remaining useful life of the i-th sample;

[0238] Through the loss function Update the model until it achieves good prediction results on both the training set data and the test set data in the source domain, and save the trained model.

[0239] The technical solution of the present invention does not simply convert one-dimensional data into two-dimensional image data, but uses the Gram angular field and information gain to screen features, clean the data segments, reduce the outliers caused by bearing degradation at different stages, extract high-quality features from the cleaned data, and input them into subsequent models to improve the model prediction accuracy.

[0240] To further verify the technical effect of the present invention, this embodiment applies the present invention to the bearing data in Table 1 and Table 2 to predict the remaining life. The results are as follows: Figure 3 As shown in the figure, the oblique line is the actual value, and the broken line is the predicted value of the present invention. The two are very close, indicating that the present invention can combine the information gain time series trend segmentation method to divide the degradation stage of the bearing, combine multi-segment data cleaning, extract high-quality features, reduce the impact of outliers on the remaining service life prediction, and improve the accuracy of the prediction task.

[0241] Table 3 Data comparison between the present invention and the existing solution

[0242]

[0243] This embodiment uses the same data processing method to compare multiple models, as shown in Table 3, Figures 4 to 8The results show that the same sliding window, learning rate and learning rate decay, the same training data, data set partitioning, similar epoch and other hyperparameters, and the same output layer are used for training. It is found that after data processing, the number of parameters of this model is at a medium level among various models, but the fitting effect is the best.

Claims

1. A bearing remaining life prediction method based on TESAformer, characterized in that: The following steps are involved: Step 1: Collect original vibration data, extract time domain features and frequency domain features, and normalize the time domain features and frequency domain features respectively; Step 2: For the processed time domain features and frequency domain features, use the Gram angular field structure similarity method to perform feature screening to obtain the filtered features. The specific method is as follows: First, a feature similar to the bearing degradation process is selected as a reference feature and normalized. After obtaining a scaled new time series X, the value is encoded as the angular cosine and the timestamp is encoded as the radius, thereby representing the new time series in polar coordinates. Next, the SSIM image structural similarity metric is calculated for the two sets of time series, and the structural similarity between the Gram angle field images generated by the two features is calculated. Features whose structural similarity exceeds a preset threshold are considered redundant features, and the filtered features are obtained. Step 3: For the filtered features obtained in step 2, the bearing degradation stage is divided using the bearing degradation stage method based on information gain to obtain the degradation point index on the original data; Step 4: For the original data degradation point index obtained in step 3, use the segmented data cleaning method to clean abnormal outliers, and perform a second feature extraction on the cleaned data. Calculate the corresponding label value for the secondary filtered features obtained in this extraction, and divide the data into training set, validation set, and test set; Step 5: Build a bearing remaining service life prediction network based on TESAformer. Input the secondary screening features obtained in step 4 into the bearing remaining service life prediction network. The output is the remaining service life of the bearing. The bearing remaining service life prediction network includes dimension embedding position coding, TESA attention mechanism and predictor MLP. The dimension embedding position coding is used to project the secondary filtered features into a high-dimensional space. The TESA attention mechanism is used to extract features. The predictor MLP includes multiple fully connected layers to obtain the predicted value of the remaining service life. Step 6: Use the training set data to train the bearing remaining service life prediction network constructed in step 5, and use the loss function Update the bearing remaining service life prediction network.

2. The method for predicting remaining life of a bearing based on TESAformer according to claim 1, characterized in that: The detailed method of step 2 is: Step 2.

1. Select a feature similar to the bearing degradation process as a reference feature , and normalize the reference features to obtain : ； Step 2.2: Encode the value as the cosine of the angle and the timestamp as the radius to represent the new time series in polar coordinates. ; ； in, is the radius, timestamp, is a constant used to normalize the span of polar coordinates; As time goes by, the corresponding values ​​will be distorted at different corners on the span circle. To solve this problem, the sine of the angle difference in the Gram angle difference field is calculated; ； Where I is the unit row vector, and are different row vectors; Step 2.3, use the mean as an estimate of brightness, the standard deviation as an estimate of contrast, and the covariance as an estimate of structural similarity to calculate the mean of the two images. , standard deviation and covariance Three sets of values; ； n is the sample data size, is the data value of the i-th data point; ； ； Step 2.4: Based on the average value , standard deviation and covariance , calculate the structural similarity measure SSIM between the Gram angle field images generated by two features, the formula is as follows: ； in, Represent the mean of image A and image B respectively; Represent the standard deviation of image A and image B respectively, Denote the variance of image A and image B respectively, Represent the covariance of image A and image B respectively; C1 and C2 are constants; Features with a structural similarity metric of 0.85 or above are considered redundant features, and the filtered features can be obtained by deleting the redundant features.

3. The method for predicting remaining life of a bearing based on TESAformer according to claim 1, characterized in that: The detailed method of step 3 is: Step 3.1: Segment the time series based on information gain L. First, calculate the sum of the total information entropy and the weighted entropy of each segment under the current segmentation scheme. The formula is as follows: ； Among them, k represents the number of segments, which is divided into k+1 segments, H(S) represents the information entropy of the entire sequence, and S is the total data. The data is a certain time series window; Then calculate the information entropy, the formula is as follows: ； in, represents the entropy of the jth segment, represents the probability of the feature of the jth segment, represents the value of the qth vibration signal of the i-th feature sequence, and m represents the number of features. Traverse all data points and find the point with the largest information gain among all data points; Step 3.2: Use dynamic programming method to transform the information gain L. The formula is as follows: 。 4. The method for predicting remaining life of a bearing based on TESAformer according to claim 1, characterized in that: The detailed method of step 4 is as follows: Step 4.1: Use the trend segmentation method IGTS of multivariate time series to divide the original vibration signal sequence X into three subsequences ,Different penalty factors are applied in each stage to filter out outliers; Take the first subsequences No. points ; Calculate first The local reachable density of : ； ； ； in express and k-distance, express and The reachable distance, express The k-proximity distance, express The local reachable density of ; Step 4.2, calculation The local outlier factor of : ； in, express Local factors in the k-neighborhood; Again, the local anomaly factors are sorted in descending order and accumulated in ascending order; Finally, find the maximum number of anomalies, ; Where LOFthreshold represents the threshold of the local outlier factor; The proportion of anomalies is calculated as follows: ; According to the given contamination value, the original data will be cleaned to varying degrees, outliers will be removed, and the deleted outliers will be replaced by the average of the surrounding 20 data points; Obtain vibration signal data after cleaning; Step 4.3: Extract features from the cleaned vibration signal using the sliding window operation and use the time window method for time stepping. The calculation formula is: ; Normalize the extracted signal and calculate the RUL label corresponding to each time step as follows: ; In the formula is the current time step, is the total number of time steps over the entire life cycle of the bearing, The remaining life of the current time step is the percentage of the total life cycle; The above data is divided into training set, validation set, and test set.

5. The method for predicting remaining life of a bearing based on TESAformer according to claim 1, characterized in that: The remaining useful life prediction network of the bearing in step 5 includes dimension embedding position encoding, TESA attention mechanism and predictor MLP; Step 5.1: Project the input secondary filtered features into high-dimensional space through dimension embedding and position encoding to obtain feature representation : , in, refers to the learnable weight matrix used for embedding, and is the projection input; k is the feature dimension of the original input data, The feature dimension after dimension embedding is projected to a high dimension through embedding and then position encoding is performed; Then perform position encoding to generate a position encoding matrix ,as follows: ； ； in is the position index of the time step, i is the dimension index; each position in the sequence Each has a unique encoding, and the final input to the encoding layer is obtained by adding the positional encoding to the projection input: ； Step 5.2: Linear projection is combined with convolution operations to effectively capture long-term dependencies while enhancing the ability to learn local fine-level features. First, Q, K, V are generated from the embedding by combining linear projection and convolution; ； ； ； in, represents the trainable parameters of the linear projection, Represent the weight matrix and bias of the convolution kernel respectively , c represents the number of convolution kernels, z represents the size of the convolution kernel; Step 5.3, Selective Convolution Module The SKN unit consists of three parts: segmentation, fusion, and selection; First, convolution kernels of different scales are used to extract features from the input information, generating features that contain information of different scales and correspond to neurons with different receptive fields. ; Then, the feature information generated by all neurons is fused to generate composite information U: Then compress the composite information U to generate channel statistics , ; In addition, create a compact feature z ∈ Rd×1, ; δ is the ReLU function, B represents batch normalization, ; ， ； in , a, b represent the and The soft attention vector of and is the corresponding soft attention weight; The final output feature map V of the selective convolution module is obtained by the attention weights on various kernels: ； ； final, , ; ; Then, for each element in the Q matrix, by using the matrix Q and the matrix The dot product of is used to calculate the attention score of the current position and other local fields, and the softmax function is used to obtain the weights of different local fields relative to the current position along the time dimension, as shown below: ； in, Represents the learned attention weight of the current position with the local field; Finally, the output of TESA is calculated as and The product is as follows: ； in, is the output of TESA, which is connected to the input , the dimensions of TESA are the same; In addition, the process of long TESA is as follows: ； in, It is the output of multi-head TESA, represents the linear projection matrix parameters, h is the number of heads, and the output of each head for: ； Step 5.4: Output layer (RUL prediction module) outputs the final predicted lifespan , The calculation formula is as follows: ； in, is the RUL predicted lifespan, is the activation function, and is a learnable parameter matrix.

6. The method for predicting remaining life of a bearing based on TESAformer according to claim 1, characterized in that: The detailed method of step 6 is: Step 6.1: Input the training bearing data samples into the bearing remaining service life prediction network according to the corresponding batch size, perform forward reasoning, and the final fully connected output is the predicted value of the sample ; Step 6.2: Use the MSE loss function to calculate the model prediction output Lifespan label corresponding to the sample The loss between them, the MSE calculation formula is as follows: ； Where, is the batch size, is the model’s predicted output value of the remaining useful life of the i-th sample, is the actual value of the remaining useful life of the i-th sample; Through the loss function Update the model until it achieves prediction results on both the training set data and the test set data in the source domain, and save the trained model. Step 6.3: After performing the above steps S1-S4 on the newly collected bearing vibration data, the trained model can be input and the optimal model data can be read to predict the remaining service life of the bearing.

Images