Multi-mode bearing residual life prediction method and system based on balance optimization

Abstract

The invention discloses a multi-mode bearing residual life prediction method and system based on balance optimization. The method comprises the following steps: firstly, in an offline stage, acquiring time sequence data of a vibration signal from a bearing operation data set, and generating a time-frequency image through wavelet transform to ensure that the data comprises two modes of a time sequence and an image; secondly, extracting time sequence features of the time sequence data by using a long short-term memory (LSTM) network, and extracting spatial features of a time-frequency image by using a graph convolutional network (GCN); then, features extracted by the LSTM and the GCN are spliced to generate a joint feature vector, and the joint feature vector is input into a full connection layer to output an RUL predicted value; in the training process, the loss contribution difference of the LSTM mode and the GCN mode is monitored in real time through an adaptive optimization strategy, the gradient update amplitude of each mode is adjusted through back propagation, the optimization of the dominant mode inhibiting the vulnerable mode is avoided, and the full extraction of the multi-mode features is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of remaining useful life (RUL) prediction of mechanical equipment, and in particular to a bearing RUL prediction method that combines time series and image data for multimodal feature fusion and adopts a gradient adaptive optimization strategy. Background Art

[0002] With the rapid development of industrial technology, bearing health monitoring and remaining useful life (RUL) prediction are crucial for ensuring safe equipment operation, as key components in mechanical systems. Currently, data-driven approaches have attracted widespread attention in the field of RUL prediction. However, most existing methods rely primarily on single-modal data or fail to fully consider the imbalance in learning features from different modalities when fusing multimodal data, resulting in limited prediction performance.

[0003] Current RUL prediction methods are mainly divided into physical model-based methods and data-driven methods. Physical models rely on mathematical modeling, such as the Wiener process, particle filtering, Kalman filtering, and hidden Markov models. However, due to the complexity of the bearing degradation process, accurately establishing a physical model is a significant challenge. Data-driven methods do not require professional knowledge and have gradually become mainstream in recent years with the rise of deep learning. For example, time series feature extraction methods based on deep long short-term memory (LSTM) networks and feature extraction methods based on convolutional neural networks (CNNs) have demonstrated excellent performance in RUL prediction.

[0004] However, in multimodal learning, a single modality may dominate model optimization, limiting feature learning in another modality and thus affecting overall performance. Therefore, effectively integrating time series and image data while optimizing the learning weights of each modality is a key challenge in the current RUL prediction field. Summary of the Invention

[0005] To address the challenges of existing technologies, this paper provides a bearing remaining useful life (RUL) prediction method and system based on multimodal adaptive optimization. This approach aims to address the limitations of single-modal data, imbalanced optimization across modalities, and insufficient prediction accuracy in existing technologies. By combining the multimodal features of time series data and time-frequency images and employing an adaptive optimization strategy, this method significantly improves the accuracy and generalization of bearing RUL prediction.

[0006] The technical solution of the present invention is a method for predicting the remaining service life of a bearing based on multi-modal adaptive optimization, which includes the following steps:

[0007] Step 1: Obtain the time series data of the vibration signal from the bearing operation data set and generate a time-frequency image through wavelet transform;

[0008] Step 2: In terms of temporal modality, the long short-term memory network (LSTM) is used to extract the temporal features of the time series data; in terms of spatial modality, the graph convolutional network (GCN) is used to extract the spatial features of the time-frequency image.

[0009] Step 3: Concatenate the extracted temporal features and spatial features along the feature dimension to generate a joint feature vector, which is then fed into the fully connected layer to output the predicted remaining useful life (RUL) of the bearing.

[0010] Step 4: Use an adaptive optimization strategy to monitor the difference in loss contributions between LSTM and GCN in real time, dynamically adjust the gradient update amplitude of each modality, and obtain the model weight;

[0011] Step 5: Combine the model weights and loss function to train the bearing remaining life prediction model constructed by steps 2-3;

[0012] Step 6: Use the trained model to predict the remaining life of the bearing.

[0013] Furthermore, the time series data of the vibration signal is input into the LSTM network, and the timing features are extracted through multi-layer LSTM units. Each LSTM unit controls the information flow through the input gate, forget gate and output gate.

[0014] Furthermore, each layer of GCN updates node information through matrix multiplication, and uses multi-layer GCN to extract the spatial features of the time-frequency image. The output of each layer is used as the input of the next layer, and finally the spatial feature vector is output. , the formula is as follows:

[0015]

[0016] in, is an adjacency matrix with self-loops, is the degree matrix, is a learnable parameter, is the input feature matrix.

[0017] Furthermore, the specific implementation of step 4 includes the following sub-steps:

[0018] (4.1) Calculate the loss values ​​of the time mode and the space mode respectively. Based on the loss values ​​of the two modes, calculate the difference ratio of the time mode to the space mode and the difference ratio of the space mode to the time mode.

[0019] (4.2) Calculate the dynamic adjustment coefficient of each mode based on the difference ratio ;

[0020] (4.3) In each iteration, the dynamic adjustment coefficient is used Adjust the gradient update amplitude of each mode to obtain the model weight.

[0021] Furthermore, for each mini-batch of data , calculate the loss values ​​of time mode and space mode respectively:

[0022]

[0023]

[0024] in, and are the input of time series data and time-frequency images respectively, is the loss value of the time mode, is the loss value of the spatial mode, It is a real RUL label. Indicates that time series data Input the LSTM model and output the predicted value of RUL. Indicates that the image data Input the GCN model and output the predicted value of RUL. criterion() is the loss function, using MSE mean square error.

[0025] Furthermore, based on the loss values ​​of the two modalities, the difference ratio of the time modality to the spatial modality is calculated. , the specific formula is as follows:

[0026]

[0027] The difference ratio of the spatial mode to the temporal mode Then The reciprocal of , the specific formula is as follows:

[0028]

[0029] in, is a constant used to prevent the denominator from being zero.

[0030] Furthermore, the dynamic adjustment coefficient The specific formula is as follows:

[0031]

[0032] in, is a hyperparameter that controls the intensity of gradient regulation. Represents the difference ratio.

[0033] Furthermore, the calculation formula for the gradient update amplitude is as follows:

[0034]

[0035] in, is a mini-batch of data randomly sampled in the t-th iteration, is a sample in a small batch of data, Use the current model parameters On the sample The calculated loss value, is the gradient of the loss value with respect to the parameter, is the average of all sample gradients;

[0036]

[0037] in, is the learning rate, represents the parameter vector of the u-th mode at the t-th iteration, i.e., the model weight, is the dynamic adjustment coefficient of the u-th mode in the t-th iteration, and u takes the values ​​of 1 and 2, where 1 represents the time mode and 2 represents the space mode.

[0038] Furthermore, the mean square error is used as the loss function , measures the difference between the predicted value and the true value. The specific formula is as follows:

[0039]

[0040] in, is the real RUL value, To predict the RUL value, N is the number of samples.

[0041] The present invention also provides a multi-modal bearing remaining life prediction system based on balance optimization, comprising:

[0042] one or more processors;

[0043] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the multi-modal bearing remaining life prediction method based on balance optimization as described in the above technical solution.

[0044] This method extracts time series data of vibration signals from a bearing operating dataset and generates time-frequency images through wavelet transform, ensuring that the data contains both time series and image modalities. The time series data captures the degradation trend of the bearing, while the time-frequency images provide the spatial characteristics of the vibration signals in the frequency domain. The combination of the two can more comprehensively characterize the health status of the bearing.

[0045] A long short-term memory (LSTM) network is used to extract the time series features of the vibration signal. LSTM controls information flow through input, forget, and output gates to avoid the vanishing gradient problem and effectively capture the long-term dependencies of bearing degradation. A graph convolutional network (GCN) is used to extract the spatial features of the time-frequency image. GCN transfers information through adjacency and degree matrices, capturing spatial dependencies in the image and effectively characterizing the local characteristics of bearing faults.

[0046] In addition, by real-time monitoring of the difference in loss contribution between LSTM and GCN, the difference ratio is calculated. and , and adjust the gradient update amplitude of each mode according to the difference ratio back propagation. Specifically, when the loss contribution of a certain mode is large (i.e. ), through the gradient modulation coefficient Suppress its gradient update to avoid the optimization of the dominant mode suppressing the weak mode, and ensure the full extraction of multimodal features;

[0047] The time series features extracted by LSTM and spatial features extracted by GCN Concatenate into joint feature vector , input to the fully connected layer and output the RUL prediction value. Through multimodal feature fusion, it can fully utilize the information of time series and time-frequency images to significantly improve the prediction accuracy;

[0048] The mean square error (MSE) is used as the loss function and the adaptive optimization strategy is combined to train the model. Through multiple iterative training, the model parameters are optimized to improve the prediction accuracy and generalization ability.

[0049] Finally, the root mean square error (RMSE) and mean absolute error (MAE) were used as evaluation metrics to evaluate the final prediction results. Experimental results show that the RMSE and MAE of this method on the IEEE PHM 2012 bearing dataset significantly outperform traditional single-modal methods (such as CNN and LSTM) and multimodal methods (such as T-GCN).

[0050] Compared with the prior art, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0051] (1) This paper proposes a bearing remaining useful life (RUL) prediction method based on multimodal data fusion and adaptive optimization. By combining the multimodal features of time series data and time-frequency images, the health status of the bearing is comprehensively characterized, significantly improving the accuracy and generalization ability of RUL prediction.

[0052] (2) An adaptive optimization strategy is designed to monitor the difference in loss contribution between the LSTM and GCN modalities in real time, and adaptively adjust the gradient update amplitude of each modality to avoid the dominant modality suppressing the optimization of the weak modality, thus ensuring the full extraction of multimodal features and balanced training.

[0053] (3) By fusing the temporal features extracted by LSTM with the spatial features extracted by GCN, the multimodal information of time series and time-frequency images is fully utilized, significantly improving the performance of the prediction model. Experimental results show that the RMSE and MAE of this method are significantly better than those of traditional single-modal and multimodal methods.

[0054] (4) This method has strong generalization ability and can adapt to a variety of industrial application scenarios, providing reliable technical support for bearing health management and intelligent operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart provided by an embodiment of the present invention;

[0056] Figure 2 This is an ablation experiment provided by an embodiment of the present invention, in which (A) the method proposed by the present invention; (B) only using LSTM; (C) using GCN; and (D) no adaptive optimization strategy. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0058] To address the imbalance in multimodal data fusion and optimization in bearing remaining useful life (RUL) prediction, this paper provides a bearing remaining useful life prediction method based on multimodal adaptive balanced optimization. This method significantly improves the accuracy and generalization of RUL prediction by combining the multimodal features of time series data and time-frequency images and employing an adaptive optimization strategy. First, in the offline phase, time series data of vibration signals are acquired from a bearing operation dataset and time-frequency images are generated via wavelet transform, ensuring that the data contains both time series and image modalities. A long short-term memory (LSTM) network is then used to extract temporal features from the time series data, while a graph convolutional network (GCN) is used to extract spatial features from the time-frequency images. The features extracted by the LSTM and GCN are then concatenated to generate a joint feature vector, which is then fed into a fully connected layer to output the RUL prediction value. During training, an adaptive optimization strategy monitors the difference in loss contributions between the LSTM and GCN modalities in real time. Backpropagation adjusts the gradient update amplitude of each modality to prevent the dominant modality from suppressing the optimization of the weaker modality, ensuring sufficient extraction of multimodal features. Finally, the root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation indicators to evaluate the final prediction results. Figure 1 The flowchart of the present invention is shown below, which is a specific implementation example.

[0059] (1) Data collection and processing

[0060] This study used a bearing degradation life dataset provided by the PRONOSTIA experimental platform from the IEEE PHM 2012 Challenge. This platform collects operating data from multiple bearings under various operating conditions through accelerated testing, effectively reflecting the complete operating cycle from initial state to failure.

[0061] In the experiment, sensors collected bearing vibration signals at a sampling frequency of 25.6 kHz, acquiring time series data. This data was collected every 10 seconds, with each acquisition lasting 0.1 seconds. Consequently, each sample contained 2,560 data points. This data collection process continued until the bearing reached the end of its lifecycle, ensuring that the data covered the entire lifecycle, from healthy state to failure, facilitating modeling of degradation trends and prediction of remaining useful life (RUL).

[0062] (2) Multimodal feature extraction

[0063] (2.1) Time Series Feature Extraction: A long short-term memory (LSTM) network is used to extract temporal features from time series data. LSTM controls information flow through input, forget, and output gates, preventing the vanishing gradient problem and effectively capturing the long-term dependencies of bearing degradation. The specific steps are as follows:

[0064] (2.1.1) LSTM unit structure: The LSTM unit consists of an input gate, a forget gate, an output gate, and a unit state. The specific formula is as follows:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] in, is the unit state, For output, 、 、 They are forget gate, input gate and output gate respectively. is the Sigmoid activation function, Represents element-wise multiplication.

[0071] (2.1.2) LSTM network structure:

[0072] A multi-layer LSTM network is used to extract the temporal features of the time series. The output of each layer is used as the input of the next layer, and the final output is the temporal feature vector. ,in is the feature dimension of LSTM output.

[0073] (2.2) Image feature extraction: Use a graph convolutional network (GCN) to extract the spatial features of the time-frequency image. GCN transfers information through the adjacency matrix and degree matrix to capture the spatial dependencies in the image. The specific steps are as follows:

[0074] (2.2.1) GCN information transmission formula: The information transmission process of GCN can be expressed as:

[0075]

[0076] in, is an adjacency matrix with self-loops, is the degree matrix, is a learnable parameter, is the input feature matrix;

[0077] (2.2.2) GCN network structure:

[0078] Use multi-layer GCN to extract the spatial features of the time-frequency image. The output of each layer is used as the input of the next layer, and the spatial feature vector is finally output. ,in is the feature dimension of GCN output.

[0079] (3) Multimodal feature fusion

[0080] (3.1) The time series features extracted by LSTM and spatial features extracted by GCN The concatenation is done into a joint feature vector, which is then fed into the fully connected layer to output the RUL prediction value. The specific steps are as follows:

[0081] (3.1.1) Feature splicing: The temporal features extracted by LSTM and spatial features extracted by GCN Feature dimension splicing to generate feature vector The specific formula is as follows:

[0082]

[0083] in, represents the concatenation operation of feature dimensions, and These are the feature dimensions of LSTM and GCN outputs respectively.

[0084] (3.1.2) Fully connected layer prediction: the joint feature vector Input the fully connected layer and output the RUL prediction value. The specific formula is as follows:

[0085]

[0086] in, and are the weights and biases of the fully connected layer, respectively.

[0087] (4) Adaptive balance optimization

[0088] (4.1) Difference Ratio Calculation: During multimodal training, feature extraction and optimization across different modalities may be unbalanced, resulting in features from some modalities being ignored or suppressed. To address this issue, this paper introduces an adaptive balance optimization strategy that dynamically adjusts the gradient update amplitude for each modality by monitoring the difference in loss across different modalities in real time.

[0089] (4.1.1) Calculate the modal loss: For each mini-batch data , calculate the loss values ​​of time series modality (LSTM) and image modality (GCN) respectively:

[0090]

[0091]

[0092] in, and are the input of time series data and time-frequency images respectively, It is a real RUL label. Time series data Input LSTM model and output the predicted value of RUL, Image data Input the GCN model and output the predicted value of RUL. criterion(...) is the loss function, which measures the error between the model prediction value and the true value using MSE mean square error.

[0093] (4.1.2) Calculate the difference ratio: Based on the loss values ​​of the two modalities, calculate the difference ratio of the time series modality to the image modality , the specific formula is as follows:

[0094]

[0095] in, Used to prevent the denominator from being zero.

[0096] The difference ratio of the image modality relative to the time series modality Then The reciprocal of , the specific formula is as follows:

[0097]

[0098] in, Used to prevent the denominator from being zero.

[0099] (4.2) Gradient modulation coefficient

[0100] Based on the difference ratio , calculate the dynamic adjustment coefficient of each mode , used to adjust the amplitude of gradient update. The specific formula is as follows:

[0101]

[0102] in, is a hyperparameter used to control the intensity of gradient regulation. ), its gradient update amplitude will be appropriately reduced to avoid this mode dominating the training process.

[0103] (4.3) Parameter update

[0104] In each iteration, a dynamic adjustment coefficient is used Adjust the gradient update amplitude of each mode. The specific steps are as follows:

[0105] (4.3.1) Gradient update formula:

[0106] For the parameters of LSTM and GCN, the update formula is as follows:

[0107]

[0108] in, is a mini-batch of data randomly sampled in the t-th iteration, is a sample in a small batch of data, Use the current model parameters On the sample The calculated loss value, is the gradient of the loss with respect to the parameters, is the average of all sample gradients.

[0109]

[0110] in, is the learning rate, is the gradient estimate for the mini-batch. represents the parameter vector (model weight) of the u-th mode at the t-th iteration, is the gradient modulation coefficient of the u-th mode in the t-th iteration, which comes from the adaptive balance optimization strategy.

[0111] (5) Model training

[0112] (5.1) Loss function calculation

[0113] During model training, the mean square error (MSE) is used as the loss function to measure the difference between the predicted value and the true value. The specific formula is as follows:

[0114]

[0115] in, is the real RUL value, To predict the RUL value, N is the number of samples.

[0116] (5.2) Model training

[0117] (5.2.1) Initialization parameters

[0118] Initialize the network parameters of the LSTM and GCN, including the weight matrix and bias term. The initialization parameters of the LSTM include the weights and biases of the input gate, forget gate, and output gate. The initialization parameters of the GCN include the weights of the adjacency matrix and degree matrix.

[0119] (5.2.2) Forward propagation

[0120] For each mini-batch of data , forward propagation is performed through LSTM and GCN respectively to extract the features of time series and time-frequency image. The features of the two modalities are then concatenated into a joint feature vector , input the fully connected layer and output the RUL prediction value.

[0121] (5.2.3) Backpropagation

[0122] The loss between the predicted value and the true value is calculated, and the parameters of LSTM and GCN are updated through the back-propagation algorithm. During the back-propagation process, the adaptive balance optimization strategy is used to adjust the gradient update amplitude of each modality.

[0123] (5.2.4) Iterative training

[0124] Repeat the forward propagation and backpropagation process until the model converges or reaches the predetermined number of training rounds (epochs).

[0125] (6) Performance evaluation

[0126] (6.1) Evaluation indicators

[0127] In order to evaluate the prediction performance of the model, the root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation indicators. The specific formulas are as follows:

[0128]

[0129]

[0130] in, is the real RUL value, To predict the RUL value, N is the number of samples.

[0131] (6.2) Experimental setup

[0132] (6.2.1) Dataset Partitioning

[0133] The IEEE PHM 2012 bearing dataset is divided into a training set and a test set. The training set is used for model training, and the test set is used to evaluate the model's prediction performance.

[0134] (6.2.2) Hyperparameter settings

[0135] During model training, the learning rate is set to 0.01, the number of training epochs is 200, and the batch size is 32. The Adam optimizer is used to minimize the mean squared error (MSE) loss, and normalized linear labels are used for all experiments.

[0136] (6.2.3) Experimental platform

[0137] The experiments were conducted on the following hardware platforms:

[0138] CPU: Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz

[0139] Graphics processor: NVIDIA GeForce RTX 3090

[0140] Programming language: Python 3.7

[0141] (6.3) Experimental results

[0142] To verify the effectiveness of the multimodal RUL prediction method proposed in this paper, we conducted experiments on the IEEE PHM Challenge 2012 bearing dataset. The experiment is divided into two parts: a comparative experiment and an ablation experiment.

[0143] (6.3.1) The comparative experimental results are shown in Table 1 and Table 2.

[0144] Table 1 RMSE values ​​of different models and bearings

[0145] Table 2 MAE values ​​of different models and bearings

[0146] (6.3.1.1) RMSE result analysis

[0147] As can be seen from Table 1, the proposed method outperforms the other comparison models (CNN, CNNLSTM, and T-GCN) in RMSE values ​​for all bearings. In particular, the proposed method achieves significantly lower RMSE values ​​than the other methods for bearings 1_1, 1_2, 1_3, 1_4, 1_6, and 1_7.

[0148] On bearing 1_5, the RMSE value of T-GCN is slightly better than that of our method, but our method still performs well with an RMSE value of 0.288, which is close to T-GCN's 0.286.

[0149] (6.3.1.2) MAE result analysis

[0150] As can be seen from Table 2, the proposed method outperforms the other comparison models in terms of MAE values ​​for all bearings. In particular, the MAE values ​​of the proposed method are significantly lower than those of the other methods for bearings 1_1, 1_2, 1_3, 1_4, 1_5, 1_6, and 1_7.

[0151] The MAE value of this method on bearing 1_5 is 0.278, which is better than 0.283 of T-GCN, further demonstrating the superiority of this method.

[0152] (6.3.1.3) Overall performance

[0153] The average RMSE of this method on seven bearings was 0.285, and the average MAE was 0.274, significantly outperforming other comparison models. This demonstrates that the proposed multimodal feature fusion and adaptive balance optimization strategy can effectively improve the accuracy of RUL prediction.

[0154] (6.3.2) Ablation Experiment Results

[0155] In order to verify the importance of each module, this paper designed an ablation experiment, including the complete model, the model using only single modality data, and the model without adaptive balance optimization. The specific experimental results are shown in Table 3 and Figure 2 As shown:

[0156] Table 3 Ablation experiment number

[0157] The specific results are analyzed as follows:

[0158] (6.3.2.1) LSTM and GCN:

[0159] The model using only LSTM performs better than the model using only GCN, indicating that time series data contributes more significant features in RUL prediction.

[0160] (6.3.2.2) Importance of Adaptive Optimization:

[0161] The model without adaptive optimization (D) performs slightly worse than the complete model (A), indicating that the adaptive optimization strategy can effectively balance the training process of different modalities and improve the prediction accuracy of the model.

[0162] (6.3.2.3) Advantages of multimodal fusion:

[0163] The complete model (A) performs best in all ablation experiments, indicating that multimodal feature fusion can significantly improve the accuracy of RUL prediction.

[0164] On the other hand, an embodiment of the present invention further provides a multi-modal bearing remaining life prediction system based on balance optimization, comprising:

[0165] one or more processors;

[0166] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the multi-modal bearing remaining life prediction method based on balance optimization as described in the above technical solution.

[0167] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions and modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A multi-modal bearing remaining life prediction method based on balance optimization, characterized in that: The steps include: Step 1: Obtain the time series data of the vibration signal from the bearing operation data set and generate a time-frequency image through wavelet transform; Step 2: In terms of temporal modality, the long short-term memory network (LSTM) is used to extract the temporal features of the time series data; in terms of spatial modality, the graph convolutional network (GCN) is used to extract the spatial features of the time-frequency image. Step 3: Concatenate the extracted temporal features and spatial features along the feature dimension to generate a joint feature vector, which is then fed into the fully connected layer to output the predicted remaining useful life (RUL) of the bearing. Step 4: Use an adaptive optimization strategy to monitor the difference in loss contributions between LSTM and GCN in real time, dynamically adjust the gradient update amplitude of each modality, and obtain the model weight; Step 5: Combine the model weights and loss function to train the bearing remaining life prediction model constructed by steps 2-3; Step 6: Use the trained model to predict the remaining life of the bearing.

2. The method for predicting remaining life of a multi-modal bearing based on balance optimization according to claim 1, characterized in that: The time series data of the vibration signal is input into the LSTM network, and the timing features are extracted through multi-layer LSTM units. Each LSTM unit controls the information flow through the input gate, forget gate and output gate.

3. The method for predicting remaining life of a multi-modal bearing based on balance optimization according to claim 1, wherein: Each layer of GCN updates node information through matrix multiplication, and uses multi-layer GCN to extract the spatial features of the time-frequency image. The output of each layer is used as the input of the next layer, and the spatial feature vector is finally output. , the formula is as follows: ； in, is an adjacency matrix with self-loops, is the degree matrix, is a learnable parameter, is the input feature matrix.

4. The method for predicting remaining life of a multi-modal bearing based on balance optimization according to claim 1, wherein: The specific implementation of step 4 includes the following sub-steps: (4.1) Calculate the loss values ​​of the time mode and the space mode respectively. Based on the loss values ​​of the two modes, calculate the difference ratio of the time mode to the space mode and the difference ratio of the space mode to the time mode. (4.2) Calculate the dynamic adjustment coefficient of each mode based on the difference ratio ; (4.3) In each iteration, the dynamic adjustment coefficient is used Adjust the gradient update amplitude of each mode to obtain the model weight.

5. The method for predicting the remaining life of a multi-modal bearing based on balance optimization according to claim 4, characterized in that: For each mini-batch of data , calculate the loss values ​​of time mode and space mode respectively: ； ； in, and are the input of time series data and time-frequency images respectively, is the loss value of the time mode, is the loss value of the spatial mode, It is a real RUL label. Indicates that time series data Input the LSTM model and output the predicted value of RUL. Indicates that the image data Input the GCN model and output the predicted value of RUL. criterion() is the loss function, using MSE mean square error.

6. The method for predicting the remaining life of a multi-modal bearing based on balance optimization according to claim 5, characterized in that: Based on the loss values ​​of the two modes, calculate the difference ratio of the time mode to the spatial mode , the specific formula is as follows: ； The difference ratio of the spatial mode to the temporal mode Then The reciprocal of , the specific formula is as follows: ； in, is a constant used to prevent the denominator from being zero.

7. The method for predicting the remaining life of a multi-modal bearing based on balance optimization according to claim 4, characterized in that: Dynamic adjustment coefficient The specific formula is as follows: ； in, is a hyperparameter that controls the intensity of gradient regulation. Represents the difference ratio.

8. The method for predicting the remaining life of a multi-modal bearing based on balance optimization according to claim 4, characterized in that: The gradient update amplitude is calculated as follows: ； in, is a mini-batch of data randomly sampled in the t-th iteration, is a sample in a small batch of data, Use the current model parameters On the sample The calculated loss value, is the gradient of the loss value with respect to the parameter, is the average of all sample gradients; ； in, is the learning rate, represents the parameter vector of the u-th mode at the t-th iteration, i.e., the model weight, is the dynamic adjustment coefficient of the u-th mode in the t-th iteration, and u takes the values ​​of 1 and 2, where 1 represents the time mode and 2 represents the space mode.

9. The method for predicting the remaining life of a multi-modal bearing based on balance optimization according to claim 1, wherein: Use mean squared error as loss function , measures the difference between the predicted value and the true value. The specific formula is as follows: ； in, is the real RUL value, To predict the RUL value, N is the number of samples.

10. A multi-modal bearing remaining life prediction system based on balance optimization, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the multi-modal bearing remaining life prediction method based on balance optimization as described in any one of claims 1 to 9.

Images