Method and system for predicting the residual life of a multi-modal bearing based on equilibrium optimization

Abstract

The application discloses a multi-modal bearing residual life prediction method and system based on balance optimization. First, in the offline stage, the time series data of the vibration signal is obtained from the bearing operation data set, and the time-frequency image is generated through wavelet transform, so as to ensure that the data contains two modes of time series and image. Then, the long short-term memory network (LSTM) is used to extract the time sequence characteristics of the time series data, and the graph convolution network (GCN) is used to extract the spatial characteristics of the time-frequency image. Then, the features extracted by the LSTM and the GCN are spliced to generate a joint feature vector, which is input into a full connection layer to output an RUL prediction value. In the training process, the loss contribution difference of the two modes of the LSTM and the GCN is monitored in real time through an adaptive optimization strategy, the gradient update amplitude of each mode is adjusted through back propagation, the optimization of the dominant mode suppressing the weak mode is avoided, and the full extraction of the multi-modal characteristics is ensured.

Description

Technical Field

[0001] This invention relates to the field of remaining useful life (RUL) prediction technology for mechanical equipment, and in particular to a bearing RUL prediction method that combines time series and image data for multimodal feature fusion and employs a gradient adaptive optimization strategy. Background Technology

[0002] With the rapid development of industrial technology, bearings, as key components in mechanical systems, are crucial for ensuring the safe operation of equipment through health status monitoring and remaining useful life (RUL) prediction. Currently, data-driven methods have received widespread attention in the field of RUL prediction. However, most existing methods are mainly based on single-modal data, or fail to fully consider the imbalance in the learning of different modal features when fusing multimodal data, resulting in limited prediction performance.

[0003] Current RUL prediction methods are mainly divided into physical model-based methods and data-driven methods. Physical models rely on mathematical modeling, such as Wiener processes, particle filtering, Kalman filtering, and hidden Markov models. However, due to the complexity of bearing degradation processes, accurately establishing physical models presents significant challenges. Data-driven methods do not require specialized knowledge and have gradually become mainstream in recent years with the rise of deep learning. For example, temporal feature extraction methods based on deep long short-term memory (LSTM) networks and feature extraction methods based on convolutional neural networks (CNNs) have demonstrated superior performance in RUL prediction.

[0004] However, in multimodal learning, a single modality may dominate model optimization, limiting feature learning for other modalities and thus affecting overall performance. Therefore, effectively fusing time series and image data while optimizing the learning weights of each modality is a key challenge in the field of RUL prediction. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a bearing remaining service (RUL) prediction method and system based on multimodal adaptive optimization, aiming to solve the problems of single-modal data limitations, inter-modal optimization imbalance, and insufficient prediction accuracy in the prior art. This method significantly improves the accuracy and generalization ability of bearing RUL prediction by combining multimodal features from time series data and time-frequency images and using an adaptive optimization strategy.

[0006] The technical solution of this invention is a bearing remaining service life prediction method based on multimodal adaptive optimization, which includes the following steps:

[0007] Step 1: Obtain time series data of vibration signals from the bearing operation dataset and generate time-frequency images through wavelet transform;

[0008] Step 2: In terms of temporal modality, the Long Short-Term Memory (LSTM) network is used to extract the temporal features of the time series data; in terms of spatial modality, the Graph Convolutional Network (GCN) is used to extract the spatial features of the time-frequency images.

[0009] Step 3: Concatenate the extracted temporal and spatial features along the feature dimensions to generate a joint feature vector, and input it into the fully connected layer to output the predicted remaining service life (RUL) of the bearing.

[0010] Step 4: Use an adaptive optimization strategy to monitor the difference in loss contribution between LSTM and GCN in real time, dynamically adjust the gradient update magnitude of each modality, and obtain the model weights.

[0011] Step 5: Train the bearing remaining life prediction model constructed from steps 2-3 by combining model weights and loss function;

[0012] Step 6: Use the trained model to predict the remaining life of the bearing.

[0013] Furthermore, the time-series data of the vibration signal is input into the LSTM network, and the temporal features are extracted through multi-layer LSTM units. Each LSTM unit controls the information flow through an input gate, a forget gate, and an output gate.

[0014] Furthermore, each layer of the GCN updates node information through matrix multiplication, and multi-layer GCNs are used to extract spatial features from time-frequency images. The output of each layer serves as the input to the next layer, ultimately outputting a spatial feature vector. The formula is as follows:

[0015]

[0016] in, Given an adjacency matrix with self-loops, For degree matrix, For learnable parameters, This is the input feature matrix.

[0017] Furthermore, the specific implementation of step 4 includes the following sub-steps:

[0018] (4.1) Calculate the loss values ​​of the temporal mode and the spatial mode respectively. Based on the loss values ​​of the two modes, calculate the difference ratio of the temporal mode relative to the spatial mode and the difference ratio of the spatial mode relative to the temporal mode.

[0019] (4.2) Calculate the dynamic adjustment coefficient for each mode based on the difference ratio. ;

[0020] (4.3) In each iteration, use the dynamic adjustment coefficient. Adjust the gradient update magnitude for each modality to obtain the model weights.

[0021] Furthermore, for each mini-batch of data Calculate the loss values ​​for the time mode and the spatial mode respectively:

[0022]

[0023]

[0024] in, and The inputs are time series data and time-frequency images, respectively. The loss value for the time modality. The loss value is for the spatial modes. It's a genuine RUL label. This indicates that time series data Input an LSTM model and output the predicted value for RUL. This indicates that the image data Input the GCN model and output the predicted value of RUL. The criterion() is the loss function, using the mean squared error (MSE).

[0025] Furthermore, based on the loss values ​​of the two modes, the difference ratio between the temporal mode and the spatial mode is calculated. The specific formula is as follows:

[0026]

[0027] The difference ratio between spatial modes and temporal modes Then it is The reciprocal of is given by the following formula:

[0028]

[0029] in, It is a constant used to prevent the denominator from being zero.

[0030] Furthermore, the dynamic adjustment coefficient The specific formula is as follows:

[0031]

[0032] in, It is a hyperparameter used to control the strength of gradient adjustment. Indicates the difference ratio.

[0033] Furthermore, the formula for calculating the gradient update magnitude is as follows:

[0034]

[0035] in, It is a small batch of data randomly sampled in the t-th iteration. It is a single sample from a small batch of data. Is it using the current model parameters? On the sample The calculated loss value, It is the gradient of the loss value with respect to the parameters. It is the average gradient of all samples;

[0036]

[0037] in, It's the learning rate. Let represent the parameter vector, i.e., the model weights, for the u-th mode at the t-th iteration. It is the dynamic adjustment coefficient of the u-th mode in the t-th iteration, where u takes the values ​​1 and 2, where 1 represents the time mode and 2 represents the spatial mode.

[0038] Furthermore, mean squared error is used as the loss function. The formula for measuring the difference between predicted and actual values ​​is as follows:

[0039]

[0040] in, This is the true RUL value. To predict the RUL value, N is the number of samples.

[0041] This invention also provides a multimodal bearing remaining life prediction system based on balance optimization, comprising:

[0042] One or more processors;

[0043] A storage device is provided for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the balance-optimized multimodal bearing remaining life prediction method as described in the above technical solution.

[0044] This invention acquires time-series data of vibration signals from bearing operation datasets and generates time-frequency images through wavelet transform, ensuring that the data includes both time-series and image modalities. The time-series data captures the degradation trend of the bearing, while the time-frequency images provide the spatial characteristics of the vibration signals in the frequency domain. The combination of the two can more comprehensively characterize the health status of the bearing.

[0045] A Long Short-Term Memory (LSTM) network is used to extract time-series features of vibration signals. LSTM controls the information flow through input gates, forget gates, and output gates, avoiding the gradient vanishing problem and effectively capturing the long-term dependencies of bearing degradation. A Graph Convolutional Network (GCN) is used to extract spatial features of the time-frequency image. GCN transmits information through adjacency and degree matrices, capturing spatial dependencies in the image and effectively characterizing the local features of bearing faults.

[0046] In addition, by monitoring the difference in loss contributions between the LSTM and GCN modes in real time, the difference ratio is calculated. and And adjust the gradient update magnitude of each mode based on the difference ratio through backpropagation. Specifically, when the loss contribution of a certain mode is large (i.e., ), through gradient modulation coefficients Suppress its gradient update to avoid the dominant mode suppressing the optimization of the weak mode, and ensure the full extraction of multimodal features;

[0047] Temporal features extracted by LSTM Spatial features extracted by GCN Concatenate into a joint feature vector The input is a fully connected layer, and the output is a RUL prediction value. By fusing multimodal features, information from time series and time-frequency images can be fully utilized, significantly improving prediction accuracy.

[0048] The model is trained using mean squared error (MSE) as the loss function and combined with an adaptive optimization strategy. Through multiple iterations of training, the model parameters are optimized to improve prediction accuracy and generalization ability.

[0049] Finally, the root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation metrics to evaluate the final prediction results. Experimental results show that the RMSE and MAE of this method are significantly better than traditional single-modal methods (such as CNN and LSTM) and multi-modal methods (such as T-GCN) on the IEEE PHM 2012 bearing dataset.

[0050] Compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0051] (1) This invention proposes a bearing remaining service (RUL) prediction method based on multimodal data fusion and adaptive optimization. By combining the multimodal features of time series data and time-frequency images, the health status of the bearing is comprehensively characterized, which significantly improves the accuracy and generalization ability of RUL prediction.

[0052] (2) An adaptive optimization strategy was designed to monitor the difference in loss contribution between the two modes, LSTM and GCN, in real time, and to adaptively adjust the gradient update magnitude of each mode to avoid the dominant mode suppressing the optimization of the weak mode, thus ensuring the full extraction of multimodal features and balanced training.

[0053] (3) By fusing the temporal features extracted by LSTM with the spatial features extracted by GCN, the multimodal information of time series and time-frequency images is fully utilized, significantly improving the performance of the prediction model. Experimental results show that the RMSE and MAE of this method are significantly better than those of traditional single-modal and multimodal methods.

[0054] (4) This method has strong generalization ability and can adapt to diverse industrial application scenarios, providing reliable technical support for the health management and intelligent operation and maintenance of bearings. Attached Figure Description

[0055] Figure 1 This is a flowchart provided in an embodiment of the present invention;

[0056] Figure 2 The ablation experiment provided in the embodiments of the present invention includes (A) the method proposed in the present invention; (B) using only LSTM; (C) using GCN; and (D) without adaptive optimization strategy. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0058] To address the imbalance between multimodal data fusion and optimization in bearing remaining useful life (RUL) prediction, this invention provides a bearing RUL prediction method based on multimodal adaptive balance optimization. This method significantly improves the accuracy and generalization ability of RUL prediction by combining multimodal features from time-series data and time-frequency images, and employing an adaptive optimization strategy. First, in the offline phase, time-series data of vibration signals is obtained from the bearing operation dataset, and time-frequency images are generated through wavelet transform, ensuring that the data includes both time-series and image modalities. Then, a Long Short-Term Memory (LSTM) network is used to extract the temporal features of the time-series data, while a Graph Convolutional Network (GCN) is used to extract the spatial features of the time-frequency images. Next, the features extracted by LSTM and GCN are concatenated to generate a joint feature vector, which is input into a fully connected layer to output the predicted RUL value. During training, an adaptive optimization strategy is used to monitor the difference in loss contributions between the LSTM and GCN modalities in real time, and backpropagation is used to adjust the gradient update magnitude of each modality, avoiding the suppression of weaker modal optimization by the dominant modality and ensuring the full extraction of multimodal features. Finally, the root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation metrics to evaluate the final prediction results. Figure 1 The flowchart illustrates the present invention, and the following are specific implementation examples.

[0059] (1) Data collection and processing

[0060] This study used the bearing degradation life dataset provided by the PRONOSTIA experimental platform in the IEEE PHM 2012 Challenge. This platform accelerates the acquisition of operating data for multiple bearings under different working conditions, realistically reflecting the complete operating cycle from the initial state to the failure stage.

[0061] In the experiment, the sensor collected the vibration signal of the bearing at a sampling frequency of 25.6 kHz, acquiring time-series data of the vibration signal. Data was collected every 10 seconds, with each acquisition lasting 0.1 seconds, resulting in 2560 data points per sample. The entire data acquisition process continued until the bearing reached the end of its lifespan, ensuring that the data covered the entire process from healthy condition to failure, facilitating the modeling of degradation trends and prediction of remaining useful life (RUL).

[0062] (2) Multimodal feature extraction

[0063] (2.1) Time Series Feature Extraction: A Long Short-Term Memory (LSTM) network is used to extract the temporal features of the time series data. LSTM controls the information flow through input gates, forget gates, and output gates, avoiding the gradient vanishing problem and effectively capturing long-term dependencies in bearing degradation. The specific steps are as follows:

[0064] (2.1.1) LSTM cell structure: An LSTM cell consists of an input gate, a forget gate, an output gate, and cell states. The specific formula is as follows:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] in, For the unit state, For output, , , These are the forget gate, input gate, and output gate, respectively. It is the Sigmoid activation function. This indicates element-wise multiplication.

[0071] (2.1.2) LSTM network structure:

[0072] A multi-layer LSTM network is used to extract time series features, with the output of each layer serving as the input to the next layer, ultimately outputting a time series feature vector. ,in The feature dimension output by LSTM.

[0073] (2.2) Image Feature Extraction: Spatial features of time-frequency images are extracted using a Graph Convolutional Network (GCN). GCN transmits information through adjacency and degree matrices, capturing spatial dependencies in the image. The specific steps are as follows:

[0074] (2.2.1) GCN Information Transmission Formula: The information transmission process of GCN can be represented as:

[0075]

[0076] in, Given an adjacency matrix with self-loops, For degree matrix, For learnable parameters, The input feature matrix;

[0077] (2.2.2) GCN network structure:

[0078] A multi-layer GCN is used to extract spatial features from time-frequency images. The output of each layer serves as the input to the next layer, ultimately outputting a spatial feature vector. ,in The feature dimension output by GCN.

[0079] (3) Multimodal feature fusion

[0080] (3.1) Temporal features extracted by LSTM Spatial features extracted by GCN The concatenated feature vectors are input into a fully connected layer, which then outputs the RUL prediction value. The specific steps are as follows:

[0081] (3.1.1) Feature concatenation: The temporal features extracted by LSTM are concatenated. Spatial features extracted by GCN Feature dimensions are concatenated to generate feature vectors. The specific formula is as follows:

[0082]

[0083] in, This represents the concatenation operation of feature dimensions. and These are the feature dimensions output by LSTM and GCN, respectively.

[0084] (3.1.2) Fully connected layer prediction: joint feature vector The input layer is a fully connected layer, and the output is the predicted RUL value. The specific formula is as follows:

[0085]

[0086] in, and These are the weights and biases of the fully connected layer, respectively.

[0087] (4) Adaptive balance optimization

[0088] (4.1) Difference Ratio Calculation: During multimodal training, the feature extraction and optimization of different modalities may become unbalanced, causing features of some modalities to be ignored or suppressed. To address this issue, this invention introduces an adaptive balancing optimization strategy, which dynamically adjusts the gradient update magnitude of each modality by monitoring the loss differences between different modalities in real time.

[0089] (4.1.1) Calculate modal loss: For each mini-batch of data Calculate the loss values ​​for the time series mode (LSTM) and the image mode (GCN) respectively:

[0090]

[0091]

[0092] in, and The inputs are time series data and time-frequency images, respectively. It is a genuine RUL label. Time series data Input an LSTM model and output the predicted value of RUL. Image data The input is the GCN model, and the output is the predicted value of RUL. The criterion(...) is the loss function, and the mean squared error (MSE) is used to measure the error between the model's predicted value and the true value.

[0093] (4.1.2) Calculate the difference ratio: Based on the loss values ​​of the two modes, calculate the difference ratio of the time series mode relative to the image mode. The specific formula is as follows:

[0094]

[0095] in, Used to prevent the denominator from being zero.

[0096] The difference ratio between image modalities and time series modalities Then it is The reciprocal of is given by the following formula:

[0097]

[0098] in, Used to prevent the denominator from being zero.

[0099] (4.2) Gradient modulation coefficient

[0100] According to the difference ratio Calculate the dynamic adjustment coefficient for each mode. This is used to adjust the magnitude of gradient updates. The specific formula is as follows:

[0101]

[0102] in, It is a hyperparameter used to control the strength of gradient adjustment. When the loss of a certain mode is large (i.e., The gradient update magnitude will be appropriately reduced to prevent this mode from dominating the training process.

[0103] (4.3) Parameter update

[0104] In each iteration, a dynamic adjustment coefficient is used. Adjust the gradient update magnitude for each mode. The specific steps are as follows:

[0105] (4.3.1) Gradient update formula:

[0106] The update formulas for the parameters of LSTM and GCN are as follows:

[0107]

[0108] in, It is a small batch of data randomly sampled in the t-th iteration. It is a single sample from a small batch of data. Is it using the current model parameters? On the sample The calculated loss value, It is the gradient of the loss with respect to the parameters. It is the average gradient of all samples.

[0109]

[0110] in, It's the learning rate. It is gradient estimation for small batches of data. This represents the parameter vector (model weights) of the u-th mode at the t-th iteration. It is the gradient modulation coefficient of the u-th mode in the t-th iteration, derived from the adaptive equilibrium optimization strategy.

[0111] (5) Model training

[0112] (5.1) Loss function calculation

[0113] During model training, mean squared error (MSE) is used as the loss function to measure the difference between the predicted and actual values. The specific formula is as follows:

[0114]

[0115] in, This is the true RUL value. To predict the RUL value, N is the number of samples.

[0116] (5.2) Model Training

[0117] (5.2.1) Initialize parameters

[0118] Initialize the network parameters of LSTM and GCN, including the weight matrix and bias terms. The initialization parameters of LSTM include the weights and biases of the input gate, forget gate, and output gate, while the initialization parameters of GCN include the weights of the adjacency matrix and degree matrix.

[0119] (5.2.2) Forward propagation

[0120] For each mini-batch of data The time series and time-frequency images are extracted by forward propagation using LSTM and GCN respectively. The features from the two modalities are then concatenated into a joint feature vector. The input fully connected layer outputs the RUL prediction value.

[0121] (5.2.3) Backpropagation

[0122] The loss between the predicted and actual values ​​is calculated, and the parameters of LSTM and GCN are updated using the backpropagation algorithm. During backpropagation, an adaptive balancing optimization strategy is used to adjust the gradient update magnitude for each mode.

[0123] (5.2.4) Iterative training

[0124] Repeat the forward and backward propagation process until the model converges or reaches the predetermined number of training epochs.

[0125] (6) Performance evaluation

[0126] (6.1) Evaluation indicators

[0127] To evaluate the predictive performance of the model, root mean square error (RMSE) and mean absolute error (MAE) are used as evaluation metrics. The specific formulas are as follows:

[0128]

[0129]

[0130] in, This is the true RUL value. To predict the RUL value, N is the number of samples.

[0131] (6.2) Experimental setup

[0132] (6.2.1) Dataset partitioning

[0133] The IEEE PHM 2012 bearing dataset was divided into a training set and a test set. The training set was used for model training, and the test set was used to evaluate the model's predictive performance.

[0134] (6.2.2) Hyperparameter settings

[0135] During model training, the learning rate was set to 0.01, the number of training epochs was 200, and the batch size was 32. The Adam optimizer was used to minimize the mean squared error (MSE) loss, and normalized linear labels were used for all experiments.

[0136] (6.2.3) Experimental Platform

[0137] The experiment was conducted on the following hardware platform:

[0138] Central Processing Unit: Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz

[0139] Graphics Processor: NVIDIA GeForce RTX 3090

[0140] Programming language: Python 3.7

[0141] (6.3) Experimental Results

[0142] To verify the effectiveness of the proposed multimodal RUL prediction method, we conducted experiments on the IEEE PHM Challenge2012 bearing dataset. The experiments consisted of two parts: comparative experiments and ablation experiments.

[0143] (6.3.1) The comparative experimental results are shown in Table 1 and Table 2.

[0144] Table 1. RMSE values ​​for different models and bearings

[0145]

[0146] Table 2 MAE values ​​for different models and bearings

[0147]

[0148] (6.3.1.1) Analysis of RMSE results

[0149] As shown in Table 1, the proposed method outperforms other comparative models (CNN, CNNLSTM, T-GCN) in RMSE values ​​across all bearings. In particular, the RMSE values ​​of this method are significantly lower than those of other methods on bearings 1_1, 1_2, 1_3, 1_4, 1_6, and 1_7.

[0150] On bearing 1_5, the RMSE value of T-GCN is slightly better than that of this method, but this method still performs well with an RMSE value of 0.288, which is close to T-GCN's 0.286.

[0151] (6.3.1.2) MAE Result Analysis

[0152] As shown in Table 2, the proposed method outperforms other comparative models in terms of MAE values ​​for all bearings. In particular, the MAE values ​​of this method are significantly lower than those of other methods for bearings 1_1, 1_2, 1_3, 1_4, 1_5, 1_6, and 1_7.

[0153] The MAE value of this method on bearing 1_5 is 0.278, which is better than T-GCN's 0.283, further proving the superiority of this method.

[0154] (6.3.1.3) Overall performance

[0155] Our method achieves an average RMSE of 0.285 and an average MAE of 0.274 across seven bearings, significantly outperforming other comparative models. This demonstrates that the proposed multimodal feature fusion and adaptive balance optimization strategy can effectively improve the accuracy of RUL prediction.

[0156] (6.3.2) Ablation test results

[0157] To verify the importance of each module, this paper designed ablation experiments, including a complete model, a model using only single-modal data, and a model without adaptive equilibrium optimization. Specific experimental results are shown in Table 3 and... Figure 2 As shown:

[0158] Table 3 Ablation Experiment Numbers

[0159]

[0160] The specific results are analyzed as follows:

[0161] (6.3.2.1) LSTM and GCN:

[0162] The model using only LSTM performed better than the model using only GCN, indicating that time series data contributes more significant features to RUL prediction.

[0163] (6.3.2.2) The importance of adaptive optimization:

[0164] The model without adaptive optimization (D) performed slightly worse than the complete model (A), indicating that the adaptive optimization strategy can effectively balance the training process of different modalities and improve the prediction accuracy of the model.

[0165] (6.3.2.3) Advantages of multimodal fusion:

[0166] The complete model (A) performed best in all ablation experiments, indicating that multimodal feature fusion can significantly improve the accuracy of RUL prediction.

[0167] On the other hand, embodiments of the present invention also provide a multimodal bearing remaining life prediction system based on balance optimization, including:

[0168] One or more processors;

[0169] A storage device is provided for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the balance-optimized multimodal bearing remaining life prediction method as described in the above technical solution.

[0170] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art can make substitutions and modifications under the guidance of this invention without departing from the scope of protection of the claims of this invention, and all such substitutions and modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for predicting the remaining life of a multimodal bearing based on balance optimization, characterized in that, Includes the following steps: Step 1: Obtain time series data of vibration signals from the bearing operation dataset and generate time-frequency images through wavelet transform; Step 2: In terms of temporal modality, the Long Short-Term Memory (LSTM) network is used to extract the temporal features of the time series data; in terms of spatial modality, the Graph Convolutional Network (GCN) is used to extract the spatial features of the time-frequency images. Step 3: Concatenate the extracted temporal and spatial features along the feature dimensions to generate a joint feature vector, and input it into the fully connected layer to output the predicted remaining service life (RUL) of the bearing. Step 4: Use an adaptive optimization strategy to monitor the difference in loss contribution between LSTM and GCN in real time, dynamically adjust the gradient update magnitude of each modality, and obtain the model weights. Step 4 includes the following sub-steps: Step 4.1: Calculate the loss values ​​for the temporal mode and the spatial mode respectively. Based on the loss values ​​of the two modes, calculate the difference ratio of the temporal mode relative to the spatial mode and the difference ratio of the spatial mode relative to the temporal mode. Step 4.2: Calculate the dynamic adjustment coefficient for each mode based on the difference ratio. ; Step 4.3: In each iteration, use the dynamic adjustment coefficient. Adjust the gradient update magnitude for each modality to obtain the model weights; Step 5: Train the bearing remaining life prediction model constructed from steps 2-3 by combining model weights and loss function; Step 6: Use the trained model to predict the remaining life of the bearing.

2. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: The time-series data of the vibration signal is input into the LSTM network, and the temporal features are extracted through multiple LSTM units. Each LSTM unit controls the information flow through an input gate, a forget gate, and an output gate.

3. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: Each layer of the GCN updates node information through matrix multiplication. Multi-layer GCNs are used to extract spatial features from time-frequency images. The output of each layer serves as the input to the next layer, ultimately outputting a spatial feature vector. The formula is as follows: ； in, Given an adjacency matrix with self-loops, For degree matrix, For learnable parameters, This is the input feature matrix.

4. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: For each mini-batch of data Calculate the loss values ​​for the time mode and the spatial mode respectively: ； ； in, and The inputs are time series data and time-frequency images, respectively. The loss value for the time modality. The loss value is for the spatial modes. It's a genuine RUL label. This indicates that time series data Input an LSTM model and output the predicted value for RUL. This indicates that the image data Input the GCN model and output the predicted value of RUL. The criterion() is the loss function, using the mean squared error (MSE).

5. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 4, characterized in that: Based on the loss values ​​of the two modes, the difference ratio between the temporal mode and the spatial mode is calculated. The specific formula is as follows: ； Spatial mode versus temporal mode difference ratio Then it is The reciprocal of is given by the following formula: ； in, It is a constant used to prevent the denominator from being zero.

6. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: Dynamic adjustment coefficient The specific formula is as follows: ； in, It is a hyperparameter used to control the strength of gradient adjustment. Indicates the difference ratio.

7. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: The formula for calculating the gradient update magnitude is as follows: ； in, It is a small batch of data randomly sampled in the t-th iteration. It is a single sample from a small batch of data. Is it using the current model parameters? On the sample Calculated loss value, It is the gradient of the loss value with respect to the parameters. It is the average gradient of all samples; ； in, It's the learning rate. Let represent the parameter vector, i.e., the model weights, for the u-th mode at the t-th iteration. It is the dynamic adjustment coefficient of the u-th mode in the t-th iteration, where u takes the values ​​1 and 2, where 1 represents the time mode and 2 represents the spatial mode.

8. The multimodal bearing remaining life prediction method based on balance optimization as described in claim 1, characterized in that: Using mean squared error as the loss function The formula for measuring the difference between predicted and actual values ​​is as follows: ； in, This is the true RUL value. To predict the RUL value, N is the number of samples.

9. A multimodal bearing remaining life prediction system based on balance optimization, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the balance-optimized multimodal bearing remaining life prediction method as described in any one of claims 1 to 8.

Images