RUL prediction framework efficiency optimization method based on DDPG fusion AM

Abstract

This invention provides an efficiency optimization method for the RUL prediction framework based on DDPG fusion AM, relating to the field of mechanical equipment life prediction. The method includes: generating continuous functional signal data using the FAR method; calculating the similarity between the current test sequence and historical data using an attention-weighted dynamic time warping algorithm, dynamically selecting the M most relevant historical data segments to construct a degradation training dataset; constructing an attention-enhanced CNN and LSTM parallel hybrid model, inputting the degradation training dataset into the model to extract equipment degradation features, and outputting initial RUL prediction values; using the performance evaluation index of the initial prediction values ​​and the current RUL prediction framework parameters as state inputs to the DDPG agent, outputting actions to adjust the RUL prediction framework parameters, and dynamically optimizing the size of the degradation training dataset and the length of the input time window of the parallel hybrid model. This invention achieves global optimization of RUL prediction accuracy and operational efficiency.

Description

Technical Field

[0001] This invention relates to the field of mechanical equipment life prediction technology; specifically, it relates to an efficiency optimization method for a RUL prediction framework based on DDPG fusion AM. Background Technology

[0002] As an important means of ensuring the reliability of mechanical equipment, one of the key tasks of Prognostics and Health Management (PHM) technology is to predict Remaining Useful Life (RUL). RUL prediction aims to estimate the time from the current moment to failure of equipment by continuously monitoring the operating status and analyzing the degradation trend, thereby providing decision support for preventive maintenance, avoiding sudden failures and reducing operation and maintenance costs.

[0003] In data-driven RUL prediction methods, deep learning technology has become a research hotspot due to its powerful nonlinear modeling and feature extraction capabilities. Models such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolutional Neural Network (CNN) are widely used to extract device degradation features from multidimensional sensing time series.

[0004] However, these existing methods typically rely on fixed time windows or preset weight allocation strategies, making it difficult to adaptively capture the dynamic changes in the contribution of sensor signals to the degradation process at different operating stages. This results in limited generalization ability of the model when faced with complex operating conditions and individual differences.

[0005] In recent years, reinforcement learning (RL) has demonstrated significant advantages in sequential decision-making problems due to its ability to autonomously learn optimal decision-making policies through interaction with the environment. Deep Deterministic Policy Gradient (DDPG) algorithms, as a class of actor-critic algorithms applicable to continuous action spaces, combine deep neural networks with deterministic policy search mechanisms, exhibiting outstanding performance in control and optimization tasks. DDPG improves training stability through experience replay mechanisms and target network structures, effectively handling policy optimization problems in high-dimensional state and action spaces.

[0006] Although existing research has attempted to apply DDPG to RUL prediction—for example, by constructing Markov Decision Processes (MDPs) to model the degradation process of devices or by combining attention mechanisms to highlight key degradation stages—these methods still face challenges such as large policy training variance, hyperparameter sensitivity, and low computational efficiency. Especially in real-world industrial scenarios, the massive amounts of data generated by high sampling frequencies and long-term monitoring place even higher demands on the real-time performance and resource consumption of prediction models. Summary of the Invention

[0007] Therefore, the purpose of this invention is to propose an efficiency optimization method for the Remaining Life (RUL) prediction framework of mechanical equipment based on Deep Deterministic Policy Gradient (DDPG) fused attention mechanism. This method uses a reinforcement learning approach based on the DDPG algorithm to fuse an attention mechanism, adaptively optimizing the efficiency of the RUL prediction framework. It enhances feature extraction and fusion capabilities through the attention mechanism and achieves adaptive hyperparameter optimization by improving the exploration strategy and gradient update mechanism of the DDPG algorithm, reducing variance and oscillations during training. Simultaneously, it introduces a lightweight network structure and parameter sharing strategy, significantly reducing model complexity and computational overhead. This addresses the limitations in engineering applications caused by strong hyperparameter dependence, low efficiency of multi-source data fusion, and high computational complexity in existing methods, achieving global optimization of prediction accuracy and operational efficiency, and providing an efficient and reliable RUL prediction solution for intelligent operation and maintenance of industrial equipment.

[0008] This invention provides an efficiency optimization method for a RUL prediction framework based on DDPG fusion AM, comprising the following steps:

[0009] S1. The original multimodal sensor time-series data is subjected to noise suppression, heterogeneity processing and length normalization using the Function Aligned Resampling (FAR) method to generate continuous functional signal data;

[0010] S2. For the current functional signal data test sequence, using the Attention-weighted Dynamic Time Warping (Attention-DTW) algorithm, calculate the similarity between the current functional signal data and historical data (e.g., ...). Figure 4 As shown in the figure, the most relevant M historical data segments are dynamically selected based on similarity to construct a degraded training dataset (a personalized training dataset of variable size).

[0011] Specifically, the Attention-DTW algorithm is used to adaptively identify the device degradation stage, filter samples with similar degradation patterns, and provide a robust data foundation for subsequent feature extraction.

[0012] S3. Construct a parallel hybrid model (HybridMultiModalRULNet) that combines attention-enhanced multi-scale convolutional neural networks (CNN) and long short-term memory networks (LSTM). Input the degradation training dataset into the parallel hybrid model, extract the degradation features of the device (extract local temporal features and global degradation patterns respectively), and output the initial predicted value of RUL (remaining lifetime).

[0013] Specifically, the parallel hybrid model's RUL prediction network adopts a dual-branch structure (attention-enhanced multi-scale deep separable convolutional network), aiming to effectively capture the short-term dependency characteristics of data at multiple scales and reduce computational complexity. It uses convolutional kernels of different sizes (3×1, 5×1, 7×1, 9×1) to extract local features, and then uses channel and spatial attention (CSAM) modules for weighted fusion. By dynamically weighting key features, it achieves efficient fusion of multi-level features. Multi-layer bidirectional LSTM combined with temporal attention mechanism captures long-term dependencies and improves the ability to extract global degradation features. It uses three hidden layers of different dimensions, namely 64, 128, and 256.

[0014] A multi-head attention mechanism is introduced to enhance the perception of key time steps, and an adaptive Bayesian probability model is used to achieve optimal fusion of multi-source features, forming a joint feature representation with strong discriminative power.

[0015] S4. The performance evaluation index of the initial prediction value and the current RUL prediction framework parameters are used as the state and input into the DDPG agent. The DDPG agent outputs actions to adjust the RUL prediction framework parameters, dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model, so as to achieve a global balance between prediction efficiency and accuracy.

[0016] Specifically, the optimization problem of key hyperparameters (such as time window length L and number of similar samples M) in RUL prediction is modeled as a Markov decision process (MDP). The dynamic adaptive adjustment of parameters is achieved by using the experience replay mechanism and target network soft update technology in the DDPG algorithm.

[0017] By improving the exploration strategy (such as adding noise perturbation) and gradient update mechanism, local optima are avoided, significantly improving convergence speed and stability.

[0018] Preferably, the feature extraction network and the DDPG agent are trained together to achieve synergistic optimization of deep learning and reinforcement learning. In the inference stage, the optimized parameter configuration is used to reduce redundant calculations, significantly reduce prediction latency, and maintain high prediction accuracy.

[0019] Further, step S1 includes:

[0020] Raw multimodal sensor time-series data is defined as ;

[0021] Where t is the length of the time series. ;

[0022] x represents the number of mechanical equipment. ;

[0023] u represents the number of sensors. ;

[0024] Calculate all values ​​within the local window h average The formula is as follows:

[0025] ;

[0026] Among them, the local window h is Centered on the data, the data first undergoes functional adaptive regression (FAR) processing, followed by local weighted scatter smoothing (LOWESS) to remove high-frequency noise. The smoothing value of LOWESS is... The locally weighted regression formula is:

[0027] ;

[0028] in, and These are the regression coefficients obtained using the weighted least squares method;

[0029] Preferably, a continuous and smooth functional signal is generated by cubic natural spline interpolation (CNS) to solve the problem of unequal data length;

[0030] CNS interpolation method is used for multi-channel functional signals Perform global fitting to generate continuous and smooth functional signal data; the fitting function for global fitting. The formula is:

[0031] ;

[0032] Among them, coefficient , , , The objective function is solved by minimizing it; its formula is:

[0033] ;

[0034] Global fitting is performed on the smoothed data to generate continuous functional signal data. The formula is:

[0035] .

[0036] Further, step S2 includes:

[0037] By using a multi-head attention layer, the attention weights of each sensor channel are dynamically learned to highlight key degradation features; the most relevant M historical data segments are selected based on similarity to construct a personalized training dataset, thereby improving the predictive accuracy and robustness.

[0038] Define the multi-channel time series of each sensor channel as input. ;

[0039] in, This represents the time-series data of the u-th sensor. For the test sequence, For the historical sequence, R represents the real number domain where the time series signal space or sensor readings are located, and T represents the set of time steps, i.e., the length of the time series or the set of sampling times {t1, t2, …, t}. n}; Calculate the attention score for each sensor channel using the following formula:

[0040] ;

[0041] in, For a multilayer perceptron, a single-channel sequence is mapped to a feature vector; Indicates test sample The time series of the u-th sensor, Representing historical samples The time series of the u-th sensor, Indicates test sample Data from the k-th sensor Representing historical samples The time series of the k-th sensor; This reflects the importance of the u-th sensor to the similarity metric;

[0042] Weight Substitute multiple channels and calculate DTW distance similarity;

[0043] DTW distance similarity metric can adaptively focus on degradation-sensitive sensors and suppress noise interference;

[0044] During system degradation, tail data usually reflects the current state and latest trends of the system. Reducing noise interference with prediction results helps to detect problems in a timely manner.

[0045] Select the M most similar segments, expressed as:

[0046] ;

[0047] in, This represents the m-th similar segment between the v-th test sample and the i-th historical sample, where i is the index of the historical sample, j is the index of the time segment, and v is the index of the test sample.

[0048] Calculate predicted labels based on the formula for calculating the most similar segments. Construct a degraded training dataset:

[0049] ;

[0050] in, This represents the predicted label of the m-th segment in the i-th historical sample corresponding to the v-th test sample; This indicates the expiration time of the segment; Indicates the start time point or time offset of the segment;

[0051] This builds a complete training dataset. This will be used in subsequent CNN feature extraction networks. Indicates the test sequence A personalized training dataset was constructed. This represents the set of labels corresponding to the dataset, i.e., the predicted remaining lifetime value for each similar segment.

[0052] Preferably, an independent multi-head attention layer is used to dynamically learn the weights of each channel in the multi-sensor sequence to highlight the contribution of key degradation features in similarity calculation.

[0053] Furthermore, the weighting Methods for calculating DTW distance similarity by substituting multiple channels include:

[0054] Use a fixed window L to capture each unlabeled element. The last L time points of the sample were obtained. Subsequently, in the historical data Xh (where Xh represents the overall set of historical samples, but only specific samples from it are used in the formula derivation), (To perform calculations, Xh no longer appears explicitly) Slide window L to extract all candidate fragments. According to the Attention-DTW similarity calculation formula, calculate The formula is:

[0055] ;

[0056] Where t is the test sequence The time step, t', is the historical sequence. The corresponding matching time step, Represents the test sequence The observation value of the u-th sensor at time step t; Represents historical sequence The observation value of the u-th sensor at time step t'.

[0057] Specifically, It is a weighted dynamic time warping (DTW) distance metric, where different sensor channels are weighted by attention. Weighting is used to calculate the similarity between multi-channel sequences.

[0058] Further, step S3 includes:

[0059] To further enhance the model's ability to capture degradation features and improve the robustness of prediction results, a multi-head attention mechanism is introduced. This mechanism adaptively focuses on key time steps by dynamically allocating weights and adaptively paying attention to the time-scale features most important for RUL prediction. Feature fusion is achieved through a Bayes probabilistic model, integrating the outputs of CNN and LSTM to calculate the joint feature distribution. Finally, the RUL prediction value is output through a fully connected layer, expressed as:

[0060] ;

[0061] in, The feature distribution after fusion, This represents the global degradation features extracted by attention-enhanced multi-scale LSTM (MSALSTM); This represents local degradation features extracted by multi-scale convolution (MSCNN); This represents the comprehensive feature representation obtained through Bayesian fusion; This represents the final RUL prediction output value.

[0062] Further, step S4 includes:

[0063] The DDPG agent uses the current prediction performance parameters (mean squared error MSE, AIC, DTW similarity, window length L, and number of samples M) as its state, and outputs continuous actions ΔL and ΔM to adjust the parameters. The formula for calculating AIC is:

[0064] ;

[0065] Where Φ is the number of parameters in the DDPG agent model, U is the number of sensor channels, L is the sliding window size, and M is the number of similar segments. It is the likelihood function value of the maximum likelihood estimate of the model;

[0066] Design a composite reward function R, including: basic reward, improvement reward, and stability reward;

[0067] The basic reward is negatively correlated with MSE and AIC, and positively correlated with DTW distance similarity, as shown in the formula:

[0068] ;

[0069] in, , , These are the weighting coefficients. The dynamic time-warped similarity index in similarity measurement reflects the degree of shape similarity between the test sequence and historical sequences after time alignment.

[0070] The improvement reward incentivizes performance metrics to improve relative to the previous step. The formula for the improvement reward is:

[0071] ;

[0072] in, , , These are the weighting coefficients. and These represent the mean squared errors of the predictions at the previous and current times, respectively. and This represents the Akaike information criterion value between the previous moment and the current moment; and It represents the dynamic time warping similarity between the previous moment and the current moment;

[0073] The stability reward is a penalty for recent performance fluctuations, and the formula for the stability reward is:

[0074] ;

[0075] in, It is the penalty coefficient for stability reward items; It represents the standard deviation of the mean square error over multiple recent time steps and is used to measure the stability of model predictions.

[0076] The formula for the total reward function is:

[0077] .

[0078] Specifically, the composite reward function R comprehensively improves the relative performance of prediction accuracy, model complexity, and historical performance. The state space S and action space A of the DDPG agent are defined as continuous values ​​for fine-tuning the frame parameters. ΔL and ΔM are the adjustment amounts for the window length L and the dataset size M, respectively.

[0079] Furthermore, the method for implementing the action of the DDPG agent outputting in step S4 to adjust the parameters of the RUL prediction framework, and dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model, includes:

[0080] During the training of the DDPG agent, the DDPG agent iteratively interacts with the RUL prediction environment. Based on its current state, the DDPG agent selects an action in the Actor network to adjust L and M. The environment executes this action, updates the values ​​of L and M, re-evaluates the performance metrics of the RUL prediction model (including MSE, AIC, and DTW distance similarity), and calculates the corresponding reward and new state. The agent stores the current state, action, reward, next state, and termination signal in an experience replay buffer. When the buffer accumulates sufficient experience, the agent randomly samples a batch of data from the buffer to update the parameters of the Actor and Critic networks, and synchronizes the target network through a soft update mechanism. This process is repeated until the agent learns an optimal policy that can adaptively adjust L and M.

[0081] After the DDPG-driven parameter adaptive optimization step, an iterative loop is entered. The degraded training dataset (personalized dynamic dataset) construction step and the attention-enhanced hybrid network prediction step are re-executed using the optimized new parameters (L', M') until the prediction performance meets the preset termination condition or the maximum number of iterations is reached.

[0082] Further, the method for step S3, which involves inputting the degraded training dataset into the parallel hybrid model, extracting the degradation features of the device, and outputting the initial predicted value of RUL, includes:

[0083] S31. Extract multi-scale local features of the input signal using parallel convolutional layers with convolutional kernels of different sizes;

[0084] S32. Adaptive weighting of extracted multi-scale features is performed through the channel and spatial attention module of the multi-head attention layer;

[0085] S33. Capture long-term temporal dependencies of device degradation by combining bidirectional long short-term memory networks with temporal attention mechanisms;

[0086] S34. Based on Bayes' theorem, the output features of the convolutional neural network branch and the long short-term memory network branch are probabilistically fused to generate the final remaining lifetime prediction value.

[0087] This invention provides end-to-end efficiency-accuracy balance optimization, applicable to real-time lifetime prediction in practical industrial scenarios.

[0088] Experimental results on public datasets (such as engine and railway freight car wheel data) show that the method of this invention outperforms the traditional CNN-LSTM model in both root mean square error (RMSE) and prediction accuracy on public engine datasets and private railway freight car wheel datasets, while reducing training time by about 30% and improving inference efficiency by about 25%. This RUL prediction model has been applied to the railway freight car condition repair diagnosis and decision system.

[0089] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the above-described method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM.

[0090] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the above-described method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM.

[0091] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0092] The RUL prediction framework efficiency optimization method based on DDPG fusion AM provided by this invention uses a reinforcement learning method based on the DDPG algorithm to fuse an attention mechanism, adaptively optimizing the efficiency of the mechanical equipment remaining life prediction framework. It enhances feature extraction and fusion capabilities through the attention mechanism and achieves adaptive hyperparameter optimization by improving the exploration strategy and gradient update mechanism of the DDPG algorithm, reducing variance and oscillations during training. Simultaneously, it introduces a lightweight network structure and parameter sharing strategy, significantly reducing model complexity and computational overhead. This solves the engineering application limitations of existing methods caused by strong hyperparameter dependence, low efficiency of multi-source data fusion, and high computational complexity. This invention addresses the issue of achieving global optimization of prediction accuracy and operational efficiency, providing an efficient and reliable RUL prediction solution for intelligent operation and maintenance of industrial equipment. By transforming the hyperparameter optimization process into a reinforcement learning task through the DDPG algorithm, it reduces the cost of manual parameter tuning and improves the model's generalization ability and robustness under varying operating conditions. Combining attention mechanisms and Bayesian fusion methods, it effectively solves the problems of heterogeneity and temporal inconsistency in multi-source sensor data, achieving accurate segmentation and feature extraction of equipment degradation stages. This invention's RUL prediction framework does not rely on a large amount of labeled data and can achieve efficient training through unsupervised or weakly supervised methods, supporting migration applications across different mechanical equipment and exhibiting wide compatibility and applicability. Attached Figure Description

[0093] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0094] In the attached diagram:

[0095] Figure 1 A complete flowchart of RUL prediction based on DDPG fusion attention mechanism provided for embodiments of the present invention;

[0096] Figure 2 A flowchart of attention-enhanced hybrid network RUL prediction provided for embodiments of the present invention;

[0097] Figure 3 This is a structural diagram of the DDPG network provided in an embodiment of the present invention;

[0098] Figure 4 This is a network structure diagram of the multi-head attention layer provided in an embodiment of the present invention;

[0099] Figure 5 This is a multidimensional data preprocessing diagram provided in an embodiment of the present invention;

[0100] Figure 6 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation

[0101] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and products consistent with some aspects of this disclosure as detailed in the appended claims.

[0102] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0103] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0104] The embodiments of the present invention will be described in further detail below.

[0105] This invention provides an efficiency optimization method for a RUL prediction framework based on DDPG fusion AM, comprising the following steps:

[0106] S1. The original multimodal sensor time-series data is subjected to noise suppression, heterogeneity processing, and length normalization using the Functionally Aligned Resampling (FAR) method to generate continuous functional signal data; Step S1 includes:

[0107] Raw multimodal sensor time-series data is defined as ;

[0108] Where t is the length of the time series. ;

[0109] x represents the number of mechanical equipment. ;

[0110] u represents the number of sensors. ;

[0111] Calculate all values ​​within the local window h average The formula is as follows:

[0112] ;

[0113] Among them, the local window h is Centered on the data, the data first undergoes Functional Adaptive Regression (FAR) processing, followed by LOWESS (Locally Weighted Scatter Smoothing) to remove high-frequency noise. The smoothing value of LOWESS is... The locally weighted regression formula is:

[0114] ;

[0115] in, and These are the regression coefficients obtained using the weighted least squares method;

[0116] In this embodiment, a continuous and smooth functional signal is generated by cubic natural spline interpolation (CNS) to solve the problem of unequal data length.

[0117] CNS interpolation method is used for multi-channel functional signals Perform global fitting to generate continuous and smooth functional signal data; the fitting function for global fitting. The formula is:

[0118] ;

[0119] Among them, coefficient , , , The objective function is solved by minimizing it; its formula is:

[0120] ;

[0121] Global fitting is performed on the smoothed data to generate continuous functional signal data. The formula is:

[0122] .

[0123] S2. For the current functional signal data test sequence, using the Attention-weighted Dynamic Time Warping (Attention-DTW) algorithm, calculate the similarity between the current functional signal data and historical data (e.g., ...). Figure 4 As shown in the figure, the most relevant M historical data segments are dynamically selected based on similarity to construct a degraded training dataset (a personalized training dataset of variable size).

[0124] The Attention-DTW algorithm is used to adaptively identify the device degradation stage, filter samples with similar degradation patterns, and provide a robust data foundation for subsequent feature extraction.

[0125] Step S2 includes:

[0126] By using a multi-head attention layer, the attention weights of each sensor channel are dynamically learned to highlight key degradation features; the most relevant M historical data segments are selected based on similarity to construct a personalized training dataset, thereby improving the predictive accuracy and robustness.

[0127] Define the multi-channel time series of each sensor channel as input. ;

[0128] in, This represents the time-series data of the u-th sensor. For the test sequence, For the historical sequence, R represents the real number domain where the time series signal space or sensor readings are located, and T represents the set of time steps, i.e., the length of the time series or the set of sampling times {t1, t2, …, t}. n}; Calculate the attention score for each sensor channel using the following formula:

[0129] ;

[0130] in, For a multilayer perceptron, a single-channel sequence is mapped to a feature vector; Indicates test sample The time series of the u-th sensor, Representing historical samples The time series of the u-th sensor, Indicates test sample Data from the k-th sensor Representing historical samples The time series of the k-th sensor; This reflects the importance of the u-th sensor to the similarity metric;

[0131] Weight Substitute multiple channels and calculate DTW distance similarity;

[0132] DTW distance similarity metric can adaptively focus on degradation-sensitive sensors and suppress noise interference;

[0133] During system degradation, tail data usually reflects the current state and latest trends of the system. Reducing noise interference with prediction results helps to detect problems in a timely manner.

[0134] Use a fixed window L to capture each unlabeled element. The last L time points of the sample were obtained. Subsequently, in the historical data Xh (where Xh represents the overall set of historical samples, but only specific samples from it are used in the formula derivation), (To perform calculations, Xh no longer appears explicitly) Slide window L to extract all candidate fragments. According to the Attention-DTW similarity calculation formula, calculate The formula is:

[0135] ;

[0136] Where t is the test sequence The time step, t', is the historical sequence. The corresponding matching time step, Represents the test sequence The observation value of the u-th sensor at time step t; Represents historical sequence The observation value of the u-th sensor at time step t'.

[0137] Select the M most similar segments, expressed as:

[0138] ;

[0139] in, This represents the m-th similar segment between the v-th test sample and the i-th historical sample, where i is the index of the historical sample, j is the index of the time segment, and v is the index of the test sample.

[0140] Calculate predicted labels based on the formula for calculating the most similar segments. Construct a degraded training dataset:

[0141] ;

[0142] in, This represents the predicted label of the m-th segment in the i-th historical sample corresponding to the v-th test sample; This indicates the expiration time of the segment; Indicates the start time point or time offset of the segment;

[0143] This builds a complete training dataset. This will be used in subsequent CNN feature extraction networks. Indicates the test sequence A personalized training dataset was constructed. This represents the set of labels corresponding to the dataset, i.e., the predicted remaining lifetime value for each similar segment.

[0144] In this embodiment, an independent multi-head attention layer is used to dynamically learn the weights of each channel in the multi-sensor sequence, highlighting the contribution of key degradation features in similarity calculation.

[0145] S3. Construct a parallel hybrid model (HybridMultiModalRULNet) combining an attention-enhanced multi-scale convolutional neural network (CNN) and a long short-term memory network (LSTM). Input the degradation training dataset into the parallel hybrid model, extract the device's degradation features (extract local temporal features and global degradation patterns respectively), and output the initial predicted value of RUL (Remaining Lifetime). Step S3 includes:

[0146] To further enhance the model's ability to capture degradation features and improve the robustness of prediction results, a multi-head attention mechanism is introduced. This mechanism adaptively focuses on key time steps by dynamically allocating weights and adaptively paying attention to the time-scale features most important for RUL prediction. Feature fusion is achieved through a Bayes probabilistic model, integrating the outputs of CNN and LSTM to calculate the joint feature distribution. Finally, the RUL prediction value is output through a fully connected layer, expressed as:

[0147] ;

[0148] in, The feature distribution after fusion, This represents the global degradation features extracted by attention-enhanced multi-scale LSTM (MSALSTM); This represents local degradation features extracted by multi-scale convolution (MSCNN); This represents the comprehensive feature representation obtained through Bayesian fusion; This represents the final RUL prediction output value.

[0149] The method for inputting the degradation training dataset into the parallel hybrid model, extracting the degradation features of the device, and outputting the initial predicted value of RUL includes:

[0150] S31. Extract multi-scale local features of the input signal using parallel convolutional layers with convolutional kernels of different sizes;

[0151] S32. Adaptive weighting of extracted multi-scale features is performed through the channel and spatial attention module of the multi-head attention layer;

[0152] S33. Capture long-term temporal dependencies of device degradation by combining bidirectional long short-term memory networks with temporal attention mechanisms;

[0153] S34. Based on Bayes' theorem, the output features of the convolutional neural network branch and the long short-term memory network branch are probabilistically fused to generate the final remaining lifetime prediction value.

[0154] S4. Using the performance evaluation metrics of the initial predicted values ​​and the current RUL prediction framework parameters as the state, input them into the DDPG agent. The DDPG agent outputs actions to adjust the RUL prediction framework parameters, dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model, achieving a global balance between prediction efficiency and accuracy. Step S4 includes:

[0155] The DDPG agent uses the current prediction performance parameters (mean squared error MSE, AIC, DTW similarity, window length L, and number of samples M) as its state, and outputs continuous actions ΔL and ΔM to adjust the parameters. The formula for calculating AIC is:

[0156] ;

[0157] Where Φ is the number of parameters in the DDPG agent model, U is the number of sensor channels, L is the sliding window size, and M is the number of similar segments. It is the likelihood function value of the maximum likelihood estimate of the model;

[0158] Design a composite reward function R, including: basic reward, improvement reward, and stability reward;

[0159] The basic reward is negatively correlated with MSE and AIC, and positively correlated with DTW distance similarity, as shown in the formula:

[0160] ;

[0161] in, , , These are the weighting coefficients. The dynamic time-warped similarity index in similarity measurement reflects the degree of shape similarity between the test sequence and historical sequences after time alignment.

[0162] The improvement reward encourages performance metrics relative to the previous step {"previous step" refers to the previous time step, i.e., the state and metric value calculated in the previous iteration of reinforcement learning, which is related to the formula for the improvement reward}. The optimization formula for the reward in the "pre" section is as follows:

[0163] ;

[0164] in, , , These are the weighting coefficients. and These represent the mean squared errors of the predictions at the previous and current times, respectively. and This represents the Akaike information criterion value between the previous moment and the current moment; and It represents the dynamic time warping similarity between the previous moment and the current moment;

[0165] The stability reward is a penalty for recent performance fluctuations, and the formula for the stability reward is:

[0166] ;

[0167] in, It is the penalty coefficient for stability reward items; It represents the standard deviation of the mean square error over multiple recent time steps and is used to measure the stability of model predictions.

[0168] The formula for the total reward function is:

[0169] .

[0170] The composite reward function R comprehensively improves the relative performance of prediction accuracy, model complexity, and historical performance. The state space S and action space A of the DDPG agent are defined as continuous values ​​for fine-tuning the frame parameters. ΔL and ΔM are the adjustment amounts for the window length L and the dataset size M, respectively.

[0171] The RUL prediction network of the parallel hybrid model adopts a two-branch structure, such as... Figure 2 As shown, an attention-enhanced multi-scale deep separable convolutional network aims to effectively capture short-term dependencies in data across multiple scales while reducing computational complexity. It extracts local features using convolutional kernels of different sizes (3×1, 5×1, 7×1, 9×1) and fuses them through channel-space attention (CSAM) modules with weighted fusion. This dynamic weighting of key features enables efficient fusion of multi-level features. A multi-layer bidirectional LSTM combined with a temporal attention mechanism captures long-term dependencies, enhancing the extraction of globally degenerate features. Three hidden layers with different dimensions (64, 128, and 256) are employed. By introducing a multi-head attention mechanism, the network enhances its ability to perceive key time steps, and an adaptive Bayesian probability model is used to achieve optimal fusion of multi-source features, forming a highly discriminative joint feature representation.

[0172] The optimization problem of key hyperparameters (such as time window length L and number of similar samples M) in RUL prediction is modeled as a Markov decision process (MDP). The empirical replay mechanism and target network soft update technique from the DDPG algorithm are used to achieve dynamic adaptive adjustment of the parameters. By improving the exploration strategy (such as adding noise perturbations) and gradient update mechanism, local optima are avoided, significantly improving convergence speed and stability.

[0173] By jointly training the feature extraction network and the DDPG agent, we achieve synergistic optimization of deep learning and reinforcement learning. In the inference stage, we use the optimized parameter configuration to reduce redundant computation, significantly reduce prediction latency, and maintain high prediction accuracy.

[0174] The DDPG agent outputs actions to adjust the parameters of the RUL prediction framework, and the implementation method for dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model includes:

[0175] During the training of the DDPG agent, the DDPG agent iteratively interacts with the RUL prediction environment. Based on its current state, the DDPG agent selects an action in the Actor network to adjust L and M. The environment executes this action, updates the values ​​of L and M, re-evaluates the performance metrics of the RUL prediction model (including MSE, AIC, and DTW distance similarity), and calculates the corresponding reward and new state. The agent stores the current state, action, reward, next state, and termination signal in an experience replay buffer. When the buffer accumulates sufficient experience, the agent randomly samples a batch of data from the buffer to update the parameters of the Actor and Critic networks, and synchronizes the target network through a soft update mechanism. This process is repeated until the agent learns an optimal policy that can adaptively adjust L and M.

[0176] DDPG-driven parameter adaptive optimization and the DDPG adaptive parameter adjustment process are as follows: Figure 3 As shown. After the DDPG-driven parameter adaptive optimization step, an iterative loop is entered, and the degenerate training dataset (personalized dynamic dataset) construction step and the attention-enhanced hybrid network prediction step are re-executed using the optimized new parameters (L', M') until the prediction performance meets the preset termination condition or the maximum number of iterations is reached.

[0177] Figure 1This embodiment illustrates the complete process of RUL prediction based on the DDPG fusion attention mechanism. The efficiency optimization method for the mechanical equipment remaining life (RUL) prediction framework based on the Deep Deterministic Policy Gradient (DDPG) fusion attention mechanism in this embodiment firstly smooths, interpolates, and standardizes the original multi-sensor time-series data through data preprocessing to form a standardized functional signal. Then, for a specific test sequence, an attention-weighted dynamic time warping algorithm is used to calculate its similarity to historical data, adaptively selecting the most relevant historical data segments to construct a personalized training dataset. Subsequently, the data is input into a multi-scale convolutional and long short-term memory (LSTM) hybrid network that integrates a multi-head attention mechanism to extract degradation features and achieve initial RUL prediction. Finally, an innovative DDPG reinforcement learning agent is introduced, using prediction performance indicators and current framework parameters as states to dynamically adjust key hyperparameters (such as input window length and dataset size). By maximizing the designed composite reward function, global adaptive optimization of prediction accuracy and computational efficiency is achieved. This effectively solves the adaptability problem of fixed-parameter models under different degradation stages and individual differences, significantly improving the overall efficiency and practicality of the RUL prediction framework.

[0178] The specific implementation process of this embodiment in practical application includes: inputting raw sensor data, and then... Figure 5 After preprocessing, standardized functional signals are obtained. For each test sequence, similarity is calculated using Attention-DTW to construct a personalized dataset, which is then input into the dataset. Figure 2 The parallel hybrid model shown extracts multi-scale features and predicts RUL. Based on the prediction results, it calculates performance indicators as input to the DDPG state. The DDPG agent generates parameter adjustment actions, updates L and M, re-optimizes the prediction process, and iterates until the termination condition is met. Finally, it outputs the predicted RUL value and the optimized parameter configuration.

[0179] This invention also provides a computer device. Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 6 As shown, the computer device includes: an input device 23, an output device 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the RUL prediction framework efficiency optimization method based on DDPG fusion AM provided in the above embodiments; wherein the input device 23, the output device 24, the memory 22, and the processor 21 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0180] The memory 22, as a read / write storage medium for a computing device, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the RUL prediction framework efficiency optimization method based on DDPG fusion AM described in this embodiment of the invention. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created according to the use of the device, etc. Furthermore, the memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include memory remotely located relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0181] Input device 23 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the device; output device 24 may include display devices such as a display screen.

[0182] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, thereby realizing the above-mentioned efficiency optimization method of RUL prediction framework based on DDPG fusion AM.

[0183] The computer equipment provided above can be used to execute the RUL prediction framework efficiency optimization method based on DDPG fusion AM provided in the above embodiments, and has corresponding functions and beneficial effects.

[0184] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the RUL prediction framework efficiency optimization method based on DDPG fusion AM as provided in the above embodiments. The storage medium can be any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (e.g., in different computer systems connected via a network). The storage medium can store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0185] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the DDPG fusion AM-based RUL prediction framework efficiency optimization method described in the above embodiments, but can also execute related operations in the DDPG fusion AM-based RUL prediction framework efficiency optimization method provided in any embodiment of the present invention.

[0186] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0187] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An efficiency optimization method for RUL prediction framework based on DDPG fusion AM, characterized in that, Includes the following steps: S1. For the original multimodal sensor time-series data, the function-aligned resampling method is used to suppress noise, process heterogeneity and normalize length to generate continuous functional signal data; S2. For the current functional signal data test sequence, combined with the attention-weighted dynamic time warping algorithm, calculate the similarity between the current functional signal data and historical data, and dynamically select the M most relevant historical data segments based on the similarity to construct the degradation training dataset. S3. Construct a parallel hybrid model of attention-enhanced multi-scale convolutional neural network and long short-term memory network, input the degradation training dataset into the parallel hybrid model, extract the degradation features of the device, and output the initial prediction value of RUL. S4. The performance evaluation index of the initial prediction value and the current RUL prediction framework parameters are used as the state and input into the DDPG agent. The DDPG agent outputs actions to adjust the RUL prediction framework parameters, dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model, so as to achieve a global balance between prediction efficiency and accuracy.

2. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 1, characterized in that, Step S1 includes: Raw multimodal sensor time series data is defined as ; Where t is the length of the time series. ; x represents the number of mechanical equipment. ; u represents the number of sensors. ; Calculate all values ​​within the local window h average The formula is as follows: ； Among them, the local window h is Centered on this, the system first undergoes functional adaptive regression processing, then uses LOWESS to remove high-frequency noise. The smoothing value of LOWESS is... The locally weighted regression calculation formula is: ； in, and These are the regression coefficients obtained using the weighted least squares method; CNS interpolation method is used for multi-channel functional signals Perform global fitting to generate continuous and smooth functional signal data; the fitting function for global fitting. The formula is: ； Among them, coefficient , , , The objective function is solved by minimizing it; its formula is: ； Global fitting is performed on the smoothed data to generate continuous functional signal data. The formula is: 。 3. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 1, characterized in that, Step S2 includes: By using a multi-head attention layer, the attention weights of each sensor channel are dynamically learned to highlight key degradation features; the most relevant M historical data segments are selected based on similarity to construct a personalized training dataset. Define the multi-channel time series of each sensor channel as input. ; in, This represents the time-series data of the u-th sensor. For the test sequence, For the historical sequence, R represents the real number domain where the time series signal space or sensor readings are located, and T represents the set of time steps, i.e., the length of the time series or the set of sampling times {t1, t2, …, t}. n }; Calculate the attention score for each sensor channel using the following formula: ； in, For a multilayer perceptron, a single-channel sequence is mapped to a feature vector; Indicates test sample The time series of the u-th sensor, Representing historical samples The time series of the u-th sensor, Indicates test sample Data from the k-th sensor Representing historical samples The time series of the k-th sensor, This reflects the importance of the u-th sensor to the similarity metric; Weight Substituting multiple channels, calculate the DTW distance similarity, select the M most similar segments, and the expression is: ； in, This represents the m-th similar segment between the v-th test sample and the i-th historical sample, where i is the index of the historical sample, j is the index of the time segment, and v is the index of the test sample. Calculate predicted labels based on the formula for calculating the most similar segments. Construct a degraded training dataset: ； in, This represents the predicted label of the m-th segment in the i-th historical sample corresponding to the v-th test sample; This indicates the expiration time of the segment; Indicates the start time point or time offset of the segment; This builds a complete training dataset. This will be used in subsequent CNN feature extraction networks. Indicates the test sequence A personalized training dataset was constructed. This represents the set of labels corresponding to the dataset, i.e., the predicted remaining lifetime value for each similar segment.

4. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 3, characterized in that, The weight Methods for calculating DTW distance similarity by substituting multiple channels include: Use a fixed window L to capture each unlabeled element. The last L time points of the sample were obtained. Then, a sliding window L is used on the historical data Xh to extract all candidate fragments. According to the Attention-DTW similarity calculation formula, calculate The formula is: ； Where t is the test sequence The time step, t', is the historical sequence. The corresponding matching time step, Represents the test sequence The observation value of the u-th sensor at time step t; Represents historical sequence The observation value of the u-th sensor at time step t'.

5. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 1, characterized in that, Step S3 includes: A multi-head attention mechanism is introduced to adaptively focus on key time steps. By dynamically allocating weights, it adaptively focuses on the time-scale features most important for RUL prediction. Feature fusion is performed using a Bayes probabilistic model, integrating the outputs of CNN and LSTM to calculate the joint feature distribution. Finally, the RUL prediction value is output through a fully connected layer, expressed as: ； in, The feature distribution after fusion, This represents the global degradation features extracted by attention-enhanced multi-scale LSTM (MSALSTM); This represents local degradation features extracted by multi-scale convolution (MSCNN); This represents the comprehensive feature representation obtained through Bayesian fusion; This represents the final RUL prediction output value.

6. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 1, characterized in that, The S4 step includes: The DDPG agent uses the current prediction performance parameters (mean squared error MSE, AIC, DTW similarity, window length L, and number of samples M) as its state, and outputs continuous actions ΔL and ΔM to adjust the parameters. The formula for calculating AIC is: ； Where Φ is the number of parameters in the DDPG agent model, L is the number of sensor channels, L is the sliding window size, and M is the number of similar segments. It is the likelihood function value of the maximum likelihood estimate of the model; Design a composite reward function R, including: basic reward, improvement reward, and stability reward; The basic reward is negatively correlated with MSE and AIC, and positively correlated with DTW distance similarity, as shown in the formula: ； in, , , These are the weighting coefficients. The dynamic time-warped similarity index in similarity measurement reflects the degree of shape similarity between the test sequence and historical sequences after time alignment. The improvement reward incentivizes performance metrics to improve relative to the previous step. The formula for the improvement reward is: ； in, , , These are the weighting coefficients. and These represent the mean squared errors of the predictions at the previous and current times, respectively. and This represents the Akaike information criterion value between the previous moment and the current moment; and It represents the dynamic time warping similarity between the previous moment and the current moment; The stability reward is a penalty for recent performance fluctuations, and the formula for the stability reward is: ； in, It is the penalty coefficient for stability reward items; It represents the standard deviation of the mean square error over multiple recent time steps and is used to measure the stability of model predictions; The formula for the total reward function is: 。 7. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 6, characterized in that, The method for dynamically optimizing the size M of the degraded training dataset and the length L of the input time window of the parallel hybrid model in step S4, where the DDPG agent outputs actions to adjust the parameters of the RUL prediction framework, includes: During the training of the DDPG agent, the DDPG agent iteratively interacts with the RUL prediction environment. Based on its current state, the DDPG agent selects an action in the Actor network to adjust L and M. The environment executes this action, updates the values ​​of L and M, re-evaluates the performance metrics of the RUL prediction model, and calculates the corresponding reward and new state. The agent stores the current state, action, reward, next state, and termination signal in an experience replay buffer. When the buffer accumulates sufficient experience, the agent randomly samples a batch of data from the buffer to update the parameters of the Actor and Critic networks, and synchronizes the target network through a soft update mechanism. This process is repeated until the agent learns an optimal policy that can adaptively adjust L and M.

8. The method for optimizing the efficiency of the RUL prediction framework based on DDPG fusion AM according to claim 1, characterized in that, The method for step S3, which involves inputting the degraded training dataset into the parallel hybrid model, extracting the degradation features of the device, and outputting the initial predicted value of RUL, includes: S31. Extract multi-scale local features of the input signal using parallel convolutional layers with convolutional kernels of different sizes; S32. Adaptive weighting of extracted multi-scale features is performed through the channel and spatial attention module of the multi-head attention layer; S33. Capture long-term temporal dependencies of device degradation by combining bidirectional long short-term memory networks with temporal attention mechanisms; S34. Based on Bayes' theorem, the output features of the convolutional neural network branch and the long short-term memory network branch are probabilistically fused to generate the final remaining lifetime prediction value.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the RUL prediction framework efficiency optimization method based on DDPG fusion AM as described in any one of claims 1-8.

10. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the RUL prediction framework efficiency optimization method based on DDPG fusion AM as described in any one of claims 1-8.

Images