A fault diagnosis method of online alignment missing scene scenario reinforcement learning combined optimization

Abstract

This invention provides a fault diagnosis method based on reinforcement learning joint optimization for online missing value scenarios, comprising: acquiring time-series data from multiple sensors such as bearings; constructing a dataset using a sliding window technique and employing a random single-channel complete masking strategy; constructing a distribution-consistent perception generative adversarial imputation network, using a temporal convolutional network (TCN) as the backbone to capture long-range spatiotemporal dependencies, and combining a weighted physical reconstruction loss to achieve high-fidelity reconstruction of missing dimensions; constructing a TCN-based fault diagnosis module to extract deep discriminative features from the reconstructed signals; introducing a deep deterministic policy gradient algorithm to construct a diagnostic feedback-driven joint optimization closed loop, and using a progressive training strategy and an asymmetric reward mechanism to adaptively adjust the feature attention weights of the imputation network through diagnostic feedback to achieve coordinated fine-tuning of the imputation and diagnostic models. Real-time monitored data containing missing values ​​is input into the jointly optimized model to achieve accurate fault identification under sensor failure conditions.

Description

Technical Field

[0001] This invention relates to the field of industrial equipment condition monitoring and fault diagnosis, and in particular to a fault diagnosis method based on reinforcement learning joint optimization for online missing alignment scenarios. Background Technology

[0002] Deep learning-based fault diagnosis techniques for rotating machinery typically assume that the acquired multi-sensor signals are complete. However, in complex industrial environments, due to electromagnetic interference, sensor loosening, or transmission failures, monitoring data often exhibits persistent and complete signal loss from specific sensor channels. This data incompleteness can lead to a sharp decline in the performance of standard diagnostic models that rely on complete input, or even cause them to fail.

[0003] Existing solutions typically employ a two-stage strategy of "incomplete first, then diagnosed," meaning that generative models are used to repair missing data before classification. However, traditional incomplete methods primarily aim to minimize signal reconstruction errors, focusing on restoring the physical waveform of the data while neglecting the ability of reconstructed features to distinguish fault categories. Because the incomplete process and the diagnostic task are disconnected, the recovered signal, while visually realistic, often loses crucial discriminative information, making it difficult to improve the final diagnostic accuracy. Therefore, there is an urgent need for a method that can synergistically optimize data incompleteness and fault diagnosis to ensure that the reconstructed data serves the diagnostic task to the greatest extent possible. Summary of the Invention

[0004] The purpose of this invention is to address the problem of decreased diagnostic accuracy in existing industrial equipment fault diagnosis technologies when sensor failure scenarios occur due to the disconnect between the completion and diagnostic task objectives, and to provide a fault diagnosis method that uses reinforcement learning joint optimization for online alignment missing scenarios.

[0005] The above-mentioned objective of the present invention is achieved through the following technical solution:

[0006] A fault diagnosis method for online array missing scenarios using reinforcement learning joint optimization includes the following steps:

[0007] Step 1: Obtain historical multi-sensor time series data of the bearing during operation and perform preprocessing to obtain the bearing fault dataset X;

[0008] Step 2: Randomly mask the samples in the bearing failure dataset X to construct missing data simulating sensor failure. Use the missing data to pre-train the generative adversarial incomplete module generated by the distribution consistency to obtain the pre-trained incomplete module and reconstructed data.

[0009] Step 3: Use the reconstructed data to perform supervised training on the fault diagnosis module to obtain the pre-trained diagnosis module;

[0010] Step 4: Construct a deep deterministic policy gradient reinforcement learning module, and perform synchronous fine-tuning of the filling module and the diagnosis module in dynamic interaction to obtain the final optimized robust fault diagnosis model.

[0011] Step 5: Obtain real-time multi-channel monitoring data of the bearing under actual operating conditions, input it into the robust fault diagnosis model to identify specific fault modes, and finally output the classification results.

[0012] Furthermore, in step 1, we assume historical multi-sensor time series data. K represents the total length of the data, and p represents the dimension of the feature;

[0013] The dataset M is processed using Z-score standardization to convert the original signal into a standard normal distribution with a mean of 0 and a standard deviation of 1. Then, a sliding window technique is used to window the dataset with a window size of W and a sliding step size of s, resulting in the dataset... W represents the length of the data, and .

[0014] Furthermore, the specific implementation of step 2 includes:

[0015] S21, based on a deterministic exclusive adversarial masking strategy, utilizes the dataset Pre-train the distribution consistency-aware generative adversarial incomplete module; specifically, construct the mask matrix. In the training iteration, a target index q is randomly and uniformly sampled from p feature dimensions, and set as follows: The q-th column is all 0s, and the rest of the columns are 1s;

[0016] S22, Construct a generator based on Temporal Convolutional Network (TCN) Its backbone is composed of multiple stacked one-dimensional dilated convolutional residual blocks to expand the temporal receptive field. Each residual block contains a one-dimensional convolution and activation function and uses residual connections. The end is mapped to the output dimension through a one-dimensional point convolution.

[0017] S23, Construct the discriminator The discriminator Also composed of TCN, it further introduces a cueing mechanism to help the discriminator distinguish whether each component in the input data is a true value actually observed or a generated value filled in by the generator; the discriminator receives the final interpolated output of the generator. As input, and outputting a probability matrix with the same dimension as the data, it is used to evaluate the authenticity of the reconstructed data, thereby forcing the generator to approximate the real data distribution during the adversarial process;

[0018] S24, Construct the overall optimization objective that includes adversarial loss and weighted physical reconstruction loss:

[0019]

[0020] Among them, combating losses The aim is to ensure that the generated data conforms to the target distribution; This is a hyperparameter used to balance the weights of adversarial loss and reconstruction loss in the overall objective; weighted reconstruction loss. Defined as:

[0021]

[0022] In this formula, the first term constrains the fidelity of the observed data, and the second term utilizes weighting coefficients. The focus is on penalizing reconstruction errors in areas masked by the code.

[0023] Furthermore, define the input tensor of the generator. The true observations preserved for the mask and the random noise at the missing locations Combinations:

[0024]

[0025] in, Represents the Hadamard product; the final interpolation output of the generator. It is reconstructed from the true portion of the original observations and the generator's estimate of the missing portion:

[0026] .

[0027] Furthermore, the specific implementation of step 3 includes:

[0028] S31, let the reconstructed data obtained after the filling module be... ;

[0029] S32, using this reconstructed data Supervised training is performed on the fault diagnosis module to extract discriminative features with high inter-class separability, resulting in a pre-trained fault diagnosis module. The fault diagnosis module establishes a foundation from the reconstructed data space. To device state space The nonlinear mapping employs a stacked TCN residual block structure. Subsequently, in the fault diagnosis module, a global average pooling layer is introduced after the last TCN layer to condense the high-dimensional temporal features into a fixed-dimensional global feature vector v. Finally, the feature vector v is input into a fully connected classifier, and the probability of a sample belonging to each fault class is calculated using the Softmax function.

[0030]

[0031] in, and Here, represents the weights and biases of the classifier, and k represents the fault category index. The cross-entropy classification loss is used to supervise the classifier in learning discriminative features, and the category with the highest probability is selected as the final diagnostic result, thus realizing the identification of fault states from incomplete data.

[0032] Furthermore, the specific implementation of step 4 includes:

[0033] S41, establish an agent architecture comprising a policy network (Actor) and a value network (Critic), where both the Actor and Critic are implemented using a multilayer perceptron, and the generative adversarial mitigation module and the fault diagnosis module are defined as the interaction environment; under this architecture, the state space... Defined as a cascaded combination of the complete time series output by the generative network-filling module and the deep discriminative features extracted by the fault diagnosis module within the current time window, the action space... Defined as a deterministic continuous action vector output by the Actor network based on the current state, used to represent the feature dimension attention weights for the loss function of the generative imputation module, and the reward function. It is defined as the gain based on the accuracy change of the fault diagnosis module between adjacent rounds, in order to evaluate the contribution of the current filling module to the diagnostic task;

[0034] S42, firstly, fix the parameters of the fault diagnosis module, and use only its feedback signal to drive the Deep Deterministic Policy Gradient Reinforcement Learning (DDPG) module to update; specifically, calculate the reward value based on the change in diagnostic accuracy, update the Critic network parameters by minimizing the temporal difference error, and update the Actor network parameters along the Q-value gradient boosting direction using the chain rule;

[0035] S43 utilizes the updated Actor output action vector and generates global feature attention weights through a global aggregation strategy. :

[0036]

[0037] Where N represents the total number of states;

[0038] Subsequently, a weighted average absolute error loss function based on this weight is constructed. This gradient signal is used as the gradient signal for backpropagation to perform targeted updates of the parameters of the generative adversarial insufficiency module:

[0039]

[0040] in, To counteract the output of the fill module, It represents the Hadamah accumulation. The balancing coefficient is used to adjust the weights of observation reconstruction error and missing value inference error; Mask is a binary mask matrix; by introducing... The system focuses on penalizing reconstruction deviations in missing areas to generate more discriminative supplementary data for fault diagnosis.

[0041] The above steps S42 to S43 are iterated n times.

[0042] S44. After completing n iterations, the parameters of the fault diagnosis module are unfixed, and the generative incomplete module and the fault diagnosis module are trained together. All network parameters are fine-tuned by the loss of the generative incomplete module and the loss of the fault diagnosis module until the model converges as a whole.

[0043] Further updates to the Critic network include:

[0044] The goal of a Critic network is to estimate the value of the current state-action pair. Fit to the true reward value By minimizing the mean square error loss function Update Critic network parameters:

[0045]

[0046] in: The loss function for the Critic network; This indicates that when the parameter is The time-based Critic network for the current state and actions The estimated value below; This represents the expectation on the state-action sample distribution.

[0047] Further updates to the Actor network include:

[0048] The Actor network maximizes the value score output by the Critic network. To optimize the action strategy, the chain rule is used to update the strategy parameters along the gradient boosting direction. :

[0049]

[0050] in: The gradient of the objective function of the Actor network; Indicates Critic value Action The gradient; This indicates that when the parameter is The Actor network outputs the gradient of the action with respect to its own parameters; Represents the state sample The mathematical expectation.

[0051] Furthermore, step 5 specifically includes: acquiring real-time multi-channel monitoring data of the bearing under actual operating conditions, which includes missing values ​​due to the failure of some sensors; inputting the real-time data sequence containing missing values ​​into the distribution consistency generative adversarial imputation module co-optimized by the deep deterministic policy gradient reinforcement learning module DDPG, using the pre-learned spatiotemporal correlation of multiple sensors to reconstruct the key features of the missing dimensions, and outputting the reconstructed complete signal sequence; subsequently, inputting the reconstructed complete signal sequence into the fault diagnosis module, which extracts deep discriminative features from the signal, identifies specific fault modes, and finally outputs the classification result, thereby obtaining the bearing fault diagnosis result under sensor failure conditions.

[0052] The present invention also provides a computer-readable storage medium, including a readable storage medium on which a computer program is stored, wherein when the computer program is executed, it implements a fault diagnosis method for online reinforcement learning joint optimization in the scenario of missing columns as described in the above technical solution.

[0053] The beneficial effects of the technical solution provided by this invention are:

[0054] (1) To address the problem of missing data in online data processing, this paper breaks through the limitations of the traditional "offline interpolation and independent diagnosis" separation model, integrates data filling and fault diagnosis into a collaborative optimization closed-loop system, establishes a joint mapping from incomplete data under sensor failure to fault state, and effectively solves the problem of model performance degradation caused by the complete failure of a single sensor in online monitoring.

[0055] (2) Construct a generative adversarial imputation model with "distribution consistency perception". In view of the distribution gap between offline training (random missing) and online deployment (complete missing), a "deterministic exclusive adversarial masking strategy" is proposed. Adversarial failure scenarios are explicitly constructed in the latent space, which forces the model to learn spatiotemporal features with distribution invariance, thereby improving the model's adaptability to unknown sensor failures.

[0056] (3) Using the DDPG algorithm, the accuracy of the diagnostic results is used as a feedback signal to dynamically adjust the attention of the interpolation network to different features. This makes the interpolation process no longer simply pursue data restoration, but prioritizes the recovery of key features that are more useful for distinguishing fault types, thereby improving the final diagnostic accuracy. Attached Figure Description

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0058] Figure 1 This is a step diagram in an embodiment of the present invention;

[0059] Figure 2- Figure 8 This presents a t-SNE visualization of the high-dimensional features extracted by different diagnostic models under sensor failure conditions on the HUST dataset; among them, Figures 2 to 7 These correspond to six comparison models: LSTM, Transformer, CGNBD, ISONet, CSST-Net, and ParC-ResMNet. Figure 8 Corresponding to the method proposed in this invention; each scatter point in the figure represents a test sample, and different colors correspond to different categories. Detailed Implementation

[0060] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0061] Embodiments of the present invention provide a fault diagnosis method based on reinforcement learning joint optimization for online missing columns scenarios.

[0062] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of a fault diagnosis method using reinforcement learning joint optimization for online missing columns in an embodiment of the present invention, including:

[0063] Step S1: Obtain historical multi-sensor time series data of the bearing during operation and preprocess it to obtain the bearing fault dataset X; construct a robust fault diagnosis model for handling missing data. This model consists of three parts: a generative adversarial imputation module generated by distribution consistency, a fault diagnosis module, and a reinforcement learning-based imputation and diagnosis co-optimization module.

[0064] Specifically: Assume historical multi-sensor time series data K represents the total length of the data, and p represents the dimension of the feature;

[0065] The dataset M is processed using Z-score standardization to convert the original signal into a standard normal distribution with a mean of 0 and a standard deviation of 1. Then, a sliding window technique is used to window the dataset with a window size of W and a sliding step size of s, resulting in the dataset... W represents the length of the data, and .

[0066] Step S2: Randomly mask the samples in the bearing failure dataset X to construct missing data simulating sensor failure; use this missing data to pre-train the generative adversarial imputation module generated by the distribution consistency, capture the potential distribution of the data and reconstruct the missing information through adversarial learning, and obtain the pre-trained imputation module and the reconstructed complete signal.

[0067] Specifically, it includes:

[0068] S21: Based on a deterministic exclusive adversarial masking strategy, utilizing the dataset The distribution consistency-aware generative adversarial incomplete module is pre-trained. Specifically, a mask matrix is ​​constructed. In the training iteration, a target index q is randomly and uniformly sampled from p feature dimensions, and set as follows: The q-th column is all 0s, and the rest of the columns are 1s;

[0069] S22: Construct a generator based on a Temporal Convolutional Network (TCN) Its backbone consists of multiple stacked one-dimensional dilated convolutional residual blocks to expand the temporal receptive field. Each residual block contains a one-dimensional convolution and an activation function, connected by residuals. The ends are mapped to the output dimension via one-dimensional point convolutions. The input tensor of the generator is defined. The true observations preserved for the mask and the random noise at the missing locations Combinations:

[0070]

[0071] in, This represents the Hadamard product. The generator's final interpolation output. It is reconstructed from the true portion of the original observations and the generator's estimate of the missing portion:

[0072]

[0073] S23: Construct the discriminator Discriminator Composed of a TCN, a further cueing mechanism is introduced to assist the discriminator in distinguishing whether the components in the input data are actual observed values ​​or generated values ​​filled in by the generator. The discriminator receives the final interpolated output from the generator. As input, and outputting a probability matrix with the same dimension as the data, it is used to evaluate the authenticity of the reconstructed data, thereby forcing the generator to approximate the real data distribution during the adversarial process;

[0074] S24: To enhance the physical reconstruction capability of the failure dimension and utilize the complete labels of the offline stage, optimization is performed through minimizing maxima game theory. A total optimization objective is constructed, incorporating adversarial loss and weighted physical reconstruction loss:

[0075]

[0076] Among them, combating losses The aim is to ensure that the generated data conforms to the target distribution; This is a hyperparameter used to balance the weights of adversarial loss and reconstruction loss in the overall objective; weighted reconstruction loss. Defined as:

[0077]

[0078] In this formula, the first term constrains the fidelity of the observed data, and the second term utilizes weighting coefficients. The focus is on penalizing the reconstruction error of the masked area (i.e., the simulated failed sensor).

[0079] Step S3: Mask the bearing fault dataset X and input it into a generative adversarial incomplete module with fixed parameters to obtain the reconstructed complete signal; use the reconstructed data to supervise the training of the fault diagnosis module, extract discriminative features with high inter-class separability, and obtain the pre-trained fault diagnosis module.

[0080] S31: Data set of bearing faults The signal is masked and then input into a generative adversarial infilling module with fixed parameters to obtain the reconstructed complete signal. ;

[0081] S32: Utilize this reconstructed data Supervised training is performed on the fault diagnosis module to extract discriminative features with high inter-class separability, resulting in a pre-trained fault diagnosis module. Specifically, this involves establishing a system based on the reconstructed data space. To device state space The nonlinear mapping is achieved by using a stacked TCN residual block structure in the main body of the network. The exponential expansion of the receptive field is achieved by using the dilated convolution mechanism, which can model both local transient impacts and long-span time dependencies without significantly increasing the computational cost.

[0082] Subsequently, in the fault diagnosis module, a global average pooling layer is introduced after the last TCN layer to condense the high-dimensional temporal features into a fixed-dimensional global feature vector v. This operation effectively reduces the number of model parameters and encourages the model to focus on statistical significance within the overall time window to improve robustness. Finally, the feature vector v is input to a fully connected classifier, and the probability of a sample belonging to each fault class is calculated using the Softmax function.

[0083]

[0084] in, and These represent the weights and biases of the classification layer, respectively, and k represents the fault category index.

[0085] To supervise the classifier in learning discriminative features, a batch cross-entropy classification loss is introduced:

[0086]

[0087] Where E represents the batch size. For the first The true category of each sample.

[0088] The model selects the category with the highest probability as the final diagnostic result, thus realizing the identification of fault states from incomplete data.

[0089] Step S4: Introduce reinforcement learning to construct a joint optimization closed loop driven by "diagnostic feedback" and adopt a progressive optimization strategy: First, fix the parameters of the pre-trained fault diagnosis model and use only its feedback signal to drive reinforcement learning to adjust the feature attention weights of the filling module, and establish the discriminative orientation of data reconstruction; then carry out collaborative joint training so that the filling and diagnosis models are simultaneously fine-tuned in dynamic interaction to obtain the final optimized robust fault diagnosis model.

[0090] S41: Constructing a Deep Deterministic Policy Gradient (DDPG) reinforcement learning module: An agent architecture comprising a policy network (Actor) and a value network (Critic) is established, where both the Actor and Critic are implemented using a Multilayer Perceptron (MLP), and the generative adversarial incomplete module and the fault diagnosis module are defined as the interaction environment; within this framework, the state space... Defined as the complete time series output by the generative filling network module within the current time window, action space Defined as a deterministic continuous action vector output by the Actor network based on the current state, used to represent the feature dimension attention weights for the loss function of the generative imputation module, and the reward function. It is defined as the gain based on the accuracy change of the fault diagnosis module between adjacent rounds, in order to evaluate the contribution of the current filling module to the diagnostic task.

[0091] S42: First, fix the parameters of the fault diagnosis module and drive the DDPG update only using its feedback signal. Specifically, calculate the reward value based on the change in diagnostic accuracy, update the Critic network parameters by minimizing the temporal difference error (TD-Error), and update the Actor network parameters along the Q-value gradient boosting direction using the chain rule;

[0092] Given that the current optimization scenario is a single-step decision-making environment without state transitions, immediate rewards are used as the direct supervision signal. First, the parameters of the fault diagnosis model are fixed, and only its feedback signal is used to drive the DDPG update.

[0093] Critic (Value) Network Update: The goal of the Critic network is to estimate the value of the current state-action pair. Fit to the true reward value The Critic network parameters are updated by minimizing the mean squared error loss function:

[0094]

[0095] in: The loss function for the Critic network; This indicates that when the parameter is The time-based Critic network for the current state and actions The estimated value below This represents the expectation on the state-action sample distribution.

[0096] Actor (Strategy) Network Update: The Actor network updates by maximizing the value score output by the Critic network. To optimize the action strategy, the chain rule is used to update the strategy parameters along the gradient boosting direction. :

[0097]

[0098] in: The gradient of the objective function of the Actor network; Indicates Critic value Action The gradient indicates the direction in which weights should be adjusted to obtain higher diagnostic returns; This indicates that when the parameter is At that time, the Actor network outputs the gradient of the action with respect to its own parameters; Represents the state sample The mathematical expectation.

[0099] S43: Utilize the updated Actor network to output action vectors, and generate global feature attention weights through a global aggregation strategy. :

[0100]

[0101] Where N represents the total number of states.

[0102] Subsequently, a weighted average absolute error loss function based on this weight is constructed. This gradient signal is used as the gradient signal for backpropagation to perform targeted updates of the parameters of the generative adversarial insufficiency module:

[0103]

[0104] in, is a balancing coefficient used to adjust the weights of observation reconstruction error and missing value inference error; Mask is a binary mask matrix. By introducing... The system focuses on penalizing reconstruction biases in missing regions, prompting the network to generate more discriminative supplementary data for fault diagnosis.

[0105] The above steps S42 to S43 are iterated n times. Through continuous trial and error and strategy adjustment by the reinforcement learning agent, the filling module is guided to learn the feature reconstruction mode that is most conducive to subsequent fault diagnosis.

[0106] S44: After completing n iterations, remove the fixation on the parameters of the fault diagnosis module, and jointly train the generative incomplete module and the fault diagnosis module. Fine-tune all network parameters through the loss of the generative incomplete module and the loss of the fault diagnosis module until the model converges as a whole.

[0107] Step S5: Acquire real-time multi-channel monitoring data of the bearing under actual operating conditions. This data includes missing values ​​due to the failure of some sensors. Input the real-time data sequence containing missing values ​​into the distribution consistency generative adversarial imputation module, which has been co-optimized by DDPG. Utilize the multi-sensor spatiotemporal correlation learned in advance by the model to reconstruct the key features of the missing dimensions and output a complete signal sequence. Subsequently, input the reconstructed complete signal sequence into the fault diagnosis module. This module extracts deep discriminative features from the signal to identify specific fault modes. Finally, output the classification result to obtain the bearing fault diagnosis result under sensor failure conditions.

[0108] In one embodiment of this invention, the experiment used the SEU rotating machinery dataset and the HUST bearing fault dataset. The SEU dataset, sourced from the dynamic simulator for power transmission systems at Southeast University, selected two speed conditions (20Hz and 30Hz), covering five modes: healthy, rolling element fault, inner ring fault, outer ring fault, and combined fault. Vibration signals from channels 1, 7, and 8 were extracted as input. The HUST dataset, sourced from the bearing fault simulation platform at Huazhong University of Science and Technology, selected acceleration signals in the X, Y, and Z orthogonal directions under a 20Hz speed condition, also including typical categories such as normal, inner ring fault, outer ring fault, rolling element fault, and combined fault.

[0109] The experimental environment used the PyTorch framework throughout the model. The server used to run the model had one Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz, 125GB of RAM, and two NVIDIA GeForce RTX 3090 GPUs.

[0110] Offline Phase: In this experiment, the architecture parameters were uniformly configured based on a grid search strategy. For the HUST and SEU datasets, the generator and discriminator were stacked with 8 and 3 layers of dilated causal convolutional structures, respectively, and the reconstruction loss weights were set to [value missing]. =5. The main difference between the two lies in the setting of the adversarial loss weights: HUST dataset setting = 1, while the SEU dataset is set to = 2. During the model training phase, the batch size is set to 32, and the Adam optimizer with a learning rate of 0.001 is used to drive parameter updates. The training process follows three stages from shallow to deep: first, the generative adversarial incomplete module is pre-trained to complete the basic data repair; then, the fault diagnosis module is trained to establish feature mapping relationships; finally, the DDPG agent is introduced to iteratively optimize the policy for feature weights, and finally, collaborative joint training is performed.

[0111] At the current stage: After the model is trained, real-time multi-channel monitoring data of the bearing under actual operating conditions (including sensor signals with missing values) is obtained; this data is then sequentially input into the generative adversarial imputation module generated by distribution consistency to reconstruct the missing key features, and then input into the fault diagnosis module to identify the specific fault mode, finally obtaining the bearing fault diagnosis result under sensor failure conditions.

[0112] To comprehensively verify the effectiveness and generalization ability of the proposed method in industrial sensor failure scenarios, this invention conducted comparative experiments with six other representative advanced fault diagnosis models. These comparative methods include: LSTM, a classic variant of recurrent neural networks, used to establish a performance benchmark for fault diagnosis based on imputed data; Transformer, representing a pure attention mechanism, used to evaluate the effectiveness of this architecture in mining fault patterns from reconstructed time-series data; CGNBD, representing a physically guided denoising approach, aimed to evaluate the feature extraction capability of classifier-guided blind deconvolution in noisy environments; ISONet, an improved 1D-CNN architecture, used to verify the ability of input space parameterization strategies to mine spatial correlations of multi-channel signals; CSST-Net, combining spatially augmented convolution and Vision Transformer, utilizing ViT's global attention mechanism to capture long-range dependencies; and ParC-ResMNet, overcoming the limitations of traditional CNNs' local receptive fields through position-aware recurrent convolution and residual masking strategies. All methods maintained consistent data input and output methods, and the same hyperparameter settings were used in the experiments to ensure fairness. Evaluation metrics include Accuracy, Precision, Recall, and F1-score. To intuitively evaluate the model's performance at the feature extraction level, Figures 2-8 The high-dimensional features extracted by different diagnostic models under sensor failure conditions in the HUST dataset are visualized using t-SNE visualization. Detailed comparison of diagnostic result evaluation indicators is shown in Table 1. The results in bold in the table show that the method proposed in this invention outperforms other methods in terms of performance.

[0113] Table 1. Comparison of fault diagnosis performance of different methods based on three-dimensional feature averaging

[0114]

[0115] As shown in Table 1, the experimental results comparing different methods demonstrate that the proposed method significantly outperforms the other six methods in all four evaluation metrics—Accuracy, Precision, Recall, and F1-score—on both the HUST and SEU datasets. On the HUST dataset, the proposed method achieves a score of 0.9962 for all metrics, representing a performance improvement of over 16% compared to the second-best performing ParC-ResMNet, showcasing extremely high diagnostic reliability even with partial sensor failures. On the more challenging SEU dataset, the severe feature loss due to sensor failure significantly disrupts the data distribution, causing a sharp decline in the performance of traditional models such as LSTM and Transformer. However, the proposed method still maintains a high accuracy of 0.8310 and an F1 score of 0.8093. Overall, the proposed method exhibits generally high diagnostic accuracy and strong robustness under various conditions, demonstrating significantly better overall performance than current mainstream advanced algorithms.

[0116] Furthermore, to verify the contribution of the DDPG adaptive weighting mechanism and channel-level mask training strategy in the proposed framework to the final diagnostic performance, ablation experiments were conducted on the HUST dataset. Three variant schemes were set up for comparison: the DDPG removal scheme removed the reinforcement learning agent and directly used fixed weights for joint training to verify the necessity of dynamic weights; the random mask scheme retained the same architecture but replaced the channel-level mask with a random point mask to evaluate the impact of a specific masking strategy on capturing sensor spatial correlations; the mean-filling scheme used only mean filling as a baseline control. The specific results of the ablation experiments are shown in Table 2.

[0117] Table 2 Evaluation Indicators for Ablation Comparison Experiment

[0118]

[0119] The ablation experiment results (Table 2) strongly validate the necessity of each key component in the framework. Mean imputation, by completely ignoring the spatiotemporal correlation of the signal, resulted in the worst diagnostic performance and was essentially unable to cope with sensor failure scenarios. Although the random masking strategy was superior to simple imputation, it could not simulate real channel-level missing signals, leading to insufficient generalization ability of the model when faced with the loss of entire signal columns. The model performance declined after removing DDPG, proving that without the agent's dynamic adjustment of feature weights, the model's accurate capture of key fault features would be limited. Ultimately, this method achieved optimal performance through deep collaboration among the modules, establishing the effectiveness of the joint optimization strategy.

[0120] On the other hand, embodiments of the present invention also provide a computer-readable storage medium, including a readable storage medium on which a computer program is stored, wherein when the computer program is executed, it implements a fault diagnosis method for online reinforcement learning joint optimization of missing columns scenario as described in the above technical solution.

[0121] The above description is merely a specific embodiment of the present invention, used to illustrate the technical solution of the present invention and not to limit it. Those skilled in the art can make various modifications, equivalent substitutions, or additions to the above embodiments without departing from the spirit and scope of the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A fault diagnosis method for online array missing scenarios using reinforcement learning joint optimization, characterized in that, Includes the following steps: Step 1: Obtain historical multi-sensor time series data of the bearing during operation and perform preprocessing to obtain the bearing fault dataset X; Step 2: Randomly mask the samples in the bearing failure dataset X to construct missing data simulating sensor failure. Use the missing data to pre-train the generative adversarial incomplete module generated by the distribution consistency to obtain the pre-trained incomplete module and reconstructed data. Step 3: Use the reconstructed data to perform supervised training on the fault diagnosis module to obtain the pre-trained diagnosis module; Step 4: Construct a deep deterministic policy gradient reinforcement learning module, and perform synchronous fine-tuning of the filling module and the diagnosis module in dynamic interaction to obtain the final optimized robust fault diagnosis model. The specific implementation of step 4 includes: S41, establish an agent architecture comprising a policy network (Actor) and a value network (Critic), where both the Actor and Critic are implemented using a multilayer perceptron, and the generative adversarial mitigation module and the fault diagnosis module are defined as the interaction environment; under this architecture, the state space... Defined as a cascaded combination of the complete time series output by the generative network-filling module and the deep discriminative features extracted by the fault diagnosis module within the current time window, the action space... Defined as a deterministic continuous action vector output by the Actor network based on the current state, used to represent the feature dimension attention weights for the loss function of the generative imputation module, and the reward function. It is defined as the gain based on the accuracy change of the fault diagnosis module between adjacent rounds, in order to evaluate the contribution of the current filling module to the diagnostic task; S42, firstly, fix the parameters of the fault diagnosis module, and use only its feedback signal to drive the Deep Deterministic Policy Gradient Reinforcement Learning (DDPG) module to update; specifically, calculate the reward value based on the change in diagnostic accuracy, update the Critic network parameters by minimizing the temporal difference error, and update the Actor network parameters along the Q-value gradient boosting direction using the chain rule; S43 utilizes the updated Actor output action vector and generates global feature attention weights through a global aggregation strategy. : ； Where N represents the total number of states; Subsequently, a weighted average absolute error loss function based on this weight is constructed. This gradient signal is used as the gradient signal for backpropagation to perform targeted updates of the parameters of the generative adversarial insufficiency module: ； in, To counteract the output of the fill module, It represents the Hadamah accumulation. The balancing coefficient is used to adjust the weights of observation reconstruction error and missing value inference error; Mask is a binary mask matrix; by introducing... The system focuses on penalizing reconstruction deviations in missing areas to generate more discriminative supplementary data for fault diagnosis. The above steps S42 to S43 are iterated n times. S44. After completing n iterations, the parameters of the fault diagnosis module are unfixed, and the generative incomplete module and the fault diagnosis module are trained together. All network parameters are fine-tuned by the loss of the generative incomplete module and the loss of the fault diagnosis module until the model converges as a whole. Step 5: Obtain real-time multi-channel monitoring data of the bearing under actual operating conditions, input it into the robust fault diagnosis model to identify specific fault modes, and finally output the classification results.

2. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 1, characterized in that: In step 1, we assume historical multi-sensor time series data. K represents the total length of the data, and p represents the dimension of the feature; The dataset M is processed using Z-score standardization to convert the original signal into a standard normal distribution with a mean of 0 and a standard deviation of 1. Then, a sliding window technique is used to window the dataset with a window size of W and a sliding step size of s, resulting in the dataset... W represents the length of the data, and .

3. The fault diagnosis method for online array missing scenarios using reinforcement learning joint optimization as described in claim 1, characterized in that: The specific implementation of step 2 includes: S21, based on a deterministic exclusive adversarial masking strategy, utilizes the dataset Pre-train the distribution consistency-aware generative adversarial incomplete module; specifically, construct the mask matrix. In the training iteration, a target index q is randomly and uniformly sampled from p feature dimensions, and set as follows: The q-th column is all 0s, and the rest of the columns are 1s; S22, Construct a generator based on Temporal Convolutional Network (TCN) Its backbone is composed of multiple stacked one-dimensional dilated convolutional residual blocks to expand the temporal receptive field. Each residual block contains a one-dimensional convolution and activation function and uses residual connections. The end is mapped to the output dimension through a one-dimensional point convolution. S23, Construct the discriminator The discriminator It is also composed of TCN, and further introduces a prompting mechanism to help the discriminator distinguish whether each component in the input data is the actual observed value or the generated value filled in by the generator; the discriminator receives the final interpolated output of the generator. As input, and outputting a probability matrix with the same dimension as the data, it is used to evaluate the authenticity of the reconstructed data, thereby forcing the generator to approximate the real data distribution during the adversarial process; S24, Construct the overall optimization objective that includes adversarial loss and weighted physical reconstruction loss: ； Among them, combating losses The aim is to ensure that the generated data conforms to the target distribution; This is a hyperparameter used to balance the weights of adversarial loss and reconstruction loss in the overall objective; weighted reconstruction loss. Defined as: ； In this formula, the first term constrains the fidelity of the observed data, and the second term utilizes weighting coefficients. The focus is on penalizing reconstruction errors in areas masked by the code.

4. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 3, characterized in that: Define the input tensor of the generator The true observations preserved for the mask and the random noise at the missing locations Combinations: ； in, Represents the Hadamard product; the final interpolation output of the generator. It is reconstructed from the true portion of the original observations and the generator's estimate of the missing portion: 。 5. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 1, characterized in that: The specific implementation of step 3 includes: S31, let the reconstructed data obtained after the filling module be... ; S32, using this reconstructed data Supervised training is performed on the fault diagnosis module to extract discriminative features with high inter-class separability, resulting in a pre-trained fault diagnosis module. The fault diagnosis module establishes a foundation from the reconstructed data space. To device state space The nonlinear mapping employs a stacked TCN residual block structure. Subsequently, in the fault diagnosis module, a global average pooling layer is introduced after the last TCN layer to condense the high-dimensional temporal features into a fixed-dimensional global feature vector v. Finally, the feature vector v is input into a fully connected classifier, and the probability of a sample belonging to each fault class is calculated using the Softmax function. ； in, and Here, represents the weights and biases of the classifier, and k represents the fault category index. The cross-entropy classification loss is used to supervise the classifier in learning discriminative features, and the category with the highest probability is selected as the final diagnostic result, thus realizing the identification of fault states from incomplete data.

6. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 1, characterized in that: Critic network updates include: The goal of a Critic network is to estimate the value of the current state-action pair. Fit to the true reward value By minimizing the mean square error loss function Update Critic network parameters: ； in: The loss function for the Critic network; This indicates that when the parameter is The time-based Critic network for the current state and actions The estimated value below; This represents the expectation on the state-action sample distribution.

7. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 1, characterized in that: Actor network updates include: The Actor network maximizes the value score output by the Critic network. To optimize the action strategy, the chain rule is used to update the strategy parameters along the gradient boosting direction. : ； in: The gradient of the objective function of the Actor network; Indicates Critic value Action The gradient; This indicates that the parameter is The Actor network outputs the gradient of the action with respect to its own parameters; Represents the state sample The mathematical expectation.

8. The fault diagnosis method for online array missing scenario reinforcement learning joint optimization as described in claim 1, characterized in that: Step 5 specifically includes: acquiring real-time multi-channel monitoring data of the bearing under actual operating conditions, which includes missing values ​​due to the failure of some sensors; inputting the real-time data sequence containing missing values ​​into the distribution consistency generative adversarial imputation module co-optimized by the deep deterministic policy gradient reinforcement learning module DDPG, using the pre-learned spatiotemporal correlation of multiple sensors to reconstruct the key features of the missing dimensions, and outputting the reconstructed complete signal sequence; subsequently, inputting the reconstructed complete signal sequence into the fault diagnosis module, which extracts deep discriminative features from the signal, identifies specific fault modes, and finally outputs the classification result, thereby obtaining the bearing fault diagnosis result under sensor failure conditions.

9. A computer-readable storage medium, characterized in that, It includes a readable storage medium on which a computer program is stored, and when the computer program is executed, it implements a fault diagnosis method for online reinforcement learning joint optimization in the scenario of missing columns as described in any one of claims 1-8.

Images