Dynamic perception and adaptive decision fault diagnosis system based on deep reinforcement learning

Abstract

The invention relates to the technical field of decision fault diagnosis, in particular to a dynamic perception and adaptive decision fault diagnosis system based on deep reinforcement learning. According to the technical scheme, the system comprises a multi-source sensor module, a real-time state encoder, a depth Q network decision module, a dynamic execution controller and a feedback learning loop, and the multi-source sensor module is used for collecting and preprocessing vibration, temperature and acoustic signals of equipment; the real-time state encoder is used for dynamically adjusting the data sampling rate and the sliding window length, extracting time domain, frequency domain and frequency domain features, fusing the environmental noise level and the CPU occupancy rate, and generating a low-dimensional state vector. According to the method, the problems of data dynamic change and complex fault mode recognition in mechanical equipment fault diagnosis can be effectively solved, meanwhile, the computing resource utilization rate is remarkably optimized, and an efficient and accurate solution is provided for intelligent fault diagnosis in an actual industrial scene.

Description

Technical Field

[0001] The present invention relates to the technical field of decision-making fault diagnosis, and in particular to a dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning. Background Art

[0002] In industrial production, fault diagnosis of mechanical equipment is a key link to prevent accidents, ensure personal and equipment safety, and improve economic benefits. Fault diagnosis technology can reduce the probability of major emergencies, and at the same time, through prediction technology, it can achieve the purpose of extending the service life of mechanical equipment and reducing the probability and time of downtime. However, traditional fault diagnosis methods usually rely on fixed threshold judgments or static rule guidance, which is difficult to adapt to the detection needs of equipment degradation, environmental changes, and complex fault modes. In the prior art, the selection of sampling rate, feature extraction method, and diagnosis model is often fixed, and it is impossible to dynamically adjust according to the real-time status of the equipment, resulting in low utilization of available resources and insufficient diagnostic accuracy. In addition, the data fusion and real-time processing capabilities of multi-source sensors in complex industrial scenarios are limited, which further limits the accuracy and real-time nature of fault diagnosis. Therefore, there is a need for an intelligent fault diagnosis method that can dynamically perceive the equipment status and adaptively adjust the decision to improve the accuracy of fault diagnosis, rationally utilize resources, and meet the real-time requirements of industrial scenarios. Summary of the invention

[0003] The purpose of the present invention is to propose a dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning to address the problem that traditional fault diagnosis methods in the background technology usually rely on fixed threshold judgments or static rule guidance, which are difficult to adapt to the detection needs of equipment degradation, environmental changes and complex fault modes.

[0004] The technical solution of the present invention: A dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning, comprising: a multi-source sensor module for collecting vibration, temperature and acoustic signals of the equipment and performing pre-processing; Real-time state encoder dynamically adjusts data sampling rate and sliding window length, extracts time domain, frequency domain and time-frequency domain features, integrates environmental noise level and CPU occupancy rate, and generates low-dimensional state vector; The deep Q network (DQN) decision module designs a hierarchical action space and reward function based on the state space. The state space includes root mean square (RMS), kurtosis, peak frequency, energy entropy, noise level, and CPU occupancy. It uses the ε-greedy strategy to balance exploration and utilization and output the optimal decision action. A dynamic execution controller adjusts hardware parameters and schedules diagnostic algorithms according to the optimal decision action to perform action combination; The feedback learning loop dynamically updates the network parameters of the deep Q network (DQN) and evaluates the decision confidence through online incremental learning and uncertainty processing, forming a closed-loop optimization system.

[0005] Optionally, the multi-source sensor module includes: Three-axis acceleration sensor, used to collect vibration signals; K-type thermocouple sensor, used to collect temperature signals; A microphone sensor for collecting acoustic signals; The preprocessing includes aligning the multimodal data signal synchronization with hardware timestamps and performing noise reduction filtering and normalization processing using a wavelet threshold method.

[0006] Optionally, the operation of the real-time state encoder includes: (a) Dynamic sampling rate adjustment: low-frequency sampling is used when the device is in a stable state, and high-frequency sampling is switched to capture transient features when the device is in an abnormal state; (b) Sliding window length optimization: adaptively adjust the window length according to the signal period and real-time requirements; (c) Multi-dimensional feature extraction, including: Time domain characteristics: root mean square (RMS), kurtosis; Frequency domain features: Fast Fourier transform (FFT) peak, wavelet packet energy; Time-frequency domain features: wavelet packet node energy.

[0007] Optionally, the hierarchical action space specifically includes: Sampling frequency: including the sampling frequency of temperature sensor F0, acoustic sensor F1, and vibration sensor F2; Feature set: time domain feature set SetA, frequency domain feature set SetB, time-frequency domain feature set SetC; Diagnostic model: threshold alarm M0, support vector machine (SVM) M1, one-dimensional convolutional neural network (1D-CNN) M2.

[0008] Optionally, the reward function is: in, As the basic reward item, is the penalty item, is the accuracy weight, Penalty for resource consumption, Punishment for false reporting, is the fault detection accuracy, To calculate resource utilization, The number of false positives.

[0009] Optionally, the action combination of the dynamic execution controller includes: Hardware adjustment: dynamically switch sensor channels and adjust analog-to-digital converter (ADC) sampling rate; Algorithm scheduling: Generate action options by cross-combining sampling rate, feature set and diagnostic model.

[0010] Optionally, the action options include: {F0_SetA_M0}: low frequency sampling, time domain characteristics, threshold alarm; {F1_SetB_M1}: intermediate frequency sampling, frequency domain features, SVM classification; {F2_SetC_M2}: high-frequency sampling, time-frequency domain features, and 1D-CNN model.

[0011] Optionally, the feedback learning loop specifically includes: Online incremental learning: store immediate reward data through the experience replay buffer and regularly update DQN parameters through random sampling; Uncertainty handling: When the decision confidence is lower than the preset threshold, redundant detection or manual intervention is triggered.

[0012] Compared with the prior art, the present invention has at least one of the following beneficial technical effects: Through the collaborative data collection of multi-modal sensors such as vibration, temperature, and acoustics, combined with hardware timestamp alignment and wavelet noise reduction technology, the synchronization and accuracy of the data are ensured, and the operation status of the equipment is fully captured; the sampling rate and sliding window length are dynamically adjusted, low-frequency sampling is used when the equipment is stable to save resources, and high-frequency sampling is switched to capture transient fault characteristics when abnormal, ensuring the timeliness and accuracy of signal analysis. Combining multi-dimensional feature extraction methods in time domain, frequency domain, and time-frequency domain, it can effectively identify complex fault modes and improve diagnostic accuracy.

[0013] The reward function weighs the diagnostic accuracy, resource consumption and false alarm rate to achieve the optimal balance between diagnostic performance and cost. The experience replay buffer continuously stores real-time data, dynamically updates the DQN model parameters, adapts to equipment degradation, environmental noise changes and new failure modes, and avoids model obsolescence. Through confidence-based evaluation, when the decision confidence is low, redundant detection or manual intervention is triggered to reduce the risk of misjudgment and ensure the high reliability of the diagnostic results.

[0014] The present invention proposes a dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning, which can not only effectively solve the problems of dynamic data changes and complex fault pattern recognition in mechanical equipment fault diagnosis, but also significantly optimize the utilization of computing resources, providing an efficient and accurate solution for intelligent fault diagnosis in actual industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1This is the structural diagram of the dynamic perception and adaptive decision-making system.

[0016] Figure 2 This is the execution action distribution diagram proposed by the present invention.

[0017] Figure 3 Updated flowchart for dynamic perception and adaptive decision-making system. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0019] Example

[0020] like Figure 1 As shown, the dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning proposed in the present invention includes a multi-source sensor module, a real-time state encoder, a deep Q network (DQN) decision module, a dynamic execution controller and a feedback learning loop. Each module is described in detail below.

[0021] In this embodiment, the multi-source sensor module uses hardware timestamp alignment, wavelet threshold noise reduction and normalization to preprocess the collected vibration, temperature and acoustic signals. The real-time state encoder dynamically adjusts the sampling rate and sliding window length, extracts time domain (RMS, kurtosis), frequency domain and time-frequency domain features, and generates a low-dimensional state vector containing the equipment health score and environmental noise level.

[0022] The calculation formula of the root mean square value is: in, To represent the signal Sample values, is the total number of sampling points; The function of is to normalize the total energy (or total power) of the signal to obtain the average energy of the signal; The calculation formula of kurtosis is: Where: N is the total number of samples, is the ith data value, represents the mean of the data, and -3 is an adjustment term used to define the kurtosis of the normal distribution as 0.

[0023] The decision module defines the state space and hierarchical action space, and uses the ε-greedy strategy to balance exploration and utilization. At the beginning of training, a random number between 0 and 1 is generated. If the random number is less than the exploration rate, a random action is executed. Otherwise, the best action value function is selected based on the learned information. ,in , To accumulate rewards in the long term; the reward function is designed to be divided into basic rewards and penalties: In the above formula: As the basic reward item, is the penalty item, is the accuracy weight, Penalty for resource consumption, Punishment for false reporting, is the fault detection accuracy, To calculate resource utilization, The number of false positives.

[0024] The reward function is designed to combine the diagnostic accuracy and computational cost to achieve intelligent decision-making. The mean square error is used as the loss function during training to find the optimal parameter value. The formula is: In the above formula: is the optimal parameter, N is the total number of samples, r is the immediate reward, is the discount factor, Q(s,a) is the action value function at the current moment, is the action value function at the next moment.

[0025] After calculating the loss function, the gradient descent is performed, and the formula is: In the above formula: is the parameter vector, is the learning rate, is the gradient of the loss function with respect to the parameters.

[0026] like Figure 2 and Figure 3As shown in the figure, the dynamic execution controller executes the output action of the decision module through hardware adjustment and algorithm scheduling. There are 27 selectable actions, such as {F0_SetA_M0)}{F1_SetB_M1}{F2_SetC_M2}, which respectively represent the sampling rate of F0, time domain feature extraction, diagnostic model as threshold method, sampling rate of F1, frequency domain feature extraction, diagnostic model as SVM, and sampling rate of F2, time-frequency domain feature extraction, diagnostic model as 1D-CNN. The three-level linkage of sampling rate, feature extractor, and diagnostic model, combined with their respective advantages, is suitable for scenarios with different complexity, computing speed and requirements. Through dynamic adjustment and optimization, the system significantly reduces the consumption of computing resources while ensuring the diagnostic accuracy. It is suitable for mechanical equipment fault diagnosis in complex industrial environments, can significantly reduce maintenance costs, extend equipment life, and improve the reliability and safety of production lines. The feedback learning loop realizes online incremental learning and uncertainty processing mechanism, dynamically updates model parameters and evaluates decision confidence to form a closed-loop optimization system. Through dynamic adjustment and optimization, the system significantly reduces computing resource consumption while ensuring diagnostic accuracy. It is suitable for mechanical equipment fault diagnosis in complex industrial environments, and can significantly reduce maintenance costs, extend equipment life, and improve the reliability and safety of production lines.

[0027] In this implementation, through the collaborative collection of multimodal sensors such as vibration, temperature, and acoustics, combined with hardware timestamp alignment and wavelet noise reduction, high-precision data synchronization and redundancy removal are achieved to fully capture device status information. The sampling rate and sliding window length are dynamically adjusted to save resources when the equipment is stable, and high-frequency sampling is used to capture transient features during abnormalities to ensure the timeliness and accuracy of signal analysis. Multi-dimensional features in the time domain (RMS, kurtosis), frequency domain (FFT peak, wavelet packet energy), and time-frequency domain (wavelet packet node energy) are fused to effectively identify complex fault modes (such as nonlinear faults and multiple fault superpositions). Through the dynamic perception and adaptive decision-making architecture of deep reinforcement learning, the limitations of traditional static diagnosis methods have been broken through, and high-precision, low-latency, and low-cost fault diagnosis has been achieved in complex industrial environments.

[0028] It is worth noting that the experience replay buffer continuously stores real-time data, dynamically updates DQN model parameters, adapts to equipment degradation, environmental noise changes and new fault modes, and avoids model obsolescence. Based on confidence assessment, when the decision confidence is low, redundant detection or manual intervention is triggered to reduce the risk of misjudgment and ensure the high reliability of the diagnosis results. By dynamically adjusting the sampling rate and diagnostic model, computing power is saved during the stable period of the equipment, and resources are concentrated for rapid response during the abnormal period, which significantly reduces computing resource consumption and reduces average power consumption by 20%-30%. It supports millisecond-level fault detection and decision-making to meet the real-time monitoring needs of the production line; the modular design facilitates the expansion of new sensors or diagnostic algorithms to adapt to different industrial scenarios. Through accurate prediction and early intervention, unplanned downtime can be reduced by more than 50%, the service life of equipment can be extended, maintenance costs can be reduced, and production efficiency can be improved.

[0029] The above specific embodiments are only several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning, characterized in that: include: Multi-source sensor module, used to collect vibration, temperature and acoustic signals of equipment and perform pre-processing; Real-time state encoder dynamically adjusts data sampling rate and sliding window length, extracts time domain, frequency domain and time-frequency domain features, integrates environmental noise level and CPU occupancy rate, and generates low-dimensional state vector; The deep Q network decision module designs a hierarchical action space and reward function based on the state space. The state space includes root mean square, kurtosis, peak frequency, energy entropy, noise level, and CPU occupancy. It balances exploration and utilization through the ε-greedy strategy and outputs the optimal decision action. A dynamic execution controller adjusts hardware parameters and schedules diagnostic algorithms according to the optimal decision action to perform action combination; The feedback learning loop dynamically updates the network parameters of the deep Q network and evaluates the decision confidence through online incremental learning and uncertainty processing, forming a closed-loop optimization system.

2. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 1 is characterized in that: The multi-source sensor module comprises: Three-axis acceleration sensor, used to collect vibration signals; K-type thermocouple sensor, used to collect temperature signals; A microphone sensor for collecting acoustic signals; The preprocessing includes aligning the multimodal data signal synchronization with hardware timestamps and performing noise reduction filtering and normalization processing using a wavelet threshold method.

3. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 1 is characterized in that: The operation of the real-time state encoder includes: (a) Dynamic sampling rate adjustment: low-frequency sampling is used when the device is in a stable state, and high-frequency sampling is switched to capture transient features when the device is in an abnormal state; (b) Sliding window length optimization: adaptively adjust the window length according to the signal period and real-time requirements; (c) Multi-dimensional feature extraction, including: Time domain characteristics: root mean square, kurtosis; Frequency domain features: Fast Fourier transform peak, wavelet packet energy; Time-frequency domain features: wavelet packet node energy.

4. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 1 is characterized in that: The hierarchical action space specifically includes: Sampling frequency: including the sampling frequency of temperature sensor F0, acoustic sensor F1, and vibration sensor F2; Feature set: time domain feature set SetA, frequency domain feature set SetB, time-frequency domain feature set SetC; Diagnostic model: threshold alarm M0, support vector machine M1, one-dimensional convolutional neural network M2.

5. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 4 is characterized in that: The reward function is: in, As the basic reward item, is the penalty item, is the accuracy weight, Penalty for resource consumption, Punishment for false reporting, is the fault detection accuracy, To calculate resource utilization, The number of false positives.

6. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 5 is characterized in that: The action combination of the dynamic execution controller includes: Hardware adjustment: dynamically switch sensor channels and adjust analog-to-digital converter sampling rate; Algorithm scheduling: Generate action options by cross-combining sampling rate, feature set and diagnostic model.

7. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 6 is characterized in that: The action options include: {F0_SetA_M0}: low frequency sampling, time domain characteristics, threshold alarm; {F1_SetB_M1}: intermediate frequency sampling, frequency domain features, SVM classification; {F2_SetC_M2}: high-frequency sampling, time-frequency domain features, and 1D-CNN model.

8. The dynamic perception and adaptive decision-making fault diagnosis system based on deep reinforcement learning according to claim 1 is characterized in that: The feedback learning loop specifically includes: Online incremental learning: store immediate reward data through the experience replay buffer and regularly update DQN parameters through random sampling; Uncertainty handling: When the decision confidence is lower than the preset threshold, redundant detection or manual intervention is triggered.

Images