Intelligent global dynamic maintenance method for multi-component system considering imperfect maintenance based on deep reinforcement learning
By constructing degradation and maintenance decision models for multi-component systems using deep reinforcement learning, the problems of global collaborative optimization and dynamic adaptation in multi-component systems are solved, achieving efficient resource utilization and improved system availability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2026-04-16
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies struggle to achieve global collaborative optimization in multi-component systems. Traditional maintenance strategies lack dynamic collaborative capabilities, and imperfect maintenance effects are not adequately considered, leading to redundant investment of maintenance resources and reduced system availability.
A deep reinforcement learning-based approach is adopted to construct a stochastic process model of component degradation, perform RUL prediction through Bayesian updates, establish a maintenance timing bias penalty model, and construct a deep reinforcement learning decision model to achieve joint optimization of dynamic global group maintenance and opportunistic imperfect maintenance.
It improves the efficiency of maintenance resource utilization in multi-component systems, reduces long-term average maintenance costs, and enhances system availability and the real-time implementability of maintenance strategies.
Smart Images
Figure CN122367445A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of equipment health management and intelligent operation and maintenance technology, specifically involving an intelligent global dynamic maintenance method for multi-component systems that considers imperfect maintenance based on deep reinforcement learning. Background Technology
[0002] Predictive Health Management (PHM) technology has been widely applied to condition monitoring, fault diagnosis, and remaining useful life prediction, gradually forming a closed-loop operation and maintenance system of "condition awareness—health assessment / life prediction—maintenance decision—maintenance execution feedback". In this closed loop, maintenance decisions need to transform dynamic monitoring data and health assessment results into executable maintenance plans to achieve proactive risk control and lifecycle cost optimization.
[0003] Traditional maintenance strategies typically include time-based maintenance (TBM) and condition-based maintenance (CBM). Time-based maintenance relies on preset time intervals or equipment lifespan for scheduling, offering simplicity but lacking responsiveness to changes in equipment health, easily leading to over-maintenance or delayed maintenance. Condition-based maintenance, triggered by sensor monitoring data and health assessment results, can improve the targeting and cost-effectiveness of maintenance to some extent. However, in multi-component systems, a single maintenance strategy struggles to meet the collaborative optimization needs of complex systems. Multi-component systems are usually composed of heterogeneous components from multiple sources, with significant differences in degradation mechanisms, degradation rates, and failure thresholds among the components. Traditional single-component-based maintenance optimization methods struggle to achieve global optimization at the system level. Furthermore, in actual operation and maintenance, systems generally exhibit resource coupling and downtime coupling characteristics; a single maintenance preparation or shutdown operation can simultaneously serve multiple components. If independent triggering of maintenance is still used, it will lead to redundant investment of maintenance resources and reduce the overall availability of the system.
[0004] Furthermore, existing maintenance methods are mostly based on the assumption of perfect maintenance, which assumes that components can be fully restored to their initial state after maintenance. However, in engineering practice, maintenance results are often imperfect; components only achieve partial performance recovery, and the recovery effect decreases with the number of maintenance cycles. Such idealized assumptions will cause maintenance timing and maintenance combination decisions to deviate from actual operational requirements, thereby reducing the robustness and applicability of maintenance strategies.
[0005] Furthermore, multi-component systems may experience sudden failure events during operation, triggering unplanned maintenance opportunities. Traditional maintenance combination methods based on rolling optimization or fixed rules often struggle to effectively integrate unplanned maintenance opportunities with established maintenance plans, lacking dynamic coordination capabilities and resulting in deficiencies in the real-time nature and engineering feasibility of maintenance decisions.
[0006] Although current predictive maintenance and multi-component maintenance optimization methods have made some progress in the field of equipment health management ([1] D. Wu, K. Gao, R. Peng, J. Sun, L. Yang, and F, Wei, “Optimal preventive maintenance policy considering technical-innovation-empoweredrenewals,” Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 1748006X251412931, 2025; [2] J. Zhu, B. Li, Z. Zhu, and X. Zhao, “Dynamic predictive maintenance strategy for the multi-state system based on remaining life prediction,” Reliab. Eng. Syst. Saf., vol. 264, p. 111289, Dec. 2025), existing research is mostly focused on single-component modeling or static strategy optimization, and its global collaborative ability and dynamic adaptability in complex multi-component systems have not yet been fully utilized. Summary of the Invention
[0007] To overcome the shortcomings of existing technologies, the present invention aims to provide an intelligent global dynamic maintenance method for multi-component systems that considers imperfect maintenance based on deep reinforcement learning. This method achieves joint optimization of predictive and opportunistic maintenance for multi-component systems without relying on fixed maintenance intervals or fixed maintenance combination rules, thereby reducing long-term average maintenance costs and improving resource utilization efficiency and system availability.
[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for intelligent global dynamic maintenance of multi-component systems considering imperfect maintenance based on deep reinforcement learning is proposed. First, a degradation observation sequence of each component in the multi-component system is obtained based on periodic state monitoring or inspection, and a unified health status representation is constructed. Second, a stochastic process model of component degradation oriented towards degradation heterogeneity is established, and iterative RUL prediction is achieved through offline training and online Bayesian updates. Then, based on the RUL prediction results, the baseline predictive maintenance timing for each component is determined, and a penalty cost model for maintenance timing deviation in group maintenance is constructed. Furthermore, a deep reinforcement learning decision model with long-term average maintenance cost as the optimization objective is constructed, and the maintenance combination and maintenance delay time of Dynamic Global Group Maintenance (DGGM) are output in a rolling manner. Finally, when a sudden failure occurs during the operation of the multi-component system, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered. While performing corrective maintenance, the set of components for opportunistic maintenance is determined, and subsequent group maintenance plans are updated synchronously, achieving intelligent global dynamic maintenance in an infinite time domain.
[0009] A method for intelligent global dynamic maintenance of multi-component systems considering imperfect maintenance based on deep reinforcement learning includes the following steps: Step 1: Perform periodic status monitoring or inspection on the multi-component system, obtain degradation observation data of each component at each inspection time, form a degradation observation sequence, and set component failure thresholds, maintenance cost parameters and inspection cycle parameters to construct the health status representation required for maintenance decisions. Step 2: To address the degradation heterogeneity of multi-component systems, establish a stochastic process model for component degradation, perform offline parameter training, and update the model using Bayesian methods based on newly acquired degradation observation data during online operation. Output the RUL probability distribution information of each component at the current inspection time. Step 3: Based on the RUL probability distribution information obtained in Step 2, determine the baseline predictive maintenance timing for each component at the current inspection time, and determine the baseline delayed maintenance time under the condition of considering the imperfect maintenance effect. At the same time, establish a deviation penalty cost model caused by the early or delayed maintenance timing in group maintenance. Step 4: Construct a deep reinforcement learning optimization model with the health status of multiple components as the state space, the maintenance combination and maintenance delay time as the action space, and the long-term maintenance cost as the reward function. Use the proximal policy optimization algorithm to perform rolling optimization on the dynamic global group maintenance (DGGM) and output the maintenance combination and maintenance execution time for each decision cycle. Step 5: During the operation of the multi-component system, if a sudden failure event occurs, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered. Corrective maintenance is performed on the failed component, and the set of opportunistic maintenance components is determined based on the latest RUL probability distribution information. The subsequent DGGM maintenance plan is updated synchronously. Steps 1-5 above are executed repeatedly to achieve dynamic maintenance decision-making in an infinite time domain.
[0010] Step 1 specifically involves: accumulating runtime and conducting periodic inspections on a multi-component system containing multiple degraded components, setting the inspection cycle to [missing information]. At each inspection moment Collect degradation observation data for each component; the degradation observation data consists of health indicators, degradation measurements, or characteristic sequences related to the degradation level obtained by sensors or monitoring systems; set a failure threshold for each component. Corrective maintenance costs Predictive maintenance costs Imperfect predictive maintenance costs And the fixed costs of maintenance preparation / resource mobilization This forms the set of input parameters for maintenance decisions.
[0011] Step 2 specifically involves: establishing a stochastic process model of component degradation to characterize the deterministic trend and stochastic fluctuations in degradation evolution, and using it for RUL prediction; components Degenerative state Described by a Wiener degradation process including drift and diffusion terms, and expressed by a monotonic time transformation function. The nonlinear degenerate behavior is characterized by the following formal representation: in, It is a set of parameters used to characterize the degradation properties of a component. Indicates the error term; It is a drift parameter of the degradation rate, used to describe the degradation heterogeneity among different individuals; It is a spatiotemporal transformation function, the form of which is defined by expert experience or historical data from similar devices; This represents standard Brownian motion, used to characterize the stochastic uncertainty in the degradation process; in the offline phase, the component's historical inspection time sequence and corresponding degradation observation sequence are used to analyze common parameters. and hyperparameters in the prior distribution The model parameters are subjected to maximum likelihood estimation or equivalent statistical training to obtain offline parameters; in the online phase, at each inspection time... After obtaining new degradation observations, the drift parameters are... Gaussian distribution parameters , By introducing a prior distribution and performing Bayesian updates, the posterior distribution of parameters is iteratively updated with the latest observations, thereby obtaining the degradation state estimate and RUL probability distribution information of the component at the current time. When the component degradation state reaches the failure threshold... The component failure is determined by time. RUL is defined as a random variable that takes time from the current moment until the degradation state first reaches the failure threshold. The RUL probability density function and RUL cumulative distribution function are obtained from this and used as input for subsequent maintenance decisions.
[0012] Step 3 specifically involves: based on the RUL probability distribution information obtained in Step 2, namely the RUL probability density function and the RUL cumulative distribution function, determining the baseline timing for predictive maintenance and using it as the baseline point in group maintenance optimization; under perfect maintenance conditions, minimizing the average maintenance cost rate per unit time for components. Optimal delay maintenance time This achieves an optimal trade-off between predictive maintenance and failure risk while meeting reliability and cost constraints; and introduces a degradation acceleration factor to account for imperfect maintenance scenarios. This study characterizes the potential degradation rate changes caused by imperfect maintenance. Based on this, it also uses minimizing the average maintenance cost rate as the criterion to determine the optimal delayed maintenance time under imperfect maintenance conditions. A maintenance deviation penalty cost model is constructed: when the actual group maintenance execution time is earlier than the baseline maintenance timing, the penalty cost includes "sacrificing remaining lifetime value" and the potential impact of maintenance type conversion; when the actual group maintenance execution time is delayed compared to the baseline maintenance timing, the penalty cost includes "the risk cost of corrective maintenance due to increased failure probability" and the change in lifetime value during the delay period. The deviation penalty function is unified into the same analytical form for both the earlier and delayed scenarios. In the optimization of group maintenance, a comparability evaluation of different maintenance combinations and maintenance times is conducted.
[0013] Step 4 specifically involves: constructing a Dynamic Global Group Maintenance (DGGM) based on the long-term rolling optimization concept; updating the maintenance combination and maintenance time according to the latest health status in each decision cycle; modeling the DGGM as a reinforcement learning model; the state space consists of the health status of each component at the current inspection time, wherein the health status includes at least a degradation state estimate, a degradation rate estimate, or a RUL prediction statistic; and the action space includes maintenance combination decisions and maintenance delay time decisions. Maintaining the vector for combined decision-making It means that among them Representation Component Whether to include in this group maintenance, the maintenance delay time represents the time from the current inspection time to the execution of group maintenance; the reward function is set to maximize the cost savings brought by group maintenance; the PPO algorithm is used to train the policy network and value network to obtain the maintenance decision strategy, and the optimal maintenance combination and maintenance delay time are output based on the latest state in each decision cycle, and the rolling optimization and dynamic update in the infinite time domain are realized according to the rule of "if the planned execution time is earlier than the next inspection point, then execute; otherwise, wait until the next inspection to re-evaluate".
[0014] Step 5 specifically involves: when a sudden failure event occurs during the operation of a multi-component system, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered; firstly, corrective maintenance is immediately performed on the failed component to restore the system's operating state; simultaneously, using mobilized maintenance resources, opportunistic imperfect maintenance is performed on other non-failed components within the same maintenance window; the set of other non-failed components is divided into "components that should be prioritized for opportunistic maintenance" and "components that need further evaluation for inclusion in opportunistic maintenance" based on their relationship with the imperfect maintenance threshold, and the opportunistic maintenance component set is determined with the goal of maximizing the net cost savings brought by opportunistic maintenance. This sub-problem is modeled as a binary classification decision process of reinforcement learning, and the PPO strategy outputs the decision on whether to include in opportunistic maintenance based on the real-time degradation characteristics of each component; after DOIM is executed, the health status and maintenance records of each component are updated synchronously, and the subsequent DGGM plan is reset and rolled over, thereby completing the dynamic coordination of DGGM and DOIM.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention enables joint optimization of maintenance combinations and timing at the system level under conditions of multi-component degradation heterogeneity and coupled maintenance resources. By introducing imperfect maintenance improvement factors and explicitly considering the uncertainty of maintenance effects in maintenance decisions, the maintenance strategy becomes closer to engineering practice. By constructing a maintenance timing deviation penalty cost model, the lifetime value loss or failure risk cost caused by the time alignment of group maintenance can be quantified and uniformly weighed in the optimization. By using deep reinforcement learning to learn strategies for high-dimensional maintenance combinations and timing decisions, the computational burden of traditional exhaustive / analytic optimization in complex state spaces can be avoided, thereby improving the online adaptability and real-time implementability of the strategy. By coordinating planned group maintenance (DGGM) with opportunity imperfect maintenance (DOIM) triggered by sudden failures, the maintenance combination can be dynamically adjusted and subsequent plans updated under sudden events, which helps to reduce long-term average maintenance costs and improve maintenance resource utilization efficiency and system availability. Attached Figure Description
[0016] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.
[0017] Figure 2 This is a schematic diagram illustrating the rolling optimization of dynamic global group maintenance of DGGM in this invention.
[0018] Figure 3 This is a timing diagram illustrating the collaboration between DGGM and DOIM in an embodiment of the present invention. Detailed Implementation
[0019] The present invention will now be described in detail with reference to the embodiments and accompanying drawings.
[0020] Reference Figure 1 A method for intelligent global dynamic maintenance of multi-component systems considering imperfect maintenance based on deep reinforcement learning includes the following steps: Step 1: Perform periodic status monitoring or inspection on the multi-component system, obtain degradation observation data of each component at each inspection time, form a degradation observation sequence, and set component failure thresholds, maintenance cost parameters and inspection cycle parameters to construct the health status representation required for maintenance decisions. For multi-component systems containing multiple degraded components, cumulative runtime and periodic inspections are performed, with the inspection cycle set as follows: At each inspection moment Collect degradation observation data for each component; the degradation observation data can be health indicators, degradation measurements, or feature sequences related to the degradation level obtained by sensors or monitoring systems; set a failure threshold for each component. Corrective maintenance costs Predictive maintenance costs Imperfect predictive maintenance costs And the fixed costs of maintenance preparation / resource mobilization This forms the set of input parameters for maintenance decisions; Step 2: To address the degradation heterogeneity of multi-component systems, establish a stochastic process model for component degradation, perform offline parameter training, and update the model using Bayesian methods based on newly acquired degradation observation data during online operation. Output the RUL probability distribution information of each component at the current inspection time. A stochastic process model of component degradation is established to characterize the deterministic trend and stochastic fluctuations in degradation evolution, and used for RUL prediction; components Degenerative state It can be described by a Wiener degradation process containing drift and diffusion terms, and by a monotonic time transformation function. The nonlinear degenerate behavior can be characterized in the following form: in, It is a set of parameters used to characterize the degradation properties of a component. This represents the error term. Specifically, It is a drift parameter of the degradation rate, used to describe the degradation heterogeneity among different individuals. It is a spatiotemporal transformation function, the form of which can be defined by expert experience or historical data from similar devices; This represents standard Brownian motion, used to characterize the stochastic uncertainty in the degradation process; in the offline phase, the model parameters (including common parameters) are evaluated using the component's historical inspection time sequence and the corresponding degradation observation sequence. and hyperparameters in the prior distribution Perform maximum likelihood estimation or equivalent statistical training to obtain offline parameters; during the online phase, at each inspection time... After obtaining new degradation observations, key parameters (such as drift parameters) are analyzed. Introducing a prior distribution and performing Bayesian updates ensures that the posterior distribution of parameters is updated iteratively with the latest observations, thereby obtaining the component's degradation state estimate and RUL probability distribution information at the current moment; when the component's degradation state reaches the failure threshold... The component failure is determined by time. RUL is defined as a random variable that takes time from the current moment until the degradation state first reaches the failure threshold. From this, the probability density function and cumulative distribution function of RUL can be obtained and used as input for subsequent maintenance decisions. Step 3: Based on the RUL probability distribution information obtained in Step 2, determine the baseline predictive maintenance timing for each component at the current inspection time, and determine the baseline delayed maintenance time under the condition of considering the imperfect maintenance effect. At the same time, establish a deviation penalty cost model caused by the early or delayed maintenance timing in group maintenance. Based on the RUL probability distribution information obtained in step 2, namely the RUL probability density function and the RUL cumulative distribution function, the baseline timing for predictive maintenance is determined and used as the baseline point in group maintenance optimization. Under the perfect maintenance scenario, the component is selected by minimizing the average maintenance cost rate per unit time. Optimal delay maintenance time This achieves an optimal trade-off between predictive maintenance and failure risk while meeting reliability and cost constraints; and introduces a degradation acceleration factor to account for imperfect maintenance scenarios. The degradation rate change caused by imperfect maintenance is characterized, and the optimal delayed maintenance time under imperfect maintenance conditions is obtained based on minimizing the average maintenance cost rate. Furthermore, to quantify the economic impact of advancing or delaying component maintenance timing during group maintenance alignment, a maintenance deviation penalty cost model is constructed: when the actual group maintenance execution time is earlier than the baseline maintenance timing, the penalty cost includes "sacrificing remaining lifetime value" and the potential impact of maintenance type conversion; when the actual group maintenance execution time is delayed compared to the baseline maintenance timing, the penalty cost includes "corrective maintenance risk cost due to increased failure probability" and the change in lifetime value during the delay period. The model unifies the two cases of advance and delay into a single analytical form of the deviation penalty function. The bias penalty can be divided into three cases based on the failure time falling into different intervals and combined into a unified function form for reward calculation in reinforcement learning. Step 4: Construct a deep reinforcement learning optimization model with the health status of multiple components as the state space, the maintenance combination and maintenance delay time as the action space, and the long-term maintenance cost as the reward function. Use the proximal policy optimization algorithm to perform rolling optimization on the dynamic global group maintenance (DGGM) and output the maintenance combination and maintenance execution time for each decision cycle. A Dynamic Global Group Maintenance (DGGM) is constructed based on the long-term rolling optimization concept. In each decision cycle, the maintenance combination and maintenance time are updated according to the latest health status. The DGGM is modeled as a reinforcement learning model; its state space consists of the health status of each component at the current inspection time, where each health status includes at least a degradation state estimate, a degradation rate estimate, or a RUL prediction statistic. The action space includes maintenance combination decisions and maintenance delay time decisions. Maintaining the combined decision can be done using vectors It means that among them Representation Component Whether to include in this group maintenance depends on the maintenance delay time, which represents the time from the current inspection moment until the group maintenance is executed. The reward function is set to maximize the cost savings brought by group maintenance. The PPO algorithm is used to train the policy network and value network to obtain the maintenance decision strategy. In each decision cycle, the optimal maintenance combination and maintenance delay time are output based on the latest state. According to the rule of "if the planned execution time is earlier than the next inspection point, then execute; otherwise, wait until the next inspection for re-evaluation," rolling optimization and dynamic updates are achieved in the infinite time domain. At the execution level, refer to... Figure 2As can be seen, DGGM adopts a long-term rolling execution mechanism based on inspection points. At each inspection moment, it outputs the group maintenance combination of the current cycle and its planned execution time based on the latest health status, and uses the "relationship between the planned execution time and the next inspection point" as the execution criterion. If the planned execution time is earlier than the next inspection point, DGGM is executed according to the plan and the component status and maintenance records are updated.
[0021] Step 5: During the operation of the multi-component system, if a sudden failure event occurs, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered. Corrective maintenance is performed on the failed component, and the set of opportunistic maintenance components is determined based on the latest RUL probability distribution information. The subsequent DGGM maintenance plan is updated synchronously. Steps 1-5 above are executed repeatedly to achieve dynamic maintenance decision-making in an infinite time domain. When a sudden failure event occurs during the operation of a multi-component system, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered. First, corrective maintenance is immediately performed on the failed component to restore the system to an operational state. Simultaneously, mobilized maintenance resources are used to perform opportunistic imperfect maintenance on other non-failed components within the same maintenance window, reducing the cost of subsequent short-term downtime and repeated mobilization. In some embodiments, the set of other non-failed components is divided into "components that should be prioritized for opportunistic maintenance" and "components that need further evaluation for inclusion in opportunistic maintenance" based on their relationship with the imperfect maintenance threshold. The opportunistic maintenance component set is determined with the goal of maximizing the net cost savings brought by opportunistic maintenance. To improve online decision-making efficiency, this sub-problem can be modeled as a binary classification decision process using reinforcement learning, where the PPO strategy outputs a decision on whether to include the component in opportunistic maintenance based on the real-time degradation characteristics of each component. (Refer to...) Figure 3 As can be seen, when a sudden failure event occurs within the inspection interval, the system will trigger DOIM and complete the joint execution of "corrective maintenance of failed components" and "opportunistic imperfect maintenance of other candidate components" in the same maintenance window. At the same time, the original DGGM plan will be interrupted and reconstructed. The execution of DOIM will interrupt the original DGGM plan and reset / update the subsequent DGGM plan after DOIM. After the execution of DOIM, the health status and maintenance records of each component will be updated synchronously, and the subsequent DGGM plan will be reset and rolled over, thereby completing the dynamic collaboration between DGGM and DOIM.
[0022] This embodiment uses a multi-component system of the main drive train of a wind turbine generator as an example to verify the effectiveness of the method of the present invention. The main drive train system includes key components such as gears, high-speed shafts, and bearings. Failure of any key component may lead to generator shutdown and significant losses. At the same time, since the setup costs for maintenance preparation / shutdown / scheduling can be shared by multiple components, it is suitable for verifying the economy and feasibility of the joint optimization strategy for group maintenance and maintenance timing.
[0023] To facilitate degradation modeling for different component types, this embodiment selects three types of component samples: gears, shafts, high-speed shafts / main shafts (shafts), and bearings, and constructs a multi-component system consisting of six components (#1–#6), where components #1 and #2 represent gears, components #3 and #4 represent shafts, and components #5 and #6 represent bearings. The cumulative number of days the wind turbine has been running is used as the time metric for degradation observation and parameter estimation, and the state is updated with a fixed inspection cycle set to Δ. T =5. Several training samples were selected to fit the degradation process parameters (shaft, gear, bearing) of three key components (shaft, gear, bearing) in the main drive system of the wind turbine, using the time-degradation scale function. Expressed in the form of a power function The maximum likelihood estimation was used to train the training set samples offline, and the fitting results are shown in Table 1.
[0024] Table 1. Offline training parameters for all components The PPO algorithm was used to train the maintenance decision strategy, and Adam was selected as the optimizer. The discount factor was set to... The GAE parameter is set to The shear coefficient of PPO is taken as The value function loss coefficient is set to The learning rate is set to Regarding data sampling and policy updates, the step size for each policy rollover is set to... The small batch sample size is The parameters are updated 10 times after each scroll.
[0025] The maintenance settings cost parameters at the system level are shown in Table 2. Starting with a clear degradation trend, parameters were dynamically updated. Based on the system-level group maintenance strategy, a fault-free DGGM maintenance plan was simulated, and the results are shown in Table 3.
[0026] Table 2 Cost Parameters (RMB) Table 3 Trouble-Free Maintenance Plan In another scenario, if component #3 experiences a sudden failure on day 216, Opportunistic Imperfect Group Maintenance (DOIM) is triggered in real time, and the maintenance mix is readjusted. The relevant results are shown in Table 3. From the 8th plan, it can be observed that components #1, #2, and #4 underwent imperfect opportunistic maintenance, while component #3 underwent corrective maintenance. Comparing Tables 3 and 4 shows that the introduction of opportunistic maintenance significantly changed the subsequent maintenance strategy. This mechanism plays an indispensable role in optimizing the overall system maintenance plan and improving cost savings.
[0027] Table 4 Maintenance Plan under Unexpected Failure
Claims
1. A method for intelligent global dynamic maintenance of multi-component systems considering imperfect maintenance based on deep reinforcement learning, characterized in that: First, based on periodic state monitoring or inspection, the degradation observation sequence of each component in the multi-component system is obtained and a unified health status representation is constructed. Second, a component degradation stochastic process model oriented towards degradation heterogeneity is established, and iterative RUL prediction is achieved through offline training and online Bayesian update. Then, based on the RUL prediction results, the baseline predictive maintenance timing for each component is determined, and a penalty cost model for maintenance timing deviation in group maintenance is constructed. Furthermore, a deep reinforcement learning decision model with long-term average maintenance cost as the optimization objective is constructed to continuously output the maintenance combination and maintenance delay time of dynamic global group maintenance (DGGM). Finally, when a sudden failure occurs during the operation of the multi-component system, dynamic opportunistic imperfect maintenance (DOIM) is triggered. While performing corrective maintenance, the set of components for opportunistic maintenance is determined, and subsequent group maintenance plans are updated synchronously to achieve intelligent global dynamic maintenance in the infinite time domain.
2. A method for intelligent global dynamic maintenance of multi-component systems considering imperfect maintenance based on deep reinforcement learning, characterized in that, Includes the following steps: Step 1: Perform periodic status monitoring or inspection on the multi-component system, obtain degradation observation data of each component at each inspection time, form a degradation observation sequence, and set component failure thresholds, maintenance cost parameters and inspection cycle parameters to construct the health status representation required for maintenance decisions. Step 2: Establish a stochastic process model of component degradation, perform offline parameter training, and perform Bayesian updates based on newly acquired degradation observation data during online operation, outputting the RUL probability distribution information of each component at the current inspection time; Step 3: Based on the RUL probability distribution information obtained in Step 2, determine the baseline predictive maintenance timing for each component at the current inspection time, and determine the baseline delayed maintenance time under the condition of considering the imperfect maintenance effect. At the same time, establish a deviation penalty cost model caused by the early or delayed maintenance timing in group maintenance. Step 4: Construct a deep reinforcement learning optimization model, use the PPO algorithm to perform rolling optimization on the DGGM, and output the maintenance combination and maintenance execution time for each decision cycle; Step 5: During the operation of the multi-component system, if a sudden failure event occurs, DOIM is triggered to perform corrective maintenance on the failed component, and the set of opportunistic maintenance components is determined based on the latest RUL probability distribution information, and the subsequent DGGM maintenance plan is updated synchronously.
3. The method according to claim 2, characterized in that, Step 1 specifically involves: accumulating operating time and conducting periodic inspections of an industrial system containing multiple degraded components, setting the inspection cycle as follows: At each inspection moment Collect degradation observation data for each component; Degradation observation data are health indicators, degradation measurements, or characteristic sequences related to degradation levels obtained by sensors or monitoring systems; Set the failure threshold for each component. Corrective maintenance costs Predictive maintenance costs Imperfect predictive maintenance costs And the fixed costs of maintenance preparation / resource mobilization This forms the set of input parameters for maintenance decisions.
4. The method according to claim 2, characterized in that, Step 2 specifically involves: establishing a stochastic process model of component degradation to characterize the deterministic trend and stochastic fluctuations in degradation evolution, and using it for RUL prediction; components Degenerative state Described by the Wiener degradation process including drift and diffusion terms, and expressed by a monotonic time transformation function. The nonlinear degenerate behavior is characterized by the following formal representation: in, It is a set of parameters used to characterize the degradation properties of a component. Indicates the error term; It is a drift parameter of the degradation rate, used to describe the degradation heterogeneity among different individuals; It is a spatiotemporal transformation function, the form of which is defined by expert experience or historical data from similar devices; This represents standard Brownian motion, used to characterize the stochastic uncertainty in the degradation process; in the offline phase, the component's historical inspection time sequence and corresponding degradation observation sequence are used to analyze common parameters. and hyperparameters in the prior distribution The model parameters are subjected to maximum likelihood estimation or equivalent statistical training to obtain offline parameters; in the online phase, at each inspection time... After obtaining new degradation observations, the drift parameters are... Gaussian distribution parameters , By introducing a prior distribution and performing Bayesian updates, the posterior distribution of parameters is iteratively updated with the latest observations, thereby obtaining the degradation state estimate and RUL probability distribution information of the component at the current time. When the component degradation state reaches the failure threshold... The component failure is determined by time. RUL is defined as a random variable that takes time from the current moment until the degradation state first reaches the failure threshold. The probability density function and cumulative distribution function of RUL are obtained from this and used as input for subsequent maintenance decisions.
5. The method according to claim 2, characterized in that, Step 3 specifically involves: based on the RUL probability distribution information obtained in Step 2, namely the RUL probability density function and the RUL cumulative distribution function, determining the baseline timing for predictive maintenance and using it as the baseline point in group maintenance optimization; under perfect maintenance conditions, minimizing the average maintenance cost rate per unit time for components. Optimal delay maintenance time This achieves an optimal trade-off between predictive maintenance and failure risk while meeting reliability and cost constraints; and introduces a degradation acceleration factor to account for imperfect maintenance scenarios. This study characterizes the potential degradation rate changes caused by imperfect maintenance. Based on this, it also uses minimizing the average maintenance cost rate as the criterion to determine the optimal delayed maintenance time under imperfect maintenance conditions. A maintenance deviation penalty cost model is constructed: when the actual group maintenance execution time is earlier than the baseline maintenance timing, the penalty cost includes "sacrificing remaining lifetime value" and the potential impact of maintenance type conversion; when the actual group maintenance execution time is delayed compared to the baseline maintenance timing, the penalty cost includes "the risk cost of corrective maintenance due to increased failure probability" and the change in lifetime value during the delay period. The deviation penalty function is unified into the same analytical form for both the earlier and delayed scenarios. This is to enable a comparable evaluation of different maintenance combinations and maintenance times in subsequent group maintenance optimization.
6. The method according to claim 2, characterized in that, Step 4 specifically involves: constructing a dynamic global group maintenance DGGM based on the long-term rolling optimization idea; updating the maintenance combination and maintenance time according to the latest health status in each decision cycle; modeling the DGGM as a reinforcement learning model state space composed of the health status of each component at the current inspection time, wherein the health status includes at least degradation state estimation, degradation rate estimation or RUL prediction statistics. The action space includes maintenance combination decisions and maintenance delay time decisions. Maintaining the combined decision can be done using vectors It means that, among them Representation Component Whether to include in this group maintenance, the maintenance delay time represents the time from the current inspection time to the execution of group maintenance; the reward function is set to maximize the cost savings brought by group maintenance; the PPO algorithm is used to train the policy network and value network to obtain the maintenance decision strategy, and the optimal maintenance combination and maintenance delay time are output based on the latest state in each decision cycle, and the rolling optimization and dynamic update in the infinite time domain are realized according to the rule of "if the planned execution time is earlier than the next inspection point, then execute; otherwise, wait until the next inspection to re-evaluate".
7. The method according to claim 2, characterized in that, Step 5 specifically involves: when a sudden failure event occurs during the operation of a multi-component system, Dynamic Opportunistic Imperfect Maintenance (DOIM) is triggered; firstly, corrective maintenance is immediately performed on the failed component to restore the system to an operational state; simultaneously, using mobilized maintenance resources, opportunistic imperfect maintenance is performed on other non-failed components within the same maintenance window; the set of other non-failed components is divided into "the set of components that should be prioritized for opportunistic maintenance" and "the set of components that need further evaluation for inclusion in opportunistic maintenance" according to the relationship between their RUL and imperfect maintenance threshold, and the set of components for opportunistic maintenance is determined with the goal of maximizing the net cost savings brought by opportunistic maintenance. This sub-problem is modeled as a binary classification decision process of reinforcement learning, and the PPO policy outputs the decision on whether to include in opportunistic maintenance based on the real-time degradation characteristics of each component; after DOIM is executed, the health status and maintenance records of each component are updated synchronously, and the subsequent DGGM plan is reset and rolled over, thereby completing the dynamic coordination of DGGM and DOIM.