Maintenance decision-making method and system for multi-state complex system under partial observable information and computer program product

Abstract

The invention relates to the technical field of mechanical engineering, in particular to a multi-state complex system maintenance decision-making method under partial observable information, and the method comprises the steps: constructing a continuous time homogeneous Markov chain model of a multi-state degradation process based on state observation data; the expectation maximization algorithm is utilized to jointly estimate to-be-estimated state transition parameters and observation parameters, the Bayesian theorem is combined to update the posterior probability of the system at each sampling moment in real time, and the optimal condition reliability control limit is sought with the calculation of the condition reliability of the system and the minimization of the long-range expectation average cost as the target; and deriving and calculating the transition probability, the expected residence time and the expected maintenance cost in the semi-Markov decision process so as to determine the maintenance decision of the multi-state complex system. Therefore, the problems that in the related technology, the difference of failure rates of all states of the system is not considered, and the optimal shutdown threshold value is a virtual composite index, so that a maintenance decision is lack of accuracy and pertinence, and the reliability of the system is affected are solved.

Description

Technical Field

[0001] The present application relates to the technical field of fault prediction and maintenance, and particularly relates to a maintenance decision-making method, system and computer program product for a multi-state complex system under partially observable information. Background Art

[0002] With the increasing complexity of modern equipment, condition monitoring has also become a key guarantee for the normal operation of the system. Most systems are equipped with online monitoring sensors. However, the data obtained by the sensors can only reflect partial information of the system state, and the true state of the system is usually unknown, so it is called partially observable information.

[0003] In related technologies, the Markov process is mainly used to describe the mapping relationship between monitoring data and system states and the stochastic process of system state transition. Macroscopically, the system is divided into a healthy state, an unhealthy state (alarm state) and a failure state, which can clearly show the transition process of the system state, and provide the optimal shutdown threshold of the system to determine the maintenance decision, and can make maintenance decisions in a timely manner.

[0004] However, in related technologies, the residence time of each unobservable state is assumed to be an exponential distribution, a Weibull distribution, an Erlang(2,λ) or a normal distribution, etc., without considering the difference in the failure rate of each state, which is likely to lead to the inability to accurately reflect the true process of the system, increase the prediction error, and the index of the optimal shutdown threshold of the system is usually a virtual composite index, which is difficult to accurately quantitatively describe the maintenance decision, increases the uncertainty of the decision, and reduces the accuracy and efficiency of the maintenance decision, and urgent improvement is needed. Summary of the Invention

[0005] The present application provides a maintenance decision-making method for a multi-state complex system under partially observable information to solve the problems in related technologies that the maintenance decision lacks accuracy and pertinence, affecting the reliability and operation efficiency of the system, etc.

[0006] The first aspect of the present application provides a maintenance decision-making method for a multi-state complex system under partially observable information, including the following steps: constructing a continuous-time homogeneous Markov chain model of a multi-state degradation process based on the state observation data of the multi-state complex system; using the expectation maximization algorithm to jointly estimate the state transition parameters and observation parameters to be estimated in the Markov chain model to obtain updated state transition parameters and updated observation parameters; based on the updated state transition parameters and the updated observation parameters, combining Bayes' theorem to update the posterior probability of the multi-state complex system at each sampling moment in real time to calculate the conditional reliability of the multi-state complex system at each sampling moment; using a semi-Markov decision process to seek the optimal conditional reliability control limit with the goal of minimizing the long-term expected average cost; and deriving and calculating the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process according to the size relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit to determine the maintenance decision of the multi-state complex system.

[0007] Optionally, in an embodiment of the present application, the constructing of the continuous-time homogeneous Markov chain model of the multi-state degradation process includes: constructing the multi-state degradation process into a continuous-time homogeneous Markov chain model with a state space of , where the state space , is defined as the healthy state set, is defined as the unhealthy state set, is defined as the failure state set. In the healthy state , the failure rate of a random failure is a constant, and in the unhealthy state , the failure rate increases with time.

[0008] Optionally, in an embodiment of the present application, in the state, the observation data of the system follows a multivariate normal distribution with a mean of and a covariance of , and its expression is as follows:

[0009]

[0010] where is the observation data of the system in the state, which follows a multivariate normal distribution with a mean of and a covariance of ; is the sampling interval; is the time of the kth sampling; and d is the dimension of the data.

[0011] In the state The sojourn time under [condition] follows an Erlang distribution with order \(k_1\) and transition rate \(\lambda_1\), and the expression of its probability density function \(f_1(t)\) is as follows:

[0012]

[0013] where \(f_1(t)\) is the probability density function in state ; \(\lambda_1\) is the transition rate when the system is in the healthy state; \(t\) is the sojourn time.

[0014] Optionally, in an embodiment of the present application, in state, the observed data of the system follows a multivariate normal distribution with mean and covariance , and its expression is as follows:

[0015]

[0016] where is the observed data of the system in state, which follows a multivariate normal distribution with mean and covariance ; is the sampling interval; is the time of the \(k\)th sampling; \(d\) is the dimension of the data.

[0017] In state, the sojourn time follows a hyper-Erlang distribution, and the expression of its cumulative distribution function is as follows:

[0018]

[0019] where is the cumulative distribution function of the sojourn time in state following a hyper-Erlang distribution; \(\lambda_2\) is the transition rate when the system is in the unhealthy state; is the cumulative distribution function of the hyper-Erlang distribution; is the number of exponential phases and the cumulative distribution function of the Erlang distribution with transition rate \(\lambda_2\).

[0020] Optionally, in an embodiment of the present application, the expression for minimizing the long-term expected average cost is as follows:

[0021]

[0022] where \(R\) * is the optimal conditional reliability control limit; The long - term expected average cost when the conditional reliability is R * ; and are respectively the expected cycle cost and the expected cycle length when the conditional reliability is R * ;

[0023]

[0024] Wherein, is the long - term expected average cost when the conditional reliability is ; is the given conditional reliability control limit, obtained by solving the following linear equation:

[0025]

[0026] Wherein, is the relative value of the given current state x; is the relative value of the value function when the semi - Markov decision process is in state l; l is a certain state of the semi - Markov decision process; is the expected maintenance cost at the next decision moment given the current state x; S is the state space of the semi - Markov decision process; is the expected sojourn time at the next decision moment given the current state x; is given the current state , and at the next moment the system will be in state with probability; ; is the relative value of the system when it is in state of the semi - Markov decision process.

[0027] Optionally, in an embodiment of the present application, the derivation and calculation of the transition probability in the semi - Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit includes: If , the transition probability in the semi - Markov decision process is calculated by the following formula:

[0028]

[0029] Wherein, is the conditional reliability of the k - th sampling; is the given conditional reliability control limit; is the transition probability from state to state in the semi - Markov decision process at the k - th sampling moment; , where Denote the state space where the sum of the sub - states of the semi - Markov decision process is less than L; L is the fixed number of sub - intervals for partitioning the state space of the semi - Markov decision process; is the sub - state of the semi - Markov decision process at the k - th sampling time; is the sub - state of the semi - Markov decision process at the next sampling time; Denote the m - th sub - state in the state space of the semi - Markov decision process; k1 is the number of exponential phases of the sojourn time distribution in the healthy state; k2 is the number of exponential phases of the sojourn time distribution in the unhealthy state approximated by the hyper - Erlang distribution; Denote at time, given the observed data the posterior probability that the system is in sub - state i; , , , Denote that the next state is in the a - th sub - state in the state space of the semi - Markov decision process; is the failure time of the system; is the probability that the system is in sub - state i at the next time under the condition of obtaining the monitoring data at time; is at time, the state of the system; is the conditional reliability of the system under the condition of obtaining the monitoring data at time; is the posterior probability that the system is in each sub - state under the condition of obtaining the monitoring data at time; is the sampling interval; is the time of the k - th sampling; d is the dimension of the data.

[0030] If , the probability that the system is in the unhealthy state is:

[0031]

[0032] where, is the conditional reliability of the system at time; is the transition probability from state to state PM in the semi - Markov decision process, is the sum of the posterior probabilities that the system is in the unhealthy state, PM represents that the system is in the unhealthy state after a comprehensive inspection;

[0033] The probability that the system is in the healthy state is:

[0034]

[0035] Among them, is the transition probability from state to state in the semi - Markov decision process, indicates that the system is in a healthy state at the initial moment;

[0036] The probability of system failure is:

[0037]

[0038] Among them, is the transition probability from state to state F in the semi - Markov decision process; F indicates that the system is in a failure state after a comprehensive inspection;

[0039] After the system fails and is replaced and preventive maintenance measures are taken, the state process starts from a new system cycle:

[0040]

[0041]

[0042] Among them, is the transition probability from state F to state in the semi - Markov decision process, is the transition probability from state PM to state in the semi - Markov decision process.

[0043] Optionally, in an embodiment of the present application, according to the size relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the expected maintenance cost in the semi - Markov decision process is deduced and calculated. The expected maintenance cost includes: If , a comprehensive inspection of the system is required, generating a corresponding inspection cost per unit time C I and a production loss cost C LP , then the expected maintenance cost is:

[0044]

[0045] Among them, is the expected cost when the conditional reliability is lower than the control limit threshold at the moment and the semi - Markov decision process is in state ; T I is the inspection time; C I is the inspection cost per unit time generated by a comprehensive inspection of the system; C LP is the production loss cost generated by a comprehensive inspection of the system;

[0046] If , then the system may fail or may not take effect at the next moment. If it is in an unhealthy state, additional operating costs C AO and maintenance costs C AM will also be incurred. Then the expected maintenance cost is:

[0047]

[0048] Where, At When the conditional reliability at the moment is higher than the control limit threshold, the expected cost in the semi-Markov decision process in state ; C S Is the cost required for each sampling; C F Is the after-fact maintenance cost; T F Is the component replacement time; C AO , C AM Are the additional operating cost and maintenance cost respectively; j is a sub-state when the system is in an unhealthy state; Is at The posterior probability that the system is in sub-state i at the moment; Is the probability that the system transfers from sub-state i to sub-state j within t time;

[0049] The expected maintenance costs in the PM state and the F state are respectively:

[0050]

[0051]

[0052] Where, C PM And T PM Are the preventive maintenance cost and maintenance time;

[0053] Optionally, in an embodiment of the present application, according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the expected sojourn time in the semi-Markov decision process is deduced and calculated, including: If , a comprehensive inspection of the system is required, then the expected sojourn time is:

[0054]

[0055] Where, Is at When the conditional reliability at the moment is lower than the control limit threshold, the expected sojourn time in the semi-Markov decision process in state ;

[0056] If , then the system may fail or may not take effect at the next moment, and the expected sojourn time is:

[0057]

[0058] where is the expected sojourn time when the conditional reliability at time is higher than the control limit threshold and the semi - Markov decision process is in state ; is the probability density function of the remaining life of the system updated after obtaining the monitoring data at time ;

[0059] The expected sojourn times in the PM state and the F state are respectively:

[0060]

[0061]

[0062] where represents the expected sojourn time in the PM state; represents the expected sojourn time in the F state.

[0063] The second - aspect embodiment of the present application provides a maintenance decision - making system for a multi - state complex system under partially observable information, including: a degradation model construction unit: used to construct a continuous - time homogeneous Markov chain model of a multi - state degradation process based on the state observation data of the multi - state complex system; an estimation unit: used to jointly estimate the to - be - estimated state - transition parameters and to - be - estimated observation parameters in the Markov chain model by using the expectation - maximization algorithm to obtain updated state - transition parameters and updated observation parameters; a calculation unit: used to combine the updated state - transition parameters and the updated observation parameters, and update the posterior probability of the multi - state complex system at each sampling moment in real - time according to Bayes' theorem to calculate the conditional reliability of the multi - state complex system at each sampling moment; a control - limit determination unit: used to seek the optimal conditional reliability control limit by using a semi - Markov decision process with the goal of minimizing the long - term expected average cost; a policy determination unit: used to deduce and calculate the transition probability, expected sojourn time, and expected maintenance cost in the semi - Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit to determine the maintenance decision of the multi - state complex system.

[0064] The third - aspect embodiment of the present application provides a computer program product, including computer programs / instructions, which, when executed by a processor, are used to implement the above - mentioned maintenance decision - making method for a multi - state complex system under partially observable information.

[0065] Embodiments of the present application can construct a continuous-time homogeneous Markov chain model for a multi-state degradation process based on the state observation data of a multi-state complex system, jointly estimate the state transition parameters and observation parameters to be estimated in the Markov chain model using the expectation maximization algorithm. Furthermore, based on the updated state transition parameters and observation parameters, combined with Bayes' theorem, the posterior probability of the system at each sampling moment is updated in real time to calculate the conditional reliability of the system. With the goal of minimizing the long-term expected average cost, a semi-Markov decision process is used to find the optimal conditional reliability control limit. Thus, based on the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process are deduced and calculated, realizing an accurate quantitative description of the maintenance decision of the multi-state complex system, which helps to optimize the system performance, reduce the maintenance cost, and improve the overall operation efficiency. Thereby, it solves the problems in the related technologies that due to the failure to consider the differences in the failure rates of the system states and the optimal shutdown threshold index being a virtual composite index, it is easy to lead to the lack of accuracy and pertinence in the maintenance decision, affecting the reliability and operation efficiency of the system, etc.

[0066] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0068] Figure 1 FIG. is a flowchart of a maintenance decision-making method for a multi-state complex system under partially observable information according to an embodiment of the present application;

[0069] Figure 2 FIG. is a flowchart of a maintenance decision-making method for a multi-state complex system under partially observable information according to an embodiment of the present application;

[0070] Figure 3 FIG. is a schematic structural diagram of a maintenance decision-making system for a multi-state complex system under partially observable information according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0072] The following describes a maintenance decision-making method for a multi-state complex system with partially observable information in embodiments of the present application. In view of the problems in the related technologies mentioned in the above background art, where the difference in the failure rates of the system in various states is not considered and the index of the optimal shutdown threshold is a virtual composite index, which easily leads to the lack of accuracy and pertinence in maintenance decision-making and affects the reliability and operating efficiency of the system, the present application provides a maintenance decision-making method, system, and computer program product for a multi-state complex system with partially observable information. In this method, a continuous-time homogeneous Markov chain model of the multi-state degradation process can be constructed based on the state observation data of the multi-state complex system, and the expectation-maximization algorithm is used to jointly estimate the to-be-estimated state transition parameters and observation parameters in the Markov chain model. Furthermore, based on the updated state transition parameters and observation parameters, combined with Bayes' theorem, the posterior probability of the system at each sampling moment is updated in real time to calculate the conditional reliability of the system. With the goal of minimizing the long-term expected average cost, the semi-Markov decision process is used to find the optimal conditional reliability control limit. Thus, based on the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process are derived and calculated, realizing an accurate and quantitative description of the maintenance decision-making of the multi-state complex system, which helps to optimize the system performance, reduce the maintenance cost, and improve the overall operation efficiency. Thereby, the problems in the related technologies, such as the lack of accuracy and pertinence in maintenance decision-making due to the failure to consider the difference in the failure rates of the system in various states and the virtual composite index of the optimal shutdown threshold, affecting the reliability and operating efficiency of the system, are solved.

[0073] Specifically, Figure 1 is a schematic flow chart of a maintenance decision-making method for a multi-state complex system with partially observable information provided by an embodiment of the present application.

[0074] As Figure 1 shown, the maintenance decision-making method for the multi-state complex system with partially observable information includes the following steps:

[0075] In step S101, based on the state observation data of the multi-state complex system, a continuous-time homogeneous Markov chain model of the multi-state degradation process is constructed.

[0076] It can be understood that for the state observation data of the system in embodiments of the present application, a Markov process with 3 states (state 1, 2, 3) can be used to describe the degradation process of the system. Among them, state 1 represents the normal or healthy state, state 2 represents the abnormal or unhealthy state, and state 3 represents the failure state. At the same time, since the state monitoring information obtained by the sensor cannot directly reflect the true state of the system and only provides partial observable information, states 1 and 2 are unobservable, and only the failure state can be observed.

[0077] Specifically, in combination with Figure 2 As shown, considering the actual degradation process of most mechanical systems in the embodiments of the present application, the operating time in the healthy state is usually longer than that in the abnormal state, and it is easier to transfer to the failure state in the abnormal state. Therefore, it is assumed that the failure rate λ of random failures occurring in state 1 is a constant, and the failure rate in state 2 increases.

[0078] Optionally, in an embodiment of the present application, in state 1, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of , and its expression is as follows:

[0079] (1)

[0080] where is the observed data of the system in the state, which obeys a multivariate normal distribution with a mean of and a covariance of ; is the sampling interval; is the time of the kth sampling; d is the dimension of the data. is the time of the kth sampling; d is the dimension of the data.

[0081] In state 2, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of , and its expression is as follows:

[0082] (2)

[0083] where is the observed data of the system in the state, which obeys a multivariate normal distribution with a mean of and a covariance of ; is the sampling interval; is the time of the kth sampling; d is the dimension of the data. is the time of the kth sampling; d is the dimension of the data.

[0084] The sojourn time in state 1 follows an Erlang distribution with order k1 and transition rate λ1, and its probability density function f1(t) is as follows:

[0085] (3)

[0086] where f1(t) is the probability density function of state Lower probability density function; λ1 is the transition rate when the system is in the healthy state; k1 is the number of exponential phases in the residence time distribution in the healthy state; t is the residence time.

[0087] In the unhealthy state, that is, the residence time in state 2 follows a hyper-Erlang distribution, which is approximated by a mixture of a series of Erlang distributions, and its cumulative distribution function is:

[0088] (4)

[0089] Wherein, is the cumulative distribution function of the residence time in state following a hyper-Erlang distribution; λ2 is the transition rate when the system is in the unhealthy state; is the cumulative distribution function of the hyper-Erlang distribution; is the number of exponential phases is and the cumulative distribution function of the Erlang distribution with a transition rate of λ2.

[0090] In the above formula, there is a positive integer function related to λ2, that is , such that when then , so that formula (2) can be written in the following form:

[0091] (5)

[0092] Where: n2 is the positive integer order.

[0093] For a continuous-time homogeneous Markov chain with n2 + 1 states, its state space is {1, 2,..., n2 + 1}. The starting state of this Markov chain is 1. For any state 1 ≤ i ≤ n2, the next state will transfer to state i + 1 or state n2 + 1 (absorbing state). The residence time in any state 1 ≤ i ≤ n2 follows an exponential distribution with a transition rate of γ. When the system makes a state transition, with probability it transfers from state i to state i + 1, and with probability it transfers from state i to the absorbing state n2 + 1. The probability is calculated as follows:

[0094] (6)

[0095] The residence time in state 1 follows an Erlang distribution, and the residence time in state 2 follows a hyper-Erlang distribution. Both have the characteristics of a phase-type distribution. Therefore, the state transition process of the system can be described by a Markov process. Optionally, in an embodiment of the present application, the new state space is , where represents the set of states where the system is in a healthy condition, represents the set of states where the system is in an unhealthy condition, represents that the system is in a failed state. Therefore, the state process ( ) is modeled as a continuous-time homogeneous Markov chain with state space , and its transition rate matrix is:

[0096] (7)

[0097] where: , and in other cases .

[0098] For t ≥ 0, the transition probability matrix P(t) of the system is defined as follows:

[0099] (8)

[0100] The matrix elements in P(t) can be calculated from the Chapman-Kolmogorov equation through the transition rate matrix:

[0101] (9)

[0102] Based on the state observation data of a multi-state complex system, the embodiments of this application construct a continuous-time homogeneous Markov chain model of a multi-state degradation process, and adopt different distribution models for the healthy state and the unhealthy state to describe the residence time in these two states, which can more accurately reflect the difference in failure rates of the system in each state, more accurately describe the degradation process, and provide a more accurate information basis for subsequent determination of the system's maintenance strategy.

[0103] In step S102, the expectation-maximization algorithm is used to jointly estimate the state transition parameters to be estimated and the observation parameters to be estimated in the Markov chain model, and the updated state transition parameters and the updated observation parameters are obtained.

[0104] In the actual execution process, as shown in Figure 2 , the embodiments of this application collect groups of state monitoring failure history data, which can be represented by . groups of censored history data are collected, which can be represented by . is used to represent the observation data, is the corresponding likelihood function, where and They are the state and observation parameters to be estimated respectively. The Expectation-Maximization (EM) algorithm is used to estimate the parameters of the model, which is divided into the E-step and the M-step. Let and be the initial values of the parameters to be estimated.

[0105] E-step: Calculate the pseudo log-likelihood function:

[0106] (10)

[0107] where is the complete data set, that is, the observed data set each group of failure data and censored data augment the unobservable sample path information of the state process.

[0108] M-step: Select such that

[0109] (11)

[0110] The parameter updated at each step is then used as the initial value and substituted into the E-step, so that the E-step and the M-step are iteratively operated until the Euclidean norm .

[0111] In the embodiment of the present application, the Expectation-Maximization algorithm is used to jointly estimate the state transition parameters to be estimated and the observation parameters to be estimated in the Markov chain model, which can make the parameter estimation more accurate, help to more accurately describe the state transition and observation process of the system, and thus improve the prediction ability and reliability of the model.

[0112] In step S103, based on the updated state transition parameters and the updated observation parameters, the posterior probabilities of the multi-state complex system at each sampling moment are updated in real time in combination with Bayes' theorem to calculate the conditional reliability of the multi-state complex system at each sampling moment.

[0113] Specifically, as shown in Figure 2 , in the partially observable Markov process of the embodiment of the present application, the posterior probabilities of the system in each state at the k-th sampling moment are often used for maintenance decision-making. Let represent the posterior probability that the system is in the sub-state i at time given the observed data :

[0114] (12)

[0115] where: is the failure time of the system, Represents the posterior probability vector.

[0116] The posterior probabilities of the system being in each state satisfy the following equation:

[0117] (13)

[0118] Furthermore, by Bayes' theorem, at each sampling moment, the posterior probability can be iteratively updated by the following equation:

[0119] (14)

[0120] In the above equation (14):

[0121] (15)

[0122] (16)

[0123] Assume that at the system has not failed yet, that is, , for any , at the conditional reliability of the remaining useful life of the system is:

[0124] (17)

[0125] For a sampling interval of , the conditional reliability at

[0126] (18)

[0127] By updating the posterior probability in real time in the embodiments of the present application, the actual state at each sampling moment can be more accurately reflected, which helps to capture the change of the state in time, and by updating the conditional reliability in real time, it helps to enhance the prediction ability of the system reliability and improve the efficiency and accuracy of system decision-making.

[0128] In step S104, with the goal of minimizing the long-term expected average cost, a semi-Markov decision process is used to find the optimal conditional reliability control limit.

[0129] Specifically, the embodiments of the present application construct a maintenance decision optimization function with the goal of minimizing the long-term expected average cost. At each sampling moment, the state observation data of the system is obtained, and two decisions need to be made according to the conditional reliability of the system. One is that the system continues to run until the next sampling moment, and the cost required for each sampling is C S . The other is that the system stops for inspection, and preventive maintenance measures are taken according to the inspection results. The corresponding inspection cost and inspection time are C Iand T I If a decision to perform a shutdown inspection is made, the true potential state of the system becomes known. If the system is in state 1 after inspection, the system makes a minor preventive maintenance decision, such as minor adjustment, lubrication, etc. If the system is in state 2 after inspection, the system makes a major preventive maintenance decision, such as repair, etc., and the corresponding preventive maintenance costs and maintenance times are C PM and T PM respectively. In state 2, additional operating cost rates and maintenance cost rates C AO and C AM will be incurred. If the system fails during this period, then a corrective maintenance decision is made, and the corresponding maintenance costs and component replacement times are C F and T F respectively. In addition, when the system is undergoing a shutdown inspection, preventive maintenance, or corrective maintenance, production loss costs C LP per unit time will be incurred. After preventive maintenance or corrective maintenance, the system returns to the as-good-as-new state.

[0130] Optionally, in an embodiment of the present application, the optimization objective of the optimal shutdown inspection timing is to seek the optimal remaining life conditional reliability control limit, that is, the optimal conditional reliability control limit R * , such that the long-term expected average cost is minimized:

[0131] (19)

[0132] where R* is the optimal conditional reliability control limit; is the long-term expected average cost when the conditional reliability is R*; and are the expected cycle cost and expected cycle length when the conditional reliability is R*, respectively.

[0133] The embodiments of the present application provide a model for constructing the long-term expected average cost, which can more comprehensively consider the influence of various factors on the cost, improve the accuracy and reliability of cost prediction, and help to comprehensively determine the optimal shutdown time.

[0134] In step S105, according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process are deduced and calculated to determine the maintenance decision of the multi-state complex system.

[0135] Specifically, as shown in Figure 2 , the embodiments of the present application can seek the optimal conditional reliability control limit R* in the semi-Markov decision process SMDP. Since is a probability simplex. To use the SMDP method to find the optimal policy, it is necessary to discretize the space [0, 1] of the posterior probability. For a fixed number of sub-intervals L, define that the SMDP is in the state , that is, if the posterior probability is in the interval , then the SMDP is in the state . If the posterior probability is in the interval , then the SMDP is in the state , etc. If , and the system is in an unhealthy state after a comprehensive inspection, define that the SMDP is in the state . If the system fails, define that the SMDP is in the state .

[0136] Use . Then define the state space of the SMDP as , where indicates that the system is in a healthy state at the initial moment.

[0137] Optionally, in an embodiment of the present application, for the cost minimization optimization objective, given the control limit , the long-term expected average cost can be obtained by solving the following linear equation:

[0138] (20)

[0139] where is the relative value given the current state x; is the relative value of the value function when the semi-Markov decision process is in the state l; l is a certain state of the semi-Markov decision process; is the expected maintenance cost at the next decision moment given the current state x; S is the state space of the semi-Markov decision process; is the expected sojourn time at the next decision moment given the current state x; is the probability that given the current state , the system will be in the state at the next moment; ; is the relative value of the system when it is in the state in the semi-Markov decision process.

[0140] Optionally, in an embodiment of the present application, the semi-Markov decision process algorithm is used to calculate the state transition probability, the expected maintenance cost, and the expected sojourn time, which can be specifically as follows:

[0141] ① SMDP transition probability calculation. For each , and , , where , when , the transition probability of the semi-Markov decision process (SMDP) is calculated by the following formula:

[0142] (21)

[0143] where is the conditional reliability at the k-th sampling; is the given conditional reliability control limit; is the transition probability of the semi-Markov decision process from state to state at the k-th sampling time; , where represents the state space where the sum of the sub-states of the semi-Markov decision process is less than L; L is the fixed sub-interval number for partitioning the state space of the semi-Markov decision process; is the sub-state of the semi-Markov decision process at the k-th sampling time; is the sub-state of the semi-Markov decision process at the next sampling time; represents the m-th sub-state in the state space of the semi-Markov decision process; k1 is the number of exponential phases of the sojourn time distribution in the healthy state; k2 is the number of exponential phases of the sojourn time distribution in the unhealthy state approximated by the hyper-Erlang distribution; represents at time the posterior probability that the system is in sub-state i given the observed data , , , represents that the next state is the a-th sub-state in the state space of the semi-Markov decision process; is the failure time of the system; is the probability that the system is in sub-state i at the next moment given the monitoring data at time; is at time the state of the system; is the conditional reliability of the system given the monitoring data at time; is the posterior probability that the system is in each sub-state given the monitoring data at time; is the sampling interval; is the time of the k-th sampling; d is the dimension of the data.

[0144] Let , at the sampling time , the posterior probabilities of the system in each state can be written in the following form:

[0145] (22)

[0146] Where:

[0147] In Equation (21), for , the probability is calculated as follows:

[0148] (23)

[0149] Where: , .

[0150] Similarly, for :

[0151] (24)

[0152] Where: , .

[0153] If , then immediately conduct a comprehensive inspection of the system. After the inspection, the probability of "true warning", that is, the system is in an unhealthy state, is:

[0154] (25)

[0155] Where, is the conditional reliability of the system at time ; is the transition probability from state to state PM in the semi-Markov decision process, is the sum of the posterior probabilities that the system is in an unhealthy state. PM indicates that the system is in an unhealthy state after a comprehensive inspection.

[0156] After a comprehensive inspection, it is a "false warning", that is, the probability that the system is in a healthy state is:

[0157] (26)

[0158] Where, is the transition probability from state to state in the semi-Markov decision process, indicates that the system is in a healthy state at the initial time.

[0159] If , then the probability of system failure is:

[0160] (27)

[0161] Among them, is the transition probability from state to state F in the semi-Markov decision process; F represents that the system is in a failure state after a comprehensive inspection.

[0162] After the system fails and is replaced and preventive maintenance measures are taken, the state process starts from a new system cycle:

[0163] (28)

[0164] (29)

[0165] Among them, is the transition probability from state F to state , is the transition probability from state PM to state .

[0166] ② Calculation of the expected maintenance cost of SMDP. If , a comprehensive inspection of the system is required, resulting in corresponding inspection costs per unit time and production loss costs. The expected maintenance cost is:

[0167] (30)

[0168] Among them, is the expected cost when the conditional reliability is lower than the control limit threshold at time in the semi-Markov decision process and the system is in state ; T I is the inspection time; C I is the inspection cost per unit time for a comprehensive inspection of the system; C LP is the production loss cost for a comprehensive inspection of the system.

[0169] If , then the system may fail or may not take effect at the next moment. If it is in an unhealthy state, additional operation costs and maintenance costs will also be generated. The expected maintenance cost is:

[0170] (31)

[0171] Among them, is the expected cost when the conditional reliability is higher than the control limit threshold at time in the semi-Markov decision process and the system is in state ; CS The cost required for each sampling; C F The cost of after-sales maintenance; T F The component replacement time; C AO and C AM are the additional operation cost and maintenance cost respectively; j is a certain sub-state when the system is in an unhealthy state; is at The posterior probability that the system is in sub-state i at time; is the probability that the system transfers from sub-state i to sub-state j within time t.

[0172] The expected maintenance costs in the PM state and the F state are respectively:

[0173] (32)

[0174] (33)

[0175] Among them, C PM and T PM are the preventive maintenance cost and maintenance time.

[0176] ③ Calculation of the expected sojourn time of the semi-Markov decision process (SMDP). If , a comprehensive inspection of the system is required, and the expected sojourn time is:

[0177] (34)

[0178] Among them, is at The expected sojourn time in the semi-Markov decision process when the conditional reliability is lower than the control limit threshold at time and in state .

[0179] If , then the system may fail or may not take effect at the next moment. The expected sojourn time is:

[0180] (35)

[0181] Among them, is at The expected sojourn time in the semi-Markov decision process when the conditional reliability is higher than the control limit threshold at time and in state ; At The probability density function of the remaining life of the system updated after obtaining the monitoring data at time.

[0182] The expected sojourn times in the PM state and the F state are respectively:

[0183] (36)

[0184] (37)

[0185] Among them, represents the expected sojourn time of the PM state; represents the expected sojourn time of the F state.

[0186] It should be noted that the embodiments of the present application can calculate the optimal conditional reliability control limit, which is specifically as follows:

[0187] (38)

[0188] It is obtained by assigning different control limits The minimum average cost is solved by iterative calculation from Equation (20) and obtained.

[0189] The embodiments of the present application use the method of combining SMDP with the long-term cost model, which can accurately and quantitatively describe the maintenance decision. Moreover, on the basis of considering the actual system degradation characteristics, by dynamically optimizing the maintenance strategy, the balance between reliability and economy can be achieved.

[0190] According to the maintenance decision method for multi-state complex systems under partially observable information proposed by the embodiments of the present application, a continuous-time homogeneous Markov chain model of the multi-state degradation process can be constructed based on the state observation data of the multi-state complex system. The expectation maximization algorithm is used to jointly estimate the state transition parameters and observation parameters to be estimated in the Markov chain model. Furthermore, based on the updated state transition parameters and observation parameters, combined with Bayes' theorem, the posterior probability of the system at each sampling moment is updated in real time to calculate the conditional reliability of the system. With the goal of minimizing the long-term expected average cost, the semi-Markov decision process is used to find the optimal conditional reliability control limit. Thus, based on the size relationship between the conditional reliability and the optimal conditional reliability control limit at each sampling moment, the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process are derived and calculated, realizing the accurate and quantitative description of the maintenance decision of the multi-state complex system, which helps to optimize the system performance, reduce the maintenance cost, and improve the overall operation efficiency. Therefore, the problems in the related art are solved. Since the difference in the failure rates of the system in each state is not considered and the index of the optimal shutdown threshold is a virtual composite index, it is easy to lead to the lack of accuracy and pertinence in the maintenance decision, affecting the reliability and operation efficiency of the system.

[0191] Secondly, refer to the drawings to describe the maintenance decision system for multi-state complex systems under partially observable information proposed by the embodiments of the present application.

[0192] Figure 3It is a schematic structural diagram of a multi-state complex system maintenance decision-making system under partial observable information of an embodiment of the present application.

[0193] As Figure 3 shown, the multi-state complex system maintenance decision-making system 10 under partial observable information includes: a degradation model construction unit 100, an estimation unit 200, a calculation unit 300, a control limit determination unit 400, and a policy determination unit 500.

[0194] Specifically, the degradation model construction unit 100 is used to construct a continuous-time homogeneous Markov chain model of the multi-state degradation process based on the state observation data of the multi-state complex system.

[0195] The estimation unit 200 is used to jointly estimate the state transition parameters to be estimated and the observation parameters to be estimated in the Markov chain model by using the expectation maximization algorithm, and obtain the updated state transition parameters and the updated observation parameters.

[0196] The calculation unit 300 is used to update the posterior probability of the multi-state complex system at each sampling moment in real time based on the updated state transition parameters and the updated observation parameters, and combine with Bayes' theorem to calculate the conditional reliability of the multi-state complex system at each sampling moment.

[0197] The control limit determination unit 400 is used to seek the optimal conditional reliability control limit by using the semi-Markov decision process with the goal of minimizing the long-term expected average cost.

[0198] The policy determination unit 500 is used to derive and calculate the transition probability, the expected sojourn time, and the expected maintenance cost in the semi-Markov decision process according to the size relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, so as to determine the maintenance decision of the multi-state complex system.

[0199] It should be noted that the foregoing explanation of the embodiment of the multi-state complex system maintenance decision-making method under partial observable information also applies to the multi-state complex system maintenance decision-making system under partial observable information of this embodiment, and will not be elaborated here.

[0200] The multi-state complex system maintenance decision-making system under partially observable information proposed according to the embodiments of the present application can construct a continuous-time homogeneous Markov chain model of the multi-state degradation process based on the state observation data of the multi-state complex system, use the expectation maximization algorithm to jointly estimate the to-be-estimated state transition parameters and observation parameters in the Markov chain model. Furthermore, based on the updated state transition parameters and observation parameters, combined with Bayes' theorem, the posterior probability of the system at each sampling moment is updated in real time to calculate the conditional reliability of the system. With the goal of minimizing the long-term expected average cost, the semi-Markov decision process is used to find the optimal conditional reliability control limit. Thus, based on the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process are derived and calculated, realizing an accurate quantitative description of the maintenance decision-making of the multi-state complex system, which helps to optimize the system performance, reduce the maintenance cost, and improve the overall operation efficiency. Thereby, it solves the problems in the related technologies that, due to the failure to consider the differences in the failure rates of the system in each state and the index of the optimal shutdown threshold being a virtual composite index, it is easy to lead to the lack of accuracy and pertinence in the maintenance decision-making, affecting the reliability and operation efficiency of the system, etc.

[0201] The embodiments of the present application also provide a computer program product, including a computer program / instructions, which when executed by a processor, implement the above multi-state complex system maintenance decision-making method under partially observable information.

[0202] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or N embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0203] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0204] Any process or method description represented in a flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logic function or process. The scope of the preferred embodiments of the present application includes additional implementations, where functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0205] Logic and / or steps represented in a flowchart or otherwise described herein, for example, may be considered a sequenced list of executable instructions for implementing a logical function, and may be embodied specifically in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. As used in this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection having one or N wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, as the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0206] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0207] Those of ordinary skill in the art can understand that all or part of the steps carried out in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0208] In addition, in each of the embodiments of the present application, the functional units can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0209] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

Claims

1. A maintenance decision-making method for multi-state complex systems under partially observable information, characterized in that, The method includes the following steps: Based on the state observation data of a multi-state complex system, construct a continuous-time homogeneous Markov chain model for the multi-state degradation process; Use the expectation maximization algorithm to jointly estimate the state transition parameters to be estimated and the observation parameters to be estimated in the Markov chain model, and obtain the updated state transition parameters and the updated observation parameters; Based on the updated state transition parameters and the updated observation parameters, combine Bayes' theorem to update the posterior probability of the multi-state complex system at each sampling moment in real time, so as to calculate the conditional reliability of the multi-state complex system at each sampling moment; With the goal of minimizing the long-term expected average cost, use the semi-Markov decision process to seek the optimal conditional reliability control limit; According to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, deduce and calculate the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process to determine the maintenance decision of the multi-state complex system.

2. The method according to claim 1, wherein The construction of the continuous-time homogeneous Markov chain model for the multi-state degradation process includes: Construct the multi-state degradation process into a continuous-time homogeneous Markov chain model with a state space of , where the state space is defined as , is the set of healthy states, is the set of unhealthy states, is the set of failure states. The failure rate of a random failure occurring in the healthy state is a constant, and the failure rate in the unhealthy state increases with time.

3. The method according to claim 2, wherein In state, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of The expression is as follows: Among them, In the state, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of ; is the sampling interval; is the time of the k-th sampling; d is the dimension of the data; The residence time in the state follows an Erlang distribution with order k1 and transition rate λ1, and the expression of its probability density function f1(t) is as follows: where f1(t) is the state lower probability density function; λ1 is the transition rate of the system in the healthy state; t is the residence time.

4. The method according to claim 2, wherein Under conditions, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of , and its expression is as follows: Among them, in the state, the observed data of the system obeys a multivariate normal distribution with a mean of and a covariance of ; is the sampling interval; is the time of the k-th sampling; d is the dimension of the data; In the residence time in the state follows a hyper-Erlang distribution, and the expression of its cumulative distribution function is as follows: wherein, is the cumulative distribution function of the sojourn time in state following a hyper-Erlang distribution; λ2 is the transition rate when the system is in an unhealthy state; is the cumulative distribution function of a hyper-Erlang distribution; is the number of exponential phases and is the cumulative distribution function of an Erlang distribution with a transition rate of λ2.

5. The method according to claim 1, wherein The expression for minimizing the long-term expected average cost is as follows: Among them, R* is the optimal conditional reliability control limit; is the long-term expected average cost when the conditional reliability is R*; and are the expected cycle cost and the expected cycle length when the conditional reliability is R*, respectively; Among them, is the long-term expected average cost when the conditional reliability is ; is the reliability control limit for the given condition, obtained by solving the following linear equation: Among them, is the relative value of the given current state x; is the relative value of the value function when the semi-Markov decision process is in state l; l is a certain state of the semi-Markov decision process; is the expected maintenance cost at the next decision moment given the current state x; S is the state space of the semi-Markov decision process; is the expected sojourn time at the next decision moment given the current state x; is given the current state , the probability that the system will be in state at the next moment; ; is the relative value when the system is in state in the semi-Markov decision process.

6. The method according to claim 1, wherein The deduction and calculation of the transition probability in the semi-Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit includes: If , the transition probability in the semi-Markov decision process is calculated by the following formula: wherein, is the conditional reliability of the k-th sampling; is the given conditional reliability control limit; is the transition probability of the semi-Markov decision process from state to state at the k-th sampling time; , where represents the state space where the sum of the sub-states of the semi-Markov decision process is less than L; L is the fixed sub-interval number of the state space partition of the semi-Markov decision process; is the sub-state of the semi-Markov decision process at the k-th sampling time; is the sub-state of the semi-Markov decision process at the next sampling time; represents the m-th sub-state in the state space of the semi-Markov decision process; k1 is the number of exponential phases of the residence time distribution in the healthy state; k2 is the number of exponential phases of the residence time distribution in the unhealthy state approximated by the hyper-Erlang distribution; represents at time, given the observed data the posterior probability that the system is in sub-state i; , , , represents that the next state is in the a-th sub-state in the state space of the semi-Markov decision process; is the failure time of the system; is the probability that the system is in sub-state i at the next moment under the condition of obtaining the monitoring data at time; is the state of the system at time; is the conditional reliability of the system under the condition of obtaining the monitoring data at time; is the posterior probability that the system is in each sub-state under the condition of obtaining the monitoring data at time; is the sampling interval; is the time of the k-th sampling; d is the dimension of the data; If , the probability that the system is in an unhealthy state is: Among them, is the conditional reliability of the system at moment; is the transition probability from state to state PM in the semi-Markov decision process; is the sum of the posterior probabilities that the system is in an unhealthy state; PM indicates that the system is in an unhealthy state after a comprehensive inspection; The probability that the system is in the healthy state is: Among them, is the transition probability from state to state in the semi-Markov decision process; indicates that the system is in a healthy state at the initial moment; The probability that the system fails is: Among them, is the transition probability from state to state F; F represents that the system is in a failure state after a comprehensive inspection; When the system fails and is replaced and preventive maintenance measures are taken, the state process starts from a new system cycle: Among them, is the transition probability from state F to state in the semi-Markov decision process; is the transition probability from state PM to state in the semi-Markov decision process.

7. The method according to claim 6, characterized in that, The deduction and calculation of the expected maintenance cost in the semi-Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit. The expected maintenance cost includes: If , a comprehensive inspection of the system is required, generating a corresponding inspection cost C per unit time I and a production loss cost C LP , then the expected maintenance cost is Among them, is the expected cost when in state in the semi-Markov decision process; T I is the inspection time; C I is the unit-time inspection cost generated for a comprehensive inspection of the system; C LP is the production loss cost generated for a comprehensive inspection of the system; If , then the system may fail or may not take effect at the next moment. If it is in an unhealthy state, additional operating costs C AO and maintenance costs C AM will be incurred, and the expected maintenance cost is as follows: Among them, C S is the cost required for each sampling; C F is the cost of post - event maintenance; T F is the component replacement time; C AO and C AM are the additional operation cost and maintenance cost respectively; j is a certain sub - state when the system is in an unhealthy state; is the posterior probability that the system is in sub - state i at time; is the probability that the system transfers from sub - state i to sub - state j within t time; The expected maintenance costs in the PM state and the F state are respectively: Among them, C PM and T PM are the preventive maintenance cost and the maintenance time respectively.

8. The method according to claim 6 or 7, characterized in that The deduction and calculation of the expected sojourn time in the semi-Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit includes: If , a comprehensive system check is required, the expected residence time is: wherein, is the expected sojourn time in state during the semi-Markov decision process; If , then the system may fail or may not take effect at the next moment, and the expected residence time is: Among them, at the probability density function of the remaining system life updated after obtaining the monitoring data at the moment; The expected sojourn times in the PM state and the F state are respectively: Among them, represents the expected residence time of the PM state; represents the expected residence time of the F state.

9. A maintenance decision-making system for a multi-state complex system under partially observable information, characterized in that, Include: Degradation model construction unit: used to construct a continuous-time homogeneous Markov chain model for the multi-state degradation process based on the state observation data of the multi-state complex system; Estimation unit: used to jointly estimate the state transition parameters to be estimated and the observation parameters to be estimated in the Markov chain model by using the expectation maximization algorithm, and obtain the updated state transition parameters and the updated observation parameters; Calculation unit: used to update the posterior probability of the multi-state complex system at each sampling moment in real time based on the updated state transition parameters and the updated observation parameters, and combine Bayes' theorem to calculate the conditional reliability of the multi-state complex system at each sampling moment; Control limit determination unit: used to seek the optimal conditional reliability control limit by using the semi-Markov decision process with the goal of minimizing the long-term expected average cost; Strategy determination unit: used to deduce and calculate the transition probability, expected sojourn time, and expected maintenance cost in the semi-Markov decision process according to the magnitude relationship between the conditional reliability at each sampling moment and the optimal conditional reliability control limit, so as to determine the maintenance decision of the multi-state complex system.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1-8 are implemented.

Images