Task termination strategy generation method and system based on dynamic programming algorithm

Abstract

The invention provides a task termination strategy generation method and system based on a dynamic programming algorithm, a storage medium and electronic equipment, and relates to the field of risk control. The method comprises the following steps: firstly, defining that the deterioration state of a system obeys a multi-stage continuous time Markov chain and random task duration; secondly, constructing a value function to obtain expected costs of tasks in all possible execution durations in each deterioration state; and finally, a reverse induction method is combined with a dynamic programming algorithm, forward recursion is carried out from the end point time of the longest task duration, so that an optimal task termination decision under each time step length is determined, and global optimization can be realized. According to the method, the uncertainty of the task execution time in an actual system can be reflected more truly, the dynamic change in the task execution process can be monitored in real time, and the task termination strategy is dynamically adjusted based on real-time data, so that a complex and changeable task environment can be flexibly coped, and the survival rate of the system is maximized.

Description

Technical Field

[0001] The present invention relates to the field of risk control, and in particular to a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm. Background Art

[0002] As one of the risk control methods for safety-critical systems, the mission abort strategy has been widely used in various fields. For safety-critical engineering systems such as spacecraft, submarines, and power, system survival takes precedence over mission completion, and early mission abort is an effective way to improve system survivability. For example, when the sensor of an autonomous underwater vehicle fails, it can abort its mission to explore the ocean environment and surface to wait for rescue; when the imaging of a hospital's CT equipment is unstable, it can terminate the imaging mission and wait for repairs.

[0003] In traditional task termination strategies, some scholars use preset static thresholds as the termination conditions for task execution. For example: The paper (Zhao X, Sun J, Qiu Q, et al. Optimal inspection and mission abort policies for systems subject to degradation [J]. European Journal of Operational Research, 2021, 292 (2): 610-621.) discloses that some systems are set to automatically terminate tasks when the system deterioration state reaches a certain threshold to prevent the system from deteriorating to a fault state. Although this method is simple and intuitive, it is difficult to adapt to complex and changing task execution environments, which may result in waste of resources or failure to complete tasks as scheduled.

[0004] In addition, there are mission termination strategies based on time and system deterioration. The paper (Qiu Q, Cui L. Gamma process based optimal mission abort policy [J]. Reliability Engineering & System Safety, 2019, 190: 106496.) studies the optimal mission abort strategy for the two-stage deterioration process system simulated by the Gamma process. The deterioration-based and duration-based mission abort strategies are obtained on the basis of random failure models and mission abort strategies. The results show that the duration-based abort strategy performs better when the deterioration rate is low, and the deterioration-based abort strategy is more cost-effective when the deterioration rate is large.

[0005] In summary, traditional task termination strategies are mostly based on time-fixed tasks or static thresholds, and lack the ability to respond to dynamic changes during task execution. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the deficiencies in the prior art, the present invention provides a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm, which solves the technical problem that traditional task termination strategies are mostly based on time-fixed tasks or static thresholds and lack the ability to respond to dynamic changes during task execution.

[0008] (II) Technical solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0010] A task termination strategy generation method based on a dynamic programming algorithm, comprising:

[0011] Define the system's deterioration state to obey a multi-stage continuous-time Markov chain, and define random task durations;

[0012] Based on the state transition probability and the cost within the step length of the system, a value function is constructed to obtain the expected cost of the task in each deteriorated state with all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0013] Starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected, otherwise the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0014] Preferably, the deterioration state of the system is defined as X = {X t , t≥0} is subject to a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state;

[0015] And the state transfer rate matrix is ​​set to Q, the state transfer probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where P ij (τ) represents the probability that the system will transition from state i to state j after a period of time τ, P(X s+τ =j|X s=i) represents the probability that the system will transition from state i to state j after a period of time τ starting from time s;

[0016] Define the random task duration, the shortest task duration is LΔ, and the longest task duration is MΔ, where Δ represents the unit time step, L and M represent the maximum execution time of the shortest task and the longest task, including the number of time steps;

[0017] And the probability of completing the task at time LΔ+t is recorded as g(t), which obeys the gamma distribution Γ(α, β), expressed as g(t|α, β), where α represents the shape parameter of the gamma function and β represents the scale parameter of the gamma function.

[0018] Preferably, the state transition probability specifically includes:

[0019] 1) When n<L, the state transition probability has nothing to do with the probability of task success:

[0020] p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i) = P ij (Δ) (1)

[0021] Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transfers from state i at time nΔ to state j at time (n+1)Δ; Pi j (Δ) represents the probability that the system transitions from state i to state j after Δ time;

[0022] 2) When n>L, if the task is successfully completed, the state transition will not occur, and the state transition probability is calculated by the conditional probability that the task is not completed at this time:

[0023]

[0024] Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, and its formula is

[0025] Preferably, the intra-step cost specifically includes:

[0026] 1) When n = M-1, the system state is within the step length of (M-1, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion:

[0027] C (n，i) =CF+CM+CR+CO (3)

[0028]

[0029]

[0030]

[0031]

[0032] Here, a time step Δ is divided into K equal parts, k represents the index of K, and δ represents the step length of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is:

[0033] 2) When L≤n<M-1, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO and the expected reward CR for task completion:

[0034] C (n，i) =CF+CI+CR+CO (8)

[0035]

[0036] Among them, C I represents the inspection cost;

[0037] 3) When n<L, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF′, the expected cost of inspection CI′ and the expected cost of operation CO′:

[0038] C (n，i) =CF′+CI′+CO′ (10)

[0039] CF′=P iI (Δ)(C m +C s ) (11)

[0040]

[0041]

[0042] Where PiI(Δ) represents the probability that the system state transitions from state i to system failure state I after Δ time.

[0043] Preferably, a value function V(n, i) is constructed to represent the expected cost of the system state being (n, i); specifically, it includes:

[0044] 1) When n=M, the execution time of the task reaches the maximum task time. No matter what state the system deteriorates in, the task is considered failed. The expected cost is:

[0045] V (M，i) =C m ,i≠I (14)

[0046] Among them, C m represents the cost of task failure;

[0047] 2) When n<M, i≠I, the expected cost takes the minimum value among the sum of task failure cost, immediate cost and future cost; the expected cost is:

[0048]

[0049] Among them, min represents the minimization function, V (n+1，j ) represents the future expected cost of the system state being (n+1, j).

[0050] Preferably, starting from the end time of the longest task duration, the expected cost of the task in the current deteriorated state is solved by reverse iteration using a dynamic programming algorithm at equal time steps. If the expected cost is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached; including:

[0051] S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s , C m , C I , C i , r, Q, and let n = M, i = 0;

[0052] S32, let n=n-1;

[0053] S33, determine whether n is less than 0, if so, determine that the starting time of the task has been reached, and transfer to S38, otherwise transfer to S34;

[0054] S34, determine whether i is greater than I-1, if so, go to S32, otherwise go to S35;

[0055] S35, calculating the cost C(n, i) within the step length to calculate the expected cost V(n, i);

[0056] S36. Determine whether V(n, i) is equal to C m ,If so, select the task termination action, otherwise select the task continuation action, and save it to the task termination policy set;

[0057] S37, set i=i+1, and go to S34;

[0058] S38. End the algorithm.

[0059] A task termination strategy generation system based on a dynamic programming algorithm, comprising:

[0060] Definition module, used to define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration;

[0061] A construction module is used to construct a value function based on the state transition probability of the system and the cost within the step length to obtain the expected cost of the task in each deteriorated state in all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0062] The solution module is used to start from the end time of the longest task duration, and use a dynamic programming algorithm to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected, otherwise the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0063] A storage medium stores a computer program for generating a task termination strategy based on a dynamic programming algorithm, wherein the computer program enables a computer to execute the task termination strategy generating method as described above.

[0064] An electronic device, comprising:

[0065] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include instructions for executing the task termination policy generation method as described above.

[0066] (III) Beneficial effects

[0067] The present invention provides a task termination strategy generation method, system, storage medium and electronic device based on a dynamic programming algorithm. Compared with the prior art, it has the following beneficial effects:

[0068] In the present invention, the deterioration state of the system is first defined to obey a multi-stage continuous-time Markov chain and a random task duration; secondly, a value function is constructed to obtain the expected cost of tasks in all possible execution durations in each deterioration state; finally, the reverse induction method is combined with a dynamic programming algorithm to recursively work forward from the end time of the longest task duration to determine the optimal task termination decision at each time step, which can achieve global optimization. The present invention can more truly reflect the uncertainty of task execution time in an actual system, as well as monitor the dynamic changes in the task execution process in real time, dynamically adjust the task termination strategy based on real-time data, and thus can flexibly respond to complex and changeable task environments and maximize the survival rate of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 The embodiment of the present invention provides a block diagram of a method for generating a task termination strategy based on a dynamic programming algorithm;

[0071] Figure 2 An embodiment of the present invention provides a flow chart of combining a reverse induction method with a dynamic programming algorithm. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] The embodiments of the present application provide a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm, thereby solving the technical problem that most traditional task termination strategies are based on time-fixed tasks or static thresholds and lack the ability to respond to dynamic changes during task execution.

[0074] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0075] As mentioned in the background technology, in terms of research content, traditional task termination strategies usually assume that the task duration is fixed, which cannot reflect the randomness and uncertainty of task duration in actual applications. This assumption leads to a lack of flexibility and accuracy in the termination strategy. In addition, previous studies often assume that system state changes are linear or predictable when dealing with them, ignoring the complex dynamic changes of the system state, resulting in insufficient accuracy and effectiveness of decision-making.

[0076] In fact, in terms of research methods, due to the simple decision rules and fixed termination thresholds in traditional task termination strategies, it is impossible to provide globally optimal decision support, making it difficult for traditional methods to deal with practical problems.

[0077] Different from this, an embodiment of the present invention proposes a method for solving task termination strategy based on a dynamic programming algorithm. By introducing the randomness of task duration, using a multi-stage continuous-time Markov chain to model the system state, and combining it with a dynamic programming algorithm, a method for optimizing the task termination strategy problem is provided.

[0078] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0079] Embodiment 1:

[0080] like Figure 1 As shown, a task termination strategy generation method based on a dynamic programming algorithm is characterized by comprising:

[0081] S 1. Define the system's deterioration state to obey a multi-stage continuous-time Markov chain, and define the random task duration;

[0082] S2. Based on the state transition probability and the cost within the step length of the system, a value function is constructed to obtain the expected cost of the task in each deteriorated state with all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0083] S3. Starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0084] The embodiments of the present invention can more truly reflect the uncertainty of task execution time in an actual system, monitor the dynamic changes in the task execution process in real time, and dynamically adjust the task termination strategy based on real-time data, thereby being able to flexibly respond to complex and changing task environments and maximize the survival rate of the system.

[0085] Next, we will introduce the steps of the above scheme in detail:

[0086] In step S1, the deterioration state of the system is defined to obey a multi-stage continuous-time Markov chain, and a random task duration is defined.

[0087] In this step, the deterioration state of the system is defined as X = {X t , t≥0} is subject to a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state;

[0088] And the state transfer rate matrix is ​​set to Q, the state transfer probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where P ij (τ) represents the probability that the system will transition from state i to state j after a period of time τ, P(X s+τ =j|X s =i) represents the probability that the system will transition from state i to state j after a period of time τ starting from time s.

[0089] Therefore, it is convenient to check the system status every Δ time in the subsequent steps, and then the decision maker makes a decision to choose to continue the task or terminate the task. If it is assumed that the system failure can be observed once it occurs, in order to improve the survival rate of the system, it is understandable that the correct task termination decision needs to be made before the system failure occurs.

[0090] Next, this step also defines the random task duration, the shortest task duration is LΔ, and the longest task duration is MΔ, where Δ represents the unit time step, L and M represent the maximum execution time of the shortest task and the longest task, including the number of time steps;

[0091] And the probability of completing the task at time LΔ+t is recorded as g(t), which obeys the gamma distribution Γ(α, β), expressed as g(t|α, β), where α represents the shape parameter of the gamma function and β represents the scale parameter of the gamma function.

[0092] In step S2, a value function is constructed based on the state transition probability of the system and the cost within the step to obtain the expected cost of the task in each deteriorated state for all possible execution times; wherein the execution time is pre-divided into a number of equal time steps.

[0093] In order to construct the value function, this step introduces the state transition probability of the system and the cost within the step length in turn:

[0094] In the embodiment of the present invention, the state transition probability specifically includes:

[0095] 1) When n<L, the state transition probability has nothing to do with the probability of task success:

[0096] p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i) = P ij (Δ) (1)

[0097] Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transfers from state i at time nΔ to state j at time (n+1)Δ; P ij (Δ) represents the probability that the system transitions from state i to state j after Δ time.

[0098] 2) When n>L, if the task is successfully completed, the state transition will not occur, and the state transition probability is calculated by the conditional probability that the task is not completed at this time:

[0099]

[0100] Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, and its formula is

[0101] In the embodiment of the present invention, the intra-step cost specifically includes:

[0102] 1) When n = M-1, the system state is within the step length of (M-1, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion:

[0103] C (n，i) =CF+CM+CR+CO (3)

[0104]

[0105]

[0106]

[0107]

[0108] Here, a time step Δ is divided into K equal parts, k represents the index of K, and δ represents the step length of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is:

[0109] 2) When L≤n<M-1, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO and the expected reward CR for task completion:

[0110] C (n，i) =CF+CI+CR+CO (8)

[0111]

[0112] Among them, C I Represents the inspection cost.

[0113] 3) When n<L, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF′, the expected cost of inspection CI′ and the expected cost of operation CO′:

[0114] C (n，i) =CF′+CI′+CO′ (10)

[0115] CF′=P iI (Δ)(C m +C s ) (11)

[0116]

[0117]

[0118] Where PiI(Δ) represents the probability that the system state transitions from state i to system failure state I after Δ time.

[0119] Next, this step uses the state transition probability and the cost within the step length of the above system to construct a value function V(n, i), which represents the expected cost of the system state being (n, i); specifically, it includes:

[0120] 1) When n=M, the execution time of the task reaches the maximum task time. No matter what state the system deteriorates in, the task is considered failed. The expected cost is:

[0121] V (M，i) =C m ,i≠I (14)

[0122] Among them, C m Represents the task failure cost.

[0123] 2) When n<M, i≠I, the expected cost takes the minimum value among the sum of task failure cost, immediate cost and future cost; the expected cost is:

[0124]

[0125] Among them, min represents the minimization function, V (n+1，j) Represents the future expected cost when the system state is (n+1, j).

[0126] In step S3, starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected, otherwise the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0127] The embodiment of the present invention combines dynamic programming and reverse induction to solve the optimal decision at each time step, aiming to make correct task termination decisions before system failure occurs, thereby maximizing the survival rate of the system and improving the reliability and stability of the system.

[0128] Correspondingly, such as Figure 2 As shown, this step specifically includes:

[0129] S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s , C m , C I , C i , r, Q, and let n = M, i = 0;

[0130] S32, let n=n-1;

[0131] S33, determine whether n is less than 0, if so, determine that the starting time of the task has been reached, and transfer to S38, otherwise transfer to S34;

[0132] S34, determine whether i is greater than I-1, if so, go to S32, otherwise go to S35;

[0133] S35, calculating the cost C(n, i) within the step length to calculate the expected cost V(n, i);

[0134] S36. Determine whether V(n, i) is equal to C m ,If so, select the task termination action, otherwise select the task continuation action, and save it to the task termination policy set;

[0135] S37, set i=i+1, and go to S34;

[0136] S38. End the algorithm.

[0137] It should be noted that, since the maximum duration limit of the task has been reached when n=M, no decision needs to be made at this time. Therefore, the above algorithm provides a task termination strategy that starts from n=M-1 and iterates in a loop.

[0138] So far, the embodiment of the present invention has completed the entire process of the task termination strategy generation method based on the dynamic programming algorithm.

[0139] Embodiment 2:

[0140] The embodiment of the present invention provides a task termination strategy generation system based on a dynamic programming algorithm, comprising:

[0141] Definition module, used to define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration;

[0142] A construction module is used to construct a value function based on the state transition probability of the system and the cost within the step length to obtain the expected cost of the task in each deteriorated state in all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0143] The solution module is used to start from the end time of the longest task duration, and use a dynamic programming algorithm to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected, otherwise the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0144] Embodiment 3:

[0145] An embodiment of the present invention provides a storage medium storing a computer program for generating a task termination strategy based on a dynamic programming algorithm, wherein the computer program enables a computer to execute the task termination strategy generation method as described in Example 1.

[0146] Embodiment 4:

[0147] An embodiment of the present invention provides an electronic device, including:

[0148] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the task termination policy generation method as described in Example 1.

[0149] It can be understood that the task termination strategy generation system based on the dynamic programming algorithm, storage medium and electronic device provided in the embodiments of the present invention correspond to the task termination strategy generation method based on the dynamic programming algorithm provided in the embodiments of the present invention. The explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the task termination strategy generation method and will not be repeated here.

[0150] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0151] 1. Compared with the traditional static threshold method, the embodiment of the present invention can monitor the dynamic changes in the task execution process in real time, and dynamically adjust the task termination strategy based on real-time data, and can also avoid resource waste or early task termination caused by static threshold setting. Through the application of dynamic programming algorithm, strategy selection can be made more accurately.

[0152] 2. The task duration considered in the embodiment of the present invention is random, and the change of task duration is described by using gamma distribution. This feature enables the present invention to more truly reflect the uncertainty of task execution time in the actual system, avoiding the limitations brought by the traditional method of assuming that the task duration is fixed.

[0153] 3. The embodiments of the present invention are not limited by task types, execution conditions, etc., and can flexibly cope with complex and changing task environments, and can be dynamically adjusted according to real-time conditions to ensure system retention security while executing tasks. Since the randomness of task duration and the dynamic changes of system status are taken into account, this method can be applied to complex systems that require decision optimization, such as industrial production systems, communication networks, medical equipment, etc.

[0154] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0155] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task termination strategy generation method based on a dynamic programming algorithm, characterized in that: Including: Defining that the deterioration state of the system follows a multi-stage continuous-time Markov chain, and defining a random task duration; Based on the state transition probability of the system and the cost within a time step, constructing a value function to obtain the expected cost of the task at each deterioration state for all possible execution durations; wherein, the execution duration is pre-divided into a number of equal time steps; Starting from the end time of the longest task duration, using the dynamic programming algorithm to iteratively solve the expected cost of the task in the current deterioration state every equal time step in reverse. If it is equal to the task failure cost, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set until reaching the start time of the task.

2. The task termination policy method according to claim 1, wherein Define the deterioration state of the system X = {X t , t≥0} is subject to a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state; And the state transfer rate matrix is ​​set to Q, the state transfer probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where Pi j (τ) represents the probability that the system will transition from state i to state j after a period of time τ, P(X s+τ =j|X s =i) represents the probability that the system will transition from state i to state j after a period of time τ starting from time s; Defining a random task duration, the shortest task duration is LΔ, and the longest task duration is MΔ, where Δ represents the unit time step, and L and M respectively represent the maximum execution durations of the shortest task and the longest task including the number of time steps; And the probability that the task is completed at the moment of LΔ + t is denoted as g(t), and g(t) follows the gamma distribution Γ(α, β), expressed as g(t|α, β), where α represents the shape parameter of the gamma function, and β represents the scale parameter of the gamma function.

3. The task termination strategy method according to claim 2, characterized in that: The state transition probability specifically includes: 1) When n < L, the state transition probability is independent of the task success probability: p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i)=P ij (Δ) (1) Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transfers from state i at time nΔ to state j at time (n+1)Δ; Pi j (Δ) represents the probability that the system transitions from state i to state j after Δ time; 2) When n > L, if the task is successfully completed, the state transition does not occur, and the state transition probability is calculated from the conditional probability with the condition that the task is not completed at this time: Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, and its formula is 4. The task termination strategy method according to claim 3, characterized in that: The cost within a time step specifically includes: 1) When n = M-1, the system state is within the step length of (M-1, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion: C (n，i) =CF+CM+CR+CO (3) Here, a time step Δ is divided into K equal parts, k represents the index of K, and δ represents the step length of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is: 2) When L≤n<M-1, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO and the expected reward CR for task completion: C (n，i) =CF+CI+CR+CO (8) Among them, C I represents the inspection cost; 3) When n < L, the step - cost C of the system state in (n, i) (n，i) is composed of the expected cost of system failure CF′, the expected cost of inspection CI′, and the expected cost of operation CO′: C (n，i) =CF′+CI′+CO′ (10) CF′=P iI (Δ)(C m +C s ) (11) Among them, P iI (Δ) represents the probability that the system state changes from state i to system failure state I after Δ time.

5. The task termination policy method according to claim 4, wherein Constructing a value function V(n, i) to represent the expected cost when the system state is at (n, i); specifically including: 1) When n = M, the execution duration of the task reaches the longest task duration, and the task is regarded as failed regardless of the deterioration state of the system; the expected cost is: In (M，i) =C m ,i≠I (14) Among them, C m represents the cost of task failure; 2) When n < M and i ≠ I, the expected cost takes the minimum value between the task failure cost, the immediate cost, and the sum of future costs; the expected cost is: Among them, min represents the minimization function, V (n+1，j) Represents the future expected cost when the system state is (n+1, j).

6. The task termination strategy method according to claim 5, characterized in that: The process of starting from the end time of the longest task duration, using the dynamic programming algorithm to iteratively solve the expected cost of the task in the current deterioration state every equal time step in reverse. If it is equal to the task failure cost, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set until reaching the start time of the task; includes: S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s , C m , C I , C i , r, Q, and let n = M, τ = 0; S32. Let n = n - 1; S33. Determine whether n is less than 0. If so, it is considered that the start time of the task is reached, and transfer to S38. Otherwise, transfer to S34; S34. Determine whether i is greater than I - 1. If so, transfer to S32. Otherwise, transfer to S35; S35. Calculate the cost C(n, i) within a time step to calculate the expected cost V(n, i); S36. Determine whether V(n, i) is equal to C m ,If so, select the task termination action, otherwise select the task continuation action, and save it to the task termination policy set; S37. Let i = i + 1, and transfer to S34; S38. End the algorithm.

7. A task termination strategy generation system based on a dynamic programming algorithm, characterized in that: Including: A defining module for defining that the deterioration state of the system follows a multi-stage continuous-time Markov chain, and defining a random task duration; A construction module is used to construct a value function based on the state transition probability of the system and the cost within the step length to obtain the expected cost of the task in each deteriorated state in all possible execution times; wherein the execution time is pre-divided into a number of equal time steps; The solution module is used to start from the end time of the longest task duration, and use a dynamic programming algorithm to reversely iterate at equal time steps to solve the expected cost of the task in the current deteriorated state. If it is equal to the task failure cost, the task termination action is selected, otherwise the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

8. A storage medium, characterized in that: The computer program for generating a task termination strategy based on a dynamic programming algorithm is stored therein, wherein the computer program enables a computer to execute the task termination strategy generating method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including a program for executing the task termination policy generating method according to any one of claims 1 to 6.

Images