Task termination strategy generation method and system based on dynamic programming algorithm

Abstract

The present invention provides a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm, and relates to the field of risk control. In the present invention, it is first defined that the deterioration state of the system obeys a multi-stage continuous-time Markov chain and a random task duration; secondly, a value function is constructed to obtain the expected cost of tasks in each deterioration state with all possible execution durations; finally, a reverse induction method is combined with a dynamic programming algorithm to recursively work forward from the end time of the longest task duration to determine the optimal task termination decision at each time step, which can achieve global optimization. The present invention can more realistically reflect the uncertainty of task execution time in an actual system, as well as monitor the dynamic changes in the task execution process in real time, dynamically adjust the task termination strategy based on real-time data, and thus can flexibly respond to complex and changing task environments and maximize the survival rate of the system.

Description

Technical Field

[0001] The present invention relates to the field of risk control, and in particular to a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm. Background Art

[0002] As a risk control method for safety-critical systems, mission abort strategies have been widely adopted across various fields. For safety-critical engineering systems such as spacecraft, submarines, and power generation, system survival takes precedence over mission completion, and premature mission abort is an effective method for improving system survivability. For example, when an autonomous underwater vehicle's sensor malfunctions, it can abort its ocean exploration mission and surface to await rescue. Or when a hospital's CT equipment experiences unstable imaging, it can terminate the imaging mission and await repairs.

[0003] In the traditional task termination strategy, some scholars use a preset static threshold as the termination condition for task execution. For example: The paper (Zhao X, Sun J, Qiu Q, et al. Optimal inspection and mission abort policies for systems subject to degradation [J]. European Journal of Operational Research, 2021, 292 (2): 610-621.) discloses that some systems set the system to automatically terminate the task when the deterioration state reaches a certain threshold to prevent the system from deteriorating to a fault state. Although this method is simple and intuitive, it is difficult to adapt to the complex and changing task execution environment, which may lead to waste of resources or failure to complete the task as scheduled.

[0004] In addition, there are mission abort strategies based on time and system degradation. The paper (Qiu Q, Cui L. Gamma process based optimal mission abort policy [J]. Reliability Engineering & System Safety, 2019, 190: 106496.) studies the optimal mission abort strategy for a two-stage degradation process system simulated by the Gamma process. Degradation-based and duration-based mission abort strategies are derived based on a random failure model and mission abort strategy. The results show that the duration-based abort strategy performs better when the degradation rate is low, while the degradation-based abort strategy is more cost-effective when the degradation rate is high.

[0005] In summary, traditional task termination strategies are mostly based on time-fixed tasks or static thresholds, and lack the ability to respond to dynamic changes during task execution. Summary of the Invention

[0006] (1) Technical problems solved

[0007] In response to the shortcomings of the existing technology, the present invention provides a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm, which solves the technical problem that traditional task termination strategies are mostly based on time-fixed tasks or static thresholds and lack the ability to respond to dynamic changes during task execution.

[0008] (2) Technical solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0010] A method for generating a task termination strategy based on a dynamic programming algorithm, comprising:

[0011] Define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration;

[0012] Based on the state transition probability and the cost within the step length of the system, a value function is constructed to obtain the expected cost of the task in each deteriorated state for all possible execution times; the execution time is pre-divided into several equal time steps;

[0013] Starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0014] Preferably, the deterioration state of the system is defined as X = {X t , t≥0} obeys a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state;

[0015] And the state transition rate matrix is ​​set to Q, the state transition probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where P ij (τ) represents the probability that the system transitions from state i to state j after a period of time τ, P(X s+τ =j|X s=i) represents the probability that the system transitions from state i to state j after a period of time τ, starting from time s;

[0016] Define random task durations, with the shortest task duration being LΔ and the longest task duration being MΔ, where Δ represents the unit time step, L and M represent the maximum execution duration of the shortest and longest tasks, including the number of time steps, respectively.

[0017] The probability that the task is completed at time LΔ+t is recorded as g(t), which obeys the gamma distribution Γ(α, β) and is expressed as g(t|α, β), where α represents the shape parameter of the gamma function and β represents the scale parameter of the gamma function.

[0018] Preferably, the state transition probability specifically includes:

[0019] 1) When n < L, the state transition probability is independent of the task success probability:

[0020] p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i) = P ij (Δ) (1)

[0021] Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transitions from state i at time nΔ to state j at time (n+1)Δ; Pi j (Δ) represents the probability that the system transitions from state i to state j after Δ time;

[0022] 2) When n>L, if the task is successfully completed, the state transition will not occur. The state transition probability is calculated based on the conditional probability that the task is not completed at this time:

[0023]

[0024] Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, whose formula is

[0025] Preferably, the intra-step cost specifically includes:

[0026] 1) When n=M-1, the system state is within the step length of (M-1, i) and the cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion:

[0027] C (n，i) =CF+CM+CR+CO (3)

[0028]

[0029]

[0030]

[0031]

[0032] Among them, a time step Δ is divided into K equal parts, k represents the index of K, δ represents the step size of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is

[0033] 2) When L≤n<M-1, the system state is within the step length of (n, i) and the cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO and the expected reward CR for task completion:

[0034] C (n，i) =CF+CI+CR+CO (8)

[0035]

[0036] Among them, C I represents the inspection cost;

[0037] 3) When n<L, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF′, the expected cost of inspection CI′ and the expected cost of operation CO′:

[0038] C (n，i) =CF′+CI′+CO′ (10)

[0039] CF′=P iI (Δ)(C m +C s ) (11)

[0040]

[0041]

[0042] Where PiI(Δ) represents the probability that the system state transitions from state i to system failure state I after Δ time.

[0043] Preferably, a value function V(n, i) is constructed to represent the expected cost of the system state being (n, i); specifically, it includes:

[0044] 1) When n = M, the execution time of the task reaches the maximum task time. Regardless of the state of system deterioration, the task is considered failed. The expected cost is:

[0045] V (M，i) =C m ,i≠I (14)

[0046] Among them, C m represents the cost of task failure;

[0047] 2) When n < M, i ≠ I, the expected cost is the minimum of the sum of the task failure cost, the immediate cost, and the future cost; the expected cost is:

[0048]

[0049] Among them, min represents the minimization function, V (n+1，j ) represents the future expected cost of the system state being (n+1, j).

[0050] Preferably, starting from the end time of the longest task duration, a dynamic programming algorithm is used to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If the expected cost is equal to the task failure cost, a task termination action is selected; otherwise, a task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached; including:

[0051] S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s 、C m 、C I 、C i , r, Q, and let n = M, i = 0;

[0052] S32, set n=n-1;

[0053] S33, determine whether n is less than 0, if so, determine that the starting time of the task has been reached, and transfer to S38, otherwise transfer to S34;

[0054] S34, determine whether i is greater than I-1, if so, go to S32, otherwise go to S35;

[0055] S35. Calculate the cost C(n, i) within the step length to calculate the expected cost V(n, i);

[0056] S36. Determine whether V(n, i) is equal to C m ,If so, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set;

[0057] S37, set i=i+1, and go to S34;

[0058] S38. End the algorithm.

[0059] A task termination strategy generation system based on a dynamic programming algorithm, comprising:

[0060] Definition module, used to define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration;

[0061] A construction module is used to construct a value function based on the state transition probability and the cost within the step to obtain the expected cost of the task in each degraded state for all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0062] The solution module is used to start from the end time of the longest task duration, and use the dynamic programming algorithm to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0063] A storage medium stores a computer program for generating a task termination strategy based on a dynamic programming algorithm, wherein the computer program enables a computer to execute the task termination strategy generation method described above.

[0064] An electronic device, comprising:

[0065] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the task termination policy generation method as described above.

[0066] (3) Beneficial effects

[0067] The present invention provides a method, system, storage medium, and electronic device for generating a task termination strategy based on a dynamic programming algorithm. Compared with the prior art, the present invention has the following advantages:

[0068] In this invention, the system's deterioration state is first defined as obeying a multi-stage continuous-time Markov chain and random task durations. Secondly, a value function is constructed to obtain the expected cost of tasks in each deterioration state for all possible execution durations. Finally, a backward induction method combined with a dynamic programming algorithm is used to recursively work backward from the endpoint of the longest task duration to determine the optimal task termination decision at each time step, achieving global optimization. This invention can more realistically reflect the uncertainty of task execution time in actual systems, monitor dynamic changes during task execution in real time, and dynamically adjust task termination strategies based on real-time data. This allows for flexible response to complex and changing task environments, maximizing the system's survival rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 The embodiment of the present invention provides a block diagram of a method for generating a task termination strategy based on a dynamic programming algorithm;

[0071] Figure 2 An embodiment of the present invention provides a flow chart of a reverse induction method combined with a dynamic programming algorithm. DETAILED DESCRIPTION

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0073] The embodiments of the present application provide a method, system, storage medium and electronic device for generating a task termination strategy based on a dynamic programming algorithm, thereby solving the technical problem that traditional task termination strategies are mostly based on time-fixed tasks or static thresholds and lack the ability to respond to dynamic changes during task execution.

[0074] The technical solution in the embodiments of the present application is to solve the above technical problems, and the overall idea is as follows:

[0075] As mentioned in the background, traditional task termination strategies often assume fixed task durations, which fails to reflect the randomness and uncertainty of task durations in real applications. This assumption results in a lack of flexibility and accuracy in termination strategies. Furthermore, previous research often assumes linear or predictable system state changes, ignoring the complex dynamics of system state changes, resulting in inaccurate and ineffective decisions.

[0076] In fact, in terms of research methods, due to the simple decision rules and fixed termination thresholds in traditional task termination strategies, it is impossible to provide globally optimal decision support, making it difficult for traditional methods to deal with practical problems.

[0077] Different from this, an embodiment of the present invention proposes a method for solving task termination strategy based on a dynamic programming algorithm. By introducing the randomness of task duration, using a multi-stage continuous-time Markov chain to model the system state, and combining it with a dynamic programming algorithm, a method for optimizing the task termination strategy problem is provided.

[0078] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0079] Example 1:

[0080] like Figure 1 As shown, a task termination strategy generation method based on a dynamic programming algorithm is characterized by comprising:

[0081] S 1. Define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define the random task duration;

[0082] S2. Based on the state transition probability and the cost within the step length of the system, a value function is constructed to obtain the expected cost of the task in each degraded state for all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0083] S3. Starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0084] The embodiments of the present invention can more realistically reflect the uncertainty of task execution time in an actual system, monitor the dynamic changes during task execution in real time, and dynamically adjust the task termination strategy based on real-time data, thereby being able to flexibly respond to complex and changing task environments and maximize the survival rate of the system.

[0085] The following sections describe the various steps of the above solution in detail:

[0086] In step S1, the deterioration state of the system is defined to obey a multi-stage continuous-time Markov chain, and a random task duration is defined.

[0087] In this step, the deterioration state of the system is defined as X = {X t , t≥0} obeys a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state;

[0088] And the state transition rate matrix is ​​set to Q, the state transition probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where P ij (τ) represents the probability that the system transitions from state i to state j after a period of time τ, P(X s+τ =j|X s =i) represents the probability that the system transitions from state i to state j after a period of time τ starting from time s.

[0089] This facilitates a system status check every Δ time interval in subsequent steps, allowing the decision maker to make a decision to continue or terminate the task. Assuming that system failures are observable as soon as they occur, it is understandable that in order to improve the system's survivability, it is necessary to make accurate task termination decisions before system failures occur.

[0090] Next, this step also defines the random task duration, the shortest task duration is LΔ, and the longest task duration is MΔ, where Δ represents the unit time step, L and M represent the maximum execution time of the shortest task and the longest task, including the number of time steps;

[0091] The probability that the task is completed at time LΔ+t is recorded as g(t), which obeys the gamma distribution Γ(α, β) and is expressed as g(t|α, β), where α represents the shape parameter of the gamma function and β represents the scale parameter of the gamma function.

[0092] In step S2, a value function is constructed based on the state transition probability of the system and the cost within the step to obtain the expected cost of the task in each deteriorated state for all possible execution times; wherein the execution time is pre-divided into several equal time steps.

[0093] In order to construct the value function, this step introduces the state transition probability and the cost within the step length of the system in turn:

[0094] In this embodiment of the present invention, the state transition probability specifically includes:

[0095] 1) When n < L, the state transition probability is independent of the task success probability:

[0096] p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i) = P ij (Δ) (1)

[0097] Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transitions from state i at time nΔ to state j at time (n+1)Δ; P ij (Δ) represents the probability that the system will transition from state i to state j after Δ time.

[0098] 2) When n>L, if the task is successfully completed, the state transition will not occur. The state transition probability is calculated based on the conditional probability that the task is not completed at this time:

[0099]

[0100] Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, whose formula is

[0101] In this embodiment of the present invention, the intra-step cost specifically includes:

[0102] 1) When n=M-1, the system state is within the step length of (M-1, i) and the cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion:

[0103] C (n，i) =CF+CM+CR+CO (3)

[0104]

[0105]

[0106]

[0107]

[0108] Among them, a time step Δ is divided into K equal parts, k represents the index of K, δ represents the step size of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is

[0109] 2) When L≤n<M-1, the system state is within the step length of (n, i) and the cost C (n，i) It is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO and the expected reward CR for task completion:

[0110] C (n，i) =CF+CI+CR+CO (8)

[0111]

[0112] Among them, C I Represents the inspection cost.

[0113] 3) When n<L, the system state is within the step length of (n, i) with a cost C (n，i) It is composed of the expected cost of system failure CF′, the expected cost of inspection CI′ and the expected cost of operation CO′:

[0114] C (n，i) =CF′+CI′+CO′ (10)

[0115] CF′=P iI (Δ)(C m +C s ) (11)

[0116]

[0117]

[0118] Where PiI(Δ) represents the probability that the system state transitions from state i to system failure state I after Δ time.

[0119] Next, this step uses the state transition probability and the cost within the step length of the above system to construct a value function V(n, i), which represents the expected cost of the system state being in (n, i); specifically, it includes:

[0120] 1) When n = M, the execution time of the task reaches the maximum task time. Regardless of the state of system deterioration, the task is considered failed. The expected cost is:

[0121] V (M，i) =C m ,i≠I (14)

[0122] Among them, C m Represents the task failure cost.

[0123] 2) When n < M, i ≠ I, the expected cost is the minimum of the sum of the task failure cost, the immediate cost, and the future cost; the expected cost is:

[0124]

[0125] Among them, min represents the minimization function, V (n+1，j) represents the future expected cost of the system state being (n+1, j).

[0126] In step S3, starting from the end time of the longest task duration, the dynamic programming algorithm is used to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved to the task termination strategy set until the starting time of the task is reached.

[0127] The embodiment of the present invention combines dynamic programming and backward induction to solve the optimal decision at each time step, aiming to make correct task termination decisions before system failure occurs, thereby maximizing the system's survival rate and improving system reliability and stability.

[0128] Correspondingly, such as Figure 2 As shown, this step specifically includes:

[0129] S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s 、C m 、C I 、C i , r, Q, and let n = M, i = 0;

[0130] S32, set n=n-1;

[0131] S33, determine whether n is less than 0, if so, determine that the starting time of the task has been reached, and transfer to S38, otherwise transfer to S34;

[0132] S34, determine whether i is greater than I-1, if so, go to S32, otherwise go to S35;

[0133] S35. Calculate the cost C(n, i) within the step length to calculate the expected cost V(n, i);

[0134] S36. Determine whether V(n, i) is equal to C m ,If so, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set;

[0135] S37, set i=i+1, and go to S34;

[0136] S38. End the algorithm.

[0137] It should be noted that, since the maximum duration limit of the task has been reached when n=M, no decision needs to be made at this time. Therefore, the above algorithm provides a task termination strategy that iterates cyclically starting from n=M-1.

[0138] So far, the embodiment of the present invention has completed the entire process of the task termination strategy generation method based on the dynamic programming algorithm.

[0139] Example 2:

[0140] An embodiment of the present invention provides a task termination strategy generation system based on a dynamic programming algorithm, comprising:

[0141] Definition module, used to define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration;

[0142] A construction module is used to construct a value function based on the state transition probability and the cost within the step to obtain the expected cost of the task in each degraded state for all possible execution times; wherein the execution time is pre-divided into a number of equal time steps;

[0143] The solution module is used to start from the end time of the longest task duration, and use the dynamic programming algorithm to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

[0144] Example 3:

[0145] An embodiment of the present invention provides a storage medium storing a computer program for generating a task termination policy based on a dynamic programming algorithm, wherein the computer program enables a computer to execute the task termination policy generation method as described in Example 1.

[0146] Example 4:

[0147] An embodiment of the present invention provides an electronic device, including:

[0148] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a method for executing the task termination policy generation method as described in Example 1.

[0149] It can be understood that the task termination strategy generation system based on the dynamic programming algorithm, storage medium and electronic device provided in the embodiment of the present invention correspond to the task termination strategy generation method based on the dynamic programming algorithm provided in the embodiment of the present invention. The explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the task termination strategy generation method, and will not be repeated here.

[0150] In summary, compared with the existing technology, the present invention has the following beneficial effects:

[0151] 1. Compared with traditional static threshold methods, the embodiments of the present invention can monitor dynamic changes during task execution in real time and dynamically adjust task termination policies based on real-time data. This also avoids resource waste or premature task termination caused by static threshold settings. By applying a dynamic programming algorithm, policy selection can be made more accurately.

[0152] 2. The present invention considers random task durations and uses a gamma distribution to describe the variation in task durations. This feature enables the present invention to more realistically reflect the uncertainty of task execution times in real systems, avoiding the limitations of traditional methods that assume fixed task durations.

[0153] 3. The embodiments of the present invention are not restricted by task type or execution conditions, and can flexibly adapt to complex and changing task environments. They can dynamically adjust based on real-time conditions, executing tasks while ensuring system security. Because they account for the randomness of task duration and the dynamic changes in system state, this method can be applied to complex systems requiring decision optimization, such as industrial production systems, communication networks, and medical equipment.

[0154] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0155] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A task termination strategy generation method based on a dynamic programming algorithm, characterized in that: Comprising: Defining that the deteriorating state of the system follows a multi-stage continuous-time Markov chain, and defining random task durations; Based on the state transition probability of the system and the cost within a step, constructing a value function to obtain the expected cost of the task at each deteriorating state for all possible execution durations; wherein, the execution duration is pre-divided into a number of equal time steps; Starting from the end time of the longest task duration, using the dynamic programming algorithm to iteratively solve the expected cost of the task at the current deteriorating state every equal time step in reverse. If it is equal to the task failure cost, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set until reaching the start time of the task; Define the deterioration state of the system X={X t , t≥0} obeys a multi-stage continuous-time Markov chain Ω={0,1,…,i,…,I}, where X t represents the deterioration state at any time t, i represents the state index, and I represents the system failure state; And the state transition rate matrix is ​​set to Q, the state transition probability matrix P ij (τ) = P(X s+τ =j|X s =i) is obtained by calculating the Kolmogorov backward differential equation, where P ij (τ) represents the probability that the system transitions from state i to state j after a period of time τ, P(X s+τ =j|X s =i) represents the probability that the system transitions from state i to state j after a period of time τ, starting from time s; Defining random task durations, the shortest task duration is LΔ, the longest task duration is M△, where △ represents the unit time step, and L and M respectively represent the maximum execution durations of the shortest task and the longest task including the number of time steps; And the probability that the task is completed at time LΔ + t is denoted as g(t), and g(t) follows a gamma distribution Γ(α, β), expressed as g(t|α, β), where α represents the shape parameter of the gamma function and β represents the scale parameter of the gamma function; The state transition probability specifically includes: 1) When n < L, the state transition probability is independent of the task success probability: p (n，i)(n+1，j) =Pr(X (n+1)Δ =j|X nΔ =i)=P ij (Δ) (1) Where n represents the index of the time step; p (n，i)(n+1，j) represents the probability of the system transitioning from state (n, i) to state (n+1, j); Pr(X (n+1)Δ =j|X nΔ =i) represents the probability that the system transitions from state i at time nΔ to state j at time (n+1)Δ; P ij (Δ) represents the probability that the system transitions from state i to state j after Δ time; 2) When n > L, if the task is successfully completed, no state transition occurs, and the state transition probability is calculated from the conditional probability with the condition that the task is not completed at this time: Where x represents the integral variable; Γ(α) represents the gamma function, and its formula is γ(α, x) represents the incomplete gamma function, whose formula is 2. The task termination strategy method according to claim 1, characterized in that: The cost within a step specifically includes: 1) When n=M-1, the system state is within the step length of (M-1, i) with a cost of c (n，i) It is composed of the expected cost of system failure CF, the expected cost of task failure CM, the expected cost of operation CO and the expected reward CR for task completion: C (n，i) =CF+CM+CR+CO (3) Among them, a time step △ is divided into K equal parts, k represents the index of K, δ represents the step size of each equal part; C s represents the system failure cost, C m represents the cost of task failure; C j represents the operating cost of the system in state j; The probability that a task is not completed before time LΔ+t, G(t) represents the probability that a task is completed before time LΔ+t, and its formula is 2) When L ≤ n < M - 1, the step - within cost C of the system state being (n, i) (n，i) is composed of the expected cost of system failure CF, the expected cost of inspection CI, the expected cost of operation CO, and the expected reward for task completion CR: C (n，i) =CF+CI+CR+CO (8) Among them, C I represents the inspection cost; 3) When n < L, the step cost C within the system state (n, i) (n，i) is composed of the expected cost of system failure cF′, the expected cost of inspection cI′, and the expected cost of operation cO′: C (n，i) =CF′+CI′+CO′ (10) CF′=P iI (Δ)(C m +C s ) (11) 0 3. The task termination policy method according to claim 2, wherein Constructing a value function V(n, i) to represent the expected cost when the system state is at (n, i); specifically including: 1) When n = M, the execution duration of the task reaches the longest task duration, and the task is regarded as failed regardless of the deteriorating state of the system; the expected cost is: In (M，i) =C m ,i≠I (14) Among them, C m represents the cost of task failure; 2) When n < M and i ≠ I, the expected cost takes the minimum value between the task failure cost, the immediate cost, and the sum of future costs; the expected cost is: Among them, min represents the minimization function, V (n+1，j) It represents the expected future cost of the system state being (n+1, j).

4. The task termination strategy method according to claim 3, wherein: The step of starting from the end time of the longest task duration, using the dynamic programming algorithm to iteratively solve the expected cost of the task at the current deteriorating state every equal time step in reverse. If it is equal to the task failure cost, select the task abort action, otherwise select the task continue action, and save it to the task termination policy set until reaching the start time of the task; includes: S31, initialization parameters and system status; including initialization of L, M, I, α, β, C s 、C m 、C I 、C i , r, Q, and let n = M, i = 0; S32. Let n = n - 1; S33. Judge whether n is less than 0. If so, it is determined that the start time of the task is reached, and transfer to S38. Otherwise, transfer to S34; S34. Judge whether i is greater than I - 1. If so, transfer to S32. Otherwise, transfer to S35; S35. Calculate the cost c(n, i) within a step to calculate the expected cost V(n, i); S36. Determine whether V(n, i) is equal to C m ,If so, select the task abort action, otherwise select the task continue action and save it to the task termination ,strategy set; S37. Let i = i + 1, and transfer to S34; S38. End the algorithm.

5. A task termination strategy generation system based on dynamic programming algorithm, characterized in that: For the task termination policy method according to claim 1, including: Definition module, used to define the system's deterioration state to obey a multi-stage continuous-time Markov chain and define random task duration; A construction module is used to construct a value function based on the state transition probability and the cost within the step to obtain the expected cost of the task in each degraded state for all possible execution times; wherein the execution time is pre-divided into a number of equal time steps; The solution module is used to start from the end time of the longest task duration, and use the dynamic programming algorithm to reversely iterate and solve the expected cost of the task in the current deteriorated state at equal time steps. If it is equal to the task failure cost, the task termination action is selected; otherwise, the task continuation action is selected and saved in the task termination strategy set until the starting time of the task is reached.

6. A storage medium, characterized in that The computer program for generating a task termination strategy based on a dynamic programming algorithm is stored therein, wherein the computer program enables a computer to execute the task termination strategy generating method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including instructions for executing the task termination policy generating method according to any one of claims 1 to 4.

Images