Consider the termination strategy optimization method of phased task system

Abstract

The application provides a kind of considering the termination strategy optimization method of staged task system, it is related to task termination strategy field.First, for each stage of staged task, the degradation state of system is defined to be subject to a Wiener process, and the number of decision points of each stage is obtained;Second, the degradation state of each stage is converted into a discrete state model to solve the state transition probability of the system;Third, the task termination cost and task continuation cost of each decision point under different system states are solved, and the smaller value is selected as the corresponding expected cost;Finally, using the backward dynamic programming algorithm, based on the expected cost of the current decision point and the system state, the task termination or task continuation action is selected to obtain the task termination strategy set.The optimization method proposed in the application fills the gap in related technologies that do not consider the staged task termination strategy, and uses a stochastic dynamic programming framework to express the optimization problem, which effectively reduces the cost compared to the fixed threshold strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of task termination strategy, and in particular to a termination strategy optimization method considering a phased task system. BACKGROUND

[0002] Task termination strategy plays a crucial role in aerospace, submarine and mechanical engineering fields. Specifically, when a safety-critical system performs a task within a certain time, the survival of the system has a higher priority than completing the mission due to safety or economic considerations. When the risk of system failure becomes unacceptable, the task can be interrupted, and then the rescue program is activated immediately to ensure that the system is not damaged.

[0003] In related technologies, the research on task termination strategy can be roughly divided into two categories:

[0004] One is the termination strategy based on the number of shocks, that is, when the system is subjected to external shocks to a certain number of times, the system will be subjected to corresponding damage. Literature (Levitin G, Finkelstein M. Optimal mission abort policy for systems operating in a random environment[J]. Risk Analysis, 2018, 38(4):795-803.) uses external shocks following a non-homogeneous Poisson process to simulate the impact of random environment on the system, and uses the number of shocks as a decision parameter to evaluate the task termination strategy under random environment.

[0005] Another type is the termination strategy based on random processes, which uses Wiener, Gamma and other random processes and the obtained comprehensive information to guide the termination of the task. Literature (Zhao X, Sun J, Qiu Q, et al. Optimal inspection and mission abort policies for systems subject to degradation[J]. European Journal of Operational Research, 2021, 292(2): 610-621.) uses Gamma process to model the continuous degradation system, and analyzes the joint optimization problem of the optimal detection and dynamic task termination strategy. Literature (Yang L, Chen Y, Qiu Q, et al. Risk control of mission-critical systems: Abort decision-makings integrating health and age conditions[J]. IEEE Transactions on Industrial Informatics, 2022, 18(10): 6887-6894.) designs the task termination strategy based on the degradation degree and age information of the monitored system, and studies the optimal risk control strategy of the random degradation system through the Wiener process model.

[0006] It can be seen that the existing research on phased task termination strategy is less. In addition, although Chinese patent application CN119473779A uses "phased" Markov chain to describe the degradation process of the system and proposes a task termination strategy, the "phased" is for the "multi-state" of system degradation, not the phased of task, so the termination strategy is still for single-phase task. SUMMARY

[0007] (I) Technical problems solved

[0008] In view of the shortcomings of the prior art, the present application provides a termination strategy optimization method considering a phased task system, which solves the termination problem of the overall task composed of multiple continuous sub-tasks (or phased tasks).

[0009] (II) Technical solutions

[0010] To achieve the above purpose, the present application is realized by the following technical solutions:

[0011] A termination strategy optimization method considering a phased task system, assuming that the system executes a phased task, the termination strategy optimization method comprises:

[0012] For each stage of the phased task, the degradation state of the system is defined to follow a Wiener process; and the number of decision points for each stage is obtained based on a preset decision interval.

[0013] The degradation state at each stage is converted into a discrete state model, and the state transition probability of the system is solved under the discrete state.

[0014] Based on the state transition probability, the task termination cost and task continuation cost of each decision point under different system states are calculated, and the smaller value is selected as the corresponding expected cost; wherein if the decision point is located at the boundary of an adjacent stage, the expected cost includes at least the reward for completing the stage to which the decision point belongs.

[0015] Starting from the end point of the phased task, a reverse dynamic programming algorithm is used to traverse each decision point. Based on the expected cost of the current decision point and the system state, the action of terminating or continuing the task is selected and saved to the task termination strategy set until the starting point of the phased task is reached.

[0016] Preferably, the phased task consists of M phases, and for the m-th phase, the degradation state of the system is defined. Obey the drift parameter is The diffusion coefficient is The Wiener process, in which This indicates that it is in the m-th stage. This represents the start time of the m-th stage. This indicates the end time of the m-th stage.

[0017] Preferably, the decision interval is defined as follows: Then the number of decision points in the m-th stage is ,in It is an integer. The symbol indicates rounding up.

[0018] Preferably, the step of converting the degenerate state of each stage into a discrete state model, and solving for the state transition probability of the system in the discrete state, includes:

[0019] Define a finite set of degenerate states Where 0 represents the initial healthy state of the system and 1 represents the fault state of the system;

[0020] State division rules: The discrete interval of the degradation state is set to d. When the degradation state of the system falls within the interval [id, (i+1)d], the system is determined to be in state i; when the degradation state of the system is greater than or equal to Id, the system enters fault state I; each state i is represented by the midpoint value of the interval (i+0.5)d.

[0021] During the mth stage, the system transits from the current state i to other state k, the transition probability is calculated based on the characteristics of Wiener process, and classified as follows:

[0022] 1) if i≠I and k≠I

[0023]

[0024] 2) if i≠I and k=I

[0025]

[0026] 3) if i=I

[0027]

[0028] wherein, Pm(i,k,t) represents the probability of the system transiting from state i to state k at time t during the mth stage, and d represents a discrete interval. is the cumulative distribution function of the standard normal distribution.

[0029] Preferably, the task termination cost and the task continuation cost of each decision point in different system states are solved based on the state transition probability, and the smaller value is selected as the corresponding expected cost, comprising:

[0030] The state space of the system is defined as ; wherein n represents the cumulative number of decision points; and the action space is defined as {A, C}, wherein A represents terminating the task, and C represents continuing the task, and V(n,i) represents the expected cost at state (n,i):

[0031]

[0032] wherein, min represents the minimization function, A(n,i) represents the cost of terminating the task at state (n,i), and C(n,i) represents the cost of continuing the task at state (n,i).

[0033] The calculation of V(n,i) needs to consider the following cases:

[0034] 1) if , it indicates that the phased task has been successfully completed, and the expected cost is:

[0035]

[0036] wherein, Pm(M) represents the reward of completing the Mth stage task.

[0037] 2) if , represents the cost of continuing the task at the mth stage of execution with the system state at (n, i) consisting of the cost of continuing failure CI, the cost of continuing execution CE:

[0038]

[0039] wherein, represents the cost of failure at the mth stage of execution, represents the cost of system failure; V(n+1, k) represents the future expected cost of the system state at (n+1, k); represents the time elapsed at the mth stage of execution , the probability of the system transitioning from state i to state k;

[0040] the cost of terminating the task with the system state at (n, i) consisting of the cost of terminating operation AF:

[0041]

[0042] wherein, represents the probability of successfully rescuing the system after terminating the task at the mth stage of execution;

[0043] 3) if , represents the cost of continuing the task at the mth stage of execution with the system state at (n, i) consisting of the cost of continuing failure CI, the cost of continuing execution CE, and the stage reward PR:

[0044]

[0045] wherein, represents the reward for completing the mth stage;

[0046] the cost of terminating the task with the system state at (n, i) consisting of the cost of terminating operation AF and the stage reward PR:

[0047] (13).

[0048] Preferably, the reverse dynamic programming algorithm is used to traverse each decision point from the end point of the staged task, select the task termination or task continuation action based on the expected cost at the current decision point and the system state, and save it to the task termination strategy set until the starting point of the staged task is reached, comprising:

[0049] STEP0, initialize parameters including , and set represents the task termination strategy set with the system state at (n, i), wherein indicates a continue action, indicates a terminate action;

[0050] STEP 1, calculating the total number of decision points ;

[0051] STEP 2, for all , calculating V(n, i) by formula (5);

[0052] STEP 3, setting n = N-1;

[0053] STEP 4, if n < 0, executing STEP 12, otherwise executing STEP 5;

[0054] STEP 5, setting i = 0;

[0055] STEP 6, if i > I-1, jumping to STEP 11, otherwise executing STEP 7;

[0056] STEP 7, if there exists m ∈ {1, 2, …, M-1} such that , calculating C(n, i) and A(n, i) by formula (11), (13) respectively, otherwise calculating C(n, i) and A(n, i) by formula (6), (9);

[0057] STEP 8, calculating , if V(n, i) = C(n, i), setting , i = i+1, executing STEP 6; otherwise , executing STEP 9;

[0058] STEP 9, for all k = i+1 to I-1, if there exists such that , calculating A(n, k) by formula (13), otherwise calculating A(n, k) by formula (9);

[0059] STEP 10, setting V(n, k) = A(n, k), , jumping to STEP 11;

[0060] STEP 11, setting n = n-1, returning to STEP 4;

[0061] STEP 12, ending the algorithm, outputting the task termination strategy set.

[0062] A termination strategy optimization system considering a phased task system, assuming that a system executes a phased task, the termination strategy optimization system comprising:

[0063] a defining module configured to define, for each stage of the staged task, that a degradation state of the system is subject to a Wiener process, and obtain a number of decision points for each stage based on a preset decision interval;

[0064] a solving module configured to convert the degradation state of each stage into a discrete state model, and solve a state transition probability of the system in the discrete state;

[0065] and solve a task termination cost and a task continuation cost of each decision point in different system states based on the state transition probability, and select a smaller value as a corresponding expected cost; wherein if the decision point is located at a boundary between adjacent stages, the expected cost at least includes a reward for completing the stage to which the decision point belongs;

[0066] a decision module configured to traverse each decision point from an end point of the staged task to a start point of the staged task by using a backward dynamic programming algorithm, select a task termination action or a task continuation action based on the expected cost at the current decision point and the system state, and save the action to a task termination strategy set.

[0067] A storage medium storing a computer program for optimizing a termination strategy of a staged task system, wherein the computer program causes a computer to execute the termination strategy optimization method as described above.

[0068] An electronic device, comprising:

[0069] one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs include a program for executing the termination strategy optimization method as described above.

[0070] (Three) beneficial effects

[0071] The present application provides a termination strategy optimization method for a staged task system. Compared with the prior art, the present application has the following beneficial effects:

[0072] In the present application, firstly, for each stage of the phased task, the degradation state of the system is defined to be subject to a Wiener process, and the number of decision points of each stage is obtained; secondly, the degradation state of each stage is converted into a discrete state model to solve the state transition probability of the system; thirdly, the task termination cost and the task continuation cost of each decision point under different system states are solved, and the smaller value is selected as the corresponding expected cost; finally, the reverse dynamic programming algorithm is adopted to select the task termination or task continuation action based on the expected cost under the current decision point and the system state, and the task termination strategy set is obtained. The optimization method proposed in the present application fills the gap in the related art that does not consider the phased task termination strategy, and simultaneously uses the stochastic dynamic programming framework to express the optimization problem, which effectively reduces the cost compared with the fixed threshold strategy. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0074] Figure 1 A block diagram of a termination strategy optimization method considering a phased task system is provided for the embodiments of the present application.

[0075] Figure 2 A flowchart of a reverse dynamic programming algorithm is provided for the embodiments of the present application.

[0076] Figure 3 A termination decision process diagram of a two-stage task is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0077] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the described embodiments are some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0078] The embodiments of the present application provide a termination strategy optimization method considering a phased task system, which solves the termination problem of the overall task composed of multiple continuous sub-tasks (or stage tasks).

[0079] The technical solutions in the embodiments of the present application are to solve the above technical problems, and the general idea is as follows:

[0080] In related technologies, some studies, such as Chinese patent application CN119473779A, claim to have studied phased tasks. However, its concept of "phased" is essentially a system degradation state, rather than a concept of multiple phased tasks (i.e., multiple consecutive sub-tasks). Therefore, it fails to solve the problem of terminating the overall task composed of the aforementioned multiple consecutive sub-tasks (or phased tasks). In response, this invention proposes an optimization method for phased task termination strategies. This method discretizes the task execution process into multiple time points. At each time point, based on the degradation state, a decision is made to continue or terminate the task, thereby achieving an effective balance between task reliability and system survivability.

[0081] Furthermore, the embodiments of the present invention employ a stochastic dynamic programming framework to describe the optimization problem and use backward induction to find the optimal solution.

[0082] In summary, the embodiments of the present invention provide a unified modeling and optimization of the dual dynamic interaction between system degradation state and phased task flow, and propose a termination strategy generation method that is adaptable to system state and oriented towards phased tasks.

[0083] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0084] Example 1:

[0085] like Figure 1 As shown, this embodiment of the invention provides a termination strategy optimization method for a phased task system. Assuming the system executes a phased task, the termination strategy optimization method includes:

[0086] S1. For each stage of the phased task, define the degradation state of the system as conforming to a Wiener process; and obtain the number of decision points for each stage based on a preset decision interval.

[0087] S2. Convert the degenerate state of each stage into a discrete state model, and solve the state transition probability of the system under the discrete state.

[0088] S3. Based on the state transition probability, solve for the task termination cost and task continuation cost of each decision point under different system states, and select the smaller value as the corresponding expected cost; wherein if the decision point is located at the boundary of an adjacent stage, the expected cost shall at least include the reward for completing the stage to which the decision point belongs.

[0089] S4. Starting from the end point of the phased task, the reverse dynamic programming algorithm is used to traverse each decision point. Based on the expected cost of the current decision point and the system state, the task is terminated or the task continues, and the action is saved to the task termination strategy set until the starting point of the phased task is reached.

[0090] The optimization method provided in the embodiments of the present application makes up for the blank in the related art that the staged task termination strategy is not considered, and simultaneously uses a stochastic dynamic programming framework to express the optimization problem, thereby effectively reducing the cost compared with the fixed threshold strategy.

[0091] Next, each step of the above scheme will be described in detail:

[0092] It should be noted that in the embodiments of the present application, it is assumed that the system performs a staged task, the task is composed of M stages, each stage has a corresponding execution time, and only when the current stage is completed, the next stage will be started.

[0093] Firstly, the degradation state of the staged task system to which the present application is directed is introduced, and the introduction of step S1 is referred to:

[0094] In step S1, for each stage of the staged task, the degradation state of the system is defined to follow a Wiener process; and based on a preset decision interval, the number of decision points of each stage is obtained.

[0095] A Markov chain is a kind of stochastic process, which is widely used in stochastic process modeling and analysis in natural science, social science, engineering technology and other fields. If a process is a Markov process, and the change in unit time follows a normal distribution with an expectation of 0 and a variance of 1, the process is a Wiener process.

[0096] Considering that when the system performs the staged task, the damage degree of the system in each stage is different due to the different tasks performed in each stage. Therefore, in this step, the degradation state of the system is defined to follow a Wiener process with a drift parameter and a diffusion coefficient , wherein represents being in the mth stage, represents the start time of the mth stage, represents the end time of the mth stage.

[0097] In addition, the decision interval is defined as , i.e. every time unit, the decision maker needs to decide whether to continue the task, and the number of decision points of the mth stage is , wherein is an integer, represents the upward rounding symbol.

[0098] Secondly, the mathematical model of the task termination problem is introduced, and the introduction of steps S2 and S3 is referred to:

[0099] In step S2, the degradation state of each stage is converted into a discrete state model, and the state transition probability of the system is solved in the discrete state.

[0100] In order to facilitate practical application and decision making, the degradation state of each stage is converted into a discrete state model in this step. Specifically, a finite degradation state set is defined as where 0 represents the initial health state of the system, and I represents the failure state of the system.

[0101] State partition rule: set the discrete interval of the degradation state as d, when the degradation state of the system falls within the interval [id, (i+1)d], it is determined that the system is in state i; when the degradation state of the system is greater than or equal to Id, the system enters the failure state I; the midpoint value (i+0.5)d in the interval is used to represent each state i.

[0102] Then, the state transition probability of the system is solved in the discrete state, and the related steps are as follows:

[0103] During the mth stage, i.e. in the time interval , the system is transferred from the current state i to other state k, and the transition probability is calculated based on the characteristics of the Wiener process, which is classified as follows:

[0104] 1) if i≠I and k≠I

[0105]

[0106] 2) if i≠I and k=I

[0107]

[0108] 3) if i=I

[0109]

[0110] wherein represents the probability of the system being transferred from state i to state k after time t in the mth stage, d represents the discrete interval, is the cumulative distribution function of the standard normal distribution.

[0111] In step S3, based on the state transition probability, the task termination cost and the task continuation cost of each decision point in different system states are solved, and the smaller value is selected as the corresponding expected cost; wherein if the decision point is located at the boundary of adjacent stages, the expected cost at least includes the reward for completing the stage to which the decision point belongs.

[0112] Based on the state transition probability of the system solved in step S2, this step specifically includes:

[0113] The state space of the system is defined as where n denotes the cumulative number of decision points; and the action space is defined as {A, C}, where A denotes terminating the mission and C denotes continuing the mission, let V(n, i) denote the expected cost at state (n, i):

[0114]

[0115] where min denotes the minimization function, A(n, i) denotes the cost of terminating the mission at state (n, i), and C(n, i) denotes the cost of continuing the mission at state (n, i);

[0116] Calculating V(n, i) requires considering the following cases:

[0117] 1) If , it means that the phased mission has been successfully completed, and the expected cost is:

[0118]

[0119] where represents the reward for completing the Mth phased mission;

[0120] 2) If , it means that the system state is at (n, i) within the mth phased mission, and the cost of continuing the mission C(n, i) is composed of the continuation failure cost CI and the expected cost of continuing execution CE:

[0121]

[0122] where represents the phased mission failure cost, represents the system failure cost; V(n+1, k) represents the future expected cost of the system state at (n+1, k); represents the time elapsed at the mth phase, the probability of the system transitioning from state i to state k;

[0123] The cost of terminating the mission A(n, i) when the system state is at (n, i) is composed of the termination operation expected cost AF:

[0124]

[0125] where represents the probability of successfully rescuing the system after terminating the mission at the mth phase;

[0126] 3) If , indicates that the task in the mth stage has been successfully completed, the cost C(n,i) of continuing the task when the system state is in (n,i) is composed of the continuing failure cost CI, the continuing execution expected cost CE and the stage reward PR:

[0127]

[0128] wherein, indicates the reward of completing the mth stage;

[0129] The cost A(n,i) of terminating the task when the system state is in (n,i) is composed of the terminating operation expected cost AF and the stage reward PR:

[0130] (13).

[0131] Finally, the reverse dynamic programming algorithm proposed in the embodiment of the application is introduced, and details can be referred to the content of step S4:

[0132] In step S4, starting from the end point of the staged task, the reverse dynamic programming algorithm is used to traverse each decision point, based on the expected cost under the current decision point and the system state, the task termination or task continuation action is selected and saved to the task termination strategy set, until the starting point of the staged task is reached.

[0133] As shown in the formula (5), the step specifically includes: Figure 2

[0134] STEP0, initialize parameters including , and set indicates the task termination strategy set when the system state is in (n,i), wherein indicates the continuing action, indicates the terminating action;

[0135] STEP1, calculate the total number of decision points ;

[0136] STEP2, for all , calculate V(n,i) through the formula (5);

[0137] STEP3, let n=N-1;

[0138] STEP4, if n<0, execute STEP12, otherwise execute STEP5;

[0139] STEP5, set i=0;

[0140] STEP6, if i>I-1, jump to STEP11, otherwise execute STEP7;

[0141] ​STEP7, if there exists m e {1, 2, …, M-1} such that then compute C(n, i) and A(n, i) by equations (11), (13) respectively, otherwise compute C(n, i) and A(n, i) by equations (6), (9);

[0142] STEP8, compute , if V(n, i) = C(n, i), then set , i = i + 1, and go to STEP6; otherwise , go to STEP9;

[0143] STEP9, for all k = i + 1 to I-1, if there exists such that then compute A(n, k) by equation (13), otherwise compute A(n, k) by equation (9);

[0144] STEP10, set V(n, k) = A(n, k), and go to STEP11;

[0145] STEP11, set n = n - 1, and go to STEP4;

[0146] STEP12, end the algorithm, and output the set of task termination strategies.

[0147] It can be understood that the algorithm fundamentally solves the optimization problem of the stage characteristics and the cost structure difference between different stages in the real complex task scenario by introducing the staged task modeling framework.

[0148] It is particularly pointed out that the core innovation of the above algorithm is embodied in the design of STEP7 to STEP10, which skillfully distinguishes two key decision moments—inter-stage decision points and inter-stage transition points (i.e. boundary points of adjacent stages); in particular:

[0149] When the decision point n is located at the end of a certain stage m and the start of the next stage m+1 (i.e. ), the algorithm uses special equations (11), (13) to compute C(n, i) and A(n, i), which fully considers the particularity of the stage transition moment, including the possible risk level change and the change of decision constraints between different stages; when the decision point is located in the stage (i.e. ), the conventional formula (6), (9) is used for calculation. This double calculation mechanism enables the algorithm to accurately capture the dynamic complexity in the phased task execution process, avoiding decision bias and suboptimal solution problems when facing tasks with obvious stage characteristics. In addition, STEP9 introduces an intelligent optimization mechanism based on state degradation monotonicity: once the termination strategy is determined to be the optimal choice at a certain degradation state i, the algorithm immediately uses the monotonicity of state degradation to directly set all states k (k = i + 1 to I-1) worse than the current state as the termination strategy without repeating the calculation of the continuation cost of these states. This "one-time decision, cascading application" strategy significantly improves the calculation efficiency while ensuring the logical consistency of the decision. This design not only avoids a large amount of redundant calculation, but more importantly, it reflects the inherent logic of state-dependent decision-making in phased complex tasks, enabling the algorithm to generate an efficient phased task termination strategy while ensuring global optimality, with significant computational advantages and decision accuracy improvements.

[0150] To further illustrate the technical solutions provided by the embodiments of the present application, the following describes the implementation of the phased task termination strategy in conjunction with an actual scene implementation case.

[0151] This case takes an unmanned aerial vehicle emergency material distribution task in a city environment as an example to clearly demonstrate the application of the phased task termination strategy. The task is clearly defined as containing two logically distinct, sequentially executed, and independently targeted stages:

[0152] Stage 1: Delivery stage, the unmanned aerial vehicle takes off from the base and performs material delivery to the designated target area; the task execution time in this stage is fixed at 10 units of time, i.e. T1=10, the degradation of the unmanned aerial vehicle system (such as the battery, motor, navigation system) in this stage is simulated using a Wiener process, the drift coefficient of this process is =0.6, the diffusion coefficient =0.8, the energy consumption in this stage is high (flight load), and the degradation rate is relatively fast (reflected in the high drift coefficient and diffusion coefficient).

[0153] Stage 2: Return stage, after completing the material delivery, the unmanned aerial vehicle returns to the base. The task execution time in this stage is also fixed at 10 units of time, i.e. T2=20, the degradation of the system in stage 2 is simulated using a new Wiener process, but the initial state inherits the degradation state at the end of stage 1, the drift coefficient of this process is =0.4, the diffusion coefficient =0.6, the load is reduced in this stage (the material has been delivered), and the expected degradation rate is relatively slow (reflected in the low drift coefficient and diffusion coefficient).

[0154] In addition, the discrete interval d = 0.5, the decision interval =1, the phased task failure cost and the system failure cost are 500 and 1000 respectively, the rewards for completing the first phase and the second phase are 300 and 150 respectively, and the probabilities of rescue success after the first phase and the second phase terminate the task are 0.99 and 0.93 respectively.

[0155] As shown in Figure 3 , Figure 3 The termination decision process of the two-phase task (the optimal task termination strategy) is shown, and the dashed line divides the task into two phases, and the solid line above indicates terminating the task, and below indicates continuing the task.

[0156] Table 1 and Table 2 show the comparison between the proposed termination strategy and the fixed threshold termination strategy under different costs, and it can be seen that, regardless of the change of the task failure cost or the change of the system damage cost, the proposed strategy has certain advantages compared with the fixed threshold strategy, and can reduce the cost.

[0157] Table 1 Comparison of strategies under different phased task failure costs =1000)

[0158]

[0159] Table 2 Comparison of strategies under different system failure costs =500)

[0160]

[0161] Embodiment 2

[0162] The embodiment of the present application provides a termination strategy optimization system considering a phased task system, assuming that the system performs a phased task, and the termination strategy optimization system comprises:

[0163] A definition module is configured to define that the degradation state of the system for each phase of the phased task is subject to a Wiener process, and to obtain the number of decision points of each phase based on a preset decision interval;

[0164] A solving module is configured to convert the degradation state of each phase into a discrete state model, and to solve the state transition probability of the system in the discrete state;

[0165] and to solve the task termination cost and the task continuation cost of each decision point under different system states based on the state transition probability, and to select the smaller value as the corresponding expected cost; wherein if the decision point is located at the boundary of adjacent phases, the expected cost at least includes the reward for completing the phase to which the decision point belongs.

[0166] The decision module traverses each decision point from the end point of the staged task using a backward dynamic programming algorithm, selects a task termination or task continuation action based on the expected cost at the current decision point and the system state, and saves to a task termination strategy set until the start point of the staged task is reached.

[0167] Embodiment 3:

[0168] The embodiment of the present application provides a storage medium which stores a computer program for considering termination strategy optimization of a staged task system, wherein the computer program causes a computer to execute the termination strategy optimization method as described in Embodiment 1.

[0169] Embodiment 4:

[0170] The embodiment of the present application provides an electronic device, comprising:

[0171] one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the program comprises a program for executing the termination strategy optimization method as described in Embodiment 1.

[0172] It can be understood that the termination strategy optimization system, the storage medium and the electronic device provided by the embodiment of the present application correspond to the termination strategy optimization method provided by the embodiment of the present application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding part in the termination strategy optimization method, which will not be repeated here.

[0173] In summary, compared with the prior art, the present application has the following beneficial effects:

[0174] 1. The embodiment of the present application provides a staged task termination strategy optimization method, which makes up for the blank in the current task termination strategy without considering the staged task.

[0175] 2. The embodiment of the present application uses a stochastic dynamic programming framework to express the optimization problem, and uses a reverse induction method to solve it. Compared with the traditional fixed threshold strategy, the above experimental results show that the termination strategy proposed in the embodiment of the present application has great advantages in cost.

[0176] It is to be noted that, in the present text, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0177] The above examples are merely used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing termination strategies in a phased task system, characterized in that, Assuming the system performs a phased task, the termination strategy optimization method includes: For each stage of the phased task, the degradation state of the system is defined to follow a Wiener process; and the number of decision points for each stage is obtained based on a preset decision interval. The degradation state at each stage is converted into a discrete state model, and the state transition probability of the system is solved under the discrete state. Based on the state transition probability, the task termination cost and task continuation cost of each decision point under different system states are calculated, and the smaller value is selected as the corresponding expected cost; wherein if the decision point is located at the boundary of an adjacent stage, the expected cost includes at least the reward for completing the stage to which the decision point belongs. Starting from the end point of the phased task, a reverse dynamic programming algorithm is used to traverse each decision point. Based on the expected cost of the current decision point and the system state, the task is terminated or the task continues, and the action is saved to the task termination strategy set until the starting point of the phased task is reached. The phased task consists of M phases. For the m-th phase, the degradation state of the system is defined. Obey the drift parameter is The diffusion coefficient is The Wiener process, in which This indicates that it is in the m-th stage. This represents the start time of the m-th stage. This indicates the end time of the m-th stage; Define the decision interval as Then the number of decision points in the m-th stage is ,in It is an integer. Indicates the rounding up symbol; The process of converting the degenerate state at each stage into a discrete state model, and solving for the state transition probability of the system under the discrete states, includes: Define a finite set of degenerate states Where 0 represents the initial healthy state of the system and 1 represents the fault state of the system; State division rules: The discrete interval of the degradation state is set to d. When the degradation state of the system falls within the interval [id, (i+1)d], the system is determined to be in state i; when the degradation state of the system is greater than or equal to Id, the system enters fault state I; each state i is represented by the midpoint value of the interval (i+0.5)d. During the m-th stage, the system transitions from the current state i to other states k. The transition probabilities are calculated based on the characteristics of the Wiener process and are classified as follows: 1) If i ≠ I and k ≠ I 2) If i ≠ I and k = I 3) If i = I in, This represents the probability that, at the m-th stage, the system transitions from state i to state k after time t, where d represents the discrete interval. This is the cumulative distribution function of the standard normal distribution.

2. The termination strategy optimization method as described in claim 1, characterized in that, Based on the state transition probabilities, the process of calculating the task termination cost and task continuation cost at each decision point under different system states, and selecting the smaller value as the corresponding expected cost, includes: Define the state space of the system as Where n represents the cumulative number of decision points; and the action space is defined as {A,C}, where A represents terminating the task and C represents continuing the task. Let V(n,i) represent the expected cost at state (n,i): Where min denotes the minimization function, A(n,i) represents the cost of terminating the task at state (n,i), and C(n,i) represents the cost of continuing the task at state (n,i); Calculating V(n,i) requires considering the following cases: 1) If This indicates that the phased task has been successfully completed, and the expected cost at this point is: in, This represents the reward for completing the Mth stage of the task; 2) If This indicates that during the m-th stage of the task execution, when the system state is (n,i), the cost C(n,i) of continuing the task consists of the failure continuation cost CI and the expected continuation execution cost CE. in, This represents the cost of failure in each phase of the task. V(n+1,k) represents the system failure cost; V(n+1,k) represents the expected future cost of the system being in state (n+1,k). This indicates the elapsed time at the m-th stage. The probability that the system transitions from state i to state k; The cost A(n,i) of terminating a task when the system state is (n,i) consists of the expected cost AF of the termination operation: in, This represents the probability of successfully rescuing the system after terminating the mission at the m-th stage. 3) If The cost C(n,i) of continuing the task when the m-th stage task has been successfully completed and the system state is (n,i) consists of the failure continuation cost CI, the expected cost of continuing execution CE, and the stage reward PR. in, This represents the reward for completing the m-th stage; The cost A(n,i) of terminating a task when the system state is (n,i) consists of the expected cost of the termination operation AF and the stage reward PR: （13）。 3. The termination strategy optimization method as described in claim 2, characterized in that, Starting from the end point of the phased task, a backward dynamic programming algorithm is used to traverse each decision point. Based on the expected cost at the current decision point and under the system state, the task is terminated or continued, and this is saved to the task termination strategy set until the starting point of the phased task is reached. This includes: STEP0, Initialization parameters include and set This represents the set of task termination policies when the system state is (n, i), where Indicates the continuation of the action. Indicates the termination of the action; STEP 1: Calculate the total number of decision points. ; STEP 2, For all V(n,i) is calculated using formula (5); STEP 3: Let n = N-1; STEP4: If n < 0, then execute STEP12; otherwise, execute STEP5. STEP 5: Set i=0; STEP6: If i > I-1, jump to STEP11; otherwise, execute STEP7. STEP 7. If there exists m∈{1,2,…,M-1} such that… If so, C(n,i) and A(n,i) are calculated using formulas (11) and (13) respectively; otherwise, C(n,i) and A(n,i) are calculated using formulas (6) and (9). STEP 8, Calculation If V(n,i)=C(n,i), then set If i = i + 1, execute STEP 6; otherwise... Execute STEP9; STEP 9. For all k = i+1 to i-1, if there exists , making If the result is positive, then A(n,k) is calculated using formula (13); otherwise, A(n,k) is calculated using formula (9). STEP 10. Set V(n,k) = A(n,k). Proceed to STEP11; STEP 11: Set n = n-1, then return to STEP 4; STEP12: End the algorithm and output the set of task termination strategies.

4. A termination strategy optimization system considering a phased task system, characterized in that, Assuming the system performs a phased task, the termination strategy optimization system is used to execute the termination strategy optimization method as described in claim 1, including: The definition module is used to define the degradation state of the system as conforming to a Wiener process for each stage of the phased task; and to obtain the number of decision points for each stage based on a preset decision interval. The solver module is used to convert the degenerate state of each stage into a discrete state model, and solve the state transition probability of the system in the discrete state. And based on the state transition probability, to solve for the task termination cost and task continuation cost of each decision point under different system states, and select the smaller value as the corresponding expected cost; wherein if the decision point is located at the boundary of an adjacent stage, the expected cost includes at least the reward for completing the stage to which the decision point belongs. The decision module starts from the end point of the phased task, uses a backward dynamic programming algorithm to traverse each decision point, and selects whether to terminate the task or continue the task based on the expected cost of the current decision point and the system state, and saves it to the task termination strategy set, until the starting point of the phased task is reached.

5. A storage medium, characterized in that, It stores a computer program for considering the optimization of termination strategies for a phased task system, wherein the computer program causes the computer to execute the termination strategy optimization method as described in any one of claims 1 to 3.

6. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing the termination strategy optimization method as described in any one of claims 1 to 3.

Images