A UUV adaptive mission planning method under uncertain conditions

Abstract

The invention discloses a UUV adaptive task planning method under uncertain conditions. First, according to task requirements, the initial task target is divided into a to-be-executed target set G0 and a backup target set B0; second, according to G0, an initial task plan Π0 is generated; and the success probability of Π0 is evaluated. If the success probability of the task plan initially planned meets a given threshold value requirement, the task plan is loaded into the UUV control system and starts to be executed; otherwise, the target set needs to be reset; third, according to the current state of the UUV system, the success rate of subsequent task plans is evaluated respectively according to the task type, and compared with the set threshold value. If the task success rate is small, a task target deletion algorithm RAMG is called, otherwise, an task target addition algorithm AAMG is called to generate a new target set G, and a new task plan is generated according to G re-planning, and the UUV continues to execute the new task plan; fourth, all actions in the task plan are executed in sequence, and the task ends; the invention can effectively improve the autonomous capability of the UUV.

Description

Technical Field

[0001] The present invention belongs to the field of underwater unmanned vehicle (UUV) mission planning, and in particular relates to a UUV adaptive mission planning method under uncertain conditions. Background Art

[0002] During the execution of underwater missions, factors such as hydrology, mission objectives, and system working status may change. Relying entirely on manual remote control or pre-programmed operation cannot effectively respond to complex and changing dynamic environments in a timely manner. Therefore, UUVs need to be able to achieve real-time autonomous situational awareness, evaluation, and action planning, and make correct decisions without human intervention. Mission planning methods under uncertain conditions have been a hot research topic in recent years. The relevant research methods for solving uncertain mission planning problems mainly include Markov decision process methods, replanning methods, branch planning methods, planning problem decomposition methods, and plan modification methods.

[0003] 1. Task planning method based on Markov decision process

[0004] The Markov Decision Process (MDP) constructs a "state transition function" to assign a corresponding probability value to each uncertain effect. Given the state transition probability, it seeks the optimal strategy that can obtain the maximum return value. The main problem with the Markov decision process method is that when the state space is large, the amount of calculation required is large (NP-hard problem), especially when the state variables are continuous (the number of state spaces is infinite), which limits the scope of practical application. One solution is to discretize the continuous state space or reduce the number of state spaces, but the modeling method is related to the specific problem domain. For example, Capitan et al. used the partially observable Markov decision process method (POMDP) ​​to solve the task planning problem of multi-UAV reconnaissance of multiple targets. In order to reduce the computational complexity, each UAV planned local tasks through the POMDP method and completed the global task allocation based on the auction method; Redding et al. proposed a grouped aggregated Markov decision process method GA-Dec-MMDP, which can greatly reduce the number of state spaces and is suitable for large-scale UAV cluster task planning problems, but it cannot achieve optimization. In addition, the value iteration algorithm based on heuristic forward state space search can also improve computational efficiency. The problem is that the returned strategy is not necessarily the optimal solution and the algorithm is incomplete.

[0005] 2. Re-planning method

[0006] Yoon et al. proposed a replanning method FF-Replan to solve the problem of uncertain task planning.

[0007] FF-Replan is based on the fast forward planner FF. When the system enters an unexpected state during task execution, FF replanning is initiated to generate a new task plan. Since FF is a deterministic task planning method, FF-Replan uses two methods to deal with uncertainty: one is that there is only one deterministic result when executing an action (the result can be determined by relevant heuristic rules, such as the result with the highest probability of occurrence), and the other is that the execution of an action generates all possible results.

[0008] In the task planning problem, the use of system resources changes continuously, and there are infinite possible results when each related action is executed. Therefore, when using FF-Replan to plan tasks, only a deterministic single result processing method can be used, such as stipulating that the energy consumption value of each action is the mean of the corresponding distribution. However, according to the characteristics of probability distribution, the probability that energy consumption is equal to the mean is 0, which will cause the FF-Replan planner to judge that the system enters an unspecified state after executing each action, thereby triggering task replanning. Another feature of UUV tasks is that abnormal situations such as system failures or insufficient resources may occur, and FF-Replan does not have the ability to evaluate the probability of occurrence of special situations and the planning ability to handle abnormal events. Therefore, the FF-Replan method is not suitable for UUV task planning.

[0009] 3. Branch Planning Method

[0010] Formulating a branch plan is a solution to deal with uncertain planning problems. The main idea is to add a new task plan branch at the task execution node where the possibility of uncertain results is high, and adjust the original task at the branch node according to the actual execution of the task plan. Coles proposed a branch task planning method under uncertain resource conditions. In the offline task planning stage, this method assumes that the task environment, task objectives, emergencies and possible locations are all known, and determines the branch nodes and execution conditions in the initial plan. The problem is that the flexibility of use is poor, and it is necessary to accurately obtain the task environment and all possible event information. Gough et al. compared and analyzed the performance of two planning methods with and without branches under the same task conditions. Although the branch plan planning method can improve the task completion efficiency to a certain extent (18.31%), it takes a lot of planning time.

[0011] There are some problems with the branch planning method in practical applications: First, it is difficult to determine reasonable branch nodes. The offline task planning stage requires an evaluation of the initial task plan and the calculation of the required branch planning nodes according to certain criteria. However, the results of the action completion during task execution may differ greatly from the expected results, which is not suitable for complex and changeable task scenarios. Second, it can only process a limited number of action results and cannot meet the needs of uncertainty planning problems with continuously changing states.

[0012] 4. Problem decomposition and planning method

[0013] The basic idea of ​​decomposition planning method is to decompose complex problems into a series of relatively independent subtasks, and then merge them into a complete task plan after solving them separately. Yang proposed a task goal decomposition method GDECOMP, which divides the initial task goal into several target subsets through machine learning algorithms, plans each target subset separately to obtain relevant subtask plans, and handles the "conflict" between sub-plans through constraint satisfaction problem methods. Bibai et al. proposed a meta-heuristic evolutionary planning algorithm DAEX based on state decomposition, but the computational complexity of sub-task merging is large, and the efficiency of online task planning is greatly affected by the heuristic algorithm.

[0014] In the offline task planning stage, the problem decomposition planning method needs to conduct a detailed analysis of the specified task planning problem, comprehensively consider various possible task execution results, decompose the overall task into a series of task sub-goals, and merge the corresponding task plan subsets according to the actual situation during the task execution process. The main problem with this processing method is that it needs to comprehensively consider various task execution results in advance, which is not suitable for more complex tasks or tasks with high real-time requirements.

[0015] 5. Plan to modify the planning method

[0016] Different from task replanning, the basic idea of ​​the plan modification method is to modify some elements of the initial task plan to meet the new task requirements. Cashmore et al. proposed a benefit-based AUV task planning system that can realize online task scheduling, adjust the order of task target execution to adapt to the current working conditions of the AUV or obtain greater task benefits, but can only modify the task based on the current task execution status, and cannot make relevant adjustments in advance through predictive reasoning. Wood et al. proposed a template-based plan modification planning method. Before the task is executed, a series of plan fragments are generated offline and stored in the plan library according to the type of emergency. When the robot encounters an emergency during the task execution, the relevant plan fragments are called from the plan library and inserted into the current task plan.

[0017] Intuitively, the cost of modifying a plan (such as computational effort) should be less than task replanning. However, research by Nebel et al. shows that since task modification involves calculations such as matching, comparing, adding, and deleting actions in task plans, using task replanning methods is often more advantageous.

[0018] Classical planning methods can efficiently generate action sequences that meet planning goals, but UUV task generation belongs to the field of uncertain planning, so classical planning methods cannot be used; planning methods based on Markov decision processes have too many state spaces and low solution efficiency; traditional re-planning methods have a single processing mode and poor flexibility; existing branch planning methods, problem decomposition methods, and plan modification methods mostly need to be adjusted according to pre-set processing procedures, which have problems such as large computational complexity and complex implementation, and poor adaptability to dynamic and complex environmental conditions. By comparing the advantages and disadvantages of the above methods, it can be seen that: compared with analyzing and predicting results in the offline task planning stage, the use of online task monitoring functions can effectively integrate the actual operation information of the system, and can make more accurate predictions and judgments on the system operation process; task re-planning has relatively good real-time performance and is relatively simple to implement, but it can only be "passively" planned based on the results that have occurred. Summary of the invention

[0019] In view of this, the present invention provides a UUV adaptive mission planning method under uncertain conditions, which can combine online monitoring, evaluation prediction, and mission adjustment to adapt to changing mission conditions and obtain maximum mission benefits, thereby effectively improving the UUV's autonomous capability.

[0020] The technical solution for implementing the present invention is as follows:

[0021] A UUV adaptive mission planning method under uncertain conditions includes the following steps:

[0022] Step 1: Before the UUV is launched, the initial mission objectives are divided into a target set G to be executed according to the mission requirements. 0 and backup target set B 0 ; G 0 The target tasks that must be completed in order to achieve the purpose of the mission action belong to the task objectives to be executed; B 0 The middle one is a new task target that can be adjusted under abnormal circumstances, or when there are surplus system resources, more tasks can be completed to improve the utilization efficiency;

[0023] Step 2: The task planner is based on G 0 Generate initial mission plan 0 ; According to the resource calculation formula, Π 0The success probability is evaluated. If the success probability of the initially planned mission plan meets the given threshold value requirement, the mission plan is loaded into the UUV control system and starts to be executed; otherwise, the target set needs to be reset;

[0024] Step 3: Based on the current state of the UUV system, the success rate of the subsequent mission plan is evaluated according to the mission type, and compared with the set threshold value. If the mission success rate is relatively small, the deletion mission target algorithm RAMG is called, otherwise the addition mission target algorithm AAMG is called to generate a new target set G. A new mission plan is generated based on the target set G, and the UUV continues to execute the new mission plan.

[0025] Step 4: All actions in the task plan are executed in sequence and the task is completed.

[0026] Furthermore, the adaptive task planning method is implemented by three functional modules: a task planning module, a task target management module and a task online monitoring module.

[0027] Furthermore, the task online monitoring module has a data communication interface with the underlying control system of the unmanned system, which can obtain the working status and fault information of each system component in real time, and evaluate the task in real time according to the system working conditions; monitor the system working status in real time, update the target task set, and adjust the task plan as appropriate.

[0028] Furthermore, the task objective management module adjusts the task objective according to the system work information, and starts the task planning module after the adjustment; during the task execution process, adaptive task adjustment is achieved by selecting the task objectives to be executed; the purpose of target selection is to optimize the task plan, reduce task risks, and improve task benefits, and there are two ways of selection: adding and deleting; adding a task objective refers to selecting an objective from the backup task objective set to the task objective set to be executed; deleting a task objective refers to removing the relevant objectives in the task objective set to be executed to the backup task objective set.

[0029] Furthermore, the mission planning module generates a mission plan based on the current system status and mission objectives.

[0030] Furthermore, the adaptive mission planning method adopts a deterministic planning algorithm to generate an initial mission planning scheme according to a given mission objective. Since there are many random factors in the UUV mission planning problem, after offline plan generation or online mission replanning, it is necessary to use a mission evaluation model to calculate the mission success rate and evaluate the feasibility of the new plan to meet the expected objectives. At the same time, referring to the definition of the reward function in the Markov decision process method, each mission objective is assigned a value, and the expected reward value of the mission is evaluated according to the mission success rate.

[0031] During the execution of the task action, the planning system monitors the remaining system resources in real time and calculates the success probability of completing subsequent task objectives. If the probability of task success is lower than the set threshold, the task objective set is adjusted according to the nature of various system events, and re-planning is initiated to optimize the subsequent task plan.

[0032] Beneficial effects:

[0033] 1. Combining online monitoring, evaluation and prediction, and task re-planning and adjustment has changed the problems of traditional re-planning methods, such as single processing mode and poor flexibility.

[0034] 2. The modeling of complex uncertain task planning problems does not require the introduction of huge state space information, which solves the problems of too large state space and low solution efficiency in traditional Markov decision process-based methods.

[0035] 3. It can simultaneously monitor and adaptively adjust system resource usage, system failures, external control instructions and other factors in real time to meet complex and changing real-time task requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Composition for adaptive mission planning systems.

[0037] Figure 2 Planning system information processing for adaptive tasks.

[0038] Figure 3 The data storage space is divided into use stages.

[0039] Figure 4 Schematic diagram of the mission payload delivery process.

[0040] Figure 5 The change of task success rate of simulation example 2.

[0041] Figure 6 The change of task success rate of simulation example 3. DETAILED DESCRIPTION

[0042] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0043] The present invention provides a UUV adaptive mission planning method under uncertain conditions, and the specific process is as follows:

[0044] 1. Task representation method

[0045] According to the characteristics of UUV underwater missions, the planning problem is formally represented with reference to the relevant definitions of PDDL. The world model of PDDL is represented by various system states, which are composed of logical propositions and predicate base cases, called state variables. The goals of the planning problem can be represented by proposition instances. When all the proposition conditions of a goal are met, the goal state is achieved.

[0046] Assume that the UUV action set is A, and the definition of any UUV action a∈A is shown in Table 1.

[0047] Table 1 UUV action definition

[0048]

[0049]

[0050] The system state s∈S is defined by a set of state variables. Some examples of state variable definitions are shown in Table 2.

[0051] Table 2 System state variable definition

[0052]

[0053]

[0054] 2. Overall framework of adaptive task planning method

[0055] The adaptive task planning method consists of three functional modules: task planning module, task target management module, and task online monitoring module. Figure 1 As shown in the figure, the task planning module generates a task plan based on the current system status and task objectives; the task online monitoring module has a data communication interface with the underlying control system of the unmanned system, which can obtain the working status and fault information of each system component in real time, and evaluate the task in real time according to the system working conditions; the task objective management module adjusts the task objectives according to the system working information, and starts the task planning module after the adjustment.

[0056] The information processing flow of the adaptive task planning method is shown in Figure 2As shown. Different from the traditional uncertain planning method, this method uses a deterministic planning algorithm to generate an initial mission planning scheme (a set of continuous action sequences) according to the given mission objectives. Since there are many random factors in the UUV mission planning problem, after offline plan generation or online mission replanning, it is necessary to use a mission evaluation model to calculate the mission success rate and evaluate the feasibility of the new plan to meet the expected goals; at the same time, referring to the definition of the reward function in the Markov decision process method, assign values ​​to each mission objective, and evaluate the expected reward value of the mission according to the mission success rate.

[0057] During the execution of the task action, the planning system monitors the remaining system resources in real time and calculates the success probability of completing subsequent task objectives. If the probability of task success exceeds the set threshold, the task objective set is adjusted according to the nature of various system events, and re-planning is initiated to optimize the subsequent task plan.

[0058] (1) Mission Planning Module

[0059] The task planning module implements the initial task planning and online task re-planning functions. Its performance requirement is to have a high solution efficiency. Some task scenarios require the planner to have a certain numerical optimization reasoning ability (that is, to complete the expected task goal with less action cost). When the task goal needs to be adjusted, the task planning module is called by the task goal management module. In order to meet the task planning efficiency requirements, it is possible to consider using a mature fast classic planner as the task planner. Typical fast planners include Metric-FF, Fast Downward, etc.

[0060] (2) Task Online Evaluation Module

[0061] The main function of the task online evaluation module is to monitor the system working status in real time, update the target task set, and adjust the task plan as needed. After the task plan is determined, the task success rate mainly depends on the usage of various system resources in the actual task environment. If the system resources are not enough to achieve the task objectives to be completed, the task fails. The resource usage of each action can be modeled by the relevant probability model.

[0062] ① Modeling of resource types with continuous monotonically decreasing changes

[0063] A typical resource that changes continuously and monotonically is energy. All actions in the mission plan consume energy resources, and energy cannot be replenished during the mission execution. Therefore, the total energy used by the UUV is the sum of the random variables of energy consumption of all actions. According to the relevant conclusions of probability theory, the sum of a finite number of independent normal random variables still obeys the normal distribution, so the total energy use random variable obeys the normal distribution. During the mission execution, the UUV control system collects energy usage parameters in real time, so the remaining energy amount can be accurately calculated. Assume that the current remaining energy is Ce (a known constant), and the total energy usage of subsequent actions is Te~(TeN(μ,σ 2 ), event S indicates the successful completion of the task plan (only related to energy resources). Under the current energy Ce condition, the successful completion of the subsequent action should meet the condition 0≤Te≤Ce, and the task success rate P(S) is expressed as:

[0064]

[0065] ② Continuously scalable resource modeling

[0066] A typical resource is data storage space. Unlike energy resources, data storage space is a reusable resource. Therefore, the actions of sending and collecting data cannot be simply treated as the sum of random variables in the same way as energy resources. It is necessary to consider the changes in the task execution process as a whole. Affected by factors such as navigation errors, the storage space occupied by collected data can be regarded as a random variable that obeys a normal distribution. Although UUV can accurately control the amount of data sent, in special circumstances such as adverse hydrological and meteorological conditions, data cannot be transmitted or can only be transmitted partially and then interrupted. It is assumed that the incomplete data set cannot be used. In this case, it is treated as a transmission failure; therefore, the storage space released by UUV transmission data can be regarded as obeying a Bernoulli distribution.

[0067] According to the characteristics of data storage space resource usage, the task plan is processed in stages. For example, Figure 3 The nodes for phase division are set before each transmission action, where the numbers in the boxes represent the storage space usage (the average value for the collected data), negative values ​​represent occupied storage space, positive values ​​represent released storage space, and 0 represents no change in storage space. n Indicates that the task was successfully executed to T n Node (n is a non-negative integer, indicating the number of stages the plan is divided into), event T n ' indicates that in T n The remaining storage space of the node is not less than 0. Let C n Indicates the storage space value released by the transmission data in the nth stage (when n=1, it is the initial value of the data storage space), μ n is the mean value of the amount of data recorded in the nth stage, σ nis the standard deviation, x n represents the random variable of the data recorded in the nth stage, and the probability density function is f(x n ). Assume that the data recording process of each stage does not affect each other, so x 1 ,…,x n Independent of each other.

[0068] Assume A 1 , A 2 …A n It is an independent event that the task plan action transdata(uuv,d) is successfully executed in sequence. Event B 1 ,...,B m A complete event group is formed, where Event E indicates that the task plan is successfully completed (only related to storage space resources). According to the total probability formula, the expression of P(E) is:

[0069]

[0070] Among them, P(B i ) is calculated based on the given success probability values ​​of each transmission action; P(T 1 'T 2 '…T n '|B i ) is:

[0071]

[0072] ③ Discrete monotonically decreasing resource modeling

[0073] The typical discrete monotonically decreasing resource is the mission load, and the probability distribution of the usage of the mission load resource is related to the specific mission. Taking the weapons carried by UUV as an example, the target hit in an attack follows the Bernoulli distribution, and the number of weapons consumed is related to the target damage standard and the weapon hit probability. For ease of processing, for a specific type of target, the number of weapons required to damage the target using a certain type of weapon is a constant r (such as 2 torpedoes are needed to sink a large destroyer), so the number of weapons required to hit the target r times follows the Pascal distribution. According to the relationship between the negative binomial distribution and the Pascal distribution, the negative binomial distribution is used to describe the consumption of load resources.

[0074] Assume the current remaining load quantity is N p , you need to attack n targets, and you need to hit r targets to successfully damage each target i (i=1,…,n) loads (such as torpedoes, etc.), k i represents the number of payloads that miss the i-th target, and the hit probability is p, then k 1 +…+kn ~NB(r 1 +…+r n ,p), and 0≤k 1 +…+k n ≤N p -(r 1 +…+r n ). Event W indicates that the mission plan is successfully completed (only related to the mission payload resources). p Under the condition of , the success rate P(W) of completing the subsequent action is expressed as (where I is a regular incomplete beta function):

[0075] P(W)=I 1-p (r 1 +…+r n ,1+N p -r 1 -…-r n )

[0076] (3) Task target management module

[0077]

[0078] During the mission execution process, adaptive mission adjustment is achieved by selecting the mission objectives to be executed. The purpose of target selection is to optimize the mission plan, reduce mission risks and improve mission benefits. There are two ways to select: adding and deleting. Adding a mission objective refers to selecting a target from the backup mission objective set to the mission objective set to be executed; deleting a mission objective refers to removing the relevant target from the mission objective set to be executed to the backup mission objective set. In addition, under actual mission conditions, the mission objectives are not necessarily independent of each other, and there may be a certain correlation. Therefore, when adjusting the mission objectives, the relevant mission objectives are adjusted synchronously according to the correlation relationship. The pseudo code of the mission goal management algorithm MGMA (Mission Goals Management Algorithm) is shown in Table 3. Before the mission begins, the UUV operator determines the initial mission objectives based on specific mission requirements. According to mission requirements, the initial mission objectives can be divided into a set of targets to be executed G 0 and backup target set B 0 . G 0 It is a target task that must be completed in order to achieve the purpose of the mission action.

[0079]

[0080] B 0 It is to consider new task targets that can be adjusted under abnormal circumstances, or to complete more tasks to improve the use efficiency when there are surplus system resources.0 Generate an initial mission plan (line 3). If the success probability of the initially planned mission plan meets the given threshold requirement, the mission plan is loaded into the UUV control system and starts execution (line 11); otherwise, the target set needs to be reset.

[0081] ①Add task target algorithm

[0082] The basic process pseudo code of the addition algorithm of mission goals (AAMG) is shown in Table 4.

[0083] According to the greedy principle, the AAMG algorithm adds each target in the backup target set in descending order of benefit value (line 4). If a target that meets the task success rate requirement is found, it further searches for new targets until all targets to be searched are completed and returns the updated task target set (line 15). In the MAX_B function, for task targets with the same benefit value, they are sorted by the average value of the resources consumed, and the task targets with relatively small consumption are added first.

[0084] ②Delete task target algorithm

[0085] According to the characteristics of UUV missions, a reduction algorithm of mission goals (RAMG) is designed. The pseudo code of the basic process is shown in Table 5.

[0086] RAMG is divided into two processes: reducing and expanding the target set. First, the number of deleted task targets is increased one by one, and a minimum target set that can meet the task success rate requirement is obtained. According to the greedy principle, the target with the smallest benefit value is deleted each time, as shown in rows 6 to 14.

[0087]

[0088] The minimum target is the measurement standard, so in the deleted target set, there may be targets with less resource consumption and can meet the task success rate requirements. In order to improve the efficiency of task execution, it is necessary to backtrack all deleted task targets on the basis of the minimum target set to find feasible task targets that can be added. The extended search process is the same as the AAMG algorithm. In the MIN_B function, for task targets with the same benefit value, they are sorted by the mean value of resource consumption, and the task targets with relatively large consumption are deleted first.

[0089] (3) Task-goal relationship

[0090] According to the characteristics of the UUV mission, four types of association relationships of mission objectives are defined as shown in Table 6.

[0091] Table 6 Definition of task target association relationship

[0092]

[0093] According to this table, the relational structure of the task target set can be established. In the RAMG and AAMG algorithms, when operations involving deleting and adding targets are involved, the related targets should be processed synchronously according to the task relational structure. According to the definition of task relations, in the set of task targets to be executed, the target selection operation should comply with the following rules:

[0094] Rule 1: For mission objectives with independent relationships, adding or deleting a mission objective will not have any effect on other mission objectives.

[0095] Rule 2: Task targets with one-way dependencies 1 → 2 , you need to delete g 2 When g 1 Also deleted; need to delete g 1 When the task objectives are determined, it is determined whether to delete g 2 ; need to increase g 2 When g is increased, it is determined according to the task objective constraints. 1 , increase g while satisfying the task success rate 1 ; If g does not exist in the task target set 2 , then g cannot be increased alone 1 .

[0096] Rule 3: For task targets with bidirectional dependencies, when one task target needs to be added or deleted, all other related task targets will also be added or deleted synchronously.

[0097] Rule 4: For task objectives with incompatible relationships (appearing in the set of backup task objectives), the task benefit values ​​cannot be equal; when it is necessary to add a target, add the target with the maximum benefit value; when it is necessary to delete a target, delete the target with the minimum benefit value. (V) Specific implementation methods

[0099] The UUV mission payload cabin is loaded with material and dropped at a designated location. Figure 4 shown.

[0100] The UUV starts to dive and perform the mission at waypoint 1, and plans to deliver the mission payload at two designated locations: waypoint 2 and waypoint 4. To improve the delivery position accuracy, a satellite calibration point (waypoint 3) is set before waypoint 4, and a recovery point (waypoint 5) is set. The range between each waypoint is shown in Table 7.

[0101] Table 7 Flight distance of each segment (unit: kilometers)

[0102]

[0103] The mission goal set is G = {goal1 = (have_load waypoint2 load1), goal2 = (calibrated myauv waypoint3), goal3 = (have_load myauv waypoint4 load2), goal4 = (mission_completed myauv waypoint5)}.

[0104] Table 8 Alternative flight distance (unit: kilometers)

[0105] Flight Segment Waypoint 4→6 Waypoint 6→7 Waypoint 6 → 5 voyage 200 500 1100

[0106] In order to further improve the mission efficiency, two backup mission targets are set, one is the backup payload delivery point (waypoint 6), and the other is the backup recovery point (waypoint 7, which shortens the return mileage compared to the original planned recovery point in order to reduce the probability of being unable to recover the UUV due to insufficient energy). The backup flight distances are shown in Table 8.

[0107] Backup mission goal set backup_G = {bgoal1 = (have_load myauv waypoint6 load3), bgoal2 = (mission_completed myauv waypoint7)}. The constraint relationship between mission goals is shown in Table 9.

[0108] Table 9 Task target set association relationship

[0109]

[0110] Assume that the UUV autonomous navigation error is 0.2% of the voyage. To facilitate the sea search of the mission payload, the delivery accuracy is required to be no more than 0.5 km. Therefore, the one-way dependency goal2→(goal3, bgoal1) is subject to the delivery accuracy condition. After deleting goal2, the error of navigating to waypoint4 is 2.2 km, which is greater than the delivery accuracy requirement. Goal3 is deleted, and bgoal1 is deleted in the same way.

[0111] The mission goal benefit value adopts a scoring evaluation method, with a value range of 0-10 points, as shown in Table 10. The delivery demand of bgoal1 is relatively weak, while the recovery point of bgoal2 is far from the dock and difficult to implement, so the backup goal benefit value is relatively low.

[0112] Table 10 Benefit value of each task target

[0113]

[0114] The UUV has 500 kWh of energy remaining before launching into the water to perform its mission.

[0115] Task success rate threshold P d Set to 0.841 (corresponding to energy consumption equal to the mean plus one standard deviation), P a It is set to 0.9 (corresponding to energy consumption equal to the mean plus 1.282 times the standard deviation).

[0116] The basic process is as follows:

[0117] Step 1: Generate an initial mission plan

[0118] For the initial task target set G, the planner is used to plan and obtain the initial task plan Π 0 as follows:

[0119] action1:dive(myauv)

[0120] action2:navigate(myauv waypoint1 waypoint2)

[0121] action3:releaseload(myauv load1 waypoint2)

[0122] action4:navigate(myauv waypoint2 waypoint3)

[0123] action5:surface(myauv)

[0124] action6:calibrate(myauv waypoint3)

[0125] action7:dive(myauv)

[0126] action8:navigate(myauv waypoint3 waypoint4)

[0127] action9:releaseload(myauv load2 waypoint4)

[0128] action10:navigate(myauv waypoint4waypoint5)

[0129] action11:surface(myauv)

[0130] action12:completemission(myauv waypoint5)

[0131] The total flight distance is 2,460 kilometers.

[0132] Assuming that under still water conditions, the UUV navigation energy consumption rate is 0.2 kWh / km, and the energy consumed by the release load action is negligible (the release mechanism is generally a mechanical lock or blasting bolt, etc., and only a momentary current is required to activate the release switch). The energy consumption of each action is shown in Table 11.

[0133] Table 11 Energy consumption parameters of each action

[0134]

[0135]

[0136] Step 2: Initial Mission Plan Assessment

[0137] According to the above table, calculate the task plan Π 0 The mean energy consumed by the action is 432.5 kWh, and the standard deviation is 58.6 kWh. 0 The mission success rate is 0.875 (without considering reliability), which is greater than the specified minimum threshold P a , and the conditions for execution are met.

[0138] Step 3: Task Execution

[0139] Yes 0 Each action randomly generates energy consumption values ​​according to the specified probability distribution. Different random event sequences may lead to different task execution processes. Several typical simulation results are as follows:

[0140] 1. Simulation Example 1—No need to adjust the task

[0141] The system energy can meet the task consumption, and there is no need to adjust the task plan. 0 The energy consumption values ​​of each action are shown in Table 12. At the end of the task, the remaining energy of the system is about 39 kWh.

[0142] Table 12 Π 0 The actual energy consumption simulation results when each action is executed (unit: kWh)

[0143]

[0144] 2. Simulation Example 2—Insufficient System Resources

[0145] After completing action 4, the remaining energy of the system is 188.8 kilowatts, the average energy consumption of the actions to be executed is 206.4, the standard deviation is 50.0036, and the task success rate is 0.362, which is less than P d Therefore, the RAMG module is called to reduce the original mission objectives. The energy consumption values ​​of the executed actions are shown in Table 13. The energy consumption is greater than the expected value. The main reason is that during the voyage of this section, the simulated energy consumption value of 243.1 is much greater than the expected value of 160, indicating that the UUV may encounter an unknown large countercurrent when sailing in this sea area.

[0146] Table 13 Simulated energy consumption values ​​of executed actions (unit: kWh)

[0147] Action Number 1 2 3 4 Consumption 0.09 68.1 0 243.1

[0148] The current state input of the RAMG algorithm is: S 1 ={(at_point myauv waypoint3),(under_watermyauv),(have_load myauv waypoint2 load1)}, the current task target set input is: G 1 ={(calibrated myauv waypoint3),(have_load myauv waypoint4 load2),(mission_completed myauv waypoint5)}(After action4 is executed, goal1 has been achieved).

[0149] In the current task goal set, the goal with the minimum benefit value is (calibrated myauv waypoint3), so the RAMG algorithm first deletes goal2; according to the task goal association relationship, after goal2 is deleted, goal3 is invalid; according to the task goal goal4, planning is performed to obtain the plan Π 1 as follows:

[0150] action1:navigate(myauv waypoint3 waypoint5)

[0151] action2:surface(myauv)

[0152] action3:completemission(myauv waypoint5)

[0153] According to the current remaining energy, Π 1The mission success rate is 0.43, so continue to delete goal4, and the current mission goal set is an empty set At this time, the mission success rate is 1, so the target set needs to be expanded. The input parameters of AAMG are: current mission target set To be added task target set B 0 G 1 ∪backup_G, current state is S 1 In B 0 In the example, the goal with the maximum benefit value is goal3, so goal3 is increased first (according to task goal selection rule 2, goal2 is increased simultaneously), and the task success rate is calculated to be 0.999, which is greater than P a , continue to add new targets; B 0 In the figure, goal4 and bgoal2 are incompatible. According to task goal selection rule 4, goal4 with a relatively large benefit value is selected, but the task success rate does not meet the requirements. Therefore, after the AAMG algorithm deletes goal4, it continues to add task goals bgoal1 and bgoal2. The task success rate is 0.95, which meets the requirements. At this time, the backup goal set is empty, and the task goal adjustment process ends.

[0154] After the plan was adjusted, the UUV could not only continue to complete the satellite calibration and waypoint 4 payload delivery tasks in the initial plan, but also complete the waypoint 6 payload delivery task. However, due to insufficient energy, it could not return to the scheduled recovery point and could only choose the backup recovery point. The new mission plan is as follows:

[0155] action1:dive(myauv)

[0156] action2:navigate(myauv waypoint1 waypoint2)

[0157] action3:releaseload(myauv load1 waypoint2)

[0158] action4:navigate(myauv waypoint2 waypoint3)

[0159] action5:surface(myauv)

[0160] action6:calibrate(myauv waypoint3)

[0161] action7:dive(myauv)

[0162] action8:navigate(myauv waypoint3 waypoint4)

[0163] action9:releaseload(myauv load2 waypoint4)

[0164] action10:navigate(myauv waypoint4 waypoint6)

[0165] action11:releaseload(myauv load3 waypoint6)

[0166] action12:navigate(myauv waypoint6 waypoint7)

[0167] action13:surface(myauv)

[0168] action14:completemission(myauv waypoint7)

[0169] The change of task success probability during task execution is shown in Figure 5 As shown. After the UUV passed the unpredicted high-energy consumption navigation phase (action4), the remaining energy could not effectively complete the original planned subsequent actions. If the original mission plan is continued, the UUV will terminate the mission due to insufficient energy at a certain location in the return leg, and it will not be able to search and recover the UUV in time when it is far away from the scheduled recovery area. After the adaptive mission planning adjustment, the mission success rate met the threshold requirements again, and the total benefit value after the planning adjustment was 33, which is close to the original planned mission benefit value of 34. Although the recovery area is advanced, it is still within the effective operating range, and the number of payload delivery points is also increased. The use efficiency of UUV is effectively improved after the adaptive mission planning is adopted.

[0170] 3. Simulation Example 3—UUV System Failure

[0171] Reliability is related to the specific UUV system structure, so this paper does not carry out UUV reliability modeling and simulation work, but simplifies the processing into a fixed system failure event. After completing action 4, the remaining energy of the system is 259.1 kilowatts, the mean energy consumption of the action to be executed is 206.4, the standard deviation is 50.0036, and the mission success rate is 0.8542. The UUV is preparing to perform the satellite calibration operation as planned. Assume that the satellite buoy release mechanism of the UUV is detected to have a fault at this time, so the mission goal goal2 is judged to be invalid. According to the mission goal association relationship, goal3 and bgoal1 are also invalid. The energy consumption values ​​of the executed actions are shown in Table 14.

[0172] Table 14 Simulated energy consumption values ​​of executed actions (unit: kWh)

[0173]

[0174] The target set of the mission to be executed becomes {(mission_completed myauv waypoint4)}, and the online task management module starts the task planner for replanning. The results are as follows:

[0175] action1:navigate(myauv waypoint3 waypoint5)

[0176] action2:surface(myauv)

[0177] action3:completemission(myauv waypoint5)

[0178] The mission success rate of this program is 0.943, which is greater than P a , so the AAMG algorithm is called. Since the benefit value of bgoal2 is less than goal4, the target of the task to be executed output by AAMG is goal4. Due to the failure of the satellite calibration function, the UUV cannot deliver the mission payload according to the specified accuracy requirements, so the UUV finally returns to the original recovery point waypoint5. The actual execution of the task process is as follows:

[0179] action1:dive(myauv)

[0180] action2:navigate(myauv waypoint1 waypoint2)

[0181] action3:releaseload(myauv load1 waypoint2)

[0182] action4:navigate(myauv waypoint2 waypoint3)

[0183] action5:navigate(myauv waypoint3 waypoint5)

[0184] action6:surface(myauv)

[0185] action7:completemission(myauv waypoint5)

[0186] For changes in mission success rate, see Figure 6 As shown in the figure. After action 4, although the remaining energy can meet the mission requirements, the mission execution conditions have changed and the original planned subsequent actions have failed. After adaptive mission adjustment, the UUV returns to the destination. Since there is more remaining energy, the subsequent plan can maintain a high mission success rate. Due to the failure of the satellite calibration function, the adaptive mission planning system abandons the scheduled delivery mission that cannot be completed, and enables the UUV to achieve recovery more reliably.

[0187] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A UUV adaptive mission planning method under uncertain conditions, characterized in that: The following steps are involved: Step 1: Before the UUV is launched, the initial mission objectives are divided into a pending target set G0 and a backup target set B0 according to mission requirements; the target tasks in G0 that must be completed to achieve the mission action purpose belong to the pending mission objectives; B0 contains new mission objectives that can be adjusted under abnormal circumstances, or when there are surplus system resources, more tasks can be completed to improve the use efficiency; Step 2: The mission planner generates an initial mission plan Π0 based on G0; the success probability of Π0 is evaluated according to the resource calculation formula. If the success probability of the initial mission plan meets the given threshold value requirement, the mission plan is loaded into the UUV control system and starts execution; otherwise, the target set needs to be reset; Step 3: Based on the current state of the UUV system, the success rate of the subsequent mission plan is evaluated according to the mission type, and compared with the set threshold value. If the mission success rate is relatively small, the deletion mission target algorithm RAMG is called, otherwise the addition mission target algorithm AAMG is called to generate a new target set G. A new mission plan is generated based on the target set G, and the UUV continues to execute the new mission plan. Step 4: All actions in the task plan are executed in order and the task is completed; The adaptive task planning method is implemented by three functional modules: a task planning module, a task target management module and a task online monitoring module; The task online monitoring module has a data communication interface with the unmanned system underlying control system, which can obtain the working status and fault information of each system component in real time, and evaluate the task in real time according to the system working condition; monitor the system working status in real time, update the target task set, and adjust the task plan as needed; The task target management module adjusts the task target according to the system working information and starts the task planning module after the adjustment; During the task execution process, adaptive task adjustment is achieved by selecting the task targets to be executed; the purpose of target selection is to optimize the task plan, reduce task risks, and improve task benefits. There are two ways to select: adding and deleting. Adding a task target means selecting a target from the backup task target set to the task target set to be executed; deleting a task target means removing the relevant target from the task target set to be executed to the backup task target set. The mission planning module generates a mission plan based on the current system status and mission objectives; The adaptive mission planning method adopts a deterministic planning algorithm to generate an initial mission planning scheme according to a given mission goal. Since there are many random factors in the UUV mission planning problem, after offline plan generation or online mission replanning, it is necessary to use a mission evaluation model to calculate the mission success rate and evaluate the feasibility of the new plan to meet the expected goals. At the same time, referring to the definition of the reward function in the Markov decision process method, each mission goal is assigned a value, and the expected reward value of the mission is evaluated according to the mission success rate. During the execution of the task action, the planning system monitors the remaining system resources in real time and calculates the success probability of completing subsequent task objectives. If the probability of task success is lower than the set threshold, the task objective set is adjusted according to the nature of various system events, and re-planning is initiated to optimize the subsequent task plan.

Images