Monte Carlo tree search-based unmanned cluster reconnaissance rescue task allocation method

Abstract

The invention relates to an unmanned cluster reconnaissance rescue task allocation method based on Monte Carlo tree search, and belongs to the technical field of task planning. According to the method, the task allocation problem in reconnaissance and rescue is expressed as a sequential decision process, heuristic information of the problem is fully utilized, a state space and an action space are designed, the action completeness and the action number are balanced, and calculation resources are saved while the allocation effect is guaranteed. Besides, a self-adaptive simulation number selection mechanism is provided, simulation times can be automatically selected according to the tasks to be distributed and the number of the unmanned aerial vehicles, and a large amount of repeated simulation is avoided, so that the calculation amount of an algorithm is reduced, and the calculation efficiency of task distribution is further improved. According to the method, the task allocation effectiveness of the unmanned system can be enhanced, and the task efficiency of the unmanned system is improved.

Description

Technical Field

[0001] This invention relates to a Monte Carlo tree search-based unmanned swarm reconnaissance and rescue mission allocation method, which belongs to the field of mission planning technology. It can be used by drones to quickly conduct comprehensive searches of disaster-stricken areas, locate people awaiting rescue, and deliver necessary relief supplies to key locations, thereby improving the success rate of rescue missions. Background Art

[0002] With advances in unmanned intelligent technology, unmanned swarms have demonstrated tremendous potential for application in various fields. Unmanned swarms offer advantages such as low cost and operational flexibility, making them widely applicable for reconnaissance and surveillance missions, including power inspections, road monitoring, and search and rescue. However, the increasing number of drones and the number of missions they undertake pose further challenges to the task allocation and scheduling of drone swarms. Optimizing the selection and execution sequence of multi-robot, multi-target tasks, and maximizing the performance of unmanned swarm missions within limited resources, remains a hot topic in current drone task allocation research.

[0003] The task allocation problem in unmanned swarms is often formulated as a classic combinatorial optimization problem, addressed by both exact and heuristic algorithms. Exact algorithms rely on mathematical optimization to obtain precise solutions. While these methods guarantee optimality, they are computationally expensive and generally unsuitable for solving large-scale problems or those involving complex constraints. Heuristic algorithms leverage domain knowledge and randomness to accelerate the solution process. While heuristic algorithms generally cannot guarantee a globally optimal solution, they can effectively produce high-quality solutions, making them particularly suitable for large-scale and complex problems. Consequently, they have been widely used in real-world scenarios.

[0004] Common heuristic algorithms include genetic algorithms, ant colony algorithms, and auction algorithms. However, they typically require careful parameter tuning (for example, the mutation rate in genetic algorithms or the swarm size in particle swarm optimization) or extensive problem-specific knowledge to achieve satisfactory performance. In contrast, Monte Carlo Tree Search (MCTS), as a heuristic search algorithm, stands out for its minimal reliance on problem-specific knowledge and parameter tuning, making it widely applicable to decision-making problems under uncertainty. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this paper proposes an unmanned swarm reconnaissance and rescue task allocation method based on Monte Carlo tree search. This method formulates the task allocation problem in reconnaissance and rescue as a sequential decision-making process, fully utilizing the heuristic information of the problem. The state space and action space are designed to balance action completeness and the number of actions, thus conserving computational resources while ensuring effective allocation. Furthermore, an adaptive simulation number selection mechanism is proposed to further improve computational efficiency. This method can also enhance the effectiveness of unmanned system task allocation and improve the task efficiency of unmanned systems.

[0006] The technical solution of the present invention is: an unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search, comprising the following steps:

[0007] Step 1: Mathematical modeling of the task allocation problem: Establish a set of UAVs that can perform tasks and a set of tasks that need to be performed, and clarify the task objectives and task constraints;

[0008] Step 2: Design system actions and states to transform the task allocation problem into a sequential decision problem, that is, the complete task allocation is achieved by executing a series of system actions;

[0009] Step 3: Perform a Monte Carlo tree search to determine the action to be executed: Adopt an adaptive simulation number selection mechanism to determine the number of searches, and repeat the four steps of selection, expansion, simulation, and backtracking until the number of searches reaches a given upper limit. Based on the constructed search tree, select the action corresponding to the optimal child node under the root node as the action to be executed;

[0010] Step 4: Repeat step 3 until all tasks are assigned, and finally output the task assignment results.

[0011] Furthermore, in step (1), it is assumed that the drones depart from and return to the same airport. Each drone can perform multiple tasks, but each task is assigned to only one drone. There are N drones that can perform the tasks. The drone set is represented as , Represents the drone serial number; in the task allocation problem, the set of tasks to be performed is expressed as , Represents the task number; among them, represents task number i, Represents tasks departing from or returning to an airport; the connectivity and distance between any two tasks are expressed as a symmetric matrix Indicates that Indicates a task and The distance between represents task number i, Represents task number j; Indicates from arrive The route is prohibited; each drone has different capabilities and attributes, The flight time, flight speed, reconnaissance capability and rescue capability of a UAV are expressed as ;Task The execution time, reward, search demand and rescue demand are expressed as .

[0012] Furthermore, the maximum task completion time of all drones and the total reward of all unassigned tasks are minimized as the task objectives, and the objective function is expressed as follows:

[0013]

[0014] Among them, γ = {γ0, γ1, γ2,...,γ N} is a set of task sequences assigned to each UAV, γ0 represents the set of all unassigned tasks, and γ i =(τ0,τ1,τ2,...,τ M ) is a drone U i The assigned task sequence, τ represents the task index, τ1,τ2,...,τ M Represents tasks 1, 2…M.

[0015] Furthermore, the flight time and mission execution time of each UAV must be less than its maximum flight endurance time, i.e.

[0016]

[0017] Furthermore, the search and rescue requirements of the mission cannot exceed the capabilities of the UAV, i.e.

[0018]

[0019] Furthermore, in step (2), the system action is designed to select the jth task and assign it to the i-th UAV, where j represents the sequence number of the task among all tasks, and i represents the sequence number of the UAV among all UAVs, and automatically insert the jth task into the optimal position of the current task sequence of the i-th UAV; if the task is assigned to the 0th UAV, it means abandoning the task.

[0020] Furthermore, in step (2), the state is defined as the current task allocation result, corresponding to the task sequence of each UAV, so the initial state is represented by the initial task sequence of each UAV as (T0, T0).

[0021] Furthermore, in step (3), the adaptive simulation quantity selection mechanism sets the upper limit of the Monte Carlo tree search times to N times the product of the number of remaining tasks and the number of remaining drones according to the number of remaining tasks and the number of remaining drones in the current state, where N is a positive integer determined by the computing power of the computing device. The stronger the computing power, the larger the value of N. The default selection is 4.

[0022] Furthermore, in step (3), a Monte Carlo action search tree is constructed for each action to be executed. The root node of the tree is the current task allocation result, that is, the current system state. The edges of the tree represent the actions to be executed, and the nodes of the tree represent the new task allocation state achieved after the action is executed. During the action search process, the four steps of selection, expansion, simulation and backtracking are executed cyclically until the number of searches reaches a given upper limit. Finally, based on the constructed search tree, the action corresponding to the optimal child node under the root node is selected as the action to be executed.

[0023] Furthermore, the simulation starts from the newly expanded node and performs task allocation based on the heuristic simulation strategy until all tasks are allocated, and finally evaluates the quality of the task allocation results.

[0024] Furthermore, the heuristic simulation strategy is a random task allocation strategy designed based on the heuristic information provided by the task allocation objective function and expert experience, which consists of two parts: heuristic sampling of candidate tasks and heuristic sampling of candidate drones.

[0025] The specific method of heuristic sampling of candidate tasks is to calculate the score of each candidate task, use the softmax function to convert the task score into a probability distribution, and then randomly extract the task index according to the distribution.

[0026] Furthermore, the score of a candidate task is determined by the ratio of task reward to time, T j The task score is calculated as follows:

[0027]

[0028] Furthermore, the specific method of heuristic sampling of candidate drones is to calculate the score of each candidate drone, convert the drone score into a probability distribution using the softmax function, and then randomly extract the drone index according to the distribution.

[0029] Furthermore, the score of the candidate drone is mainly determined by the minimum distance between the drone route and the newly selected task. j U j The score is calculated as follows:

[0030]

[0031] Furthermore, the evaluation task assignment results are based on the best result of all simulations as the evaluation benchmark. If the cost of this simulation is smaller than the current benchmark, then this simulation is considered successful and receives 1 point, and the comparison benchmark is updated to the result of this simulation. If the cost of the simulation result exceeds the benchmark range, it is considered a failure and receives 0 points. Finally, the score evaluation of the terminal node is:

[0032]

[0033] Among them, α>1.0 represents the evaluation coefficient, J best Indicates the score of the current best task assignment result.

[0034] Furthermore, in step (4), the task assignment result is selected from two results: one is the action sequence consisting of the optimal actions selected in each round of the Monte Carlo tree search; the other is the optimal sequence obtained from all simulations performed in the Monte Carlo tree search, which is the evaluation benchmark mentioned in the task assignment result evaluation section. By comparing the two types of action sequences (a) and (b), the action sequence with the lower cost is selected and used as the final task assignment result.

[0035] Innovation and its advantages / effects

[0036] 1. The task allocation problem in reconnaissance and rescue is formulated as a sequential decision-making process. By making full use of the heuristic information of the problem, the state space and action space are designed, balancing the completeness of actions and the number of actions. This approach saves computing resources while ensuring the allocation effect.

[0037] 2. An adaptive simulation number selection mechanism is proposed, which can automatically select the number of simulations according to the tasks to be assigned and the number of drones, avoiding a large number of repeated simulations, thereby reducing the algorithm's computational complexity and further improving the computational efficiency of task assignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of the unmanned swarm reconnaissance and rescue task allocation method based on Monte Carlo tree search. DETAILED DESCRIPTION

[0039] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] The method of the present invention can be used for unmanned cluster task allocation in reconnaissance and rescue.

[0041] like Figure 1 As shown, the specific implementation steps of the present invention are as follows:

[0042] Step 1: Mathematical modeling of task allocation problem.

[0043] In step 1, assume that the drones depart from and return to the same airport. Each drone can perform multiple tasks, but each task is assigned to only one drone. There are N drones that can perform the tasks. The set of drones is represented as:

[0044] U={U i},i=1,...,N

[0045] Among them, i represents the serial number of the UAV. The capabilities and attributes of each UAV are different. Its flight time, flight speed, reconnaissance capability and rescue capability are expressed as

[0046] In the task allocation problem, the set of tasks that need to be performed can be expressed as:

[0047] T={T j},j=0,1,....,M

[0048] in, Represents the task number, represents task number i, Represents the tasks of departing from or returning to the airport. The connectivity and distance between any two tasks are expressed as a symmetric matrix express, Indicates a task and The distance between represents task number i, Represents task number j. Indicates from arrive The route is prohibited, the mission The execution time, reward, search demand and rescue demand are expressed as .

[0049] Furthermore, the maximum task completion time of all drones and the total reward of all unassigned tasks are minimized as the task objectives, and the objective function is expressed as follows:

[0050]

[0051] Among them, γ = {γ0, γ1, γ2,...,γ N} is a set of task sequences assigned to each UAV, γ0 represents the set of all unassigned tasks, and γ i =(τ0,τ1,τ2,...,τ M ) is a drone U i The assigned task sequence, τ * Indicates the task index, for example: τ * =1 represents task No. 1.

[0052] Furthermore, to ensure mission completion, the solution must also meet two constraints. The first is the endurance constraint, which ensures that each drone has enough energy to return to the airport. The flight time and mission execution time of each drone must be less than its maximum flight endurance, i.e.

[0053]

[0054] Furthermore, the second is the capability constraint, which means that only suitable drones can be sent to perform specific tasks, that is, the search and rescue requirements of the mission cannot exceed the capabilities of the drone, i.e.

[0055]

[0056] Step 2: Design the system actions and states. The system action is defined as assigning task j to drone i and automatically inserting the jth task into the optimal position of the ith drone's current task sequence. Assigning a task to drone 0 means abandoning the task.

[0057] Furthermore, the state is defined as the current task assignment result, corresponding to the task sequence of each drone. Therefore, the initial state is represented by the initial task sequence of each drone as (T0, T0). Based on the above definitions of action and state, the solution to the task assignment problem can be converted into a series of action sequences. By executing this action sequence, that is, selecting tasks one by one and assigning them to the corresponding drones, the task assignment can be finally completed.

[0058] Step 3: Perform a Monte Carlo tree search to determine the actions to be executed. Specifically, an adaptive simulation quantity selection mechanism is used to set the upper limit of the Monte Carlo tree search to N times the product of the number of remaining tasks and the number of remaining drones in the current state, where N is a positive integer determined by the computing power of the computing device and is set to 4 by default. Based on the above definitions of actions and states, the solution to the task allocation problem can be converted into a series of action sequences. By executing a sequence of tasks, that is, selecting tasks one by one and assigning them to the corresponding drones, task allocation can ultimately be completed.

[0059] Adaptive adjustment of the number of searches: The number of searches represents the number of times the complete selection, expansion, simulation, and backtracking process needs to be performed. Too many searches will result in a long computational time and waste of resources. Too few searches may result in missing important optimal solutions, resulting in reduced solution quality.

[0060] Furthermore, the Monte Carlo tree search constructs a Monte Carlo action search tree for each action to be executed. The root node of the tree is the current task allocation result, that is, the current system state. The edges of the tree represent the actions to be executed, and the nodes of the tree represent the new task allocation state achieved after the action is executed. During the action search process, the four steps of selection, expansion, simulation, and backtracking are executed cyclically until the number of searches reaches a given upper limit. Finally, based on the constructed search tree, the action corresponding to the optimal child node under the root node is selected as the action to be executed.

[0061] Furthermore, the simulation starts from the newly expanded node and performs task allocation based on the heuristic simulation strategy until all tasks are allocated, and finally evaluates the quality of the task allocation results. The present invention has made detailed designs for the heuristic simulation strategy and task allocation result evaluation.

[0062] Furthermore, the heuristic simulation strategy is a random task allocation strategy designed based on the heuristic information provided by the task allocation objective function and expert experience, which consists of two parts: heuristic sampling of candidate tasks and heuristic sampling of candidate drones.

[0063] The specific method of heuristic sampling of candidate tasks is to calculate the score of each candidate task, use the softmax function to convert the task score into a probability distribution, and then randomly extract the task index according to the distribution.

[0064] Furthermore, the score of a candidate task is determined by the ratio of the task reward to the time. The higher the ratio, the higher the score and the greater the probability of being selected. j The task score is calculated as follows:

[0065]

[0066] Furthermore, the specific method of heuristic sampling of candidate drones is to calculate the score of each candidate drone, convert the drone score into a probability distribution using the softmax function, and then randomly extract the drone index according to the distribution.

[0067] Furthermore, the score of the candidate drone is mainly determined by the minimum distance between the drone route and the newly selected task. The smaller the minimum distance, the more likely the candidate drone is to complete the task in a shorter time. j U j The score is calculated as follows:

[0068]

[0069] Furthermore, the task assignment results are evaluated: The purpose of task assignment result evaluation is to evaluate the reward of the final result of each simulation and use this as a basis to update the value or score of the terminal node. During the entire Monte Carlo tree search process, the present invention saves the best result of all simulations as the evaluation benchmark. If the cost of this simulation is smaller than the current benchmark, then this simulation is judged to be successful and receives 1 point, and the comparison benchmark is updated to the result of this simulation; if the cost of the simulation result exceeds the benchmark range, it will be judged to be a failure and receive 0 points. Finally, the score evaluation of the terminal node is:

[0070]

[0071] Among them, α>1.0 represents the evaluation coefficient, J best Indicates the score of the current best task assignment result.

[0072] Step 4: Repeat Step 3 until all tasks have been assigned, and then output the task assignment results. The task assignment results are selected from two options: the action sequence consisting of the optimal actions selected in each round of the Monte Carlo tree search; and the optimal sequence obtained from all simulations performed in the Monte Carlo tree search, which is the evaluation benchmark mentioned in the task assignment result evaluation section. By comparing the two action sequences (1) and (2), the action sequence with the lower cost is selected and used as the final task assignment result.

[0073] Finally, it should be noted that the above examples are only used to illustrate the method of the technology of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary technicians in this field should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. An unmanned swarm reconnaissance and rescue task allocation method based on Monte Carlo tree search, characterized in that: The steps include: Step 1: Mathematical modeling of the task allocation problem: Establish a set of UAVs that can perform tasks and a set of tasks that need to be performed, and clarify the task objectives and task constraints; Step 2: Design system actions and states to transform the task allocation problem into a sequential decision problem, that is, the complete task allocation is achieved by executing a series of system actions; Step 3: Perform a Monte Carlo tree search to determine the action to be executed: Adopt an adaptive simulation number selection mechanism to determine the number of searches, and repeat the four steps of selection, expansion, simulation, and backtracking until the number of searches reaches a given upper limit. Based on the constructed search tree, select the action corresponding to the optimal child node under the root node as the action to be executed; Step 4: Repeat step 3 until all tasks are assigned, and finally output the task assignment results.

2. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: In step (1), it is assumed that the drones depart from and return to the same airport. Each drone can perform multiple tasks, but each task is assigned to only one drone. There are N drones that can perform the tasks. The drone set is represented by , Represents the drone serial number; in the task allocation problem, the set of tasks to be performed is expressed as , Represents the task number; among them, represents task number i, Represents tasks departing from or returning to an airport; the connectivity and distance between any two tasks are expressed as a symmetric matrix Indicates that Indicates a task and The distance between represents task number i, Represents task number j; Indicates from arrive The route is prohibited; each drone has different capabilities and attributes, The flight time, flight speed, reconnaissance capability and rescue capability of a UAV are expressed as ;Task The execution time, reward, search demand and rescue demand are expressed as .

3. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: The maximum task completion time of all drones and the total reward of all unassigned tasks are minimized as the task objectives, and the objective function is expressed as follows: Among them, γ = {γ0, γ1, γ2,...,γ N } is a set of task sequences assigned to each UAV, γ0 represents the set of all unassigned tasks, and γ i =(τ0,τ1,τ2,...,τ M ) is a drone U i The assigned task sequence, τ represents the task index, τ1,τ2,...,τ M Represents tasks 1, 2…M.

4. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: The flight time and mission execution time of each drone must be less than its maximum flight endurance time, i.e.

5. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: The search and rescue requirements of the mission cannot exceed the capabilities of the drone, i.e.

6. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: In step (2), the system action is designed to select the jth task and assign it to the i-th UAV, where j represents the sequence number of the task among all tasks, and i represents the sequence number of the UAV among all UAVs, and automatically insert the jth task into the optimal position of the current task sequence of the i-th UAV; if the task is assigned to the 0th UAV, it means abandoning the task.

7. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: In step (2), the state is defined as the current task allocation result, corresponding to the task sequence of each UAV, so the initial state is represented by the initial task sequence of each UAV as (T0, T0).

8. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1 is characterized in that: In step (3), the adaptive simulation quantity selection mechanism sets the upper limit of the Monte Carlo tree search times to N times the product of the number of remaining tasks and the number of remaining drones according to the number of remaining tasks and the number of remaining drones in the current state, where N is a positive integer determined by the computing power of the computing device. The stronger the computing power, the larger the value of N. The default selection is 4.

9. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: In step (3), a Monte Carlo action search tree is constructed for each action to be executed. The root node of the tree is the current task allocation result, that is, the current system state. The edges of the tree represent the actions to be executed, and the nodes of the tree represent the new task allocation state achieved after the action is executed. During the action search process, the four steps of selection, expansion, simulation and backtracking are executed cyclically until the number of searches reaches a given upper limit. Finally, based on the constructed search tree, the action corresponding to the optimal child node under the root node is selected as the action to be executed.

10. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The simulation starts from the newly expanded node and performs task allocation based on the heuristic simulation strategy until all tasks are allocated, and finally evaluates the quality of the task allocation results.

11. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The heuristic simulation strategy is a random task allocation strategy designed based on the heuristic information provided by the task allocation objective function and expert experience. It consists of two parts: heuristic sampling of candidate tasks and heuristic sampling of candidate drones.

12. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The specific method of heuristic sampling of candidate tasks is to calculate the score of each candidate task, use the softmax function to convert the task score into a probability distribution, and then randomly extract the task index according to the distribution.

13. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The score of a candidate task is determined by the ratio of task reward to time, T j The task score is calculated as follows:

14. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The specific method of heuristic sampling of candidate drones is to calculate the score of each candidate drone, convert the drone score into a probability distribution using the softmax function, and then randomly extract the drone index based on the distribution.

15. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The score of the candidate drone is mainly determined by the minimum distance between the drone route and the newly selected task. j U j The score is calculated as follows:

16. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: The evaluation task assignment results are based on the best result of all simulations. If the cost of this simulation is smaller than the current benchmark, then this simulation is considered successful and gets 1 point, and the comparison benchmark is updated to the result of this simulation. If the cost of the simulation result exceeds the benchmark range, it is considered a failure and gets 0 points. Finally, the score evaluation of the terminal node is: Among them, α>1.0 represents the evaluation coefficient, J best Indicates the score of the current best task assignment result.

17. The unmanned cluster reconnaissance and rescue task allocation method based on Monte Carlo tree search according to claim 1, characterized in that: In step (4), the task assignment result is selected from two results: one is the action sequence consisting of the optimal actions selected in each round of Monte Carlo tree search; the other is the optimal sequence obtained from all simulations performed in Monte Carlo tree search, which is the evaluation benchmark mentioned in the task assignment result evaluation section. By comparing the two types of action sequences (1) and (2), the action sequence with the lower cost is selected and used as the final task assignment result.

Images