A method for searching for a target by a UAV cluster based on human-in-the-loop mixed decision

Abstract

The application discloses a kind of unmanned aerial vehicle cluster target hunting methods based on human-in-the-loop hybrid decision-making.The present application simplifies the regulatory responsibilities of the operator,converts online parameter adjustment into the selection of decision-making tendency preference,significantly reduces the decision-making burden.Meanwhile, the present application divides the group task decision into three stages of search-distribution-execution, and only uses the traditional planning method based on modeling and joint optimization in the first two stages, while using the distributed decision-making method based on MARL in the last execution stage, taking into account the explainability, dynamic adaptability and human-machine collaboration ability of the cluster task decision-making process, effectively improving the execution efficiency and task success rate of the unmanned aerial vehicle cluster in complex task scenarios.

Description

Technical Field

[0001] This invention belongs to the field of UAV swarm collaborative control technology, specifically involving a UAV swarm target search and capture method based on human-in-the-loop hybrid decision-making. Background Technology

[0002] With the rapid development of unmanned systems and multi-agent reinforcement learning (MARL) technologies, drone swarms have been widely used in many complex mission scenarios such as regional reconnaissance, emergency rescue, and security monitoring. Compared with single drones, drone swarms have significant advantages in mission execution efficiency and success rate due to their scale advantage and autonomous group collaboration. The group mission decision-making process in many current drone swarm applications can be abstracted as a repetitive cycle of three stages: "search-assignment-execution". That is, first, a comprehensive search of the mission area is carried out; during or after the search, the discovered mission targets are assigned tasks, and drones are assigned to perform the corresponding task processing tasks.

[0003] Traditional methods solve the swarm mission decisions of drone swarms by accurately modeling the mission scenario and jointly planning. These methods have strong interpretability and relatively stable planning results, making them naturally suitable for scenarios requiring human supervision and intervention. However, as the scale of drone swarms grows, the highly dynamic and adversarial nature of swarm missions will pose a significant challenge to them and also place a huge operational burden on operators.

[0004] MARL learns effective strategies through extensive exploration and interaction of multiple agents in the environment, demonstrating significant advantages in dynamic collaboration and adversarial game theory. It provides a promising solution for group task decision-making of swarm drones. However, its behavioral logic is not transparent, which makes it difficult for operators to predict the risks of agent behavior and implement effective supervision and intervention.

[0005] Therefore, a hybrid decision-making method is needed that effectively combines the advantages of highly controllable regulatory decision-making in traditional methods with the high dynamic scenario adaptability of the MARL method. Summary of the Invention

[0006] To address the issues of excessive operator oversight load in traditional planning methods for UAV swarm missions and the limitations of MARL methods in meeting the requirements for reliable, controllable, and intervention-friendly decision-making processes, this invention aims to provide a UAV swarm target acquisition method based on a human-environment hybrid decision-making approach. This invention simplifies manual oversight of UAV swarms to the selection of decision preferences, reducing the decision-making load. Furthermore, it effectively combines traditional planning methods with MARL distributed decision-making, balancing the interpretability, dynamic adaptability, and human-machine collaboration capabilities of the swarm mission decision-making process. Therefore, the proposed method preserves the dynamic autonomous collaboration capabilities of swarm UAVs while ensuring the safety and controllability of the overall decision-making process, thereby improving the mission success rate and system security of swarm UAVs in complex adversarial scenarios.

[0007] The technical solution adopted in this invention is:

[0008] I. A method for target acquisition in UAV swarms based on human-in-the-loop hybrid decision-making

[0009] Based on the mission area size, mission objective requirements, and drone cluster attributes, a pre-planning scheme for the drone cluster area is constructed using human-machine decision-making and distributed to the drone cluster. The global mission objective information pool and the cluster drone resource pool are initialized. The drone cluster executes the pre-planning scheme and records the status of each drone as idle.

[0010] The system continuously monitors whether the task allocation conditions are triggered. If the conditions are triggered, the system uses human-machine decision-making to allocate tasks, obtains the task allocation results, and sends them to the corresponding drones. Drones that receive task allocations switch from idle state to task ready state.

[0011] Continuously monitor whether the activation conditions for each pursuit sub-task are met. If the activation conditions for a certain pursuit sub-task are met, combine human and machine decision-making to generate the pursuit strategy corresponding to the pursuit sub-task. The drone in the task preparation state corresponding to the pursuit sub-task switches to the pursuit state and executes the pursuit sub-task based on the pursuit strategy until the pursuit sub-task ends and the drone's state is switched to the idle state.

[0012] During the execution of pre-planned schemes, task allocation, and pursuit sub-tasks, the global task target information pool and the cluster drone resource pool are continuously updated.

[0013] The global task target information pool is used to store task target records.

[0014] The process of constructing a pre-planning scheme for the drone swarm area based on the task area size, task objective requirements, and drone swarm attributes, and then distributing it to the drone swarm, includes:

[0015] Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop;

[0016] For human decision-making in the loop, different pre-planning schemes for drone cluster areas are generated according to different decision biases. The pre-planning scheme for drone cluster areas is selected and sent to the drone cluster. If none of the pre-planning schemes for drone cluster areas meet the preset requirements, the planning parameters are adjusted and the pre-planning scheme for drone cluster areas is regenerated until a pre-planning scheme for drone cluster areas that meets the preset requirements is generated and sent to the drone cluster.

[0017] For human-based decision-making, search routes for search drones and standby routes for pursuit drones are generated based on the size of the task area, the attributes of the drone swarm, and the decision biases corresponding to each task objective in the preset task objective requirements. This results in a pre-planned plan for the drone swarm area, which is then distributed to the drone swarm. If the monitoring personnel find that the current pre-planned plan for the drone swarm area does not meet expectations, the pre-planning is redone using human-based decision-making, and a new pre-planned plan for the drone swarm area is distributed to the drone swarm.

[0018] The task allocation conditions include the task allocation time period, changes in the number of task target records in the global task target information pool, changes in the number of resource information records or task status in the cluster UAV resource pool, and manual request instructions.

[0019] The process of using human-machine decision-making to allocate tasks and obtain task allocation results includes:

[0020] Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop;

[0021] For human decision-making in the loop, task allocation is performed based on multi-factor matching cost according to the latest global task objective information pool and the cluster drone resource pool, generating task allocation results with different decision-making tendencies; if none of the pre-planned schemes for all drone cluster areas meet the preset requirements, the task allocation parameters are adjusted and the task allocation results are regenerated until a task allocation result that meets the preset requirements is generated and sent to the drone cluster.

[0022] For human decision-making on the loop, task allocation is performed based on multi-factor matching costs according to the preset decision bias, and task allocation results with corresponding decision bias are generated. If the monitoring personnel find that the current task allocation results do not meet expectations, the task allocation results are re-performed based on human decision-making on the loop and the new task allocation results are sent to the drone cluster.

[0023] The drone that receives the task assignment then switches from an idle state to a task-ready state, including:

[0024] Each drone that receives a task assignment will change its task status from idle to task ready. If it is executing a pre-planned plan for the drone swarm area, it will interrupt the execution of the current pre-planned plan and update its flight path interruption position in the drone swarm resource pool. Each drone will continue to move toward the assigned task target position. If it is on its way back to the preset flight path, it will move directly toward the assigned task target position.

[0025] The activation conditions for each pursuit sub-task include:

[0026] Mission target k and search drone e that matches mission target k n Distance between r perc The effective sensing range radius of the search drone; and the search drone e n Update the record corresponding to task target k five times consecutively in the global task target information pool.

[0027] The process of generating a pursuit strategy corresponding to the pursuit sub-task by combining human and machine decision-making, and then switching the drone in the task preparation state to the pursuit state and executing the pursuit sub-task based on the pursuit strategy, includes:

[0028] Based on the preset decision tendency, a target pursuit strategy corresponding to the pursuit sub-task is generated and the target pursuit strategy is sent to the corresponding drone in the task preparation state. The drone that receives the target pursuit strategy will switch to the pursuit state. During the execution of the pursuit sub-task, the monitoring personnel control the pursuit sub-task by making decisions on the loop.

[0029] II. A computer device

[0030] The device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the UAV swarm target search method based on human-in-the-loop hybrid decision-making.

[0031] III. A computer-readable storage medium

[0032] The medium stores a computer program, which, when executed by a processor, implements the steps of the UAV swarm target search method based on human-in-the-loop hybrid decision-making.

[0033] The present invention has the following beneficial effects:

[0034] 1. This invention simplifies the complex online adjustment and intervention of human-environment parameters into decision-making tendency selection based on preset parameter templates, which significantly reduces the workload of operators.

[0035] 2. This invention combines traditional planning methods with the MARL distributed decision-making method in the group task decision-making process, taking into account both the interpretability of the overall decision-making process and its ability to adapt to complex dynamics.

[0036] 3. This invention does not impose strict constraints on the specific implementation of the flight path planning, task allocation, and target pursuit strategy learning methods for swarm drones, which facilitates cross-scenario deployment and engineering expansion. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention.

[0038] Figure 2 This is a schematic diagram of the pre-planning of the search route for the search drone in an example of the present invention.

[0039] Figure 3 This is a schematic diagram comparing pre-planning schemes for cluster area search with different decision-making tendencies in an example of the present invention.

[0040] Figure 4 This is a schematic diagram of the task allocation result in an example of the present invention.

[0041] Figure 5 This is a schematic diagram of the execution process of the pursuit subtask in an example of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the specific method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making proposed in this invention is as follows:

[0044] First, based on the task area size, task objective requirements, and drone cluster attributes, a pre-planning scheme for the drone cluster area is constructed using human-machine decision-making and distributed to the drone cluster, initializing the global task objective information pool and the cluster drone resource pool; the drone cluster executes the pre-planning scheme and records the status of each drone as idle.

[0045] This invention views the entire area search and target pursuit process as occurring within an H×W rectangular area on a 2D plane. A swarm of drones conducts a comprehensive search for K dynamically located targets with unknown positions within the task area and pursues the discovered targets. All possible target categories and corresponding processing rules are maintained by a target processing rule base.

[0046] The drone swarm consists of N search drones and M pursuit drones. Only the search drones in the swarm possess perception capabilities, enabling them to detect target information (location, speed, type, value, etc.) within their perception range and share this information with the other search and pursuit drones. Each search drone's perception area is a radius of r centered on the search drone. perc The target is a circular area. The pursuit drone has a relatively high speed and low endurance, and is responsible for pursuing and capturing detected targets. After continuously approaching the target at a certain distance, it can successfully capture it.

[0047] The task target processing rule base defines the default processing priority and the required number of drones to be captured based on the category of the task target. This rule base is configured in advance through expert review or experience analysis before the task begins. In this embodiment, the task target type ID is an integer ranging from [0, 4], its processing priority is an integer ranging from [0, 10], and the required number of drones to be captured ranges from [1, 5]. For the processing priority, the higher the number, the greater the priority.

[0048] Table 1 Examples of Task Target Rule Processing Library

[0049]

[0050] Based on the mission area size, mission objective requirements, and drone swarm attributes, a pre-planning scheme for the drone swarm area is constructed using human-machine decision-making and distributed to the drone swarm, including:

[0051] Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop;

[0052] For human decision-making within a loop, different pre-planning schemes for drone swarm areas are generated based on different decision biases, such as... Figure 3 As shown. A pre-planned plan for the drone cluster area is manually selected and sent to the drone cluster; if none of the pre-planned plans for the drone cluster area meet the preset requirements, the planning parameters are manually adjusted and a new pre-planned plan for the drone cluster area is generated, until a pre-planned plan for the drone cluster area that meets the preset requirements is generated and sent to the drone cluster.

[0053] For human decision-making on the loop, the search routes of search drones and the standby routes of pursuit drones are generated based on the size of the task area, the attributes of the drone swarm, and the decision biases corresponding to each task objective in the preset task objective requirements. This results in a pre-planned plan for the drone swarm area, which is then distributed to the drone swarm. If the monitoring personnel find that the current pre-planned plan for the drone swarm area does not meet expectations, the current task process is interrupted, and a new pre-planned plan for the drone swarm area is re-distributed to the drone swarm.

[0054] This invention does not limit the specific planning method for the coverage search route, as long as the output result is the actual route of each search UAV. In this embodiment, a simple implementation based on a bow shape is adopted, based on the sensing range radius r of the search UAV. perc and overlap rate Coverage search routes for the planned area, such as Figure 2 As shown, the spacing d of its bow-shaped flight path search It can be calculated in the following ways:

[0055]

[0056] The greater the overlap, the smaller the distance between search routes, the longer the search time, and the higher the probability of successfully finding the mission target.

[0057] Therefore, different parameter configurations can be used depending on different decision-making tendencies:

[0058] 1) Task time priority: To achieve the fastest single-round coverage search time, the overlap rate can be set to 0 and the search speed can be set to the maximum movement speed.

[0059] 2) Prioritize mission success rate: Pursue the highest target discovery efficiency, with an overlap rate set to 0.3 and a search speed set to the drone's economic cruising speed.

[0060] 3) Balanced configuration: A trade-off is made between target discovery rate and time consumption. The overlap rate can be set to 0.1, and the search speed can be set to the economic cruising speed.

[0061] This invention does not limit the specific planning method of the standby route of the pursuit drones, as long as the output result is the actual route of each pursuit drone.

[0062] In this embodiment, two standby route planning methods for the pursuit drone are set up:

[0063] 1) Dispersed standby: The pursuit drones plan their round-trip routes within the mission area at uniform intervals, and use different flight speeds to form a relatively uniform distribution within the mission area, with route intervals d. wait The calculation formula is as follows:

[0064] d wait =L / (M+1)

[0065] L=max(H,W)

[0066] Where L represents the distance of the longer side of the mission area, which is used to ensure that the length of the pursuit drone's flight path is the length of the shorter side of the mission area, so as to reduce the time interval between the flight path round trip.

[0067] The speed of the pursuit drone will be randomly sampled between 0 and economic cruising speed, thus forming an approximately random and uniform distribution of the pursuit drone in the mission area, making it easier to form an encirclement structure of the mission target in advance when launching pursuit sub-tasks.

[0068] 2) Group Follow-up: The pursuit drones are grouped according to the number of search drones. Each group consists of one search drone and M / N pursuit drones. The search drone acts as the group's navigator, and the pursuit drones form a minimum formation to follow the search drone at the same speed. This scheme is more suitable for handling a small number of sudden mission targets, initiating target pursuit immediately upon the search drone's discovery of the mission target.

[0069] Depending on different decision-making tendencies, different methods can be adopted for planning the standby flight paths of pursuit drones, as follows:

[0070] 1) Prioritize success rate: All pursuit drones are planned in a dispersed standby manner to compress the escape space of the target when initiating pursuit sub-tasks. This is suitable when there are a large number of targets in the current mission and they are widely distributed.

[0071] 2) Rapid response priority: All pursuit drones are planned in a group following manner, which can ensure that there are available pursuit drones around the search drone when a pursuit sub-task is launched;

[0072] 3) Balanced configuration: The drones are divided according to a preset ratio. Some of the pursuit drones adopt a dispersed standby mode, while the remaining pursuit drones adopt a group follow mode.

[0073] The global mission target information pool is used to store mission target records. Operations on these records include update and removal. Each mission target record includes the mission target type ID, location, speed, category, priority, and the number of drones to be pursued.

[0074] Update operation: This actually includes two execution results: adding records and updating records. The location, speed, and category of the target sensed by the search drone are used as input, and matched against existing target records in the current global target information pool. If the categories are the same and the location difference is less than a threshold, the record is considered a successful match, and the matched target record is updated with the current sensed location and speed information. If no matching record is found, a new target record is added to the global target information pool based on the current target information. Its target ID is assigned based on the largest existing target ID in the current global target information pool + 1. Its processing priority and the number of drones to be pursued are obtained from the target processing rule base.

[0075] Elimination operation: When the number of consecutive T task target records in the global task target information pool is reduced by a certain threshold. delIf the time frame is not updated, the record for that task target is directly removed from the global task target information pool, indicating that the task target has been outside the perception range of all search drones for a long time. In this embodiment, let T... del =10s.

[0076] The drone swarm resource pool stores resource information records for all search and pursuit drones in the swarm. Each resource information record includes the drone's ID, category, location, speed, remaining battery life, and mission status. Each drone's initial mission status is idle. Each drone's ID is globally unique and pre-assigned during swarm creation.

[0077] The clustered drone resource pool also supports two types of operations: updating and removing resource information records.

[0078] Resource information record update operation: Each drone finds the corresponding resource information record based on its own ID. If there is no corresponding ID, a new resource information record is added to the cluster drone resource pool based on its own ID, location, speed, remaining battery life, and mission status. If there is a corresponding ID, the current resource information record is overwritten based on its own ID, location, speed, remaining battery life, and mission status.

[0079] Resource information record removal operation: When resource information records in the cluster drone resource pool are continuously removed for T consecutive T... del If the time is not updated, the resource information record is directly removed from the cluster drone resource pool, indicating that the drone has lost its ability to continue performing tasks due to damage, loss of contact, or other reasons.

[0080] During the execution of pre-planned schemes, task allocation, and pursuit sub-tasks, the global task target information pool and the cluster drone resource pool are continuously updated.

[0081] For example, the search drone moves along a pre-planned route and at a search speed, and according to the communication time interval T. comm It continuously updates its own information in the cluster's drone resource pool. When any search drone detects a mission target within its perception range, it immediately updates the global mission target information pool.

[0082] The pursuit drone moves along a pre-planned standby route and at a standby speed, and according to the communication time interval T comm Continuously update its own information in the cluster drone resource pool (location, speed, remaining battery life, mission status).

[0083] In this invention, each drone in the drone swarm has three states:

[0084] 1) Idle state: Not assigned a task target, executing a preset route or en route to return to the preset route;

[0085] 2) Mission preparation state: Move to the assigned mission target location and wait for the pursuit sub-mission to officially begin;

[0086] 3) Pursuit Status: The pursuit sub-task has been initiated. The search drone will continue to follow the mission target; the pursuit drone will attempt to capture the mission target.

[0087] The system continuously monitors whether the task allocation conditions are triggered. If the conditions are triggered, it uses human-machine decision-making to perform task allocation based on multi-factor matching costs, obtains the task allocation results, and sends them to the corresponding drones. Drones that receive task allocations switch from idle state to task ready state.

[0088] Task allocation conditions include the task allocation time period, changes in the number of task target records in the global task target information pool, changes in the number of resource information records or task status in the cluster UAV resource pool, and manual request commands. Details are as follows:

[0089] Task assignment time cycle: T seconds since the last task assignment alloc After a certain time, the task assignment will be retried.

[0090] Changes in the number of task target records in the global task target information pool: When a task target record is added to or removed from the global task target information pool, task allocation is retried (updating the information of the task target record will not trigger this, to avoid overly frequent task allocation).

[0091] Changes in the number of resource information records or task status in the cluster drone resource pool: When the number of drones in the cluster drone resource pool changes due to damage / loss of contact / reinforcement, or when the task status of a drone in the cluster drone resource pool switches back to idle, task allocation is retried.

[0092] Manual request instruction: Based on the current task allocation situation, the supervisory personnel will proactively initiate a task allocation request after adjusting the task allocation parameter configuration.

[0093] In one feasible implementation, task allocation based on multi-factor matching costs is performed using human-machine decision-making to obtain task allocation results, including:

[0094] Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop;

[0095] For human decision-making in the loop, task allocation is performed based on multi-factor matching cost according to the latest global task objective information pool and the cluster drone resource pool, generating task allocation results with different decision-making tendencies; if none of the pre-planned schemes for all drone cluster areas meet the preset requirements, the task allocation parameters are manually adjusted and the task allocation results are regenerated until a task allocation result that meets the preset requirements is generated and sent to the drone cluster.

[0096] For human decision-making on the loop, task allocation is performed based on multi-factor matching cost according to the preset decision bias, and the corresponding task allocation result is generated. If the monitoring personnel find that the current task allocation result does not meet expectations, the current task process is interrupted, the task allocation result is re-performed with human decision-making on the loop, and the new task allocation result is sent to the drone cluster.

[0097] In one feasible implementation, task allocation based on multi-factor matching cost is performed according to a preset decision bias, generating task allocation results corresponding to the decision bias, as follows:

[0098] A multi-factor matching cost is constructed for the pursuit drone and the mission target. The matching relationship between the search drone, the pursuit drone and the mission target is calculated using the Hungarian algorithm based on the virtual mission extension. Specifically:

[0099] For each task objective K represents the number of mission targets, based on the required number of drones to be captured according to the mission target record. Calculate the corresponding task allocation combination. :

[0100]

[0101]

[0102] Among them, e n The search drone assigned to mission objective k Assigned to task target k The drone being pursued by Taiwan, where n represents the number of the search drone assigned to mission target k. This indicates the number of the pursuit drone assigned to mission target k.

[0103] Multi-factor matching cost Composed of multiple cost sub-items, its complete form can be represented as:

[0104]

[0105]

[0106] The details of each sub-item are as follows:

[0107] 1) Distance cost term Based on the location of drone i Position relative to mission target k relative distance calculate, , This indicates a modulo operation. The greater the relative distance, the higher the matching cost, thus encouraging allocation based on proximity.

[0108] 2) Target Priority Items : Set according to the priority of task objectives. The higher the priority, the lower the cost, and high-priority task objectives are encouraged to be processed first.

[0109] 3) Task switching penalty Calculated based on the current drone mission status. When a mission objective has been assigned and its ID does not match the current mission objective ID, costs will increase to encourage drones to maintain the current assignment.

[0110] 4) Battery life redundancy items Drone remaining battery life (s) i Higher than expected battery life The fewer drones, the higher the cost. We encourage prioritizing the use of drones with more remaining battery life to ensure mission success.

[0111] 5) Rule Constraints When the actual matching rules are not met, the cost is set to infinity. Specifically:

[0112] Drone type does not match virtual mission type: The search drone does not match the virtual pursuit mission target, or the pursuit drone does not match the virtual search mission target.

[0113] Insufficient remaining battery life: The drone's remaining battery life is less than the estimated battery life consumed.

[0114] In one feasible implementation, the estimated battery life consumption is... Time can be estimated based on task duration. The maximum available speed of the search and pursuit drones when performing pursuit sub-tasks. The calculation is as follows:

[0115]

[0116]

[0117] in, This refers to the battery life consumption factor. It is a time consumption factor, which is set in advance at the beginning of the task based on practical application experience.

[0118] Weights of the five cost components Different combinations of numerical values ​​can influence the final decision-making tendency of task allocation results.

[0119] Typical decision-making tendencies include:

[0120] 1) Prioritize task time consumption: Increase the weight of distance cost, reduce the weight of task switching penalty and endurance redundancy, prioritize the nearest drone to handle the task target immediately, and drones that fall behind will be replaced by other drones that are closer; in addition, according to the actual task needs, especially for ultra-high value time-sensitive targets, a more aggressive allocation scheme can be adopted, setting the weight of rule constraint items to 0, to fully ensure the nearest and timely response to the task target.

[0121] 2) Prioritize task success rate: Increase the weight of the battery life redundancy item and the task switching penalty item to ensure that the allocation result has sufficient redundancy and maintains the stability of the allocation result, while also facilitating manual decision-making;

[0122] 3) Conservative and stable priority: Increase the weight of distance cost item, adjust the weight of endurance redundancy item to a negative number, minimize endurance consumption as much as possible, and prioritize drones with insufficient remaining endurance to handle tasks, while retaining drones with high endurance.

[0123] In one feasible implementation, the Hungarian algorithm based on virtual task extension is specifically as follows:

[0124] Each task objective requires the allocation of one search drone and at least one pursuit drone. To enable the Hungarian algorithm, which is only applicable to bipartite graph matching, to support many-to-one matching in heterogeneous clusters, this invention employs virtual task expansion. For each recorded task objective k in the global task objective information pool, it is expanded to... A virtual task objective is used to construct the multi-factor matching cost.

[0125] More specifically, the present invention first analyzes the global task target information pool. The mission objectives The tasks are sorted according to processing priority, and then the recorded task objectives are expanded into virtual tasks in descending order of processing priority. Each task objective is expanded into one virtual search task objective and H. k A virtual pursuit mission target. When the total number of virtual search mission targets after this expansion exceeds the cluster drone resource pool. The total number of search drones or the total number of virtual pursuit targets is greater than the drone resource pool. When the total number of pursuit drones is determined, the virtual mission extension for that mission objective is canceled, and mission assignment is only performed on mission objectives that have already been extended.

[0126] like Figure 4 As shown, the solid line represents the currently applied task allocation scheme, which prioritizes task success rate; while the dashed line represents the task allocation scheme that prioritizes task time. Because the weight of the task switching penalty is reduced, the drone may change its current target as the drone's position changes during the pursuit sub-task.

[0127] In one feasible implementation, the drone that receives the task assignment switches from an idle state to a task-ready state, including:

[0128] Each drone that receives a task assignment will change its task status from idle to task ready. If it is executing a pre-planned plan for the drone swarm area, it will interrupt the execution of the current pre-planned plan and update its flight path interruption position in the drone swarm resource pool. Each drone will continue to move toward the assigned task target position. If it is on its way back to the preset flight path, it will move directly toward the assigned task target position.

[0129] The system continuously monitors whether the activation conditions for each pursuit sub-task are met. If the activation conditions for a certain pursuit sub-task are met, the system combines human and machine decision-making to generate the pursuit strategy corresponding to that pursuit sub-task. The drone in the task preparation state corresponding to that pursuit sub-task switches to the pursuit state and executes the pursuit sub-task based on the pursuit strategy until the pursuit sub-task ends and the drone's state is switched to the idle state.

[0130] In one feasible implementation, the activation condition for each pursuit sub-task must meet the following two sub-conditions:

[0131] 1) Mission target k and the search drone e that matches mission target k n Distance between r perc The effective sensing range radius of the search drone;

[0132] 2) Search drones n Update the record corresponding to task target k five times consecutively in the global task target information pool.

[0133] Once the conditions for initiating the pursuit mission are met, a multi-target pursuit sub-task is initiated, and the corresponding search drone and pursuit drone mission status is switched to pursuit status and updated in the cluster drone resource pool.

[0134] In one feasible implementation, a pursuit strategy corresponding to the pursuit sub-task is generated by combining human and machine decision-making. The drone in the task preparation state corresponding to the pursuit sub-task switches to the pursuit state and executes the pursuit sub-task based on the pursuit strategy, including:

[0135] Based on the preset decision tendency, a target pursuit strategy corresponding to the pursuit sub-task is generated and the target pursuit strategy is sent to the corresponding drone in the task preparation state. The drone that receives the target pursuit strategy will switch to the pursuit state.

[0136] The target pursuit sub-task is executed. The search drone continuously moves towards the target and shares the target's location with the pursuit drone. The pursuit drone loads the corresponding target pursuit strategy model based on the pursuit decision preferences in the target record and performs pursuit control based on local observations to continuously pursue the target.

[0137] The target pursuit strategy model is a target pursuit strategy model trained in advance using a multi-agent reinforcement learning method. This invention does not restrict the specific training method of the target pursuit strategy model, only requiring that the strategy model be trained using a centralized training distributed execution (CTDE) framework, taking the local observation results of the pursuing drone as input, and the drone's desired acceleration, velocity, or position as output, thus completing the motion control of the pursuit process. In this embodiment, the MAPPO algorithm is used to train the multi-drone target pursuit strategy model. During training, all pursuing drones share the same Actor network based on local observations and make action decisions, while a centralized Critic network based on global observations performs state value evaluation. Multiple Actor networks are trained using various combinations of reward function parameters and subjected to horizontal comparative evaluation. Several Actor networks with different decision-making tendencies are selected as pursuit strategy models and saved in a strategy model library. Figure 5 This invention demonstrates the process by which the present invention utilizes a pre-trained multi-machine target pursuit strategy model to execute pursuit sub-tasks.

[0138] More specifically, the local observations of each Actor network include:

[0139] 1) Tracking the drone's own position and speed;

[0140] 2) The relative positions, relative speeds, and relative distances of K adjacent pursuit drones;

[0141] 3) The relative position, relative speed, and relative distance of the mission objective.

[0142] The output of each Actor network is the normalized expected velocity direction of the drone. By scaling the maximum speed of the pursuing drone, the actual speed that the pursuing drone needs to take is obtained.

[0143] The strategy model library contains at least two classes of models with different decision-making tendencies. These different models differ in at least one of the metrics: pursuit efficiency, energy consumption control, and safety maintenance. For example:

[0144] 1) Task time consumption preference: The target has a shorter average pursuit step length, prefers to use a larger pursuit speed, and has a lower pursuit success rate and survival rate;

[0145] 2) Task success rate preference: The average pursuit step length of the target is relatively long, the pursuit speed is moderate, but the pursuit success rate and survival rate are high;

[0146] 3) Conservative safety preference: Prioritize the survival rate of the pursuit drones, and limit the pursuit success rate and average pursuit step length.

[0147] It supports switching strategy models during mission execution. When switching, it inherits the current pursuit sub-task environment state, the state of adjacent pursuit drones, and the state of the mission target, and the new pursuit strategy model takes effect in the next control cycle, continuing to make action decisions based on the constructed local observation information.

[0148] Because the target pursuit sub-task has extremely dynamic characteristics during execution, human decision-making within the loop is insufficient to meet the time-sensitive requirements of the actual task. Therefore, during the execution of this pursuit sub-task, monitoring personnel adopt a human-on-the-loop decision-making approach to control the pursuit sub-task. Specifically:

[0149] 1) Temporarily change the decision-making model and continue to execute the pursuit sub-task by adopting a strategy model with other decision-making tendencies;

[0150] 2) Interrupt the current pursuit subtask, readjust the priority, number of drones to be pursued, and pursuit strategy preference fields of the task target recorded in the global task target information pool, and reassign the task only to the task target.

[0151] 3) Cancel the current pursuit sub-task, temporarily hide its corresponding task target record in the global task target information pool, and allow it to participate in task allocation again after the response method is determined later;

[0152] More specifically, the present invention estimates the time consumption of tasks. As a reference, when the execution time of a pursuit sub-task exceeds the estimated time, an alert will be automatically issued to request human-machine decision-making, and the regulator will take corresponding measures.

[0153] In one feasible implementation, the termination condition for the pursuit sub-task is specifically as follows:

[0154] 1) An arbitrary pursuit drone approaches the mission target D. cap Continuous T within the distance range cap The time frame is considered as the time when the pursuit drone successfully locks onto and captures the target.

[0155] 2) If the mission target escapes the perception area of ​​the search drone and is removed from the global mission target information pool, the pursuit is considered a failure;

[0156] 3) If the pursuit mission lasts longer than the maximum pursuit time limit, it will be considered a failed pursuit;

[0157] 4) If the number of available pursuit drones in the current subtask is 0 due to damage / loss of contact, the pursuit is considered a failure.

[0158] 5) Manually canceling this pursuit sub-task will be considered a failed pursuit.

[0159] For a successful pursuit sub-task, this invention removes the corresponding ID of the task target record from the global task target information pool, switches the task status of the corresponding assigned search drone and pursuit drone to the idle state, and returns the route interruption position recorded in the cluster drone resource pool to re-execute the pre-planned route.

[0160] For pursuit sub-tasks that fail to be completed, this invention will adopt different processing methods depending on the actual termination method.

[0161] For situations where "the pursuit mission execution time exceeds the maximum pursuit time limit" and "the number of available pursuit drones is 0," a human-machine decision-making process is introduced. This adjusts the priority of the mission objective, the required number of pursuit drones, and the pursuit strategy bias. The corresponding search and pursuit drones are then switched to an idle state, and the mission allocation is re-executed. This supports both human-on-the-loop and human-within-the-loop modes.

[0162] Human-based decision-making on the loop: automatically increasing the processing priority in the record corresponding to the task objective, the number of drones to be captured, and selecting a strategy model with a preference for capture success rate;

[0163] Human decision-making in the loop: Manually adjust the processing priority, the number of drones to be pursued, pursuit strategy preferences, or temporarily block the record of the task target in the corresponding record.

[0164] In the case of "the mission target escaping the search drone's perception area", since the mission target has been removed from the global mission target information pool, there is no need to execute the mission target.

[0165] In the case of "manually canceling this pursuit sub-task", if the human's decision in the loop is to temporarily block the target of the task, then the pursuit sub-task will not be triggered again; otherwise, the pursuit will still be carried out again.

[0166] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention in any way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent examples. However, any simple modifications, equivalent changes, and modifications made to the above examples based on the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for target acquisition in a drone swarm based on human-in-the-loop hybrid decision-making, characterized in that, Includes the following steps: Based on the mission area size, mission objective requirements, and drone cluster attributes, a pre-planning scheme for the drone cluster area is constructed using human-machine decision-making and distributed to the drone cluster, and the global mission objective information pool and cluster drone resource pool are initialized. The drone swarm executes a pre-planned scheme and records the status of each drone as idle; The process of constructing a pre-planning scheme for the drone swarm area based on the task area size, task objective requirements, and drone swarm attributes, and then distributing it to the drone swarm, includes: Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop; For human decision-making in the loop, different pre-planning schemes for drone cluster areas are generated according to different decision biases. The pre-planning scheme for drone cluster areas is selected and sent to the drone cluster. If none of the pre-planning schemes for drone cluster areas meet the preset requirements, the planning parameters are adjusted and the pre-planning scheme for drone cluster areas is regenerated until a pre-planning scheme for drone cluster areas that meets the preset requirements is generated and sent to the drone cluster. For human decision-making on the loop, the search routes of search drones and the standby routes of pursuit drones are generated based on the size of the task area, the attributes of the drone swarm, and the decision biases corresponding to each task objective in the preset task objective requirements. This results in a pre-planned plan for the drone swarm area, which is then distributed to the drone swarm. If the monitoring personnel find that the current pre-planned plan for the drone swarm area does not meet expectations, the pre-planning is redone using human decision-making on the loop, and a new pre-planned plan for the drone swarm area is distributed to the drone swarm. The system continuously monitors whether the task allocation conditions are triggered. If the conditions are triggered, the system uses human-machine decision-making to allocate tasks, obtains the task allocation results, and sends them to the corresponding drones. Drones that receive task allocations switch from idle state to task ready state. Continuously monitor whether the activation conditions for each pursuit sub-task are met. If the activation conditions for a certain pursuit sub-task are met, combine human and machine decision-making to generate the pursuit strategy corresponding to the pursuit sub-task. The drone in the task preparation state corresponding to the pursuit sub-task switches to the pursuit state and executes the pursuit sub-task based on the pursuit strategy until the pursuit sub-task ends and the drone's state is switched to the idle state. During the execution of pre-planned schemes, task allocation, and pursuit sub-tasks, the global task target information pool and the cluster drone resource pool are continuously updated.

2. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The global task target information pool is used to store task target records.

3. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The task allocation conditions include the task allocation time period, changes in the number of task target records in the global task target information pool, changes in the number of resource information records or task status in the cluster UAV resource pool, and manual request instructions.

4. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The process of using human-machine decision-making to allocate tasks and obtain task allocation results includes: Human-machine decision-making includes human decision-making within a loop and human decision-making on a loop; For human decision-making in the loop, task allocation is performed based on multi-factor matching cost according to the latest global task objective information pool and the cluster drone resource pool, generating task allocation results with different decision-making tendencies; if none of the pre-planned schemes for all drone cluster areas meet the preset requirements, the task allocation parameters are adjusted and the task allocation results are regenerated until a task allocation result that meets the preset requirements is generated and sent to the drone cluster. For human decision-making on the loop, task allocation is performed based on multi-factor matching costs according to the preset decision bias, and task allocation results with corresponding decision bias are generated. If the monitoring personnel find that the current task allocation results do not meet expectations, the task allocation results are re-performed based on human decision-making on the loop and the new task allocation results are sent to the drone cluster.

5. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The drone that receives the task assignment then switches from an idle state to a task-ready state, including: Each drone that receives a task assignment will change its task status from idle to task ready. If it is executing a pre-planned plan for the drone swarm area, it will interrupt the execution of the current pre-planned plan and update its flight path interruption position in the drone swarm resource pool. Each drone will continue to move toward the assigned task target position. If it is on its way back to the preset flight path, it will move directly toward the assigned task target position.

6. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The activation conditions for each pursuit sub-task include: Mission target k and search drone e that matches mission target k n Distance between r perc The effective sensing range radius of the search drone; and the search drone e n Update the record corresponding to task target k five times consecutively in the global task target information pool.

7. The method for UAV swarm target acquisition based on human-in-the-loop hybrid decision-making according to claim 1, characterized in that, The process of generating a pursuit strategy corresponding to the pursuit sub-task by combining human and machine decision-making, and then switching the drone in the task preparation state to the pursuit state and executing the pursuit sub-task based on the pursuit strategy, includes: Based on the preset decision tendency, a target pursuit strategy corresponding to the pursuit sub-task is generated and the target pursuit strategy is sent to the corresponding drone in the task preparation state. The drone that receives the target pursuit strategy will switch to the pursuit state. During the execution of the pursuit sub-task, the monitoring personnel control the pursuit sub-task by making decisions on the loop.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the unmanned aerial vehicle swarm target search method based on human-in-the-loop hybrid decision-making as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the UAV swarm target search method based on human-in-the-loop hybrid decision-making as described in any one of claims 1 to 7.

Images