Multi-unmanned aerial vehicle search and rescue path planning method based on COMA method
Through multi-UAV search and rescue path planning based on the COMA method, the critic and executor network architecture is utilized, combined with the counterfactual baseline, the path planning problem of multi-UAV systems in dynamic environments is solved, and efficient search and rescue missions and good system adaptability are achieved.
Patent Information
- Application Number
- CN202510606161.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-12
- Publication Date
- 2025-09-16
AI Technical Summary
It is difficult for multi-UAV systems to achieve efficient path planning with real-time, dynamic and collaborative decision-making in complex and dynamic post-disaster environments. Existing methods are prone to falling into local optimality or high computational complexity, and are difficult to adapt to large-scale dynamic environments.
A multi-UAV search and rescue path planning based on the COMA method is adopted. Through offline centralized training and online distributed deployment, the critic and executor network architecture is utilized, combined with counterfactual baselines, to evaluate the contribution of UAVs and optimize path planning to achieve adaptive collaborative cooperation.
It improves the search and rescue efficiency, coverage and energy consumption of multi-UAV systems in dynamic environments, shows strong robustness and flexibility, adapts to different communication constraints and UAV scales, and has good generalization capabilities.
Smart Images
Figure CN120652990A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of path planning, and in particular to a multi-UAV search and rescue path planning method based on a COMA method. Background Art
[0002] In recent years, with the rapid development of drone technology, its application areas have expanded to multiple scenarios, including firefighting, military operations, agriculture, environmental monitoring, and disaster search and rescue. In particular, in post-disaster search and rescue and wide-area environmental monitoring missions, multi-UAV systems (MUVs) can significantly improve spatial coverage, flexibility, and robustness against single-point failures through collaborative operations. However, to maximize mission efficiency and ensure safety in complex and dynamic environments, path planning for MUVs still faces numerous challenges, particularly in balancing real-time, dynamic, and collaborative decision-making requirements.
[0003] The core goal of UAV path planning is to design the optimal trajectory for a multi-UAV system, enabling it to safely avoid obstacles while maximizing mission benefits during mission execution. Traditional path planning methods include artificial potential field (APF), heuristic search algorithms (such as the A* algorithm and the Dijkstra algorithm), and intelligent optimization algorithms (such as the ant colony algorithm and the genetic algorithm). Among them, the APF method has good real-time performance due to its simple calculation, but it is prone to falling into local optimality, resulting in mission failure; the heuristic search method can find the optimal path in specific scenarios, but the computational complexity is high and it is difficult to adapt to large-scale dynamic environments; although the intelligent optimization algorithm has strong global search capabilities, it is prone to falling into local optimality and the generated path is relatively tortuous. Summary of the Invention
[0004] The present invention provides a multi-UAV search and rescue path planning method based on the COMA method, which can overcome certain or some defects of the prior art.
[0005] According to the present invention, a multi-UAV search and rescue path planning method based on the COMA method includes the following steps:
[0006] The offline centralized training phase includes a coverage planning layer and a path planning layer. The coverage planning layer uses a K-means algorithm to rapidly cover the disaster area based on a real-time information entropy and uncertainty map of the disaster area. The path planning layer is based on the COMA reinforcement learning algorithm, extending and applying it to the UAV team's information path planning task. It adopts a centralized critic and decentralized actor framework and introduces a counterfactual baseline to clearly measure each UAV's contribution to the overall mission, thereby achieving collaborative adaptive path planning.
[0007] Online distributed deployment: After offline training, the trained executor network is deployed to the actual drone platform, enabling each drone to independently determine the optimal path based on local observations, achieving decentralized adaptive map uncertainty reduction. During execution, each drone calculates information gain or entropy reduction based on the continuously updated local posterior map and uses this as an important input to the policy network, enabling adaptive attention to high-information areas and dynamic measurement point adjustment.
[0008] Preferably, in the offline centralized training step:
[0009] The critic network uses globally visible information to evaluate the value of each drone's action and estimate the state-action value function;
[0010] The executor network only relies on its own local map observations and historical measurement data to output the next feasible measurement location;
[0011] By introducing a counterfactual baseline, while keeping the actions of other drones unchanged, the change in the team value function after replacing the current drone action is estimated, thereby quantifying the contribution of a single drone to the overall goal.
[0012] As a preference, in the COMA reinforcement learning algorithm:
[0013] The critic network input includes the global state s and the action information of other drones, and then evaluates the joint state-action value function Q π (s,u):
[0014] Q π (s,u)=Q π (s,(u1,...,u i ,...,u N ))
[0015] Where s is the state parameter, u is the action parameter; π (s,u) represents the expected cumulative discounted return that can be obtained by continuing to act according to the current strategy π after executing the joint action u in state s; the critic network uses the TD(λ) method to calculate the discounted return G t Perform online estimation to provide an evaluation basis for strategy optimization; the TD(λ) method estimates the target value G of the critic network t λ , defined as: Where λ∈[0,1] is the attenuation parameter, G t (n) is the n-step return;
[0016] The actor network is based on its local observation information ω iCalculate the action probability distribution π(u i |ω i ):
[0017]
[0018] Where ∈ is the exploration parameter, which is used to adjust the exploration degree during training and is set to zero during deployment; U represents the optional action of a single UAV, For the executor network function.
[0019] Preferably, the input to the executor network includes:
[0020] UAV identifier i and remaining mission budget b; a location map constructed with the current location of the UAV as the center, encoding the location and altitude information of other UAVs within the map boundary and communication range; local map state M i , the environment is represented by a grid map; the weighted entropy H(M|z 0:t ,p 0:t ):
[0021]
[0022] Among them, M j represents the jth cell in the discrete grid, p(M j ) is the posterior probability that the grid unit is in the target category, W(M j ) is a weighting coefficient related to the importance of the grid cell, which is used to prioritize and pay more attention to high-value areas in the decision-making process.
[0023] To guide the drone's focus on "potential areas of interest," we introduce an importance-weighted entropy function. This function assigns different information value weights to regions based on their posterior probabilities, thereby driving the policy network to prioritize high-value paths. w1, w2 > 0 are importance weights; weights are assigned based on the current probability threshold:
[0024]
[0025] Footprint map consisting of the field of view coverage areas of all drones within the communication range.
[0026] Preferably, the input of the critic network includes all the inputs of the performer network and additionally includes:
[0027] A global position map encoding the positions of all drones Global map state M; weighted entropy of global map state H(M|z 0:t ,p 0:t ); the map area covered by all drones’ field of view; and the movement information of other drones.
[0028] As a preference, the information theory criterion I(ψ t+1 ) is used to evaluate the UAV path, and the criterion is calculated as the sum of the entropy reduction along the path:
[0029]
[0030] Where B is the maximum number of measurement steps allowed for the task, z 0:t+m and p 0:t+m Represent all measurement data and measurement positions from the initial moment to the t+mth step respectively.
[0031] As a preferred method, the following reward function is used for drone team training:
[0032]
[0033] Here, H(·) represents the weighted calculation of map entropy, α and β are the scaling and translation factors used to stabilize training. The reward function can encourage the drone to focus on high-value areas and, through normalization, keep the reward relatively stable throughout the mission.
[0034] Preferably, the COMA reinforcement learning algorithm introduces an advantage function to evaluate the contribution of individual drones in team actions:
[0035] Advantage function of drone i when the team performs joint action u:
[0036]
[0037] Among them, Q π (s,u) represents the state-action value of the team performing the joint action in the state; Represents all possible actions Traverse; use the current strategy Give each possible action a weight; use Q value If the actions of other drones remain unchanged, only the action of the current drone will be changed. The total team reward at this time; finally, perform weighted average to obtain the expected Q value of the action under the current strategy;
[0038] According to the policy gradient theorem, the parameter update gradient of each drone is expressed as:
[0039]
[0040] Using negative log-likelihood loss function:
[0041]
[0042] The optimization process is updated via mini-batch stochastic gradient descent.
[0043] As a preference, drones use a limited range communication protocol:
[0044] When the distance between two drones i and k When ,they share measurement data;
[0045] The communication radius D is set to 25 meters unless otherwise specified;
[0046] The receiving drone uses the received measurement data and location information Updates its local map beliefs.
[0047] As a preferred method, the network architecture for processing spatial information features is:
[0048] Both the actor and critic networks consist of convolutional encoders and linear layers;
[0049] The convolutional layer is followed by a ReLU activation function, and the linear layer is also followed by a ReLU activation function;
[0050] The output layer of the executor network is a softmax layer, which generates action probability distribution.
[0051] The beneficial effects of the present invention are as follows:
[0052] This paper proposes a COMA-based adaptive path planning method for multiple UAVs. This method optimizes UAV task allocation and path selection through a counterfactual benchmark mechanism, addressing the credit allocation challenge in multi-UAV collaborative planning. Experimental results in various environmental scenarios demonstrate that compared to traditional algorithms and other reinforcement learning methods, the proposed method outperforms traditional algorithms in search and rescue efficiency, coverage, and energy consumption, demonstrating particularly strong robustness and flexibility in dynamic environments. Furthermore, experiments demonstrate that the proposed planning method generalizes well under various communication constraints and at varying UAV scales. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 Flowchart of a multi-UAV search and rescue path planning method based on the COMA method in an embodiment. DETAILED DESCRIPTION
[0054] In order to further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are merely for explaining the present invention and are not intended to limit the present invention.
[0055] Example
[0056] like Figure 1As shown, this embodiment provides a multi-UAV search and rescue path planning method based on the COMA method, which includes two stages: offline centralized training and online distributed deployment;
[0057] During the offline centralized training phase, a two-layer collaborative architecture consisting of a coverage planning layer and a path planning layer is constructed, corresponding to the two key aspects of macro-task allocation and micro-path execution, respectively. The coverage planning layer focuses on information-value-driven task point extraction and region division. Based on a real-time map of the disaster area's information entropy and uncertainty, the search and rescue area or key points within the area are used as data points for the K-means algorithm, with the number of available drones serving as the K value. The cluster centroids obtained after the algorithm runs serve as the initial assembly points for drones or the centers of their assigned areas. Each cluster represents the search area or target point set assigned to the corresponding drone. The path planning layer utilizes a multi-agent deep reinforcement learning framework, taking into account multiple sources of information, including environmental conditions, obstacle distribution, and neighboring drone behavior. The COMA algorithm is used to optimize collaborative path generation, ensuring safe obstacle avoidance while maintaining high information collection value and smooth execution. The two-layer algorithm achieves coordinated design and efficient collaboration through a unified policy interface and state information feedback mechanism. Based on the COMA reinforcement learning algorithm, this paper extends and applies it to the informational path planning task of UAV teams. It adopts a centralized critic and decentralized actor framework and introduces a counterfactual baseline to clearly measure the contribution of each UAV to the overall task, thereby achieving collaborative adaptive path planning.
[0058] Online distributed deployment: After offline training, the trained actor network is deployed to the actual drone platform. During mission execution, each drone independently determines the optimal action based on its local state (including its local map, a posteriori estimates from multiple fusion measurements, and measurements shared by other drones within limited communication range). This allows the system to achieve adaptive post-disaster rescue in a distributed manner, even in environments with imperfect communications or bandwidth / latency constraints.
[0059] In the offline centralized training step:
[0060] The critic network uses globally visible information to evaluate the value of each drone's action and estimate the state-action value function. The executor network relies only on its own local map observations and historical measurement data to output the next feasible measurement position. By introducing a counterfactual baseline, while keeping the actions of other drones unchanged, the change in the team value function after replacing the current drone's action is estimated, thereby quantifying the contribution of a single drone to the overall goal.
[0061] The COMA algorithm is an actor-critic algorithm in which a critic network evaluates each UAV's behavior using global information, while an actor network makes decentralized decisions based on local information. The core idea of the COMA algorithm is to use a counterfactual baseline to assign credit to each UAV to better measure its contribution to the overall team mission goal, thereby promoting collaborative behavior within the UAV team.
[0062] During the training process of the COMA reinforcement learning algorithm, in order to make full use of the global information available during training, the critic network input includes the global state s (such as the global map state, the positions of all drones and other global features) and the action information of other drones, and then evaluates the joint state-action value function Q π (s,u):
[0063] Q π (s,u)=Q π (s,(u1,...,u i ,...,u N ))
[0064] Where s is the state parameter, u is the action parameter; π (s,u) represents the expected cumulative discounted return that can be obtained by continuing to act according to the current strategy π after executing the joint action u in state s; the critic network uses the TD(λ) method to calculate the discounted return G t Perform online estimation to provide an evaluation basis for strategy optimization; the TD(λ) method estimates the target value G of the critic network t λ , defined as: Where λ∈[0,1] is the attenuation parameter, G t (n) is the n-step return;
[0065] The actor network is based on its local observation information ω i (e.g. local map status, drone identification, remaining mission budget and spatial features) to calculate the action probability distribution π(u i |ω i ):
[0066]
[0067] Where ∈ is the exploration parameter, which is used to adjust the exploration degree during training and is set to zero during deployment; U represents the optional action of a single UAV, For the executor network function.
[0068] The inputs to the executor network include:
[0069] UAV identifier i and remaining mission budget b; a location map constructed with the current location of the UAV as the center, encoding the location and altitude information of other UAVs within the map boundary and communication range; local map state M i , the environment is represented by a grid map; the weighted entropy H(M|z 0:t ,p 0:t ):
[0070]
[0071] Among them, M j represents the jth cell in the discrete grid, p(M j ) is the posterior probability that the grid cell is in the target category (e.g., there is a trapped person), W(M j ) is a weighted coefficient related to the importance of the grid cell, which is used to prioritize high-value (higher target probability) areas during the decision-making process. w1, w2>0 are importance weights; this weight can be assigned based on the current probability threshold:
[0072]
[0073] Footprint map consisting of the field of view coverage areas of all drones within the communication range.
[0074] The input of the critic network includes all the inputs of the performer network, and additionally includes:
[0075] A global position map encoding the positions of all drones
[0076] Global map state M;
[0077] The weighted entropy of the global map state H(M|z 0:t ,p 0:t );
[0078] All map areas covered by drone vision;
[0079] Action information of other drones.
[0080] Using the information theory criterion I(ψ t+1 ) is used to evaluate the UAV path, and the criterion is calculated as the sum of the entropy reduction along the path:
[0081]
[0082] Where B is the maximum number of measurement steps allowed for the task, z 0:t+m and p 0:t+m Represent all measurement data and measurement positions from the initial moment to the t+mth step respectively.
[0083] The following reward function is used for drone team training:
[0084]
[0085] Here, H(·) represents the weighted calculation of map entropy, α and β are the scaling and translation factors used to stabilize training. The reward function can encourage the drone to focus on high-value areas and, through normalization, keep the reward relatively stable throughout the mission.
[0086] In the COMA reinforcement learning algorithm, in order to encourage cooperative behavior of the UAV team, an advantage function is introduced to evaluate the contribution of individual drones in the team action. Specifically, the advantage function of drone i when the team performs a joint action u is:
[0087]
[0088] Among them, Q π (s,u) represents the state-action value of the team performing the joint action in the state; Represents all possible actions Traverse; use the current strategy Give each possible action a weight; use Q value If the actions of other drones remain unchanged, only the action of the current drone will be changed. The total team reward at this time; finally, perform weighted average to obtain the expected Q value of the action under the current strategy;
[0089] According to the policy gradient theorem, the parameter update gradient of each drone is expressed as:
[0090]
[0091] Using the negative log-likelihood loss function L:
[0092]
[0093] The optimization process is updated via mini-batch stochastic gradient descent.
[0094] The communication protocol between drones uses a limited communication range:
[0095] When the distance between two drones i and k When ,they share measurement data;
[0096] The communication radius D is set to 25 meters unless otherwise specified;
[0097] The receiving drone uses the received measurement data and location information Updates its local map beliefs.
[0098] The network architecture for processing spatial information features is:
[0099] Both the actor and critic networks consist of convolutional encoders and linear layers;
[0100] The convolutional layer is followed by a ReLU activation function, and the linear layer is also followed by a ReLU activation function;
[0101] The output layer of the executor network is a softmax layer, which generates action probability distribution.
[0102] The drone’s action space consists of discrete 3D actions {up, north, east, south, west, down} with a fixed step size, the altitude range is limited to between 5 and 15 meters, and the system prevents the drone from moving outside the environment boundaries or having the same 2D ground coordinates.
[0103] Experimental setup
[0104] In this example, a post-disaster search and rescue scenario was simulated, in which a drone team was tasked with covering a specific area to search for survivors. Each mission was performed 50 times, and the "area of interest" (i.e., the area where survivors might be found) changed dynamically during the mission, simulating the uncertainty in the distribution of survivors in a real post-disaster environment.
[0105] To ensure effective search and rescue missions, the simulated environment was a 50m x 50m debris area, with a map resolution of 10cm and a planning resolution of 5m. To ensure mission accuracy and data quality, the drones' flight altitude was limited to between 5m and 15m. Each drone's camera had a 60° field of view, ensuring that adjacent measurement points did not overlap at the minimum flight altitude.
[0106] Considering that different UAV flight altitudes will cause different sensor noises, sensor noise models at different altitudes are set: at an altitude of 5 meters, the measurement accuracy is 0.99, which drops to 0.735 at 10 meters and drops to 0.625 at 15 meters.
[0107] The experiment used four drones, with a communication radius set to 25 meters, ensuring real-time sharing of their respective measurements. When the measurement area is repeatedly surveyed by different drones, areas likely to contain survivors can be quickly identified. To evaluate the effectiveness of path planning, two key metrics were used: map entropy to measure the uncertainty of the search and rescue area, and the F1-score to measure map accuracy, reflecting the system's effectiveness in post-disaster search and rescue missions.
[0108] Research and comparison of credit allocation mechanisms
[0109] In multi-UAV post-disaster search and rescue missions, the credit allocation mechanism is crucial, especially when multiple UAVs work together in the same mission. It is necessary to effectively allocate the contribution of each UAV to ensure that they can fully cooperate and avoid duplication and waste of resources.
[0110] To this end, this embodiment employs a counterfactual baseline mechanism, assigning credit during training by evaluating each drone's contribution to the overall mission success. This mechanism enables each drone to not only focus on its own target area but also rationally plan its path based on global information, thereby improving the team's collaboration.
[0111] First, in the input feature ablation experiment, the impact of different input features of the executor network on system performance was evaluated by removing or adding them. The results show that "local map weighted entropy" is the most critical input affecting path planning performance, which provides an important basis for the value of local information. Footprint map input plays a positive role in avoiding path overlap in the later stages of the task, reflecting the "redundant control" mechanism in team collaboration; while the entropy input of the current measurement point can accelerate the reduction of uncertainty in the early stages of the task, if used alone it will cause myopic bias in the strategy, affecting the overall completion quality of the task. Therefore, the input feature combination designed by the present invention achieves an effective trade-off between maintaining network decision stability and task effectiveness.
[0112] Secondly, in a comparative experiment on credit allocation mechanisms, the proposed method was compared with three improved strategies: (1) Central-QV (using only global state values); (2) Actor-Independent (considering only individual actions); and (3) Decentralized (not introducing any global information). Experimental results show that the COMA algorithm using the counterfactual advantage function significantly outperforms the other three variants in the middle and late stages of the task, especially when the information hotspot area has been largely covered. It can effectively avoid path overlap and resource waste, demonstrating strong team collaboration capabilities and planning accuracy.
[0113] In experiments comparing non-learning methods, the proposed method significantly outperformed other methods in terms of both F1-score and map entropy reduction rate, compared to traditional information gain (IG), coverage, and random exploration methods. Particularly when communication is limited or the number of drones fluctuates, the proposed method demonstrates good robustness and generalization, maintaining high performance without retraining.
[0114] Further experiments also demonstrated that the learning method can adapt to different team sizes and communication ranges, and can maintain good performance in changing environments without retraining. This verifies that the RL path planning method of this embodiment has strong generalization capabilities and can be widely applied to post-disaster search and rescue missions of different scales and conditions.
[0115] This embodiment proposes a multi-UAV adaptive path planning framework based on the Counterfactual Multi-Agent Policy Gradient (COMA) method to improve the execution efficiency of search and rescue missions. The centralized training and decentralized execution (CTDE) paradigm is adopted to enable UAVs to make autonomous decisions without relying on global information. At the same time, a counterfactual baseline is introduced to optimize the credit allocation problem of UAVs, thereby improving the accuracy of individual contribution evaluation. Experimental results show that compared with traditional methods based on information gain and heuristic search, this method performs superiorly in terms of information collection efficiency, search and rescue success rate, and energy efficiency, and is applicable to different disaster environments. This study provides a new technical path for intelligent UAV search and rescue based on reinforcement learning.
[0116] The above is a schematic description of the present invention and its embodiments, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs a structure and embodiment similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.
Claims
1. A multi-UAV search and rescue path planning method based on COMA method, characterized in that: The following steps are involved: Offline centralized training phase: This includes the coverage planning layer and the path planning layer. The coverage planning layer uses the K-means algorithm to quickly cover the disaster area based on the real-time information entropy and uncertainty map of the disaster area. The path planning layer is based on the COMA reinforcement learning algorithm, extending it and applying it to informative path planning tasks for UAV teams. It adopts a centralized critic and decentralized actor framework, and introduces a counterfactual baseline to clearly measure each UAV's contribution to the overall mission, thereby achieving collaborative adaptive path planning. Online distributed deployment phase: After completing offline training, the trained executor network is deployed to the actual UAV platform, enabling each UAV to independently decide the optimal path based on local observation information, achieving decentralized adaptive map uncertainty reduction.
2. The multi-UAV search and rescue path planning method based on COMA method according to claim 1 is characterized in that: In the offline centralized training step: The critic network uses globally visible information to evaluate the value of each drone's action and estimate the state-action value function; The executor network only relies on its own local map observations and historical measurement data to output the next feasible measurement location; By introducing a counterfactual baseline, while keeping the actions of other drones unchanged, the change in the team value function after replacing the current drone action is estimated, thereby quantifying the contribution of a single drone to the overall goal.
3. The multi-UAV search and rescue path planning method based on COMA method according to claim 2 is characterized in that: In the COMA reinforcement learning algorithm: The critic network input includes the global state s and the action information of other drones, and then evaluates the joint state-action value function Q π (s,u): Q π (s,u)=Q π (s,(u1,…,u i ,…,u N )) Where s is the state parameter, u is the action parameter; π (s,u) represents the expected cumulative discounted return that can be obtained by continuing to act according to the current strategy π after executing the joint action u in state s; the critic network uses the TD(λ) method to calculate the discounted return G t Perform online estimation to provide an evaluation basis for strategy optimization; the TD(λ) method estimates the target value G of the critic network t λ , defined as: Where λ∈[0,1] is the attenuation parameter, G t (n) is the n-step return; The actor network is based on its local observation information ω i Calculate the action probability distribution π(u i |ω i ): Where ∈ is the exploration parameter, which is used to adjust the exploration degree during training and is set to zero during deployment; U represents the optional action of a single UAV, For the executor network function.
4. The multi-UAV search and rescue path planning method based on COMA method according to claim 3 is characterized in that: The inputs to the executor network include: UAV identifier i and remaining mission budget b; a location map constructed with the current location of the UAV as the center, encoding the location and altitude information of other UAVs within the map boundary and communication range; local map state M i , the environment is represented by a grid map; the weighted entropy H(M|z 0:t ,p 0:t ): Among them, M j represents the jth cell in the discrete grid, p(M j ) is the posterior probability that the grid unit is in the target category, W(M j ) is a weighted coefficient related to the importance of the grid cell, which is used to give more attention to high-value areas in the decision-making process; w1, w2>0, is the importance weight; the weight is allocated according to the current probability threshold: Footprint map consisting of the field of view coverage areas of all drones within the communication range.
5. The multi-UAV search and rescue path planning method based on COMA method according to claim 4 is characterized in that: The input of the critic network includes all the inputs of the performer network, and additionally includes: A global position map encoding the positions of all drones Global map state M; weighted entropy of global map state H(M|z 0:t ,p 0:t ); the map area covered by all drones’ field of view; and the movement information of other drones.
6. The multi-UAV search and rescue path planning method based on COMA method according to claim 5 is characterized in that: Using the information theory criterion I(ψ t+1 ) is used to evaluate the UAV path, and the criterion is calculated as the sum of the entropy reduction along the path: Where B is the maximum number of measurement steps allowed for the task, z 0:t+m and p 0:t+m Represent all measurement data and measurement positions from the initial moment to the t+mth step respectively.
7. The multi-UAV search and rescue path planning method based on COMA method according to claim 6 is characterized in that: The following reward function is used for drone team training: Here, H(·) represents the weighted calculation of map entropy, α and β are the scaling and translation factors used to stabilize training. The reward function can encourage the drone to focus on high-value areas and, through normalization, keep the reward relatively stable throughout the mission.
8. The multi-UAV search and rescue path planning method based on COMA method according to claim 7 is characterized in that: The COMA reinforcement learning algorithm introduces an advantage function to evaluate the contribution of individual drones in team actions: Advantage function of drone i when the team performs joint action u: Among them, Q π (s,u) represents the state-action value of the team performing the joint action in the state; Represents all possible actions Traverse; use the current strategy Give each possible action a weight; use Q value If the actions of other drones remain unchanged, only the action of the current drone will be changed. The total team reward at this time; finally, perform weighted average to obtain the expected Q value of the action under the current strategy; According to the policy gradient theorem, the parameter update gradient of each drone is expressed as: Using negative log-likelihood loss function: The optimization process is updated via mini-batch stochastic gradient descent.
9. The multi-UAV search and rescue path planning method based on COMA method according to claim 8, characterized in that: The communication protocol between drones uses a limited communication range: When the distance between two drones i and k When ,they share measurement data; The communication radius D is set to 25 meters unless otherwise specified; The receiving drone uses the received measurement data and location information Updates its local map beliefs.
10. The multi-UAV search and rescue path planning method based on COMA method according to claim 9, characterized in that: The network architecture for processing spatial information features is: Both the actor and critic networks consist of convolutional encoders and linear layers; The convolutional layer is followed by a ReLU activation function, and the linear layer is also followed by a ReLU activation function; The output layer of the executor network is a softmax layer, which generates action probability distribution.