A Multi-UAV Cooperative Trajectory Planning Method Based on Communication-Location Performance Fingerprint Field

Abstract

This invention discloses a multi-UAV cooperative trajectory planning method based on a communication-positioning performance fingerprint field. The method specifically includes: First, considering the complex urban environment and potential communication-positioning concave points, the three-dimensional airspace is gridded according to the distribution of urban buildings to construct a flight grid composed of feasible voxels. Unmanned Aerial Vehicles (UAVs) are only allowed to move within this grid space. Second, for each feasible voxel, its communication and positioning performance are modeled and pre-calculated based on the number of surrounding base stations, the distance to the base stations, and occlusion conditions, forming a communication-positioning fingerprint. The fingerprints of all feasible voxels together constitute the communication-positioning performance fingerprint field. Finally, based on the fingerprint field and the UAV kinematic energy consumption model, a multi-UAV cooperative route planning problem is designed, and multi-agent reinforcement learning (MARL) with centralized training and decentralized execution (CTDE) is used to solve the problem, ultimately achieving efficient and safe route planning for UAVs with integrated communication and positioning in complex urban environments. This invention adopts a communication-location fingerprint information gridded architecture, which fully considers the requirements of communication, positioning, energy consumption, collision and other factors. It combines MARL to solve the optimal strategy problem and achieves relatively stable multi-UAV cooperative route planning under multiple constraints.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control and wireless communication technology for unmanned aerial vehicles (UAVs), specifically involving a multi-UAV cooperative trajectory planning method based on spatial performance modeling. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are widely used in communication relay, emergency rescue, and urban logistics due to their high maneuverability, flexible deployment, and good line-of-sight communication capabilities. With the development of wireless communication technology, by integrating communication link information with ranging observation information, UAVs can be provided with integrated communication and positioning services in complex environments. However, in urban low-altitude scenarios, the dense distribution of tall buildings and their obstruction significantly impacts wireless signal propagation, causing a noticeable spatial discontinuity in the communication link quality between the UAV and ground base stations. This leads to insufficient observable base stations or geometric degradation, reducing positioning accuracy and even causing communication interruptions or loss of positioning lock.

[0003] To address the aforementioned issues, existing research typically improves system communication performance and positioning reliability by optimizing UAV deployment locations, resource allocation, and flight trajectories. For example, optimizing UAV positions by constructing wireless channel models or using heuristic search and graph search algorithms to plan UAV flight paths can help avoid areas with poor communication quality or insufficient positioning accuracy. These methods achieve good planning results when environmental information is stable and the state space is small. However, when considering multiple constraints such as multi-UAV flight path congestion, integrated communication and positioning, and energy consumption, existing algorithms struggle to meet the demands of these complex problems in a short time. The entire system may face risks such as positioning loss, communication disconnection, and UAV collisions during actual operation.

[0004] Therefore, this study aims to develop a method that can perform unified spatial modeling of communication and positioning performance, and make path planning decisions based on the modeling results. This is of great significance for improving the stability and mission execution efficiency of UAV systems in complex urban low-altitude environments. Summary of the Invention

[0005] The purpose of this invention is to improve the trajectory planning capabilities of multiple unmanned aerial vehicles (UAVs) under multiple constraints such as communication, positioning, and energy consumption, and to ensure the reliability of multi-UAV flight in urban low-altitude environments. The main invention is a multi-UAV cooperative trajectory planning method based on a communication-positioning performance fingerprint field, used to overcome the performance dips in communication and positioning that may be encountered in UAV trajectory planning. Here, the communication-positioning performance fingerprint refers to a set of communication and positioning performance characteristics pre-calculated for each feasible three-dimensional meshed minimum unit, including at least the number of base stations with possible links, communication link quality, and positioning accuracy indicators; the communication-positioning performance fingerprint field refers to the three-dimensional spatial performance distribution composed of the communication-positioning performance fingerprints of all feasible voxels. This performance field enables UAVs to more quickly and stably identify the communication and positioning quality of the surrounding space during route planning, avoiding entering communication and positioning degradation zones.

[0006] This invention employs a gridded architecture containing communication-location fingerprint information, transforming communication and location information from "real-time observations" into a "spatial performance field," and making path decisions within this performance field. Simultaneously, it combines Multi-Agent Reinforcement Learning (MARL) to solve the optimal policy problem, achieving relatively stable multi-UAV cooperative route planning under multiple constraints.

[0007] The multi-UAV cooperative trajectory planning method based on communication-positioning performance fingerprint field proposed in this invention has the following implementation steps:

[0008] Step 1: Based on the urban building distribution, minimum safe distance, and flight altitude constraints, the urban three-dimensional airspace is divided into a grid, discretizing the continuous three-dimensional space into several voxels. Each voxel represents a three-dimensional meshed minimum unit with a defined spatial location and size; the voxel's center coordinates and side length are used to describe its spatial extent. Based on this, each divided voxel is marked for feasibility, and adjacency relationships between voxels are established, forming a three-dimensional feasible voxel space for path planning.

[0009] Step 2: In the feasible voxel space, calculate the communication and positioning performance indicators offline for each voxel, including the number of connectable base stations, the optimal communication link quality, and the lower bound of the variance of positioning accuracy. Combine the calculation results to form the communication-positioning performance fingerprint corresponding to that voxel. The fingerprints of all voxels together constitute the communication-positioning performance fingerprint field in the three-dimensional voxel space, which is used to characterize the service availability and performance characteristics of each spatial location. The performance field is calculated offline before path planning and serves as the core description of the environmental state during the path planning process.

[0010] Step 3: Based on the aforementioned communication-positioning performance field, the multi-UAV cooperative path planning problem is modeled as an optimal path decision problem within this performance field. The MARL framework is used for solving this problem, with optimization objectives including maximizing communication and positioning performance along the path, minimizing flight energy consumption, and avoiding conflicts between UAVs. In this way, the path planning problem is transformed from a traditional geometric path search problem into a path feasibility search problem under the constraints of the communication-positioning performance field.

[0011] The voxel feasibility marking mentioned in step one is performed as follows:

[0012] S11. Based on the overlap between the voxel and the space occupied by city buildings, determine whether the voxel is occupied by a building:

[0013]

[0014] Among them, the Individual morphology Its geometric center coordinates are Its size (resolution) in three-dimensional space is Therefore, voxels Area occupied in three-dimensional space It can be represented as:

[0015]

[0016] No. Each building is recorded as Its geometric center coordinates are Its dimensions in three-dimensional space are Therefore, buildings Area occupied in three-dimensional space It can be represented as:

[0017]

[0018] For the set of all voxels, A collection of buildings. When When, it indicates voxels If a voxel spatially overlaps with at least one building, it is marked as an infeasible voxel; when Time. Represents voxels. Since there is no spatial overlap with any buildings, the determination of safe distance and height constraints can continue.

[0019] S12. Apply safety distance constraints to voxels based on the minimum spatial distance between voxel units and the space occupied by buildings:

[0020]

[0021] in, This represents the minimum Euclidean distance between the occupied area of ​​a voxel and the occupied area of ​​the nearest building. To set the minimum distance;

[0022] S13. Constrain the voxels based on the UAV's flight altitude. The voxel satisfies the constraints only when its lower boundary height is not lower than the minimum flight altitude and its upper boundary height is not higher than the maximum flight altitude.

[0023]

[0024] in, , These are the set minimum and maximum flight altitudes for the drone, respectively.

[0025] S14. Based on the above constraints, the feasible voxel set is defined as follows:

[0026]

[0027] in, This represents the set of feasible voxels that satisfy all flight constraints. (Belongs to...) Voxels can be used as candidate locations in UAV path planning; not belonging to The voxels do not participate in subsequent path planning.

[0028] The communication-location fingerprint information mentioned in step two is implemented as follows:

[0029] S21. Examine each voxel. The number of connectable base stations, which is used as a fundamental characterization in the communication-location performance fingerprint field, is expressed as:

[0030]

[0031] in, voxels Available base station set For the set of all ground base stations, the first one is... Each base station is denoted as Its antenna position coordinates are The transmission power is The carrier frequency is , voxels With base station Inter-space distance, This is the maximum communication distance set for the base station. When At that time, it was considered that the base station It is a voxel A connectable base station. As one implementation method, It can be set according to the base station's transmit power, carrier frequency, and minimum receive power threshold, for example, set to 200 m to 1000 m.

[0032] S22. At the communication layer, communication capacity is used as fingerprint information. When calculating channel gain, a probabilistic line-of-sight (LoS) / non-line-of-sight (NLoS) model is used. The path loss is weighted according to the LoS / NLoS probabilities to obtain voxels. With base station Channel gain information of the link Then, the communication capacity can be obtained through Shannon's formula. In the generation of voxels When processing fingerprint information, if there is no available base station, the communication fingerprint is set to 0, indicating that communication is not possible; otherwise, the communication capacity is mapped to a normalized communication fingerprint using the sigmoid function, as shown below:

[0033]

[0034] in, sigmoid function , voxels Communication capacity at the location, As a communication performance threshold, , These are the communication-side threshold offset parameter and the function slope control parameter, respectively. As one implementation method, It can be set from 1 Mbps to 20 Mbps. It can be set from 0 to 5 Mbps. It can be set from 0.1 to 10.

[0035] The above function first performs a non-linear mapping of the communication capacity using the sigmoid function, and then subtracts... and normalized denominator This achieves the normalization characteristic of a function taking a value of 0 at the lower bound and asymptotically approaching 1 in the high-performance region. This is achieved through parameters. and The joint adjustment can control the function when the communication performance is close to the threshold. The communication fingerprint exhibits a relatively high rate of change near the threshold, thus enhancing its sensitivity to performance variations around the threshold. As the communication capacity exceeds the threshold, the function's growth rate gradually decreases, reflecting diminishing marginal returns. This design ensures that the communication fingerprint is continuously differentiable throughout the entire domain, making it suitable for constructing immediate rewards in subsequent reinforcement learning algorithms.

[0036] S23. At the positioning level, fingerprint information is obtained using a function of the Cramér–Rao Lower Bound (CRLB) based on Time Difference of Arrival (TDOA) positioning. For feasible voxels... If the number of available base stations is less than 4, the voxel is considered not to meet the base station quantity requirement for 3D TDOA positioning, and the positioning fingerprint is set to 0; otherwise, the positioning fingerprint is determined based on the voxel. Based on the geometric relationship with the connectable base station and the ranging error model, the CRLB matrix of the current voxel TDOA positioning is calculated, and normalized using the sigmoid function based on this information. The integrated positioning fingerprint is defined as follows:

[0037]

[0038] in, voxels Positioning accuracy at the location, voxels The lower bound matrix of positioning error obtained based on the TDOA positioning model The smaller the value, the higher the positioning accuracy. To determine the performance threshold, , These are the positioning side threshold offset parameters and the function slope control parameters, respectively, and the normalization approach is the same as in S22. As one implementation method, It can be set from 1 m to 10 m. It can be set from 0 m to 5 m. It can be set from 0.1 to 10.

[0039] S24. In summary, each feasible voxel The fingerprint information is as follows:

[0040]

[0041] Fingerprints corresponding to all feasible voxels Together they constitute a fingerprint field of communication-positioning performance.

[0042] The design and reinforcement learning modeling of the multi-UAV cooperative path planning problem based on the communication-positioning performance fingerprint field described in step three are as follows:

[0043] S31. The motion model of the UAV is described in a three-dimensional discrete voxel space. Let the set of UAVs be... ,in, Indicates the first A drone. (Number) A drone in a time slot The voxel below is denoted as The constraints on the drone's movement are reflected in the fact that the speed generated by the drone's displacement in each time slot must not exceed the maximum speed set by the drone. In the mesh world, this means that the drone can only choose the next position from the set of neighboring voxels corresponding to the current position in each time slot. The set of neighboring voxels can be determined using a 6-neighborhood, 18-neighborhood, or 26-neighborhood method.

[0044] In addition, the energy consumption impact of the drone's flight process needs to be considered. Indicates drone In the time slot With speed Flight power during flight, of which For existing UAV flight power models, the UAV's flight speed can be determined by the ratio of the difference between the center positions of two consecutive voxels to the time slot, i.e.:

[0045]

[0046] in, For time slots Hypotoplasm The center position coordinates of the UAV. Therefore, the average energy consumption of the UAV due to flight during the time period from 0 to t can be expressed as:

[0047]

[0048] S32. The design of an optimization problem based on multi-UAV cooperative path planning mainly involves selecting the next feasible voxel for each UAV in each time slot while satisfying constraints, so that the multi-UAV system can achieve better communication performance, positioning performance, and energy consumption during path execution. The problem is formulated as follows:

[0049]

[0050] in, This represents the total number of time slots within the planning time domain. As the energy cost factor, , These are the communication demand factor and the location demand factor, which satisfy... , voxels The set of adjacent voxels; in the constraints, Ensure that the drone always moves within the feasible flight area, and at the same time ensure that the position of the drone voxel in the current time slot is a neighbor of the previous time slot; To prevent multiple drones from occupying the same voxel in the same time slot, thus ensuring flight safety.

[0051] S33. Within the framework of CTDE, the MARL algorithm is used to solve the above problem. Specifically, Multi-Agent Proximal Policy Optimization (MAPPO) can be used as a representative implementation method. MAPPO effectively alleviates the non-stationarity problem in multi-agent environments by introducing a centralized value function to evaluate joint actions and maintaining an independent policy network for each UAV.

[0052] (1) State-space modeling

[0053] In the time slot Under these conditions, the global state of the system is defined as follows:

[0054]

[0055] During the distributed execution phase, each agent can only observe its local state, which is defined as:

[0056]

[0057] in, , These are the adjacent voxel sets and their fingerprint information, respectively.

[0058] (2) Action space modeling

[0059] Each drone moves in a discrete grid space, and its action is defined as:

[0060]

[0061] Its essence is to select the voxel position for the next time slot:

[0062] (3) Reward function design

[0063] Constructing an instantaneous reward function based on voxel fingerprint information:

[0064]

[0065] in, Indicates drone In the time slot The distance between the voxel center where it is located and the target voxel center, therefore This indicates the amount of progress achieved by the drone as it approaches the target voxel. The reward factor for the drone approaching the target point; The penalty for multiple drones occupying the same voxel is defined as follows:

[0066]

[0067] As a collision penalty factor, For time slots Internal occupying volume element The number of drones;

[0068] The reward function utilizes the normalization properties of voxel fingerprints to achieve a unified measurement of communication and positioning performance, and constrains flight energy consumption and safety through penalty terms.

[0069] Based on the design of this invention, a multi-UAV cooperative trajectory planning method based on a communication-positioning performance fingerprint field is proposed. First, the three-dimensional space is discretized by incorporating building distribution to construct a flight grid composed of feasible voxels. Simultaneously, for each feasible voxel, its communication-positioning performance index is pre-calculated based on the number of surrounding base stations, distance relationships, and occlusion conditions to obtain communication-positioning fingerprint information. This constructs a communication-positioning performance fingerprint field defined on the three-dimensional feasible voxel space to describe the communication quality and positioning reliability at different spatial locations.

[0070] Based on the design of this invention, this invention addresses the trajectory planning problem related to communication performance, positioning accuracy, and UAV motion energy consumption, and employs a multi-agent reinforcement learning method with centralized training and distributed execution for policy learning, enabling UAVs to complete collaborative path planning under multiple constraints, thereby improving the stability of multi-UAV mission execution in complex urban environments. Attached Figure Description

[0071] Figure 1 This is an architecture diagram of a multi-UAV cooperative trajectory planning method based on communication-positioning performance fingerprint field.

[0072] Figure 2 This is a flowchart of the construction process for a three-dimensional feasible voxel space and a communication-positioning performance fingerprint field.

[0073] Figure 3 This is a schematic diagram of the MAPPO algorithm training process involved in this invention. Detailed Implementation

[0074] To enable a better understanding of the features, objectives, and functions of this invention, a more detailed description of the invention is now provided in conjunction with specific implementation examples, accompanying drawings, and tables.

[0075] This invention first outlines the basic scenario and operational process. Multiple drones need to move in a low-altitude urban environment to perform missions. The drones must reach their respective target points from their starting points while ensuring communication quality and positioning accuracy. The system first constructs a communication-positioning performance fingerprint field offline to characterize the communication quality and positioning reliability at different locations in three-dimensional voxel space. After acquiring the position and environmental state information of each drone, this information is input into a multi-agent reinforcement learning model for joint decision-making, calculating the next path selection for each drone. Through iterative decision-making, the drones complete collaborative route planning while meeting communication, positioning, and safety constraints, ultimately obtaining a stable and reliable flight path.

[0076] like Figure 2 , Figure 3 As shown, the specific implementation of this invention mainly includes the following two parts: the first part is the construction of a three-dimensional feasible voxel space and a communication-localization performance fingerprint field; the second part is the multi-UAV cooperative path planning model and solution based on the communication-localization fingerprint field. The specific implementation is as follows:

[0077] Step 1: Construction of 3D Feasible Voxel Space and Communication-Location Performance Fingerprint Field

[0078] (1) Construction of three-dimensional voxel space and screening of feasible voxels

[0079] The target spatial domain is discretized based on the urban spatial extent, dividing the continuous space into a regular three-dimensional voxel grid to obtain a voxel set. ;

[0080] Subsequently, based on the collection of urban buildings and base station set The three-dimensional distribution information is used to determine the feasibility of each voxel based on building occupancy, safety distance, and flight altitude constraints, resulting in a set of feasible voxels. .

[0081] (2) Communication-Location Fingerprint Field Construction

[0082] For each feasible voxel Calculate the number of available base stations for each. Communication fingerprint and location fingerprint information The fingerprint information is integrated into the voxel. The fingerprint information of all feasible voxels together constitutes the communication-localization performance fingerprint field defined in the three-dimensional feasible voxel space.

[0083] (3) Optimize problem construction

[0084] Combining the above performance fingerprint field and UAV kinematic energy consumption model, we construct a multi-UAV cooperative optimization problem:

[0085]

[0086] Step 2: Solve the multi-UAV cooperative path planning problem using MAPPO.

[0087] (1) Initialization

[0088] Initialize the starting position and target point of each drone, and initialize the policy network (Actor) for each agent (i.e., drone). And value networks (Critic) At the same time, set the learning rate Discount Factor Cutting factor Generalized Advantage Estimation (GAE) parameters Training batch size And hyperparameters such as maximum number of training epochs;

[0089] (2) Environmental interaction and experience collection

[0090] In each training round, each drone starts from the starting point and proceeds according to the current strategy. The system interacts with the environment under the constraints of a communication-positioning performance fingerprint field and stores the observations of each UAV at each time step. Global state ,action ,award Next state Value function The system will receive this information until the drone reaches the mission objective.

[0091] (3) Calculate returns and advantages

[0092] After completing one episode sampling, based on the reward sequence and value estimation results in the experience buffer, the cumulative reward value for each time step is calculated backward, and the advantage function is calculated in conjunction with the GAE method. This is used to characterize the quality of the current action relative to the benchmark value;

[0093] (4) Update the Critic network

[0094] Using the cumulative return value as a monitoring signal, a loss function for the value network is constructed. By minimizing the error between the value estimate and the target return, the Critic network parameters are adjusted. Optimize and update the function to improve the accuracy of the value function in fitting the environmental state benefits.

[0095] (5) Update the Actor network

[0096] By utilizing the action probabilities of the old policy recorded during the empirical sampling phase and the action probabilities output by the current policy, a proximal policy optimization (PPO) objective function based on a pruning mechanism is constructed. By limiting the update magnitude of the old and new policies, excessive changes in the policy within a single iteration are avoided, thus ensuring the stability of the training process. Based on this objective function, the Actor network parameters are adjusted. Update;

[0097] (6) Soft update target network

[0098] The parameters of the target Actor and the Critic network are updated using a soft update mechanism at intervals of n iterations.

[0099] (7) Strategy update and repeated training

[0100] After completing the current round of updates, the updated strategy parameters will be... This will serve as the strategy for the next round of environmental interaction. Repeat the above steps until the preset number of training rounds is reached, or the strategy performance meets the task requirements.

[0101] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multi-UAV cooperative trajectory planning method based on communication-positioning performance fingerprint field, characterized in that: By constructing a gridded communication-positioning performance fingerprint field offline, the system can quickly assess service availability in airway areas and avoid high-risk areas during online decision-making, thus addressing performance bottlenecks such as communication interruptions or positioning loss caused by obstructions from tall buildings and other objects in urban low-altitude environments. The steps of this method are as follows: Step 1: Based on the urban building distribution, minimum safe distance, and flight altitude constraints, the urban three-dimensional airspace is divided into a grid, discretizing the continuous three-dimensional space into several voxels. Each voxel represents a three-dimensional meshed minimum unit with a defined spatial location and size; the voxel's center coordinates and side length are used to describe its spatial extent. Based on this, each divided voxel is marked for feasibility, and adjacency relationships between voxels are established, forming a three-dimensional feasible voxel space for path planning. Step 2: In the feasible voxel space, calculate the communication and positioning performance indicators offline for each voxel, including the number of connectable base stations, the optimal communication link quality, and the lower bound of the variance of positioning accuracy. Combine the calculation results to form the communication-positioning performance fingerprint corresponding to that voxel. The fingerprints of all voxels together constitute the communication-positioning performance fingerprint field in the three-dimensional voxel space, which is used to characterize the service availability and performance characteristics of each spatial location. The performance field is calculated offline before path planning and serves as the core description of the environmental state during the path planning process. Step 3: Based on the aforementioned communication-positioning performance field, the multi-UAV cooperative path planning problem is modeled as an optimal path decision problem within this performance field. The MARL framework is used for solving this problem, with optimization objectives including maximizing communication and positioning performance along the path, minimizing flight energy consumption, and avoiding conflicts between UAVs. In this way, the path planning problem is transformed from a traditional geometric path search problem into a path feasibility search problem under the constraints of the communication-positioning performance field.

2. The feasible voxel labeling described in step one of claim includes: S21. Based on the overlap between the voxel and the space occupied by city buildings, determine whether the voxel is occupied by a building: Among them, the Individual morphology Its geometric center coordinates are Its size (resolution) in three-dimensional space is Therefore, voxels Area occupied in three-dimensional space It can be represented as: No. Each building is recorded as Its geometric center coordinates are Its dimensions in three-dimensional space are Therefore, buildings Area occupied in three-dimensional space It can be represented as: For the set of all voxels, A collection of buildings. When When, it indicates voxels If a voxel spatially overlaps with at least one building, it is marked as an infeasible voxel; when Time. Represents voxels. Since there is no spatial overlap with any buildings, the determination of safe distance and height constraints can continue. S22. Apply safety distance constraints to voxels based on the minimum spatial distance between voxel units and the space occupied by buildings: in, This represents the minimum Euclidean distance between the occupied area of ​​a voxel and the occupied area of ​​the nearest building. To set the minimum distance; S23. Constrain the voxels based on the UAV's flight altitude. The voxel satisfies the constraints only when its lower boundary height is not lower than the minimum flight altitude and its upper boundary height is not higher than the maximum flight altitude. in, , These are the set minimum and maximum flight altitudes for the drone, respectively. S24. Considering the above constraints, the feasible voxel set is defined as follows: in, This represents the set of feasible voxels that satisfy all flight constraints. (Belongs to...) Voxels can be used as candidate locations in UAV path planning; not belonging to The voxels do not participate in subsequent path planning.

3. The communication-location fingerprint information described in step two of claim 1 is specifically expressed as follows: S31. Examine each voxel. The number of connectable base stations, as a fundamental characterization in the communication-location performance fingerprint field, is expressed as: in, voxels Available base station set For the set of all ground base stations, the first one is... Each base station is denoted as Its antenna position coordinates are The transmission power is The carrier frequency is , voxels With base station Inter-space distance, This is the maximum communication distance set for the base station. When At that time, it was considered that the base station It is a voxel A connectable base station. As one implementation method, It can be set according to the base station's transmit power, carrier frequency, and minimum receive power threshold, for example, set to 200 m to 1000 m. S32. At the communication layer, communication capacity is used as fingerprint information. When calculating channel gain, a probabilistic line-of-sight (LoS) / non-line-of-sight (NLoS) model is used. The path loss is weighted according to the LoS / NLoS probabilities to obtain voxels. With base station Channel gain information of the link Then, the communication capacity can be obtained through Shannon's formula. In the generation of voxels When processing fingerprint information, if there is no available base station, the communication fingerprint is set to 0, indicating that communication is not possible; otherwise, the communication capacity is mapped to a normalized communication fingerprint using the sigmoid function, as shown below: in, sigmoid function , voxels Communication capacity at the location, As a communication performance threshold, , These are the communication-side threshold offset parameter and the function slope control parameter, respectively. As one implementation method, It can be set from 1 Mbps to 20 Mbps. It can be set from 0 to 5 Mbps. It can be set from 0.1 to 10. The above function first performs a non-linear mapping of the communication capacity using the sigmoid function, and then subtracts... and normalized denominator This achieves the normalization characteristic of a function taking a value of 0 at the lower bound and asymptotically approaching 1 in the high-performance region. This is achieved through parameters. and The joint adjustment can control the function when the communication performance is close to the threshold. The communication fingerprint exhibits a relatively high rate of change near the threshold, thus enhancing its sensitivity to performance variations around the threshold. As the communication capacity exceeds the threshold, the function's growth rate gradually decreases, reflecting diminishing marginal returns. This design ensures that the communication fingerprint is continuously differentiable throughout the entire domain, making it suitable for constructing immediate rewards in subsequent reinforcement learning algorithms. S33. At the positioning level, fingerprint information is derived using a function of the Cramér–Rao Lower Bound (CRLB) based on Time Difference of Arrival (TDOA) positioning. For feasible voxels... If the number of available base stations is less than 4, the voxel is considered not to meet the base station quantity requirement for 3D TDOA positioning, and the positioning fingerprint is set to 0; otherwise, the positioning fingerprint is determined based on the voxel. Based on the geometric relationship with the connectable base station and the ranging error model, the CRLB matrix of the current voxel TDOA positioning is calculated, and normalized using the sigmoid function based on this information. The integrated positioning fingerprint is defined as follows: in, voxels Positioning accuracy at the location, voxels The lower bound matrix of positioning error obtained based on the TDOA positioning model The smaller the value, the higher the positioning accuracy. To determine the performance threshold, , These are the positioning side threshold offset parameters and the function slope control parameters, respectively, and the normalization approach is the same as in S22. As one implementation method, It can be set from 1 m to 10 m. It can be set from 0 m to 5 m. It can be set from 0.1 to 10. S34. In summary, the fingerprint information for each voxel is as follows:

4. The design and reinforcement learning modeling of the multi-UAV cooperative path planning problem based on the communication-localization performance fingerprint field as described in step three of claim 1 are as follows: S41. The motion model of the UAV is described in a three-dimensional discrete voxel space. Let the set of UAVs be... ,in, Indicates the first A drone. (Number) A drone in a time slot The voxel below is denoted as The constraints on the drone's movement are reflected in the fact that the speed generated by the drone's displacement in each time slot must not exceed the maximum speed set by the drone. In the mesh world, this means that the drone can only choose the next position from the set of neighboring voxels corresponding to the current position in each time slot. The set of neighboring voxels can be determined using a 6-neighborhood, 18-neighborhood, or 26-neighborhood method. In addition, the energy consumption impact of the drone's flight process needs to be considered. Indicates drone In the time slot With speed Flight power during flight, of which For existing UAV flight power models, the UAV's flight speed can be determined by the ratio of the difference between the center positions of two consecutive voxels to the time slot, i.e.: in, For time slots Hypotoplasm The center position coordinates. Therefore, the average energy consumption of the UAV due to flight during the time period from 0 to t can be expressed as: S42. The design of an optimization problem based on multi-UAV cooperative path planning mainly involves selecting the next feasible voxel for each UAV in each time slot while satisfying constraints, so that the multi-UAV system can achieve better communication performance, positioning performance, and energy consumption during path execution. The problem is formulated as follows: in, This represents the total number of time slots within the planning time domain. As the energy cost factor, , These are the communication demand factor and the location demand factor, which satisfy... , voxels The set of adjacent voxels; in the constraints, Ensure that the drone always moves within the feasible flight area, and at the same time ensure that the position of the drone voxel in the current time slot is a neighbor of the previous time slot; To prevent multiple drones from occupying the same voxel in the same time slot, thus ensuring flight safety. S43. Within the framework of CTDE, the MARL algorithm is used to solve the above problem. Specifically, Multi-Agent Proximal Policy Optimization (MAPPO) can be used as a representative implementation method. MAPPO effectively alleviates the non-stationarity problem in multi-agent environments by introducing a centralized value function to evaluate joint actions and maintaining an independent policy network for each UAV. (1) State-space modeling In the time slot Under these conditions, the global state of the system is defined as follows: During the distributed execution phase, each agent can only observe its local state, which is defined as: in, , These are the adjacent voxel sets and their fingerprint information, respectively. (2) Action space modeling Each drone moves in a discrete grid space, and its action is defined as: Its essence is to select the voxel position for the next time slot: (3) Reward function design Constructing an instantaneous reward function based on voxel fingerprint information: in, Indicates drone In the time slot The distance between the voxel center where it is located and the target voxel center, therefore This indicates the amount of progress achieved by the drone as it approaches the target voxel. The reward factor for the drone approaching the target point; The penalty for multiple drones occupying the same voxel is defined as follows: As a collision penalty factor, For time slots Internal occupying volume element The number of drones; The reward function utilizes the normalization properties of voxel fingerprints to achieve a unified measurement of communication and positioning performance, and constrains flight energy consumption and safety through penalty terms.

Images