A continuous path planning method for multi-unmanned aerial vehicle internet of things data collection constraints

Abstract

This invention discloses a continuous path planning method for constrained multi-UAV IoT data acquisition. Addressing the problems of poor feasibility, lack of dynamic channel awareness, delayed safety constraints, and weak multi-UAV collaboration in existing discrete planning methods, this invention reconstructs the multi-UAV data acquisition problem into a constrained multi-objective continuous optimization problem, and transforms it into a distributed partially observable Markov game. Through spatiotemporal dual-domain multi-channel semantic feature encoding, it simultaneously extracts environmental spatial features and channel temporal features, and designs a look-ahead constrained multi-agent soft-actor critic algorithm, embedding constraints pre-processing into the continuous decision-making process to achieve multi-UAV collaborative continuous waypoint planning. Simultaneously, a meta-learning framework is introduced to achieve rapid quantization of the model with few samples. This invention can adapt to channel changes in complex environments, improve data acquisition efficiency while ensuring safety, and quickly adapt to new task scenarios, making it suitable for multi-UAV IoT data acquisition tasks without ground infrastructure coverage.

Description

Technical Field

[0001] This invention belongs to the fields of UAV collaborative control, IoT wireless communication and deep reinforcement learning. In particular, it relates to a continuous path planning method for multi-UAV IoT data acquisition constraints, which can be widely adapted to complex scenarios without ground infrastructure coverage, such as mountain photovoltaic power stations, oil and gas pipeline inspection, bridge structure monitoring, and disaster emergency rescue. Background Technology

[0002] With the rapid development of IoT technology, a large number of sensor nodes are deployed in urban environments, making the efficient and flexible collection of data generated by these nodes a key issue. Drones, with their high mobility and rapid deployment capabilities, have shown great potential in data collection tasks. However, in complex urban environments, multiple drones collaboratively performing data collection tasks face numerous challenges. Cities present various spatial constraints such as building obstructions, no-fly zones, and restrictions on takeoff and landing areas, requiring drones to plan their flight paths while avoiding obstacles and no-fly zones.

[0003] Communication is inherently uncertain. The wireless channel between drones and ground-based IoT devices is affected by building obstructions, resulting in random switching between line-of-sight and non-line-of-sight modes and causing dynamic changes in communication rates. Multiple drones are required to cooperate and avoid collisions while maximizing data collection within a limited flight time. Furthermore, in actual missions, parameters such as the number of drones, devices, device locations, data volume, and flight time frequently change. Traditional path planning methods (such as those based on genetic algorithms, convex optimization, or single-scenario training reinforcement learning) require recalculation or retraining when faced with new parameters, leading to inefficiency and difficulty in meeting rapid deployment requirements. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a continuous path planning method for multi-UAV IoT data acquisition constraints, which solves the core problems of existing technologies such as poor implementation of discrete planning, lack of dynamic channel perception, easy decision failure due to post-implementation of security constraints, weak multi-UAV collaborative capability, and high cost of adapting to new scenarios. It realizes the full-process optimization of continuous smooth path planning for multiple UAVs in complex urban environments, forward perception of channel time-varying characteristics, pre-embedding of security constraints, and global balance of cluster tasks.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] The continuous path planning method for multi-UAV IoT data acquisition constraints, as described in this invention, includes the following steps:

[0007] S1: Scenario modeling of multi-UAV IoT data acquisition system, construction of UAV continuous motion model and time-varying wireless channel model, definition of multi-objective joint optimization function of data acquisition efficiency, flight energy efficiency, task balance and safety constraints, and transformation of path planning problem into distributed random game framework.

[0008] S2: Construct a spatiotemporal dual-domain attention feature encoding module. Encode the spatial correlation features of UAVs, IoT nodes, and obstacles through a spatial domain graph attention network, and encode the dynamic change features of time-varying channels through a temporal domain temporal attention network. Fuse to generate individual observation features of UAVs.

[0009] The execution process of the spatiotemporal dual-domain attention feature encoding module is as follows:

[0010] The spatial domain graph attention network encodes the spatial correlation features of the drone, IoT nodes, and obstacles, and outputs a spatial domain feature vector. The temporal domain temporal attention network encodes the dynamic change features of the time-varying channel, and outputs a temporal domain channel feature vector. The spatial domain features, temporal domain features, and the drone's own state vector are fused to generate the individual observation feature vector of the drone.

[0011] S3: Executes a multi-agent continuous decision-making process with embedded look-ahead constraints, transforming hard constraints such as obstacle avoidance, collision prevention, and power safety into action masking mechanisms and penalty terms. Through iterative updates of the critic network, policy network, and target network, it outputs the continuous waypoint planning actions of the UAV.

[0012] S4: The model training is completed using a hierarchical collaborative meta-learning framework. The upper layer achieves load balancing for multiple drones through task allocation, while the lower layer achieves rapid model convergence and adaptation to new scenarios with few samples through hierarchical meta-learning, thus completing the entire process of building the path planning model.

[0013] Preferably, the specific steps for constructing the continuous motion model of the UAV in step S1 are as follows:

[0014] No. A drone in The continuous state vector at time t is:

[0015]

[0016] in, For the three-dimensional spatial coordinates of the UAV, For the flight speed of the drone, The horizontal heading angle of the drone's flight. The pitch angle of the drone's flight. This is the ratio of the drone's remaining battery power to its total battery power, with a value ranging from [value missing]. ;

[0017] The continuous motion vector of the drone is: , For tangential acceleration, The rate of change of heading angle, Let the pitch angle change rate be denoted as , and the state transition equation for the UAV be:

[0018]

[0019] in, For decision step size, Here is the state transition matrix. This is the ambient airflow disturbance noise vector.

[0020] The specific steps for constructing the time-varying wireless channel model in step S1 are as follows:

[0021] No. The drone and the first The time-varying channel capacity of an IoT node at time t is:

[0022]

[0023] in, For communication bandwidth, For the first The launch power of the drone For the first The transmit power of the drone, among which That is, except for the one currently being interacted with. The first IoT node communication (The transmission power of all other drones operating simultaneously, excluding the drone itself). For small-scale Rayleigh fading channel vectors, Let be the three-dimensional Euclidean distance of the i-th ∈ [ ]. The path loss index is a time-varying value, updated in real time based on line-of-sight or non-line-of-sight conditions. This represents the power of Gaussian white noise.

[0024] The multi-objective joint function in step S1 is:

[0025]

[0026] in , , , Let the weights of each optimization objective be 1, and let their sums be 1. For the total data collection revenue of the system, For swarm flight energy efficiency, To balance multi-machine task performance, For the degree of satisfaction of safety constraints.

[0027] Preferably, the specific steps for constructing the spatiotemporal dual-domain attention feature encoding module in step S2 are as follows:

[0028] (2-1) Spatial Domain Feature Encoding: Using drones, IoT nodes, and obstacles as graph nodes, and spatial distance and channel correlation between nodes as edge weights, a dynamic topology graph is constructed. A three-layer graph attention network is used to extract spatial correlation features between nodes, and the spatial domain feature vector is output. ;

[0029] (2-2) Time-domain feature coding: Collecting historical channel state sequences of UAV and surrounding nodes ,in Given the historical window length, a two-layer temporal attention network is used to output the future... Channel state prediction values ​​at step size are used to extract time-domain channel feature vectors. ;

[0030] (2-3) Multi-dimensional feature fusion: Spatial domain feature vectors, temporal domain feature vectors, and the UAV's own continuous state vectors are concatenated and nonlinearly mapped through a fully connected layer to generate the final individual observation feature vector. .

[0031] Preferably, step S3 specifically includes:

[0032] (3-1) Construct a composite reward function with constraint embedding, where the single-step immediate reward is:

[0033]

[0034] in, For data collection revenue items, To interfere with and suppress reward items, For task-balanced reward items, To avoid obstacles and prevent future penalties, For multi-machine collision avoidance forward penalty items, Foresight penalties related to battery safety;

[0035] (3-2) Design an action masking mechanism to pre-filter actions that violate hard security constraints at the policy network output layer, set the output probability of illegal actions to 0, and retain only the output distribution of compliant actions;

[0036] (3-3) The network is iteratively updated based on the maximum entropy soft actor critic framework. The loss function of the policy network in the soft actor critic network (SAC) is:

[0037]

[0038] in, The temperature coefficient of the SAC algorithm balances the exploratory and exploitative aspects of the strategy; the larger the value, the more inclined the UAV is to explore new paths. For parameters The policy network, input the first Individual observations of a drone output a Gaussian distribution of continuous flight maneuvers. For the first A drone in Individual observation vector at time; , The parameters are respectively , Two independent networks of critics; , These are the trainable parameters for the first and second critic networks, respectively.

[0039] (3-4) By sampling continuous actions through reparameterization techniques, the continuous waypoint planning results of the UAV are output to complete single-step decision-making.

[0040] Preferably, step S4 specifically includes:

[0041] (4-1) Upper-layer federated task allocation: Based on the remaining battery power, location and channel status of each UAV, distributed task allocation of IoT nodes is achieved through federated reinforcement learning, without the need for the participation of a central node, ensuring the balanced task load of multiple UAVs;

[0042] (4-2) Lower-level curriculum-based meta-training: Construct curriculum-based task sets from simple to complex, and progressively train from single-drone obstacle-free scenarios to multi-drone complex urban scenarios. Optimize the initial parameters of the model through model-independent meta-learning, so that the model has the ability to adapt to new scenarios with few samples.

[0043] (4-3) Model Deployment: After training, the policy network is deployed to each UAV's onboard computing unit. The UAVs independently output continuous waypoint actions based solely on their own observation features, without the need for real-time communication between UAVs, thus completing distributed path planning.

[0044] Beneficial Effects: This invention aims to maximize data acquisition efficiency, minimize flight energy consumption, reduce multi-aircraft co-frequency interference, and minimize task imbalance. It fully utilizes channel state and spatial location information between the UAV and nodes, making it suitable for distributed IoT data acquisition scenarios with complex interference types and strict terrain constraints. Because the benefit-risk ratio of selectable waypoints is prioritized beforehand, if selecting a particular waypoint causes a performance degradation of the entire system, all subsequent feasible waypoints are disregarded, reducing computational complexity. Simultaneously, it achieves forward-looking perception of channel dynamics through spatiotemporal dual-domain feature encoding, and incorporates hard safety constraints into the decision-making process through a constraint embedding mechanism. This significantly reduces the risk of safety violations while improving overall system performance, demonstrating strong scenario generalization ability and engineering applicability. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0046] The following section provides a detailed explanation of the specific implementation method for a particular system model.

[0047] A continuous path planning method for multi-UAV IoT data acquisition constraints, such as Figure 1 As shown, it includes the following steps:

[0048] Step 1: Consider a multi-drone IoT data acquisition system, which has a total of A number of isomorphic quadrotor UAVs, where U∈[1,6], are deployed in the operational area. A number of single-antenna IoT monitoring nodes are distributed across the entire area of ​​the urban core area / mountainous operation scenario. In the spatiotemporal joint path planning system, the UAV's trajectory planning, data acquisition, and flight control are performed simultaneously. Assume the operation area is discretized into an M×M three-dimensional grid map with a grid resolution of c×c×c. Each grid position corresponds to an optional waypoint for the UAV. The UAV can only perform flight maneuvers within flyable grids; no-fly zones and grids containing obstacles are non-flyable areas. A drone in Time to the The channel model for each IoT node is as follows:

[0049] (1)

[0050] in, Indicates the first A drone in Time to the The three-dimensional Euclidean distance between IoT nodes The time-varying path loss coefficient is used in line-of-sight (LoS) scenarios. In non-line-of-sight (NLoS) scenarios The occlusion status between the drone and the node is switched in real time. The shadowing pattern follows a log-normal distribution with a mean of 0 and a variance of 0. Gaussian distribution; For communication carrier wavelength, These represent the gains of the drone's transmitting antenna and the IoT node's receiving antenna, respectively.

[0051] Step 2: Without considering interference and multi-drone collaboration, assume the minimum signal-to-noise ratio threshold for communication between the drone and the IoT node is... Based on the channel model described above, the effective communication range for the UAV to perform data acquisition on the k-th node can be determined as follows:

[0052] (2)

[0053] in, Let σk be the transmit power of the UAV, and σ2 be the Gaussian white noise power. Therefore, waypoints whose distance to the UAV is less than the above value and whose channel conditions meet the minimum communication requirements are the optional acquisition waypoints for the UAV to the k-th node. All reachable locations of the UAV within the globally flyable grid constitute the complete set of optional waypoints for the UAV. ,in Indicates the number of elements in the set. Indicates the first The sorted set of possible waypoints for the drone 1 waypoint.

[0054] Step 3: In a multi-UAV collaborative data acquisition system, multiple UAVs simultaneously perform flight and data acquisition tasks. When a UAV plans a new flight segment, it will be constrained by obstacles and no-fly zones, as well as by co-channel interference and collision risks from other UAVs. When a UAV adds a new data acquisition waypoint, it will cause co-channel interference to the communication links of other UAVs and will also be affected by interference from other UAVs. Since the final selected flight path for each UAV is not yet determined at this point, the indirect interference impact of adding a waypoint on other UAVs is not considered. If a waypoint is selected, the ratio of its channel capacity gain to the interference power and safety risk to the system is defined as the benefit-risk ratio. Then, based on this value, the available waypoints for each UAV are sorted, and spatiotemporal dual-domain feature encoding is completed. The specific steps are as follows:

[0055] make To introduce waypoints The payoff-risk ratio, its general definition is shown in equation (3):

[0056] (3)

[0057] in, For drone waypoints The sum of the total channel capacity for all nodes. This is the spatial distance from the waypoint to the nearest drone. This is the spatial distance from the waypoint to the nearest obstacle. This represents the total co-channel interference power experienced at this waypoint. , , These are the weighting coefficients for collision risk, obstacle risk, and interference risk, respectively, and the sum of the three is 1.

[0058] Sort the available waypoints for each drone in descending order of their payoff-risk ratio to obtain the sorted set of available waypoints. ,in Indicates the number of elements in the set. Indicates the first The sorted set of possible waypoints for the drone 1 waypoint.

[0059] Step 4: Simultaneously perform spatiotemporal dual-domain attention feature encoding. The specific process is as follows:

[0060] (4-1) Spatial Domain Feature Encoding: Using drones, IoT nodes, and obstacles as graph nodes, and spatial distance and channel correlation between nodes as edge weights, a dynamic topology graph is constructed. Spatial correlation features between nodes are extracted through a 3-layer graph attention network, and spatial domain feature vectors are output. ;

[0061] (4-2) Time-domain feature encoding: Collect historical channel state sequences of the UAV and surrounding nodes. ,in Given the historical window length, a two-layer temporal attention network is used to output the future... Channel state prediction values ​​at step size are used to extract time-domain channel feature vectors. ;

[0062] (4-3) Multi-dimensional feature fusion: Spatial domain feature vectors, temporal domain feature vectors, and the UAV's own continuous state vectors are concatenated and nonlinearly mapped through a fully connected layer to generate the final individual observation feature vector. .

[0063] The channel-aware multi-UAV IoT data acquisition spatiotemporal joint path planning method described in this specific embodiment includes the following steps:

[0064] S1: The first in a multi-UAV IoT data acquisition system All waypoints in the available waypoint set for the UAV are sorted in descending order according to their benefit-risk ratio. Simultaneously, a spatiotemporal dual-domain attention feature encoding module is used to extract and fuse features in the spatial and temporal domains, generating an individual observation feature vector for the UAV. The resulting sorted set of available waypoints is then obtained. Initialize the traversal index n=1.

[0065] S2: Calculate the nth waypoint sequentially according to the sorted order of the set of available waypoints. When selected, the multi-objective optimization benefit value of the entire multi-UAV data acquisition system is as follows: If the multi-objective optimization benefit value of the system is improved after the current waypoint is selected compared with that before selection, then the waypoint is added to the set of selected waypoints of the i-th UAV. At the same time, based on the multi-agent soft actor critic algorithm with look-ahead constraints, the continuous flight actions, acquisition sequence and dwell time of the UAV at the waypoint are output, and step S3 is executed; if the benefit value is not improved, step S3 is executed directly.

[0066] S3: Judgment Is it equal to If equal, then let Then proceed to step S4; otherwise, let Then return to step S2;

[0067] S4: Determine if it equals U, where U is the total number of drones in the multi-drone IoT data acquisition system: If it equals, end; otherwise, set... n, then return to step S1.

[0068] Furthermore, in step S2, the first The first drone After each waypoint is selected, the multi-objective optimization benefit value of the entire multi-UAV IoT data acquisition system is calculated according to equation (1):

[0069] (4)

[0070] In equation (4), , , , The weight coefficients for each optimization objective satisfy... ; For the total data collection revenue of the system, For swarm flight energy efficiency, To balance multi-machine task performance, For the satisfaction of safety constraints, the general calculation method for each item is as follows:

[0071] (5)

[0072] In equation (5), Let be the amount of data collected by the i-th drone from the k-th IoT node. The total amount of data to be collected from all nodes; Let i be the total flight energy consumption of the i-th UAV. This represents the maximum usable battery capacity for a single drone. The total amount of data collected by the i-th drone. The average amount of data handled by a single drone; To constrain the number of violations of system constraints, This represents the total number of decision-making steps.

[0073] In step S2, the continuous state vector and state transition equation of the UAV are:

[0074] (6)

[0075] in, For the three-dimensional spatial coordinates of the UAV, For the flight speed of the drone, The horizontal heading angle of the drone's flight. The pitch angle of the drone's flight. This is the ratio of the drone's remaining battery power to its total battery power, with a value ranging from [value missing]. ;

[0076] The continuous motion vector of the drone is: The parameters are tangential acceleration, rate of change of heading angle, and rate of change of pitch angle, respectively. The state transition equation for the UAV is:

[0077] (7)

[0078] in, For decision step size, Here is the state transition matrix. This is the ambient airflow disturbance noise vector.

[0079] In step S2, the instantaneous channel capacity of the drone and the IoT node is calculated according to equation (8):

[0080] (8)

[0081] in, For communication bandwidth, For the first The launch power of the drone For small-scale Rayleigh fading channel vectors, Let be the three-dimensional Euclidean distance of the i-th ∈ [ ]. The path loss index is a time-varying value, updated in real time based on line-of-sight or non-line-of-sight conditions. The power of Gaussian white noise;

[0082] Furthermore, in step S2, the multi-agent soft actor critic algorithm based on look-ahead constraint embedding outputs the continuous flight actions, data acquisition sequence, and dwell time of the UAV. The specific calculation process is as follows:

[0083] Step 1: Construct a composite reward function with constraint embedding. The single-step instant reward is calculated according to equation (9):

[0084] (9)

[0085] in, For data collection revenue items, To interfere with and suppress reward items, For task-balanced reward items, To avoid obstacles and prevent future penalties, For multi-machine collision avoidance forward penalty items, The three types of penalties are calculated as shown in equation (10): (This is a forward-looking penalty for power safety.)

[0086] (10)

[0087] in, The multi-machine collision avoidance safety distance threshold, The threshold for obstacle safety warning, For penalty weighting; This is the estimated remaining battery power at the end of the drone mission. The minimum safe power threshold, , , This represents the weighting coefficient for the corresponding penalty item.

[0088] The second step is to design an action masking mechanism to pre-filter actions that violate hard security constraints at the policy network output layer. This will set the output probability of non-compliant actions to 0 and retain only the output distribution of compliant actions, thus avoiding the risk of constraint violation from the source of decision-making.

[0089] The third step involves constructing a network model based on the maximum entropy soft actor / critic framework. This model includes a dual-critic network, a single-policy network, and a dual-objective critic network. The dual-critic network takes the global state and multi-machine joint actions as input and outputs the state-action soft Q-value. The policy network takes the individual UAV observation feature vector as input and outputs the Gaussian distribution of continuous actions. The policy network and critic network are iteratively updated alternately based on a loss function. The loss function for the dual-critic network is:

[0090] (11)

[0091] In equation (11), the target value is:

[0092] γ is the discount factor. The temperature coefficient of the SAC algorithm balances the exploratory and exploitative aspects of the strategy; a larger value indicates that the UAV is more inclined to explore new paths.

[0093] For parameters The policy network, input the first Individual observations of a drone output a Gaussian distribution of continuous flight maneuvers. For the first A drone in Individual observation vector at time; , The parameters are respectively , Two independent networks of critics; , These are the trainable parameters for the first and second critic networks, respectively.

[0094] The loss function of the policy network is calculated according to equation (12):

[0095] (12)

[0096] Step 4: Sample continuous actions using reparameterization techniques; action vector:

[0097] The parameters are tangential acceleration, rate of change of heading angle, rate of change of pitch angle, dwell time, and transmit power. The system outputs the continuous waypoint planning results, data acquisition sequence, and dwell time of the UAV, and completes single-step decision-making.

Claims

1. A continuous path planning method for multi-UAV IoT data acquisition constraints, characterized in that, Includes the following steps: S1: Scenario modeling of multi-UAV IoT data acquisition system, construction of UAV continuous motion model and time-varying wireless channel model, definition of multi-objective joint optimization function of data acquisition efficiency, flight energy efficiency, task balance and safety constraints, and transformation of path planning problem into distributed random game framework. S2: Construct a spatiotemporal dual-domain attention feature encoding module. Encode the spatial correlation features of UAVs, IoT nodes, and obstacles through a spatial domain graph attention network, and encode the dynamic change features of time-varying channels through a temporal domain temporal attention network. Fuse to generate individual observation features of UAVs. S3: Executes a multi-agent continuous decision-making process with embedded look-ahead constraints, transforming hard constraints such as obstacle avoidance, collision prevention, and power safety into action masking mechanisms and penalty terms. Through iterative updates of the critic network, policy network, and target network, it outputs the continuous waypoint planning actions of the UAV. S4: The model training is completed using a hierarchical collaborative meta-learning framework. The upper layer achieves load balancing for multiple drones through task allocation, while the lower layer achieves rapid model convergence and adaptation to new scenarios with few samples through hierarchical meta-learning, thus completing the entire process of building the path planning model.

2. The method according to claim 1, characterized in that, The specific steps for constructing the continuous motion model of the UAV in step S1 are as follows: No. A drone in The continuous state vector at time t is: in, For the three-dimensional spatial coordinates of the UAV, For the flight speed of the drone, The horizontal heading angle of the drone's flight. The pitch angle of the drone's flight. This is the ratio of the drone's remaining battery power to its total battery power, with a value ranging from [value missing]. ; The continuous motion vector of the drone is: , For tangential acceleration, The rate of change of heading angle, Let the pitch angle change rate be denoted as , and the state transition equation for the UAV be: in, For decision step size, Here is the state transition matrix. This is the ambient airflow disturbance noise vector.

3. The method according to claim 2, characterized in that, The specific steps for constructing the time-varying wireless channel model in step S1 are as follows: No. The drone and the first The time-varying channel capacity of an IoT node at time t is: in, For communication bandwidth, For the first The launch power of the drone For the first The transmit power of the drone, among which That is, except for the one currently being interacted with. The first IoT node communication (The transmission power of all other drones operating simultaneously, excluding the drone itself). For small-scale Rayleigh fading channel vectors, It is a three-dimensional Euclidean distance. The path loss index is a time-varying value, updated in real time based on line-of-sight or non-line-of-sight conditions. This represents the power of Gaussian white noise.

4. The method according to claim 3, characterized in that, The multi-objective joint function in step S1 is: in , , , Let the weights of each optimization objective be 1, and let their sums be 1. For the total data collection revenue of the system, For swarm flight energy efficiency, To balance multi-machine task performance, For the degree of satisfaction of safety constraints.

5. The method according to claim 1, characterized in that, The specific steps for constructing the spatiotemporal dual-domain attention feature encoding module in step S2 are as follows: (2-1) Spatial Domain Feature Encoding: Using drones, IoT nodes, and obstacles as graph nodes, and spatial distance and channel correlation between nodes as edge weights, a dynamic topology graph is constructed. A three-layer graph attention network is used to extract spatial correlation features between nodes, and the spatial domain feature vector is output. ; (2-2) Time-domain feature coding: Collecting historical channel state sequences of UAV and surrounding nodes ,in Given the historical window length, a two-layer temporal attention network is used to output the future... Channel state prediction values ​​at step size are used to extract time-domain channel feature vectors. ; (2-3) Multi-dimensional feature fusion: Spatial domain feature vectors, temporal domain feature vectors, and the UAV's own continuous state vectors are concatenated and nonlinearly mapped through a fully connected layer to generate the final individual observation feature vector. .

6. The method according to claim 1, characterized in that, Step S3 specifically includes: (3-1) Construct a composite reward function with constraint embedding, where the single-step immediate reward is: in, For data collection revenue items, To interfere with and suppress reward items, For task-balanced reward items, To avoid obstacles and prevent future penalties, For multi-machine collision avoidance forward penalty items, Foresight penalties related to battery safety; (3-2) Design an action masking mechanism to pre-filter actions that violate hard security constraints at the policy network output layer, set the output probability of illegal actions to 0, and retain only the output distribution of compliant actions; (3-3) The network is iteratively updated based on the maximum entropy soft actor critic framework. The loss function of the policy network in the soft actor critic network (SAC) is: in, The temperature coefficient of the SAC algorithm; For parameters The policy network, input the first Individual observations of a drone output a Gaussian distribution of continuous flight maneuvers; For the first A drone in Individual observation vector at time; , The parameters are respectively , Two independent networks of critics; , These are the trainable parameters for the first and second critic networks, respectively. (3-4) By sampling continuous actions through reparameterization techniques, the continuous waypoint planning results of the UAV are output to complete single-step decision-making.

7. The method according to claim 1, characterized in that, Step S4 specifically includes: (4-1) Upper-layer federated task allocation: Based on the remaining battery power, location and channel status of each UAV, distributed task allocation of IoT nodes is achieved through federated reinforcement learning, without the need for the participation of a central node, ensuring the balanced task load of multiple UAVs; (4-2) Lower-level curriculum-based meta-training: Construct curriculum-based task sets from simple to complex, and progressively train from single-drone obstacle-free scenarios to multi-drone complex urban scenarios. Optimize the initial parameters of the model through model-independent meta-learning, so that the model has the ability to adapt to new scenarios with few samples. (4-3) Model Deployment: After training, the policy network is deployed to each UAV's onboard computing unit. The UAVs independently output continuous waypoint actions based solely on their own observation features, without the need for real-time communication between UAVs, thus completing distributed path planning.

Images