A method and system for multi-UAV path planning

Abstract

This invention discloses a multi-UAV path planning method and system. The method includes: acquiring three-dimensional spatial environment information of multiple UAVs using the UAV's onboard sensor system; constructing a deep reinforcement learning model based on the three-dimensional spatial environment information; wherein the deep reinforcement learning model includes a state space, an action space, a reward function, and a network architecture, and the action space includes actions corresponding to two different flight modes; constructing a privacy protection model to protect training data containing sensitive UAV information; interactively training the UAVs with the environment to learn the path planning decision method; and applying the learned path planning decision method to a UAV swarm perception task to achieve path planning. This invention enables UAVs to autonomously select flight modes, improving their ability to perform long-duration perception tasks.

Description

Technical Field

[0001] This invention belongs to the field of path planning technology for unmanned aerial vehicle (UAV) swarm intelligence sensing tasks, and particularly relates to a multi-UAV path planning method and system. Background Technology

[0002] In UAV swarm intelligence perception tasks, UAVs must navigate complex environments containing both fixed obstacles (such as trees and buildings) and moving obstacles (such as other UAVs) to perform perception tasks. Traditional graph theory and optimization theory methods face challenges in path planning in such scenarios, especially in dynamic environments. Deep reinforcement learning, as an advanced self-learning method, learns optimal policies through interaction with the environment, enabling it to handle high-dimensional state spaces and respond quickly to environmental changes, providing a new approach for dynamic path adjustment of UAVs. However, path planning in UAV swarm intelligence perception tasks faces two key problems.

[0003] First, drones are limited by energy constraints when performing long-duration missions, which restricts their flight time and affects the efficiency of completing these missions. Although some studies have attempted to extend flight time by introducing charging stations and integrating charging decisions into deep reinforcement learning models, these methods mix data collection and charging actions, leading to chaotic action execution and reduced energy management efficiency.

[0004] Secondly, existing research based on deep reinforcement learning mainly focuses on model design to optimize UAV flight paths, neglecting the security of sensitive UAV information during deep reinforcement learning model training. During model training, a large amount of data containing sensitive information is generated and uploaded, posing a risk of privacy leakage. Although existing research (such as using differential privacy or joint differential privacy) has attempted to protect training data and model privacy, these methods are mainly applicable to simple environments (such as Atari games) and are difficult to directly apply to highly dynamic and complex UAV swarm intelligence perception tasks. They struggle to maintain data availability while ensuring data privacy, thus affecting the path planning performance of UAV swarm intelligence perception tasks. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a multi-UAV path planning method and system to resolve the issues present in the prior art.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a multi-UAV path planning method, comprising:

[0007] By utilizing the onboard sensor system of drones, three-dimensional spatial environment information of multiple drones can be obtained;

[0008] Based on the three-dimensional spatial environment information, a deep reinforcement learning model is constructed; wherein, the deep reinforcement learning model includes a state space, an action space, and a reward function, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes;

[0009] A privacy protection model is constructed to protect training data containing sensitive information about drones.

[0010] The drone is trained to interact with the environment to learn path planning and decision-making methods.

[0011] The learned path planning decision-making method is applied to the UAV swarm perception task to achieve path planning.

[0012] Preferably, the state space includes a global state space and a local state space, wherein the global state space includes all environmental information of obstacles, charging piles, signal points, and the UAV, and the local state information includes personal information of the UAV and information of obstacles, charging piles, and signal points within the perception range, and the reward function is designed according to the task objective.

[0013] Preferably, the flight mode includes a data collection mode and a charging mode.

[0014] Preferably, the global state space in the state space includes obstacles, charging piles, signal points, and environmental information of the UAV; the local state space includes information on obstacles, charging piles, signal points within the UAV's perception range, and personal information of the UAV; and the reward function is designed according to the task objective.

[0015] Preferably, the privacy protection model includes an encoder and a decoder;

[0016] The encoder maps the actual observations of the UAV to the potential space.

[0017] Generative data is generated by the decoder based on latent variables randomly sampled from the latent space, and then replaced with real observations.

[0018] Secondly, the present invention also provides a multi-UAV path planning system, comprising:

[0019] The information acquisition module is used to acquire three-dimensional spatial environment information of multiple UAVs using the UAV's onboard sensor system;

[0020] The first model building module is used to build a deep reinforcement learning model based on the three-dimensional spatial environment information; wherein the deep reinforcement learning model includes a state space, an action space, a reward function, and a network architecture, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes;

[0021] The second model building module is used to build a privacy protection model, which protects training data containing sensitive information about drones.

[0022] The interactive training module is used to conduct interactive training between the UAV and the environment based on the training data, and to complete the learning of path planning decision-making methods without exposing real training data.

[0023] The path planning module is used to apply the learned path planning decision-making methods to UAV swarm intelligence perception tasks in order to achieve energy-sustainable path planning.

[0024] Thirdly, the present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0025] Fourthly, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0026] Fifthly, the present invention also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0027] Compared with the prior art, the present invention has the following advantages and technical effects:

[0028] This invention provides a multi-UAV path planning method, comprising: first, acquiring three-dimensional spatial environment information of multiple UAVs using the UAVs' onboard sensor system; second, constructing a deep reinforcement learning model based on the three-dimensional spatial environment information; wherein the deep reinforcement learning model includes a state space, an action space, and a reward function, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes; next, constructing a privacy protection model to protect training data containing sensitive UAV information; further, interactive training of UAVs and the environment based on the training data to complete the learning of the path planning decision method without exposing the real training data; finally, applying the learned path planning decision method to a UAV swarm intelligence perception task to achieve energy-sustainable path planning.

[0029] This invention integrates a deep reinforcement learning model and a privacy-preserving model. Through their combined action, drones can train two different flight strategies without exposing their personal training data to an untrusted central server. This protects the drone's sensitive information while achieving effective path planning for efficient completion of perception tasks. This invention addresses the privacy and security issues of training data during drone swarm intelligence perception strategy training, as well as the problem of insufficient energy replenishment when using a single strategy for simultaneous data collection and charging. By integrating a privacy-preserving model and a deep reinforcement learning model, it achieves a safe and energy-sustainable path planning method for drone swarm intelligence perception tasks.

[0030] This invention enables drones to learn two different flight modes while protecting their personal information, allowing them to autonomously select flight modes and improving their ability to perform long-duration perception tasks. Attached Figure Description

[0031] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is an overall flowchart of the multi-UAV path planning method according to an embodiment of the present invention;

[0033] Figure 2 This is a structural diagram of the deep reinforcement learning model for flight strategy according to an embodiment of the present invention;

[0034] Figure 3 This is a structural diagram of the privacy protection model according to an embodiment of the present invention;

[0035] Figure 4 This is a flowchart illustrating the interactive training process according to an embodiment of the present invention;

[0036] Figure 5 This is a flowchart illustrating the deployment process of the path planning method according to an embodiment of the present invention. Detailed Implementation

[0037] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0039] The technical terms used in the following embodiments will be explained first.

[0040] Multi-drone path planning refers to the algorithmic determination of each drone's flight path during a collaborative task, ensuring they reach the target point efficiently and safely while avoiding obstacles and collisions with other drones. The core objective of multi-drone path planning is to optimize the overall performance of the drones, including minimizing flight time and maximizing energy efficiency.

[0041] Key technologies and methods for multi-UAV path planning include:

[0042] Environmental perception: Drones acquire information about their surrounding environment through sensors, such as map data and sensor measurement results, and build environmental models for path search and optimization.

[0043] Path search: Commonly used path search algorithms include A* algorithm, Dijkstra's algorithm, and depth-first search algorithm. These algorithms construct the path of the UAV based on a graph model and use heuristic functions or priority queues for searching.

[0044] Path optimization: The path of the UAV is continuously optimized through methods such as dynamic programming, genetic algorithm and particle swarm optimization to improve efficiency and safety.

[0045] Obstacle avoidance and collision avoidance: Multi-UAV path planning requires consideration of obstacle avoidance and collision avoidance issues. Improved speed obstacle models and distributed path planning algorithms can help UAVs fly safely in complex environments.

[0046] Multi-UAV path planning is widely used in battlefield reconnaissance, target identification, tracking, and strike missions. Because multi-UAV systems involve the coordination of multiple UAVs, path planning needs to solve a large-scale constrained multi-objective optimization problem, ensuring that each UAV safely reaches the target point under obstacle avoidance and collision avoidance constraints. Furthermore, multi-UAV path planning also needs to consider factors such as the number of UAVs, their functions, and coordination methods to achieve a global objective.

[0047] Example 1

[0048] To address the two key challenges in path planning for UAV swarm intelligence sensing tasks mentioned in the background section, this embodiment provides a multi-UAV path planning method, such as... Figure 1 As shown, it includes:

[0049] Step 1: Construct a simulation environment for the perception task. The UAV uses its onboard sensor system to acquire environmental information, including signal point P, obstacles (such as trees, tall buildings, etc.) B, and charging pile C, to simulate the real-world perception task scenario as closely as possible. At time t, the UAV u uses its onboard sensor system to observe the environment and obtain local observations.

[0050] Step 2: Construct a deep reinforcement learning model for safe and energy-sustainable route planning decisions;

[0051] like Figure 2 As shown, it includes the following steps:

[0052] 2-1) Construct the state space, action space, and reward function for deep reinforcement learning.

[0053] A) The state space includes a global state space and a local state space. The global state space includes all environmental information, including obstacles, charging stations, signal points, and the drone itself, and represents global observation. The local state space represents local observations acquired by the drone, and is constructed as follows;

[0054]

[0055] in, This represents the distance between the drone and the point of impact of its m-th laser beam on an obstacle; if the obstacle is not hit, it represents the maximum length of the laser beam. This represents the height of the obstacle hit by the m-th laser beam; if it is not hit, it is represented as 0. This indicates the direction from the drone u to the signal point p; This represents the distance from the drone u to the signal point p; This represents the amount of data remaining at signal point p at time t; This represents the attractive force of signal point p on drone u; Indicates the direction the drone travels to charging station c; This indicates the attractiveness of the charging station (c) to the drone (u); H min and H max These represent the minimum and maximum permitted flight altitudes for drones, respectively. This represents the flight altitude of the drone u at time t; Indicates the flight speed of the drone; This indicates the drone's current remaining energy; This indicates the current state of the drone, either in data collection or charging mode. This indicates whether the drone is in trouble, i.e., whether it is hovering in the same place.

[0056] B) As an additional implementation method, the UAV generates two different actions based on its own local observations. and These correspond to data collection mode and charging mode, respectively;

[0057] C) As an additional implementation method, the reward function is designed based on the perception task objectives. In the UAV swarm perception task, the following objectives need to be achieved: ensuring the safety of UAV flight and the security of its training data, maximizing the sum of perception data collected by all UAVs, maximizing the diversity of collected data, and minimizing the energy consumption of UAVs.

[0058] As an additional implementation method, based on the above objectives, the reward function is constructed as follows:

[0059]

[0060] in, These are penalty items. In data collection mode, the drone will be penalized if it fails to collect data or recharge. In charging mode, the drone will be penalized if it fails to recharge or collect data. In either mode, the drone will be penalized for colliding with obstacles. It is the set of signal points within the sensing range R of the UAV at time t. It is the initial energy of the drone. This represents the amount of battery power replenished by the drone at time t. D(p) represents the action performed by the drone. Then, the amount of perception data collected by the drone. Indicates the actions performed by the drone Subsequently, the diversity of the collected data, specifically Its structure is as follows:

[0061]

[0062] in, This represents the initial data volume at signal point p. This represents the number of signal points within the sensing range R of the UAV at time t. This represents the amount of data remaining at signal point p at time t. The energy consumption function is constructed as follows:

[0063]

[0064] in, This represents the amount of data collected by the drone u at time t. α represents the distance traveled by the drone u at time t, α1 and α2 represent the power consumed by the drone in collecting a unit of data and traveling a unit of distance, respectively, and α3 is the energy consumed by the drone in communicating with the central controller.

[0065] 2-2) As an additional implementation method, a network model based on deep reinforcement learning is constructed. In deep reinforcement learning, an actor network typically only determines one action. However, in energy-sustainable drone swarm intelligence perception, drones have at least two actions: data collection and charging. If the same actor network is used to determine both actions, the intended purpose of the drone's actions becomes ambiguous, affecting training effectiveness.

[0066] As an additional implementation, this embodiment uses two independent actor networks to generate two different actions, and uses a critic network to evaluate the strategies of the two actor networks. Thus, each drone includes an actor network π in the data collection mode. coll Actor Network π in Charging Mode char Critics network Q, target actor network π' under data collection model coll Target actor network π' in charging mode char The target critic network Q' is a network of six networks, π coll π char Q, π' coll ,π' char The network parameters of Q' are respectively θ Q , θ Q‘ Among them, the drones use an actor network to generate actions based on local observations; specifically, π coll Used to generate actions that collect sensory data, π char The action of generating the charge is used by the critic network to evaluate the expected reward of taking the action in that state, in order to provide useful feedback to both actors.

[0067] 2-3) Set the composition and parameters of the deep reinforcement learning-based network model, specifically including:

[0068] A) Both actor networks consist of four fully connected layers, with 512, 256, 128, and 3 neurons per layer, respectively. Each fully connected layer is connected by a LeakyReLU activation function, and the last fully connected layer is followed by a Tanh activation function to control the generated actions between -1 and 1. The input to both actor networks is the local observations of the UAV itself. After passing through a fully connected layer and an activation function, the outputs are respectively... and

[0069] B) The critic network consists of three fully connected layers and one attention mechanism layer. The fully connected layers have 512, 256, and 1 neuron respectively. The attention mechanism layer uses a single-head attention mechanism to process all local observations from the drones, with an embedding dimension of 512. The input to the critic network is a batch of (o,a), and the output is Q(o,a|θ). Q ).

[0070] C) Target Actor Network π' coll and π' char Each with the corresponding actor network π coll and π char The target critic network Q' has the same structure as the corresponding critic network Q.

[0071] Step 3: Construct a privacy-preserving model for secure and energy-sustainable route planning decisions;

[0072] Step three includes the following steps:

[0073] 3-1) Construct a privacy-preserving model based on variational autoencoders;

[0074] As an enhanced implementation, to better capture the spatiotemporal characteristics of UAV observations, a Long Short-Term Memory (LSTM) network was incorporated into the encoder, and a multi-head attention mechanism and an LSM network were combined in the decoder. The encoder is responsible for mapping the UAV's real observations to the latent space, while the decoder is responsible for generating generated data similar to the real observations based on latent variables randomly sampled from the latent space. The network parameters of the encoder and decoder are θ. en and θ de .

[0075] 3-2) Set the composition and parameters of the privacy-preserving model based on variational autoencoders, specifically including:

[0076] A) As an additional implementation, the encoder consists of a Long Short-Term Memory (LSTM) network layer and two linear layers. The LSM network has 512 hidden neurons, and the two linear layers have 128 neurons each. The encoder's input is batches of real observations from the UAV. After processing by the LSM network, the hidden state at the last time step is obtained. Then, the mean vector and logarithmic variance are calculated using the two linear layers respectively.

[0077] B) As an additional implementation, the decoder consists of an embedding layer, a multi-head attention layer, a long short-term memory (LSTM) network layer, and an output layer. The embedding layer has 512 neurons, the multi-head attention layer uses two attention heads with an embedding dimension of 512, the LSM network layer has 512 hidden neurons, and the output layer has the same number of neurons as the dimension of the actual observation. The input to the decoder is (∈, o), where ∈ is a latent vector randomly sampled from the distribution consisting of the mean and logarithmic variance of the encoder output. First, the latent vector ∈ and the actual observation o are embedded into the embedding dimension of the multi-head attention layer through the embedding layer, thus obtaining the input to the multi-head attention layer:

[0078]

[0079] The specific steps of the multi-head attention layer are as follows:

[0080]

[0081] Concat(·) is used to concatenate the outputs of the individual attention heads, and Attention(·) is used to calculate the attention of a single attention head. and They are and The segment of the i-th head. Then, for each attention head, calculate the attention score:

[0082]

[0083] Where d k Indicates the embedding dimension.

[0084] Finally, the result is passed through a linear layer to obtain the final input to the multi-head attention layer. Then, the result is further processed by a long short-term memory network layer, taking the hidden state of the last time step as the input to the output layer. Finally, the output layer maps the data to the same dimension as the actual observation to obtain the generated data.

[0085] Step 4: Unmanned aerial vehicle (UAV) interaction training with the environment to learn safe and energy-sustainable path planning and decision-making methods;

[0086] As an additional implementation method, such as Figure 4 As shown, it includes the following steps:

[0087] 4-1) Initialize the actor network π coll π char Critics' Network Q, Target Actor Network π' coll ,π' char The target critic network Q', where π coll πchar The network weight parameters of Q are randomly initialized using random numbers, π' coll ,π' char The network weight parameters of Q' are respectively derived from the corresponding network π. coll π char Obtained by copying from Q;

[0088] 4-2) Initialize the simulation environment. Each UAV observes the environment and obtains local observation values.

[0089] 4-3) Based on the observed values Each drone utilizes π coll and π char Two actions are generated and By introducing random noise ω, the actual action variables that the UAV should perform are obtained. and The drone selects one of two actions to perform based on its current flight mode and battery level: when the drone is in data collection mode, if the battery level is below 40%, it switches to charging mode and performs the action. Otherwise, maintain this flight mode. When the drone is in charging mode, if the battery level is below 40%, it will continue to operate in this mode and perform actions. Otherwise, switch to flight mode and perform the action.

[0090] 4-4) After the drone performs its actions, the environment changes accordingly, and the drone obtains new local observations. At the same time, you will receive the corresponding rewards.

[0091] 4-5) The drone utilizes a locally deployed privacy-preserving model, based on real observations... and Generate the corresponding generated data and

[0092] 4-6) Based on the type of action performed, or Store in the corresponding experience pool M coll Or M char In, and will Store separately in experience pool M vae In this context, it is used for training privacy-preserving models;

[0093] 4-7) Determine if the current training round number is greater than the set threshold ξ1. If not, skip directly to step 4-8. If it is greater, proceed from M according to the priority-based empirical replay method. coll and M char Randomly select data of the same batch size, including or The data includes the weights of each experience, and these two sets of data are concatenated as input to the critic network to calculate the Q-value and the critic network's loss function. The Q-value is the output of the critic network, and the loss function is calculated as follows:

[0094]

[0095] Where U represents the number of drones, λ t This represents the reward over N steps, where γ is the reward factor. This indicates the final action to be performed, used to uniformly represent... or Based on the Q-values ​​provided by critics, calculate the gradients of the two actor networks respectively:

[0096]

[0097] Using formulas (6) and (7), the parameters of the three networks are determined by gradient descent. θ Q Perform the update. Then, update the target network parameters corresponding to the three networks using a soft update method. θ Q‘ :

[0098]

[0099] 4-8) Determine if the current training round number is an integer multiple of the set threshold ξ2. If not, skip directly to step 4-9. If yes, proceed from M... vae The loss function of the privacy-preserving model is calculated by randomly sampling data of a certain batch size, and then the network weight parameters of the encoder and decoder are updated using gradient descent.

[0100]

[0101] Where L re Represents the reconstruction loss, calculated using the mean squared error, L KLD This represents the KL divergence loss, calculated using KL divergence. β is a hyperparameter used to balance the importance of reconstruction loss and KL divergence loss.

[0102] 4-9) Determine whether the drone has collided with obstacles or other drones during its interaction with the environment, or whether it has run out of energy. If either of these conditions is true, terminate the current training session and proceed to step 4-2). Otherwise, continue training until the perception task is completed or the drone's movement steps reach their maximum value before terminating the current training session and proceeding to step 4-10.

[0103] 4-10) Determine whether the two actor networks, the critic network, and the privacy protection model network have converged. If they have converged, end the training; otherwise, go to step 4-2) and start a new round of training.

[0104] Step 5: Deployment and application of safe and energy-sustainable route planning methods;

[0105] As an additional implementation method, such as Figure 5 As shown, it includes the following steps:

[0106] 5-1) Initialize the simulation environment. Each UAV observes the environment and obtains local observation values.

[0107] 5-2) Based on the observed values Each drone utilizes π coll and π char Two actions are generated and The drone selects one of two actions to perform based on its current flight mode and battery level: when the drone is in data collection mode, if the battery level is below 40%, it switches to charging mode and performs the action. Otherwise, maintain this flight mode. When the drone is in charging mode, if the battery level is below 40%, it will continue to operate in this mode and perform actions. Otherwise, switch to flight mode and perform the action.

[0108] 5-3) Determine if the perception task is completed. If completed, proceed to step 5-4; otherwise, execute... And jump to step 5-2);

[0109] 5-4) End the experiment.

[0110] Compared with the prior art, the beneficial effects of this embodiment are as follows:

[0111] This embodiment integrates a deep reinforcement learning model and a privacy protection model. With the combined effect of the two, the drone can complete the training of two different flight strategies without exposing its real training data to an untrusted central server. This protects the drone's sensitive information and achieves effective path planning to efficiently complete the perception task.

[0112] This embodiment uses a privacy-preserving model to generate generated data similar to real training data to replace real data for training, which solves the problem of sensitive information leakage in the drone training process and provides protection for drone data security.

[0113] This embodiment employs an improved actor-critic architecture, utilizing two independent actor networks responsible for outputting actions for data collection and actions for charging, respectively. The same critic network evaluates the value of the actions output by the two actor networks, thus merging two strategies with completely different and conflicting objectives. This allows the UAV to select different flight strategies under different circumstances, autonomously coordinate data collection and charging, and complete the collection of perception data through the combined effect of the two strategies.

[0114] This embodiment constructs a privacy-preserving model based on attention mechanisms and long short-term memory networks. This not only enables the encoder and decoder of the generative model to process data with spatiotemporal characteristics, but also improves the stability and performance of the encoding and decoding process, thereby generating higher-quality generated data.

[0115] Example 2

[0116] Based on the same inventive concept, this embodiment also provides a multi-UAV path planning system, including:

[0117] The information acquisition module is used to acquire three-dimensional spatial environment information of multiple UAVs using the UAV's onboard sensor system;

[0118] The first model building module is used to build a deep reinforcement learning model based on the three-dimensional spatial environment information; wherein the deep reinforcement learning model includes a state space, an action space and a reward function, wherein the action space includes actions corresponding to two different flight modes;

[0119] The second model building module is used to build a privacy protection model, which protects training data containing sensitive information about drones.

[0120] The interactive training module is used to train the UAV to interact with the environment and learn path planning and decision-making methods.

[0121] The path planning module is used to apply the learned path planning decision-making methods to UAV swarm intelligence perception tasks to achieve safe and energy-sustainable path planning.

[0122] The multi-UAV path planning system provided in this embodiment has all the advantages of the multi-UAV path planning method provided in Embodiment 1.

[0123] Example 3

[0124] This embodiment also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in Embodiment 1.

[0125] Example 4

[0126] This embodiment also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0127] Example 5

[0128] This embodiment also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0129] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for multi-UAV path planning, characterized in that, The method comprises the following steps: acquiring three-dimensional space environment information of multiple unmanned aerial vehicles (UAVs) by using an on-board sensor system of the UAVs; constructing a deep reinforcement learning model according to the three-dimensional space environment information, wherein the deep reinforcement learning model comprises a state space, an action space, a reward function and a network architecture, the action space comprises actions corresponding to two different flight modes, the state space comprises a global state space and a local state space, the global state space comprises all environment information of obstacles, charging piles, signal points and the UAVs, the local state information comprises individual information of the UAVs and information of obstacles, charging piles and signal points within a perception range of the UAVs, the reward function is designed according to a task target, the flight modes comprise a data collection mode and a charging mode, and the network architecture comprises an actor-critic architecture composed of two actor networks and one critic network, wherein the two actor networks are used to learn strategies in the two flight modes, respectively; the reward function is constructed as follows: wherein, is a penalty term, in the data collection mode, the UAV does not collect data or charging will be punished; in the charging mode, the UAV does not charge or collect data will be punished; no matter which mode, the UAV will be punished if it hits the obstacle; is a set of signal points within the sensing range R of the UAV u at time t; is the initial energy of the UAV; represents the amount of electricity replenished by the UAV at time t; D(p) represents the amount of sensing data collected by the UAV after performing action ; represents the diversity of the data collected by the UAV after performing action ; represents the energy consumption function; represents the amount of data collected by the UAV u at time t, represents the distance moved by the UAV u at time t; each UAV comprises a first actor network in the data collection mode, a second actor network in the charging mode, a critic network, a first target actor network in the data collection mode, a second target actor network in the charging mode and a target critic network; each UAV observes an environment to obtain local observation values; According to the local observation value, each unmanned aerial vehicle generates a first action and a second action by using a first actor network and a second actor network, and introduces random noise to obtain a first action variable and a second action variable that the unmanned aerial vehicle should actually perform and the second action variable The unmanned aerial vehicle selects one of the two actions to perform according to the current flight mode and the power level: when the unmanned aerial vehicle is in a data collection mode, if the power is less than forty percent, the mode is switched to a charging mode, and the second action variable is executed Otherwise, the flight mode is maintained to execute the first action variable When the unmanned aerial vehicle is in a charging mode, if the power is less than forty percent, the mode is continued to be maintained to execute, and the second action variable is executed Otherwise, the mode is switched to a flight mode, and the first action variable is executed a privacy protection model is constructed, and the training data containing sensitive information of the UAVs is protected by the privacy protection model; the privacy protection model comprises an encoder and a decoder, a long short-term memory network is added in the encoder, and a multi-head attention mechanism and a long short-term memory network are combined in the decoder; the real observation values of the UAVs are mapped to a latent space by the encoder; generated data is generated from the latent space according to a randomly sampled latent variable by the decoder, and the real observation values are replaced by the generated data; the UAVs and the environment are interactively trained according to the training data, and learning of a path planning decision method is completed; the learned path planning decision method is applied to a UAV swarm intelligence perception task to realize path planning.

2. A multi-UAV path planning system, comprising: The system for implementing the method of claim 1 comprises: an information acquisition module configured to acquire three-dimensional space environment information of multiple unmanned aerial vehicles (UAVs) by using an on-board sensor system of the UAVs; a first model construction module configured to construct a deep reinforcement learning model according to the three-dimensional space environment information, wherein the deep reinforcement learning model comprises a state space, an action space, a reward function and a network architecture, the state space comprises a global state space and a local state space, and the action space comprises actions corresponding to two different flight modes; a second model construction module configured to construct a privacy protection model, and protect training data containing sensitive information of the UAVs by the privacy protection model; an interactive training module configured to interactively train the UAVs and the environment, and complete learning of a path planning decision method; a path planning module configured to apply the learned path planning decision method to a UAV swarm intelligence perception task to realize path planning.

3. A computer apparatus comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 2. The processor executes the computer program to implement the steps of the method of claim 1.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by the processor, implements the steps of the method of claim 1.

5. A computer program product comprising a computer program, characterized in that, The computer program, which when executed by the processor, implements the steps of the method of claim 1.

Images