Multi-unmanned aerial vehicle path planning method and system

Abstract

The invention discloses a multi-unmanned aerial vehicle path planning method and system, and the method comprises the steps: obtaining the three-dimensional space environment information of multiple unmanned aerial vehicles through an airborne sensor system of the unmanned aerial vehicle; constructing a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model comprises a state space, an action space, a reward function and a network architecture, and the action space comprises actions corresponding to two different flight modes; constructing a privacy protection model, and protecting training data containing unmanned aerial vehicle sensitive information through the privacy protection model; performing interactive training on the unmanned aerial vehicle and the environment to complete learning of a path planning decision method; and applying the path planning decision method obtained by learning to an unmanned aerial vehicle crowd sensing task to realize path planning. According to the invention, the unmanned aerial vehicle can autonomously select the flight mode, and the capability of the unmanned aerial vehicle to execute long-time perception tasks is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of path planning for unmanned aerial vehicle (UAV) swarm intelligence perception tasks, and in particular, relates to a multi-UAV path planning method and system. Background Art

[0002] In the UAV swarm perception mission, the UAV needs to shuttle through a complex environment containing fixed obstacles (such as trees and buildings) and mobile obstacles (such as other UAVs) to perform the perception mission. Traditional methods such as graph theory and optimization theory face challenges in path planning in such scenarios, especially in dynamic environments. Deep reinforcement learning, as an advanced self-learning method, can process high-dimensional state spaces and respond quickly to environmental changes by learning optimal strategies through interaction with the environment, providing a new way for UAV dynamic path adjustment. However, the path planning of UAV swarm perception missions faces two key problems.

[0003] First, when drones are performing long-duration missions, they are subject to energy constraints and their flight time is limited, which affects the efficiency of completing long-duration missions. Although some studies have attempted to extend flight time by introducing charging piles and integrating charging decisions into deep reinforcement learning models, these methods mix data collection and charging actions, which will lead to confusion in action execution and reduce energy management efficiency.

[0004] Secondly, existing research based on deep reinforcement learning mainly focuses on model design to optimize the flight path of drones, but ignores the security of drone sensitive information during the deep reinforcement learning model training process. During the model training process, a large amount of data containing sensitive information is generated and uploaded, which poses a risk of privacy leakage. Although existing studies (such as using differential privacy or joint differential privacy) have attempted to protect the privacy of training data and models, these methods are mainly applicable to simple environments (such as Atari games) and are difficult to directly apply to highly dynamic and complex drone swarm perception tasks. It is difficult to maintain data availability while ensuring data privacy, which in turn affects the path planning effect of drone swarm perception tasks. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a multi-UAV path planning method and system to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a multi-UAV path planning method, comprising:

[0007] Using the drone's onboard sensor system to obtain the three-dimensional spatial environment information of multiple drones;

[0008] Constructing a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model includes a state space, an action space and a reward function, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes;

[0009] Building a privacy protection model, and protecting the training data containing sensitive information of the drone through the privacy protection model;

[0010] Conduct interactive training between the drone and the environment to complete the learning of path planning decision-making methods;

[0011] The learned path planning decision method is applied to the UAV swarm perception task to realize path planning.

[0012] Preferably, the state space includes a global state space and a local state space, wherein the global state space includes obstacles, charging piles, signal points and all environmental information of the drone, and the local state information includes the personal information of the drone and information of obstacles, charging piles and signal points within the perception range, and the reward function is designed according to the mission objectives.

[0013] Preferably, the flight mode includes a data collection mode and a charging mode.

[0014] Preferably, the global state space in the state space includes obstacles, charging piles, signal points and environmental information of the drone, the local state space includes information of obstacles, charging piles, signal points within the perception range of the drone and personal information of the drone, and the reward function is designed according to the mission objectives.

[0015] Preferably, the privacy protection model includes an encoder and a decoder;

[0016] Mapping the real observations of the UAV to the latent space through the encoder;

[0017] Generate data through the decoder based on latent variables randomly sampled from the latent space and replace the real observations.

[0018] In a second aspect, the present invention further provides a multi-UAV path planning system, comprising:

[0019] An information acquisition module is used to obtain the three-dimensional spatial environment information of multiple drones using the drone's onboard sensor system;

[0020] A first model building module is used to build a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model includes a state space, an action space, a reward function and a network architecture, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes;

[0021] A second model building module is used to build a privacy protection model, through which the training data containing sensitive information of the drone is protected;

[0022] An interactive training module, used to interactively train the UAV and the environment according to the training data, and complete the learning of the path planning decision method without exposing the real training data;

[0023] The path planning module is used to apply the learned path planning decision method to the UAV swarm perception task to achieve energy-sustainable path planning.

[0024] In a third aspect, the present invention further discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0025] In a fourth aspect, the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0026] In a fifth aspect, the present invention further discloses a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0027] Compared with the prior art, the present invention has the following advantages and technical effects:

[0028] The present invention provides a multi-UAV path planning method, comprising: first, using the airborne sensor system of the UAV to obtain the three-dimensional space environment information of the multiple UAVs; second, constructing a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model comprises a state space, an action space and a reward function, wherein the state space comprises a global state space and a local state space, and the action space comprises actions corresponding to two different flight modes; then, constructing a privacy protection model, and protecting the training data containing the sensitive information of the UAV through the privacy protection model; further, interactively training the UAV and the environment according to the training data, and completing the learning of the path planning decision method without exposing the real training data; finally, applying the learned path planning decision method to the UAV group intelligence perception task to realize energy sustainable path planning.

[0029] The present invention integrates a deep reinforcement learning model and a privacy protection model. Under the joint action of the two, the drone can complete the training of two different flight strategies without exposing the real personal training data to an untrusted central server, which not only protects the sensitive information of the drone, but also realizes effective path planning to efficiently complete the perception task. The present invention solves the privacy and security issues of training data in the process of drone swarm perception strategy training, as well as the problem of not being able to replenish energy in time when using a strategy to collect data and charge at the same time. By integrating the privacy protection model and the deep reinforcement learning model, a safe and energy-sustainable path planning method for drone swarm perception tasks is realized.

[0030] The present invention can give the drone the ability to learn two different flight modes while protecting the drone's personal information from being leaked, enabling the drone to autonomously select a flight mode and improving the drone's ability to perform long-term perception tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0032] Figure 1 This is an overall flow chart of the multi-UAV path planning method according to an embodiment of the present invention;

[0033] Figure 2 This is a structural diagram of a flight strategy deep reinforcement learning model according to an embodiment of the present invention;

[0034] Figure 3 This is a structural diagram of a privacy protection model according to an embodiment of the present invention;

[0035] Figure 4 is an interactive training flow chart of an embodiment of the present invention;

[0036] Figure 5 This is a deployment flow chart of the path planning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] First, the technical terms involved in the following embodiments are explained.

[0040] Multi-UAV path planning refers to the process of multiple UAVs working together to complete a task. The algorithm determines the flight path of each UAV to ensure that they can reach the target point efficiently and safely while avoiding obstacles and collisions with other UAVs. The core goal of multi-UAV path planning is to optimize the overall performance of the UAVs, including minimizing flight time and maximizing energy efficiency.

[0041] The key technologies and methods for multi-UAV path planning include:

[0042] Environmental perception: UAVs use sensors to obtain information about their surroundings, such as map data and sensor measurements, and build environmental models for path search and optimization.

[0043] Path search: Commonly used path search algorithms include A* algorithm, Dijkstra algorithm and depth-first search algorithm, etc. These algorithms build the path of the drone based on the graph model and use heuristic functions or priority queues for search.

[0044] Path optimization: Through methods such as dynamic programming, genetic algorithm and particle swarm algorithm, the path of the drone is continuously optimized iteratively to improve efficiency and safety.

[0045] Obstacle avoidance and collision avoidance: Multi-UAV path planning needs to consider obstacle avoidance and collision avoidance. The improved speed obstacle model and distributed path planning algorithm can help UAVs fly safely in complex environments.

[0046] Multi-UAV path planning is widely used in battlefield reconnaissance, target identification, tracking and strike tasks. Since the multi-UAV system involves the coordination of multiple UAVs, path planning needs to solve large-scale constrained multi-objective optimization problems to ensure that each UAV safely reaches the target point under the constraints of obstacle avoidance and collision avoidance. In addition, multi-UAV path planning also needs to consider factors such as the number, function, and coordination mode of UAVs to achieve global goals.

[0047] Embodiment 1

[0048] In view of the two key problems faced by path planning in the UAV swarm perception task pointed out in the background technology, this embodiment provides a multi-UAV path planning method, such as Figure 1 As shown, including:

[0049] Step 1: Construct a simulation environment for the perception task. The drone uses the onboard sensor system to obtain environmental information; the environmental information includes signal points P, obstacles (such as trees, tall buildings, etc.) B, and charging piles C, simulating the real perception task scene to the greatest extent. At time t, drone u uses the onboard sensor system to observe the environment and obtain local observations.

[0050] Step 2: Build a deep reinforcement learning model for safe and energy-sustainable path planning decisions;

[0051] like Figure 2 As shown, the following steps are included:

[0052] 2-1) Construct state space, action space and reward function for deep reinforcement learning.

[0053] A) The state space includes the global state space and the local state space. The global state space includes all environmental information including obstacles, charging piles, signal points and drones, which is the global observation. The local state space is the local observation obtained by the drone, and is constructed as follows;

[0054]

[0055] in, represents the distance between the drone and the obstacle hit by its mth laser. If it does not hit, it is represented as the maximum length of the laser. Indicates the height of the obstacle hit by the mth laser. If it is not hit, it is 0; Indicates the direction from UAV u to signal point p; Indicates the distance from UAV u to signal point p; Indicates the amount of data remaining at signal point p at time t; represents the attraction of signal point p to UAV u; Indicates the direction from the drone to the charging station c; represents the attractiveness of charging pile c to drone u; H min and H max Respectively represent the minimum and maximum flight altitudes that drones are allowed to fly; represents the flight altitude of UAV u at time t; Indicates the flight speed of the drone; Indicates the current remaining energy of the drone; Indicates the current state of the drone, i.e., data collection state or charging state; Indicates whether the drone is stuck, i.e., circling in the same place.

[0056] B) As an additional implementation, the drone generates two different actions based on its local observations and Corresponding to data collection mode and charging mode respectively;

[0057] C) As an additional implementation method, the reward function is designed according to the perception task objectives. In the UAV swarm perception task, the following objectives need to be achieved: ensuring the safety of the UAV flight and the safety of its training data, maximizing the sum of the perception data collected by all UAVs, maximizing the diversity of the collected data, and minimizing the energy consumption of the UAVs.

[0058] As an additional implementation, based on the above objectives, the reward function is constructed as follows:

[0059]

[0060] in, Is a penalty item. In data collection mode, the drone will be penalized if it does not collect data or charge. In charging mode, the drone will be penalized if it does not charge or collect data. In either mode, the drone will be penalized if it hits an obstacle. It is the set of signal points within the sensing range R of UAV u at time t. is the initial energy of the drone. represents the amount of power replenished by the drone at time t. D(p) represents the action performed by the drone After that, the amount of perception data collected by the drone. Indicates that the drone performs an action Finally, the diversity of the data collected, specifically The construction is as follows:

[0061]

[0062] in, represents the initial data volume of signal point p, represents the number of signal points within the sensing range R of drone u at time t, Indicates the amount of data remaining at signal point p at time t. Represents the energy consumption function, which is constructed as follows:

[0063]

[0064] in, represents the amount of data collected by drone u at time t, It represents the distance moved by UAV u at time t, α1 and α2 represent the power consumed by the UAV to collect unit data and move unit distance respectively, and α3 is the energy consumed by the UAV to communicate with the central controller.

[0065] 2-2) As an additional implementation method, a network model based on deep reinforcement learning is constructed. In deep reinforcement learning, an actor network can usually only decide one action, while in energy-sustainable drone swarm perception, drones have at least two actions: collecting data and charging. If the same actor network is used to decide on two actions, the purpose that the drone wants to achieve by the action it performs is vague, which will affect the training effect.

[0066] As an additional implementation, this embodiment uses two independent actor networks to generate two different actions respectively, and uses a critic network to evaluate the strategies of the two actor networks. Then each drone includes an actor network π in data collection mode. coll , Actor Network π in charging mode char , critic network Q, target actor network π' in data collection mode coll , target actor network π' in charging mode char , target critic network Q' six networks, π coll , π char , Q, π' coll ,π' char The network parameters of Q' are θ Q , θ Q‘ Among them, the drone uses the actor network to generate actions based on local observations. Specifically, π coll Used to generate actions to collect sensory data, π char The action used to generate the charge is then used by the critic network to evaluate the expected reward of taking that action in that state, in order to provide useful feedback to both actors.

[0067] 2-3) Set the composition and parameters of the network model based on deep reinforcement learning, including:

[0068] A) Both actor networks consist of 4 fully connected layers, with 512, 256, 128 and 3 neurons in each layer, respectively. LeakyReLU activation function is used between each fully connected layer, and Tanh activation function is used after the last fully connected layer to control the generated action between -1 and 1. The input of both actor networks is the local observation of drone u itself. After being processed by the fully connected layer and the activation function, the outputs are and

[0069] B) The critic network consists of 3 fully connected layers and an attention mechanism layer. The number of neurons in the fully connected layers is 512, 256, and 1 respectively. The attention mechanism layer uses a single-head attention mechanism to process the local observations of all drones, and the embedding dimension is 512. The input of the critic network is a batch of (o, a), and the output is Q(o, a|θ Q ).

[0070] C) Target actor network π' coll and π' char Respectively with the corresponding actor network π coll and π char The structure of the target critic network Q' is the same as that of the corresponding critic network Q.

[0071] Step 3: Build a privacy-preserving model for secure and energy-sustainable path planning decisions;

[0072] The step three comprises the following steps:

[0073] 3-1) Construct a privacy protection model based on variational autoencoder;

[0074] As an additional implementation, in order to better capture the spatiotemporal characteristics of drone observations, a long short-term memory network is added to the encoder, and a multi-head attention mechanism and a long short-term memory network are combined in the decoder. The encoder is responsible for mapping the real observations of the drone to the latent space, and the decoder is responsible for generating generated data similar to the real observations based on the latent variables randomly sampled from the latent space. The network parameters of the encoder and decoder are θ en and θ de .

[0075] 3-2) Set the composition and parameters of the privacy protection model based on the variational autoencoder, including:

[0076] A) As an additional implementation, the encoder consists of a long short-term memory network layer and two linear layers. The number of neurons in the hidden layer of the long short-term memory network is 512, and the number of neurons in the two linear layers is 128. The input of the encoder is a batch of real observations from the drone. After being processed by the long short-term memory network, the hidden state of the last time step is obtained, and then the mean vector and logarithmic variance are calculated using two linear layers respectively.

[0077] B) As an additional implementation, the decoder consists of an embedding layer, a multi-head attention layer, a long short-term memory network layer, and an output layer. The number of neurons in the embedding layer is 512, the multi-head attention layer uses two attention heads, the embedding dimension is 512, the number of hidden layer neurons in the long short-term memory network layer is 512, and the number of neurons in the output layer is consistent with the dimension of the real observation. The input of the decoder is (∈, o), where ∈ is a latent vector randomly sampled from the distribution consisting of the mean and logarithmic variance of the encoder output. First, the latent vector ∈ and the real observation o are embedded into the embedding dimension of the multi-head attention through the embedding layer, and then the input of the multi-head attention layer is obtained:

[0078]

[0079] The specific steps of the multi-head attention layer are as follows:

[0080]

[0081] Among them, Concat(·) is used to concatenate the outputs of each attention head, and Attention(·) is used to calculate the attention of a single head. and They are and The i-th head is segmented. Then for each attention head, the attention score is calculated:

[0082]

[0083] where d k Represents the embedding dimension.

[0084] Finally, the final input result of the multi-head attention layer is obtained through a linear layer. Then, the result is further processed by the long short-term memory network layer, and the hidden layer state of the last time step is taken as the input of the output layer. Finally, the output layer maps the data to the same dimension as the real observation to obtain the generated data.

[0085] Step 4: The UAV and the environment interact to complete the learning of safe and energy-sustainable path planning decision-making methods;

[0086] As an additional implementation, Figure 4 As shown, the following steps are included:

[0087] 4-1) Initialize the actor network π coll , π char , critic network Q, target actor network π' coll ,π' char , the target critic network Q', where π coll , πchar The network weight parameters of and Q are randomly initialized by random numbers, π' coll ,π' char The network weight parameters of Q' are respectively derived from the corresponding network π coll , π char and obtained by copying from Q;

[0088] 4-2) Initialize the simulation environment, each drone observes the environment and obtains local observation values

[0089] 4-3) Based on the observed values Each drone uses π coll and π char Produces two actions and And introduce random noise ω to get the action variable that the drone should actually perform and The drone selects one of two actions to perform based on the current flight mode and battery level: When the drone is in data collection mode, if the battery level is less than 40%, it switches to charging mode and performs the action Otherwise, keep the flight mode running When the drone is in charging mode, if the battery level is less than 40%, it will continue to maintain this mode and perform actions. Otherwise, switch the mode to flight mode and execute the action

[0090] 4-4) After the drone performs an action, the environment changes and the drone obtains new local observations Get corresponding rewards

[0091] 4-5) The drone uses the locally deployed privacy protection model to and Generate the corresponding generated data and

[0092] 4-6) Depending on the type of action being performed, or Stored in the corresponding experience pool M coll or M char and will Stored separately in experience pool M vae In, it is used for training privacy-preserving models;

[0093] 4-7) Determine whether the current number of training rounds is greater than the set threshold ξ1. If not, jump directly to step 4-8). If greater, replay the experience according to the priority method from M to coll and M char Randomly extract data of the same batch size from or And the weight of each experience, then splice these two sets of data together as the input of the critic network to calculate the Q value and the loss function of the critic network, where the Q value is the output of the critic network, and the loss function of the critic network is calculated as follows:

[0094]

[0095] Among them, U represents the number of drones, λ t represents the N-step reward, γ is the reward factor, Indicates the final action to be performed, used to unify the expression or According to the feedback Q value provided by the critic, the gradients of the two actor networks are calculated separately:

[0096]

[0097] Using formulas (6) and (7), the parameters of the three networks are adjusted by gradient descent method. θ Q Then, the target network parameters corresponding to the three networks are updated through soft update. θ Q‘ :

[0098]

[0099] 4-8) Determine whether the current number of training rounds is an integer multiple of the set threshold ξ2. If not, jump directly to step 4-9). If so, start from M vae The data of batch size is randomly extracted from the calculation of the loss function of the privacy protection model, and then the network weight parameters of the encoder and decoder are updated using the gradient descent method:

[0100]

[0101] Where L re Represents the reconstruction loss, calculated using mean square error, L KLD Denotes the KL divergence loss, calculated using the KL divergence, and β is a hyperparameter used to balance the importance of reconstruction loss and KL divergence loss.

[0102] 4-9) Determine whether the drone collides with obstacles or other drones during the interaction with the environment, and whether the energy is exhausted. If one of these two conditions is true, terminate this round of training and jump to step 4-2). Otherwise, terminate this round of training until the perception task is completed or the number of steps the drone moves reaches the maximum, and proceed to step 4-10).

[0103] 4-10) Determine whether the two actor networks, the critic network, and the privacy-preserving model network have converged. If so, end the training; otherwise, jump to step 4-2) and start a new round of training.

[0104] Step 5: Deployment and application of safe and energy-sustainable path planning methods;

[0105] As an additional implementation, Figure 5 As shown, the following steps are included:

[0106] 5-1) Initialize the simulation environment, each drone observes the environment and obtains local observation values

[0107] 5-2) Based on the observed values Each drone uses π coll and π char Produces two actions and The drone selects one of two actions to perform based on the current flight mode and battery level: When the drone is in data collection mode, if the battery level is less than 40%, it switches to charging mode and performs the action Otherwise, keep the flight mode running When the drone is in charging mode, if the battery level is less than 40%, it will continue to maintain this mode and perform actions. Otherwise, switch the mode to flight mode and execute the action

[0108] 5-3) Determine whether the perception task is completed. If completed, jump to step 5-4). If not completed, execute And jump to step 5-2);

[0109] 5-4) End the experiment.

[0110] Compared with the prior art, the beneficial effects of this embodiment are:

[0111] This embodiment integrates the deep reinforcement learning model and the privacy protection model. With the joint action of the two, the drone can complete the training of two different flight strategies without exposing the individual's real training data to an untrusted central server. This not only protects the drone's sensitive information, but also realizes effective path planning to efficiently complete the perception task.

[0112] This embodiment uses a privacy protection model to generate generated data similar to real training data to replace real data for training, which solves the problem of sensitive information leakage in the drone training process and provides protection for the data security of the drone.

[0113] This embodiment adopts an improved actor-critic architecture, using two independent actor networks to output actions for collecting data and charging respectively, and using the same critic network to evaluate the value of the actions output by the two actor networks, integrating two strategies with completely different and conflicting goals, so that the drone can choose different flight strategies in different situations, autonomously coordinate data collection and charging, and complete the collection of perception data through the joint action of the two strategies.

[0114] This embodiment builds a privacy protection model based on the attention mechanism and the long short-term memory network, which not only enables the encoder and decoder of the generation model to process data with spatiotemporal characteristics, but also improves the stability and performance of the encoding and decoding process, thereby generating higher quality generated data.

[0115] Embodiment 2

[0116] Based on the same inventive concept, this embodiment also provides a multi-UAV path planning system, including:

[0117] An information acquisition module is used to obtain the three-dimensional spatial environment information of multiple drones using the drone's onboard sensor system;

[0118] A first model building module is used to build a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model includes a state space, an action space and a reward function, wherein the action space includes actions corresponding to two different flight modes;

[0119] A second model building module is used to build a privacy protection model, through which the training data containing sensitive information of the drone is protected;

[0120] The interactive training module is used to interactively train the drone and the environment to complete the learning of path planning decision methods;

[0121] The path planning module is used to apply the learned path planning decision method to the UAV swarm perception task to achieve safe and energy-sustainable path planning.

[0122] The multi-UAV path planning system provided in this embodiment has all the advantages of the multi-UAV path planning method provided in the first embodiment.

[0123] Embodiment 3

[0124] This embodiment further discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first embodiment.

[0125] Embodiment 4

[0126] This embodiment further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0127] Embodiment 5

[0128] This embodiment also discloses a computer program product, including a computer program, which implements the steps of the method described in the first embodiment when executed by a processor.

[0129] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multi-UAV path planning method, characterized in that: The following steps are involved: Using the drone's onboard sensor system to obtain the three-dimensional spatial environment information of multiple drones; Constructing a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model includes a state space, an action space, a reward function and a network architecture, wherein the action space includes actions corresponding to two different flight modes; Building a privacy protection model, and protecting the training data containing sensitive information of the drone through the privacy protection model; Based on the training data, interactive training is performed on the UAV and the environment to complete the learning of the path planning decision method; The learned path planning decision method is applied to the UAV swarm perception task to realize path planning.

2. The multi-UAV path planning method according to claim 1, characterized in that: The state space includes a global state space and a local state space, wherein the global state space includes obstacles, charging piles, signal points and all environmental information of the drone, and the local state information includes the personal information of the drone and information of obstacles, charging piles and signal points within the perception range. The reward function is designed according to the task objectives.

3. The multi-UAV path planning method according to claim 1, characterized in that: The flight mode includes a data collection mode and a charging mode.

4. The multi-UAV path planning method according to claim 1, characterized in that: The network architecture includes an actor-critic architecture consisting of two actor networks and a critic network, wherein the two actor networks are used to learn strategies under two flight modes respectively.

5. The multi-UAV path planning method according to claim 1, characterized in that: The privacy protection model includes an encoder and a decoder; Mapping the real observations of the UAV to the latent space through the encoder; Generate data through the decoder based on latent variables randomly sampled from the latent space and replace the real observations.

6. A multi-UAV path planning system, characterized in that: include: An information acquisition module is used to obtain the three-dimensional spatial environment information of multiple drones using the drone's onboard sensor system; A first model building module is used to build a deep reinforcement learning model according to the three-dimensional space environment information; wherein the deep reinforcement learning model includes a state space, an action space, a reward function and a network architecture, wherein the state space includes a global state space and a local state space, and the action space includes actions corresponding to two different flight modes; A second model building module is used to build a privacy protection model, through which the training data containing sensitive information of the drone is protected; The interactive training module is used to interactively train the drone and the environment to complete the learning of path planning decision methods; The path planning module is used to apply the learned path planning decision method to the UAV swarm perception task to achieve path planning.

7. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claims 1 to 5 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 5 are implemented.

Images