A method, device, storage medium and electronic device for unmanned aerial vehicle trajectory planning

Through the drone path planning method combined with k-means++ algorithm and DDQN algorithm, the problem of low path planning efficiency and local optimal solution in the existing technology is solved, and the path planning of efficiently covering fire points in large-scale forest areas is realized .

CN118730121BActive Publication Date: 2024-11-19SOUTHWEAT UNIV OF SCI & TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411215876.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-19
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

The existing drone path planning method is difficult to achieve maximum range of fire point coverage in the shortest time in large-scale forest areas, and fails to fully consider the differences in importance of the target, resulting in low efficiency in path planning and easy to fall into local optimality. Solutions and Q value overestimation problems.

Method used

The k-means++ algorithm is used to adaptively cluster the target points, set weights according to the importance of the target points, and path modeling is performed based on the drone's endurance and obstacles as constraints. The model is solved using the DDQN algorithm to obtain the unmanned The target track of the aircraft.

Benefits of technology

By optimizing the distribution of target points and considering the importance of different goals, avoiding collision obstacles, and achieving efficient path planning, improving the effectiveness of path planning and resource utilization efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118730121B_ABST
    Figure CN118730121B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose a method, device, storage medium and electronic device for unmanned aerial vehicle (UAV) trajectory planning, and relate to the field of forest monitoring technology. The present application firstly performs adaptive clustering of multiple target points through the k-means++ algorithm, optimizes the distribution of initial target points, avoids falling into the local optimal solution, and plans the path in combination with the different importance of each target point. Then, based on the endurance and obstacles of the UAV as constraints, it is ensured that collision with obstacles is avoided when covering the new monitoring task target point, and the coverage is completed within the range limit. Finally, the solution is performed through the DDQN algorithm. The DDQN algorithm uses two different networks to respectively select actions and evaluate action values, thereby solving the problem of over-estimation, achieving efficient path planning, and improving the path planning effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of forest monitoring technology, and specifically to a method, device, storage medium and electronic device for unmanned aerial vehicle (UAV) trajectory planning. Background Art

[0002] Traditional forest fire monitoring methods mainly rely on ground monitoring stations and satellite remote sensing technology, but these methods have certain limitations in terms of real-time performance and coverage. For example, the distribution density of ground monitoring stations is limited, making it difficult to cover vast forest areas, especially remote and difficult-to-reach areas; and although satellite remote sensing has a wide coverage, due to the long monitoring cycle, it usually takes days or even weeks to obtain the latest data, making it difficult to detect and respond to fires in a timely manner.

[0003] In recent years, the rapid development of drone technology has provided a new solution for forest fire monitoring. Drones have the advantages of strong mobility, high flexibility, and low cost. They can cover large forest areas in a short time and transmit fire monitoring data in real time. However, it is challenging to deploy drones for monitoring in large forest areas and to reasonably plan the monitoring paths of drones to achieve the largest range of fire coverage in the shortest time.

[0004] The existing UAV path planning methods are poorly effective. Most of them are based on small-scale areas and sparse target sets, and fail to fully consider the differences in the importance of targets. Since these methods usually only plan paths according to ideal conditions, they ignore the fact that a large number of targets need to be monitored in actual applications and that different targets have different importance. This leads to low efficiency of path planning in practical applications and makes it difficult to meet actual needs. Existing methods fail to fully integrate these key factors, making it difficult to achieve optimal path planning and resource utilization efficiency when dealing with large-scale, multi-target tasks. In particular, when dealing with densely distributed target points, traditional clustering methods will lead to uneven target coverage; in terms of path optimization, traditional reinforcement learning algorithms are prone to local optimal solutions and Q-value overestimation problems when facing complex multi-target environments, making it difficult to provide global optimal path planning. Summary of the invention

[0005] The main purpose of this application is to provide a UAV trajectory planning method, device, storage medium and electronic device, aiming to solve the problem of poor effect of UAV path planning methods in forest monitoring applications in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for planning a trajectory of an unmanned aerial vehicle, comprising the following steps:

[0008] In the mission scenario, the k-means++ algorithm is used to cluster the target points according to the distance between them. Several obstacles and target points are set in the mission scenario, and different weights are set for the target points according to their importance.

[0009] The cluster center after clustering is used as the new monitoring task target, and the total weight of all target points in the cluster is calculated;

[0010] According to the total weight of all target points in the cluster, and with the UAV's endurance and obstacles as constraints, the new monitoring mission target covered by the UAV is modeled;

[0011] The DDQN algorithm is used to solve the model and obtain the target track of the UAV.

[0012] In a possible implementation of the first aspect, in a task scenario, the target points are clustered using a k-means++ algorithm according to the distances between the target points, including:

[0013] In the task scenario, a target point is randomly selected to be added to the cluster center point set, and the shortest distance from the remaining target points to the target points in the cluster center point set is calculated;

[0014] Assign all target points to the cluster center with the shortest distance, and update the center of each cluster to the mean of all points in the cluster;

[0015] Determine whether the distances from all target points to their cluster centers meet the preset conditions;

[0016] If the preset conditions are not met, return to the task scene, randomly select a target point to add to the cluster center point set, and calculate the shortest distance from the remaining target points to the target points in the cluster center point set, and loop until the preset conditions are met.

[0017] In a possible implementation of the first aspect, based on the total weight of all target points in the cluster and with the endurance and obstacles of the drone as constraints, a model is built for the drone to cover a new monitoring mission target, including:

[0018] According to the total weight of all target points in the cluster, and with the UAV's endurance and obstacles as constraints, the new monitoring mission target covered by the UAV is modeled based on the current state space, current action space and state transfer function.

[0019] In a possible implementation of the first aspect, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space, and the state transfer function, before modeling the drone covering a new monitoring mission target, the method further includes:

[0020] The current state space is defined according to the current position of the UAV, the endurance of the UAV, the information of obstacles and the cluster center point information set; wherein the cluster center point information set includes the total weight of the target point.

[0021] In a possible implementation of the first aspect, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space, and the state transfer function, before modeling the drone covering a new monitoring mission target, the method further includes:

[0022] Based on the decision of whether to visit each cluster center, the current action space is defined.

[0023] In a possible implementation of the first aspect, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space, and the state transfer function, before modeling the drone covering a new monitoring mission target, the method further includes:

[0024] According to the current state space and the current action space, the next state is determined to define the state transition function.

[0025] In a possible implementation of the first aspect, before modeling the drone covering a new monitoring mission target based on the current state space, the current action space, and the state transfer function, the method further includes:

[0026] Determine the global reward function according to the reward rules; wherein the reward rules include the reward rule for visiting the shortest distance maximum weight target cluster center, the obstacle collision reward rule and the over-range reward rule;

[0027] Based on the current state space, current action space and state transfer function, the new monitoring mission objectives covered by the drone are modeled, including:

[0028] Based on the current state space, current action space, state transfer function and global reward function, the new monitoring mission objectives covered by the UAV are modeled.

[0029] In a second aspect, an embodiment of the present application provides a drone trajectory planning device, comprising:

[0030] Clustering module: The clustering module is used to cluster target points in the task scene according to the distance between target points using the k-means++ algorithm. In the task scene, several obstacles and target points are set, and different weights are set for the target points according to their importance.

[0031] Weight module: The weight module is used to take the cluster center after clustering as the new monitoring task target and calculate the total weight of all target points in the cluster;

[0032] Modeling module: The modeling module is used to model the new monitoring mission target covered by the drone based on the total weight of all target points in the cluster and with the drone's endurance and obstacles as constraints;

[0033] The solving module is used to solve the model using the DDQN algorithm to obtain the target track of the UAV.

[0034] In a third aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, the drone trajectory planning method provided in any one of the first aspects above is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein:

[0036] The memory is used to store computer programs;

[0037] The processor is used to load and execute a computer program so that the electronic device executes the UAV trajectory planning method provided in any one of the first aspects above.

[0038] Compared with the prior art, the beneficial effects of this application are:

[0039] The embodiment of the present application proposes a method, device, storage medium and electronic device for UAV trajectory planning, the method comprising: in a mission scenario, according to the distance between the target points, using the k-means++ algorithm to cluster the target points; wherein a plurality of obstacles and a plurality of target points are set in the mission scenario, and different weights are set for the target points according to their importance; the cluster center after clustering is used as a new monitoring mission target, and the total weight of all target points in the cluster cluster is calculated; according to the total weight of all target points in the cluster cluster, and with the UAV endurance and obstacles as constraints, a model is built for the UAV to cover the new monitoring mission target; and the DDQN algorithm is used to solve the model to obtain the target trajectory of the UAV. This application first uses the k-means++ algorithm to adaptively cluster multiple target points, optimizes the distribution of initial target points, avoids falling into local optimal solutions, and plans paths based on the different importance of each target point. Then, based on the drone's endurance and obstacles as constraints, it ensures that obstacles are avoided when covering new monitoring mission target points, and completes coverage within the range limit. Finally, the DDQN algorithm is used to solve the problem. The DDQN algorithm uses two different networks to select actions and evaluate action values ​​respectively, thereby solving the over-estimation problem, achieving efficient path planning, and improving path planning effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the structure of an electronic device of a hardware operating environment involved in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of a flow chart of a UAV trajectory planning method provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the monitoring coverage of a drone in the drone trajectory planning method provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the flow of the k-means++ algorithm in the UAV trajectory planning method provided in the embodiment of the present application;

[0044] Figure 5 A schematic diagram of a target cluster model after clustering in the UAV trajectory planning method provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the training process of the DDQN algorithm in the UAV trajectory planning method provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of the DDQN algorithm in the UAV trajectory planning method provided in the embodiment of the present application;

[0047] Figure 8 A schematic diagram of a module of a drone trajectory planning device provided in an embodiment of the present application;

[0048] Markings in the figure: 101 - processor, 102 - communication bus, 103 - network interface, 104 - user interface, 105 - memory. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Refer to the attached Figure 1 , attached Figure 1This is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present application. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Among them, the communication bus 102 is used to realize the connection and communication between these components. The user interface 104 may include a display (Display), an input unit such as a keyboard (Keyboard), and the user interface 104 may also include a standard wired interface and a wireless interface. The network interface 103 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 105 may optionally be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) memory, or it may be a stable non-volatile memory (NVM), such as at least one disk storage. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.

[0051] Those skilled in the art will appreciate that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] As attached Figure 1 As shown, the memory 105 as a storage medium may include an operating system, a network communication module, a user interface module, and a UAV trajectory planning device.

[0053] In the attached Figure 1 In the electronic device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in the present application can be set in the electronic device, and the electronic device calls the drone trajectory planning device stored in the memory 105 through the processor 101, and executes the drone trajectory planning method provided in the embodiment of the present application.

[0054] Refer to the attached Figure 2 Based on the hardware device of the aforementioned embodiment, the embodiment of the present application provides a method for UAV trajectory planning, comprising the following steps:

[0055] S10: In the task scene, the k-means++ algorithm is used to cluster the target points according to the distance between the target points; wherein, a number of obstacles and a number of target points are set in the task scene, and different weights are set for the target points according to their importance.

[0056] In the specific implementation process, a large-scale monitoring task scenario with dense target points is constructed, and different weights are set according to the importance of the target points. For example, a two-dimensional monitoring scenario with an area of ​​50km*50km is set and 50-400 target points are randomly deployed, denoted as P = {P1, P2,...,P N}, then set M obstacle points, each target P i Each is assigned a weight W i To indicate its importance, the weight value is randomly set between 1 and 10. The larger the value, the more important the target. The drone is equipped with a high-definition camera, and the coverage area is a circle with a radius of R kilometers and a height of H kilometers. Figure 3 As shown, without considering the size, shape and other structural features of the UAV, it is simplified to a point mass, and the motion of the UAV is approximately the motion of a point mass.

[0057] k-means++ is an algorithm for selecting initial values ​​for the k-means clustering algorithm. The original k-means algorithm randomly selects K points in the data set as cluster centers at the beginning, while k-means++ assumes that n initial cluster centers have been selected. When selecting the n+1th cluster center, the points farther away from the current n cluster centers have a higher probability of being selected as the n+1th cluster center. Specifically: In the task scenario, the k-means++ algorithm is used to cluster the target points according to the distance between the target points, including:

[0058] In the task scenario, a target point is randomly selected to be added to the cluster center point set, and the shortest distance from the remaining target points to the target points in the cluster center point set is calculated;

[0059] Assign all target points to the cluster center with the shortest distance, and update the center of each cluster to the mean of all points in the cluster;

[0060] Determine whether the distances from all target points to their cluster centers meet the preset conditions;

[0061] If the preset conditions are not met, return to the task scene, randomly select a target point to add to the cluster center point set, and calculate the shortest distance from the remaining target points to the target points in the cluster center point set, and loop until the preset conditions are met.

[0062] In the specific implementation process, as shown in the attached Figure 4As shown, first initialize the sample, randomly select a target point from all target points as the initial center, that is, the first cluster center C1, and add it to the cluster center point set {C1,...,C n-1}, for the remaining target point P i ∈P, then cluster nearby and calculate the distance D(P i ), the calculation formula is as follows:

[0063]

[0064] Among them, C j represents the jth cluster center, n represents the number of existing cluster centers, and N represents the number of random targets;

[0065] According to the squared distance D(P i ), according to the probability distribution P(P i ) Select the next cluster center and repeat this process until the initial k cluster centers are selected. The calculation formula is as follows:

[0066]

[0067] Among them, P i represents the i-th target;

[0068] Assign all target points to the nearest cluster center and update the cluster center, that is, update the center of each cluster to the mean of all points in the cluster. The specific formula is as follows:

[0069]

[0070]

[0071] in, represents the jth cluster;

[0072] Calculate the distance from each point to its cluster center, and check whether the distance of all points meets the preset condition. The preset condition is that the distance is less than or equal to the preset distance constraint δ. If the distance constraints of all target points are met and the cluster center no longer changes significantly, that is, the cluster center converges, then stop the iteration, otherwise the k value is increased by 1, and return to the previous step to select a cluster center again to complete the clustering step again. The specific formula is as follows:

[0073]

[0074] Among them, δ represents the distance constraint, P i ∈P,C j∈{C1,...,C n}.

[0075] S20: The cluster center after clustering is used as the new monitoring task target, and the total weight of all target points in the cluster is calculated.

[0076] In the specific implementation process, randomly distributed weighted target points are clustered to form new weighted cluster centers, where the weight of each cluster center is equal to the sum of the weights of each target in the cluster, and the calculation formula is as follows:

[0077]

[0078] Among them, W Pi Indicates the target P i The weight, W cj Represents the cluster center C j The weight of

[0079] After clustering, several new clusters are obtained , the drone no longer traverses each target point, but traverses each cluster center , you can visit all the target points in the cluster and obtain all the target weights of the cluster. The target cluster model after clustering is shown in the attached figure. Figure 5 shown.

[0080] S30: Based on the total weight of all target points in the cluster, and with the drone's endurance and obstacles as constraints, model the new monitoring mission target covered by the drone.

[0081] In the specific implementation process, the problem of drones covering new monitoring mission targets is modeled as a restricted Markov decision process, and its model includes state space, action space, state transfer function and reward function. That is, according to the total weight of all target points in the cluster, and with the drone's endurance and obstacles as constraints, the drone's coverage of new monitoring mission targets is modeled, including:

[0082] According to the total weight of all target points in the cluster, and with the UAV's endurance and obstacles as constraints, the new monitoring mission target covered by the UAV is modeled based on the current state space, current action space and state transfer function.

[0083] In one embodiment, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space and the state transfer function, before modeling the drone covering the new monitoring mission target, the method also includes:

[0084] The current state space is defined according to the current position of the UAV, the endurance of the UAV, the information of obstacles and the cluster center point information set; wherein the cluster center point information set includes the total weight of the target point.

[0085] In the specific implementation process, the state space is defined as a collection of the current position of the drone, the remaining endurance, the cluster center data, and the obstacle information:

[0086]

[0087] Among them, (x, y) is the current position coordinate of the UAV, r is the remaining range of the UAV, {(x i ,y i ,w i ,v i )} is the cluster center point information set, (x i ,y i ) is the cluster center C i The coordinates of W Ci is the weight of the cluster center, v ci ∈{0,1} is whether the cluster center has been visited, {(x j ,y j ,r j )} is the obstacle information set, (x j ,y j ) is the center coordinate of obstacle j, r j is the radius of the obstacle. The obstacle is a circle with a radius ranging from 1 to 3, and its number is randomly set from 1 to 3. There is no target point within the obstacle range and the drone coverage range is not within the obstacle range.

[0088] In one embodiment, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space and the state transfer function, before modeling the drone covering the new monitoring mission target, the method also includes:

[0089] Based on the decision of whether to visit each cluster center, the current action space is defined.

[0090] In the specific implementation process, the action space consists of the decision of whether to visit each cluster center:

[0091]

[0092] Among them, C i represents the i-th cluster center (i.e. cluster center) after clustering, C i =1 means to visit the target cluster center C i , C i=0 means the target cluster center C is not visited i .

[0093] In one embodiment, according to the total weight of all target points in the cluster, and with the endurance and obstacles of the drone as constraints, based on the current state space, the current action space and the state transfer function, before modeling the drone covering the new monitoring mission target, the method also includes:

[0094] According to the current state space and the current action space, the next state is determined to define the state transition function.

[0095] In the specific implementation process, the state transition function is to determine the next state s' given the current state s and a. If you choose to visit C i (i.e. C i =1), the new position (x', y') of the drone is updated to the target cluster center C i The position (x i ,y i ):

[0096] , C i =1

[0097] The remaining distance r' of the next state requires the remaining distance r minus the distance from the current cluster center to the target cluster center:

[0098]

[0099] If we visit the target cluster center C i (i.e. C i =1), then update the target cluster center C i The access status V ci Has been visited (ie V ci '=1).

[0100] In one embodiment, before modeling the drone covering a new monitoring mission target based on the current state space, the current action space, and the state transfer function, the method further includes:

[0101] The global reward function is determined according to the reward rules; wherein the reward rules include the reward rule for visiting the shortest distance maximum weight target cluster center, the obstacle collision reward rule and the over-range reward rule.

[0102] In the specific implementation process, reward rules are designed to optimize path planning, including reward rules for visiting the shortest distance and maximum weight target cluster center, obstacle collision reward rules, and over-range reward rules. The global reward function is determined based on the reward rules:

[0103]

[0104] Among them, α and β represent adjustment parameters, S represents the Euclidean distance from the current cluster center to the visited cluster center, and W Ci Indicates the weight of the target cluster center, Penalty indicates the penalty for hitting an obstacle or exceeding the range, which is set to -800. In this embodiment, the drone departs from the base and returns to the base within the endurance. If it hits an obstacle or exceeds the range, it ends early. α and β can be set to 0.4 and 0.8 respectively.

[0105] In addition, the constraints are to ensure that the UAV avoids collision with obstacles when covering the new monitoring mission target point and completes the coverage of the target point within the range limit.

[0106] Based on the above steps, the new monitoring mission target covered by the drone is modeled based on the current state space, current action space and state transfer function, including:

[0107] Based on the current state space, current action space, state transfer function and global reward function, the new monitoring mission objectives covered by the UAV are modeled.

[0108] S40: Use the DDQN algorithm to solve the model and obtain the target track of the UAV.

[0109] In the specific implementation process, the DDQN algorithm is an improved reinforcement learning algorithm, which aims to solve the overestimation problem in the DQN algorithm, thereby improving the performance and stability in reinforcement learning tasks. The DDQN algorithm improves the DQN algorithm and solves the overestimation problem that may be caused by the calculation method of the target Q value in DQN. In DQN, the target Q value is directly calculated using the network of the current policy, which may lead to an overestimation of the Q value. DDQN reduces this overestimation problem by using two different networks to select actions and evaluate action values ​​respectively. Specifically, the DDQN algorithm consists of two main parts: an online network for selecting actions and a target network for calculating the target Q value. The online network updates its weights with the latest data, while the weights of the target network are copied from the online network regularly, but there will be a certain delay. This practice helps to stabilize the learning process and reduce the problem of overestimation.

[0110] In the specific implementation process, as shown in the attached Figure 6 , Attachment Figure 7 As shown, S41: Initialize the deep neural network structure, experience playback buffer and training environment of the Q network and the target Q network, set the hyperparameters learning rate α, decay factor γ, exploration rate and the target network update frequency τ;

[0111] In the specific implementation process, the initial α, γ, and τ are 0.001, 0.99, 0.9, and 500 respectively; the reasons for setting hyperparameters are as follows: the learning rate controls the step size of network parameter updates. Setting it too large may lead to unstable training, while setting it too small may lead to slow convergence; the decay factor is used to weigh the importance of future rewards. A higher γ value means that the drone path planning pays more attention to long-term rewards; the exploration rate determines the probability of randomly selecting an action. A higher A higher value means more exploration. Gradually reducing the exploration rate helps the algorithm make more use of the learned knowledge. The target network update frequency determines the frequency at which the target network parameters are synchronized with the current Q network parameters. A higher update frequency can maintain the stability of the target network.

[0112] S42: According to -The greedy strategy selects action a;

[0113] In the specific implementation process, - Greedy strategy is used to balance exploration and exploitation, with probability ∈(0,1) randomly selects an action with probability 1- Select the action with the largest Q value in the current Q network ; Start with the initial settings To select an action, follow -The exponential decay strategy in the greedy strategy gradually decays to 0.01. The exponential decay formula is as follows:

[0114]

[0115] In the formula, min represents the minimum exploration rate, 0 represents the initial exploration rate, decay represents the decay rate, which is initially 0.99, and t represents the number of time steps;

[0116] S43: Execute action a in the current environment and record the new state s′ and reward r;

[0117] S44: Add the experience sample (s, a, r, s′) to the experience playback buffer D;

[0118] S45: Randomly extract small batches of samples from the experience replay buffer for training;

[0119] S46: Calculate the target Q value y;

[0120] In the specific implementation process, for each sample (s, a, r, s′), the current Q network is used to select the best action for the next state s′ , use the target Q network to calculate the target Q value ; Among them, r represents the reward, θ and They represent the Q network parameters and the target Q network parameters at that moment, respectively, and γ represents the attenuation factor;

[0121] S47: Evaluate the loss L of the Q network, calculated as follows: , where N represents the batch size and y i represents the target Q value for the i-th time, Indicates the Q value calculated by the current Q network;

[0122] S48: Optimize the Q network parameters θ by gradient descent to minimize the loss L;

[0123] S49: Update the target Q network parameters every τ steps .

[0124] When the number of iterations reaches the preset number of training times or the loss function value of the training model does not decrease in multiple consecutive rounds, the training ends.

[0125] After the training is completed, the current state space s is input into the trained Q network, which predicts the Q value of each possible action a, and selects the action with the largest Q value as the next action of the drone. By repeating this process, the drone can plan the optimal path to maximize monitoring efficiency and minimize the total path length.

[0126] In this embodiment, the k-means++ algorithm is first used to adaptively cluster multiple target points, optimize the distribution of initial target points, avoid falling into the local optimal solution, and plan the path based on the different importance of each target point. Then, based on the UAV's endurance and obstacles as constraints, it is ensured that obstacles are avoided when covering new monitoring mission target points, and the coverage is completed within the range limit. Finally, the DDQN algorithm is used to solve the problem. The DDQN algorithm uses two different networks to select actions and evaluate action values ​​respectively, thereby solving the over-estimation problem, achieving efficient path planning, and improving the path planning effect.

[0127] Refer to the attached Figure 8 Based on the same inventive concept as in the above-mentioned embodiment, the embodiment of the present application further provides a drone trajectory planning device, which includes:

[0128] Clustering module: The clustering module is used to cluster target points in the task scene according to the distance between target points using the k-means++ algorithm. In the task scene, several obstacles and target points are set, and different weights are set for the target points according to their importance.

[0129] Weight module: The weight module is used to take the cluster center after clustering as the new monitoring task target and calculate the total weight of all target points in the cluster;

[0130] Modeling module: The modeling module is used to model the new monitoring mission target covered by the drone based on the total weight of all target points in the cluster and with the drone's endurance and obstacles as constraints;

[0131] The solving module is used to solve the model using the DDQN algorithm to obtain the target track of the UAV.

[0132] Those skilled in the art should understand that the division of the various modules in the embodiment is only a division of logical functions. In actual application, they can be fully or partially integrated into one or more actual carriers, and these modules can be implemented entirely in the form of software called by a processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in the UAV trajectory planning device in this embodiment corresponds one-to-one to each step in the UAV trajectory planning method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned UAV trajectory planning method, which will not be repeated here.

[0133] Based on the same inventive concept as in the aforementioned embodiments, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, a drone trajectory planning method as provided in an embodiment of the present application is implemented.

[0134] Based on the same inventive concept as in the above-mentioned embodiment, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein:

[0135] The memory is used to store computer programs;

[0136] The processor is used to load and execute a computer program so that the electronic device executes the UAV trajectory planning method provided in the embodiment of the present application.

[0137] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0138] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0139] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0140] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0141] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0142] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0144] In summary, the present application provides a method, device, storage medium and electronic device for UAV trajectory planning, the method comprising: in a mission scenario, according to the distance between the target points, using the k-means++ algorithm to cluster the target points; wherein a number of obstacles and a number of target points are set in the mission scenario, and different weights are set for the target points according to their importance; the cluster center after clustering is used as the new monitoring mission target, and the total weight of all target points in the cluster cluster is calculated; according to the total weight of all target points in the cluster cluster, and with the UAV endurance and obstacles as constraints, the new monitoring mission target covered by the UAV is modeled; the DDQN algorithm is used to solve the model to obtain the target track of the UAV. This application first uses the k-means++ algorithm to adaptively cluster multiple target points, optimizes the distribution of initial target points, avoids falling into local optimal solutions, and plans paths based on the different importance of each target point. Then, based on the drone's endurance and obstacles as constraints, it ensures that obstacles are avoided when covering new monitoring mission target points, and completes coverage within the range limit. Finally, the DDQN algorithm is used to solve the problem. The DDQN algorithm uses two different networks to select actions and evaluate action values ​​respectively, thereby solving the over-estimation problem, achieving efficient path planning, and improving path planning effects.

[0145] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A UAV trajectory planning method, It is characterized in that The following steps are involved: In a task scene, the target points are clustered using a k-means++ algorithm according to the distance between the target points; wherein a plurality of obstacles and a plurality of the target points are set in the task scene, and different weights are set for the target points according to their importance; in the task scene, the target points are clustered using a k-means++ algorithm according to the distance between the target points, including: In the task scenario, a target point is randomly selected to be added to the cluster center point set, and the shortest distance from the remaining target points to the target point in the cluster center point set is calculated; All the target points are assigned to the cluster center with the shortest distance, and the center of each cluster is updated to the mean of all the points in the cluster; Determine whether the distances from all the target points to their cluster centers meet preset conditions; If the preset condition is not met, return to the step of randomly selecting a target point in the task scene to add to the cluster center point set, and calculating the shortest distance from the remaining target point to the target point in the cluster center point set, and loop until the preset condition is met; The cluster center after clustering is used as the new monitoring task target, and the total weight of all the target points in the cluster is calculated; According to the total weight of all the target points in the cluster, and taking the endurance of the drone and the obstacles as constraints, modeling the drone covering the new monitoring mission target; according to the total weight of all the target points in the cluster, and taking the endurance of the drone and the obstacles as constraints, modeling the drone covering the new monitoring mission target includes: According to the total weight of all the target points in the cluster, and with the endurance of the drone and the obstacles as constraints, based on the current state space, the current action space and the state transfer function, modeling is performed on the drone covering the new monitoring mission target; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the drone endurance and the obstacles as constraints, and the current state space, the current action space and the state transfer function, the method further includes: The current state space is defined according to the current position of the drone, the endurance of the drone, the information of the obstacle, and the cluster center point information set; wherein the cluster center point information set includes the total weight of the target point; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the drone endurance and the obstacles as constraints, and the current state space, the current action space and the state transfer function, the method further includes: Defining the current action space according to the decision of whether to visit each cluster center; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the drone endurance and the obstacles as constraints, and the current state space, the current action space and the state transfer function, the method further includes: According to the current state space and the current action space, determining a next state to define the state transition function; Before modeling the drone covering the new monitoring mission target based on the current state space, the current action space and the state transfer function, the method further includes: Determine a global reward function according to a reward rule; wherein the reward rule includes a reward rule for accessing the shortest distance maximum weight target cluster center, a collision obstacle reward rule, and an over-range reward rule; The method of modeling the drone covering the new monitoring mission target based on the current state space, the current action space and the state transfer function includes: Modeling the drone covering the new monitoring mission target based on the current state space, the current action space, the state transfer function and the global reward function; The DDQN algorithm is used to solve the model and obtain the target track of the UAV.

2. A UAV trajectory planning device, It is characterized in that include: A clustering module, wherein the clustering module is used to cluster the target points in a task scene according to the distance between the target points by using a k-means++ algorithm; wherein a plurality of obstacles and a plurality of the target points are set in the task scene, and the target points are set with different weights according to their importance; wherein in the task scene, the target points are clustered by using a k-means++ algorithm according to the distance between the target points, including: In the task scenario, a target point is randomly selected to be added to the cluster center point set, and the shortest distance from the remaining target points to the target point in the cluster center point set is calculated; All the target points are assigned to the cluster center with the shortest distance, and the center of each cluster is updated to the mean of all the points in the cluster; Determine whether the distances from all the target points to the cluster centers meet the preset conditions; If the preset condition is not met, return to the step of randomly selecting a target point in the task scene to add to the cluster center point set, and calculating the shortest distance from the remaining target point to the target point in the cluster center point set, and loop until the preset condition is met; A weight module, wherein the weight module is used to take the cluster center after clustering as a new monitoring task target and calculate the total weight of all the target points in the cluster cluster; A modeling module, the modeling module is used to model the drone covering the new monitoring task target according to the total weight of all the target points in the cluster, and taking the drone endurance and the obstacles as constraints; the modeling of the drone covering the new monitoring task target according to the total weight of all the target points in the cluster, and taking the drone endurance and the obstacles as constraints, includes: According to the total weight of all the target points in the cluster, and with the endurance of the drone and the obstacles as constraints, based on the current state space, the current action space and the state transfer function, modeling is performed on the drone covering the new monitoring mission target; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the endurance of the drone and the obstacle as constraints, based on the current state space, the current action space and the state transfer function, the method further includes: The current state space is defined according to the current position of the drone, the endurance of the drone, the information of the obstacle, and the cluster center point information set; wherein the cluster center point information set includes the total weight of the target point; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the endurance of the drone and the obstacle as constraints, based on the current state space, the current action space and the state transfer function, the method further includes: Defining the current action space according to the decision of whether to visit each cluster center; Before modeling the new monitoring mission target covered by the drone based on the total weight of all the target points in the cluster, the endurance of the drone and the obstacle as constraints, based on the current state space, the current action space and the state transfer function, the method further includes: According to the current state space and the current action space, determining a next state to define the state transition function; Before modeling the new monitoring mission target covered by the drone based on the current state space, the current action space and the state transfer function, the method further includes: Determine a global reward function according to a reward rule; wherein the reward rule includes a reward rule for accessing the shortest distance maximum weight target cluster center, a collision obstacle reward rule, and an over-range reward rule; The method of modeling the drone covering the new monitoring mission target based on the current state space, the current action space and the state transfer function includes: Modeling the drone covering the new monitoring mission target based on the current state space, the current action space, the state transfer function and the global reward function; The solution module is used to solve the model using the DDQN algorithm to obtain the target track of the UAV.

3. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is loaded and executed by the processor, the unmanned aerial vehicle trajectory planning method as claimed in claim 1 is implemented.

4. An electronic device, It is characterized in that comprising a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to load and execute the computer program so that the electronic device executes the UAV trajectory planning method as described in claim 1.