Dynamic and static combined urban logistics unmanned aerial vehicle task planning method

Abstract

The invention discloses a dynamic and static combined urban logistics unmanned aerial vehicle task planning method, which comprises the following steps of (1) inputting immediate and non-immediate distributed dynamic and static combined urban logistics unmanned aerial vehicle task planning urban logistics order data, and preliminarily selecting a logistics unmanned aerial vehicle airport position by using a k-means clustering algorithm improved based on space-time constraints; and (2) determining a static task planning result for non-instant delivery orders by using a static task planning method based on an HPSO algorithm, determining the positions of logistics delivery points, optimizing the site selection of the unmanned aerial vehicle airports according to the distribution of the logistics delivery points, and balancing the number of tasks of each airport. And (3) proposing a dynamic task planning method for improving a DDQN algorithm based on a static task planning result, determining a dynamic task planning result of the instant delivery order, and finally determining a planning and scheduling scheme of the logistics unmanned aerial vehicle fleet. According to the invention, through fusion of urban logistics orders of instant delivery and non-instant delivery, unmanned aerial vehicle facility site selection and order distribution are more reasonable.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles (UAVs) and logistics, and in particular to a method for planning urban logistics UAV missions that combines dynamic and static operations. Background Art

[0002] With the development of modern computer technology and the advancement of Internet technology, logistics drone delivery driving technology has become a reality. Compared with traditional delivery methods, drone delivery can avoid traffic jams and ground obstacles and deliver goods at a faster speed. This can play an important role in emergency situations or items that need to be delivered quickly. On the other hand, drone delivery can reduce labor and transportation costs because they do not require drivers or fuel costs. The carbon emissions generated by drones are also much less than fleet delivery, which helps to reduce the impact of logistics activities on the environment and promote sustainable development. In addition, by using sensors and monitoring technology, drone delivery can monitor the status and location of goods in real time. The delivery party can deal with any problems in a timely manner to ensure the safety and integrity of the delivery.

[0003] However, drone delivery also faces a series of problems. How to plan the flight path of logistics drones from a combination of static and dynamic perspectives to minimize losses during transportation while maintaining an efficient delivery speed is the key to the widespread application of this delivery model. Existing drone delivery is often a static delivery model. Static delivery may be difficult to deal with emergencies, while the combination of static and dynamic planning can better deal with problems and risks. At the same time, when faced with instant orders, static delivery may face air traffic congestion and conflicts, while the combination of static and dynamic planning can better plan flight paths. Compared with static delivery, the key point that needs to be solved in dynamic and static combined delivery is to determine the drone fleet size planning plan based on instant and non-instant order tasks. Summary of the invention

[0004] Purpose of the invention: The present invention proposes a dynamic and static combined urban logistics drone task planning method by integrating the working characteristics of timely delivery and non-timely delivery of logistics drones, including determining and optimizing the location of drone airports, determining the location of logistics delivery points, providing a logistics drone fleet size planning plan and performing task allocation for timely delivery orders and non-timely delivery orders, so that the site selection of drone airports is more reasonable and the use of drones for delivery is more economical.

[0005] The above purpose is achieved through the following technical solutions:

[0006] A method for planning a dynamic and static urban logistics UAV mission, the method comprising the following steps:

[0007] S1. Input the urban logistics drone mission planning urban logistics order data of the combination of dynamic and static real-time and non-real-time delivery to obtain sample data, which includes airspace attributes, weather information, store location, and customer location. Based on the store and customer location data, the k-means clustering algorithm improved based on spatiotemporal constraints is used to preliminarily select the location of the logistics drone airport;

[0008] S2. Use the static task planning method based on the HPSO (hybrid particle swarm optimization) algorithm to determine the static task planning results for non-instant delivery orders, use the DBSCAN clustering algorithm considering the walkability constraint to determine the location of the logistics delivery point, and optimize the location of the drone airport according to the distribution of logistics delivery points to balance the number of tasks at each airport;

[0009] S3. Based on the location of drone airports and logistics distribution points, and taking into account spatial information, drone performance, and weather conditions, a dynamic task planning method based on the static task planning results to improve the DDQN algorithm is proposed to determine the dynamic task planning results of instant delivery orders, thereby ultimately determining the planning and scheduling plan for the logistics drone fleet.

[0010] Furthermore, the k-means clustering algorithm considering time and space constraints in step S1.1 includes the following steps:

[0011] S1.1.1 Determination of alternative address:

[0012] (1) The k-means clustering algorithm based on Euclidean distance is used to cluster the demand points, determine the number k of alternative logistics drone airports, perform k-means clustering on the demand points, and use the cluster centers in the clustering results as the alternative logistics drone airports to be evaluated;

[0013] (2) Collect the data of each candidate logistics drone airport on the evaluation indicators to form a data set, wherein the evaluation indicators include airspace conditions, traffic characteristics, and construction conditions. Then, according to actual needs or data analysis, optimize and adjust the number of candidate take-off and landing sites, randomly select k points from the data set as the initial clustering center, assign each point in the data set to the nearest clustering center, form k clusters, recalculate the center point of each cluster as the new clustering center, and repeat the above steps until the clustering center no longer changes significantly or reaches the preset number of iterations;

[0014] S1.1.2 Establishment of multi-objective site selection model:

[0015] Use a distance-based function to characterize customer satisfaction:

[0016] a i is the average distance from point i to other points in its cluster, b iis the average distance from point i to the nearest point in the other clusters. The silhouette coefficient evaluation index is used to calculate its silhouette coefficient S i :

[0017]

[0018] The silhouette coefficient measures the compactness of points within the same cluster and the degree of separation between different clusters. Its value range is between -1 and 1. The larger the value, the better the clustering effect. Finally, the average value of the silhouette coefficient of all data points is calculated as the evaluation indicator of the entire clustering result.

[0019] Furthermore, the implementation steps of the k-means clustering algorithm based on Euclidean distance in step S1.2 are as follows:

[0020] S1.2.1 Based on the obtained sample data, obtain two arbitrary data points p 1 (x 1 ,y 1 , t 1 ) and p 2 (x 2 ,y 2 , t 2 ), where (x, y) is the spatial coordinate, t is the numerical representation of the timestamp, and subscripts 1 and 2 are used to distinguish different data points;

[0021] S1.2.2 imposes spatiotemporal constraints on data points. The spatiotemporal constraint condition is a composite distance metric that considers both time and space, called the spatiotemporal distance metric. The spatiotemporal distance metric can reflect the proximity of data points in time and space. The spatiotemporal distance metric formula is defined as:

[0022] d st (p 1 , p 2 )=αd spatial (p 1 , p 2 )+βd temporal (p 1 , p 2 )

[0023] where d spatial (p 1 , p 2 ) is a spatial distance metric, defined as the Euclidean distance:

[0024] d temporal (p 1 , p 2 ) is a temporal distance measure, defined as: d temporal (p 1 , p2 )=|t 2 -t 1 |;

[0025] α is the weight coefficient of spatial distance, β is the weight coefficient of temporal distance, and the values ​​of α and β are between 0 and 1, which are used to balance the contribution of spatial distance and temporal distance in the total distance respectively;

[0026] S1.2.3 Assume that there are n i data points, whose spatial coordinates and timestamps are (x ij ,y ij , t ij ), where j = 1, 2, ..., n i ; Then the updated cluster center of the i-th cluster ((x c ,y c , t c )) is calculated by the following formula

[0027]

[0028] Further, the static task planning method based on the HPSO (hybrid particle swarm optimization) algorithm described in step 2 comprises the following steps:

[0029] S2.1 Initialization:

[0030] S2.1.1 Define particles: Each particle represents a possible non-immediate delivery task allocation scheme. The dimension of the particle is equal to the number of task points, and the value of each dimension indicates which drone the task point is assigned to.

[0031] S2.1.2 Setting parameters: define the size of the particle swarm, the maximum number of iterations, the inertia weight, the individual optimal weight and the group optimal weight;

[0032] S2.2 Initialize particle position and velocity:

[0033] S2.2.1 Generate initial position: Randomly generate a task allocation plan for each particle to ensure that each task point is assigned to a drone;

[0034] S2.2.2 Initialization speed: Randomly set the initial speed for each particle;

[0035] S2.3 Evaluating fitness:

[0036] S2.3.1 Calculate the objective function: For each particle, calculate its fitness value, that is, the objective function value, including minimizing the total distance and the minimum time. The objective function is expressed as:

[0037]

[0038] Among them, D x is the total distance that the x-th UAV performs its mission, T y is the total time for the yth UAV to perform the mission, N and M are the number of UAVs in each particle;

[0039] S2.3.2 Check constraints: Evaluate whether each particle satisfies the constraints. Particles that do not meet the constraints will be punished by the penalty function.

[0040] S2.4 Update individual optimal solution and group optimal solution:

[0041] S2.4.1 Update individual optimal solution: If the fitness of the current particle is better than its historical optimal fitness, update the individual optimal solution;

[0042] S2.4.2 Update the optimal solution of the group: If the fitness of the current particle is better than the historical optimal group fitness, then update the optimal solution of the group;

[0043] S2.5 updates particle velocity and position:

[0044] S2.5.1 Speed ​​update: According to the formula:

[0045] v op (t+1)=Ω·v op (t)+c 1 ·r 1 ·(p op -x op (t))+c 2 ·r 2 ·(g op -x op (t))

[0046] Among them, v op is the velocity of particle o in dimension p, x op is the particle position, p op is the individual optimal solution, g op is the group optimal solution, Ω is the inertia weight, c 1 and c 2 is the learning factor, r 1 and r 2 is a random number;

[0047] S2.5.2 Position update: Update particle position according to velocity:

[0048] x op (t+1)=x op (t)+v op (t+1)

[0049] S2.5.3 Position adjustment: ensure that the updated particle position meets the task allocation constraints;

[0050] S2.6 Termination condition check:

[0051] S2.6.1 Iteration number: Check whether the maximum number of iterations has been reached;

[0052] S2.6.2 Convergence condition: Check whether the fitness reaches a predetermined threshold or converges to a stable value;

[0053] S2.7 Result output:

[0054] S2.7.1 Optimal task allocation: Output the task allocation plan of particles with optimal fitness;

[0055] S2.7.2 Performance evaluation: Evaluate the performance of the task allocation scheme, including total distance, total time, and whether all constraints are met;

[0056] S2.8 Actual deployment.

[0057] Furthermore, the dynamic task planning method of the DDQN algorithm based on the static task planning results described in step 3 is based on specific spatial information, takes into account the performance of the drone and the weather conditions at the time, and optimizes the processing of instant delivery orders based on the static task planning results, thereby improving the dynamic task planning method of the DDQN algorithm. The static task planning results and the dynamic task planning results are integrated, that is, the maximum number of fleets at each location of the two planning results is selected as the fleet size planning result to ensure that the delivery order requirements are met. Specifically, the following steps are included:

[0058] S3.1: Construct the communication scenario between the drone and the communication terminal, IoT device, and user equipment, including a drone and obstacles of different heights. Within the time η, the position of the drone is w;

[0059] S3.2: Based on the communication scenario, determine the UAV air-to-ground channel gain and establish the communication link model between the UAV and the ground user:

[0060] where α Los α QLos is the communication efficiency between the UAV and the terminal in the Los time interval and the average communication efficiency in I Los time intervals, β Los β QLos is the communication efficiency between the ground equipment and the terminal in the time interval of Los and the average communication efficiency in I time intervals; is the channel gain between the ground equipment and the UAV; η Lps,ifLos is the basic communication efficiency in the time interval of Los; η TLos,ifILos is the basic communication efficiency in the total time of I Los time intervals;

[0061] S3.3: Select action a according to the ε strategy t , that is, randomly select action a t The probability of selecting an action according to the position w of the UAV is ε, and the probability of selecting an action according to the position w of the UAV is 1-ε, to ensure that the UAV has a certain degree of exploration;

[0062] S3.4: Add a t Enter the data collection environment and get the current state s t and the next position of the drone w t+1 , where w t+1 =w t +a t ;

[0063] S3.5: Based on the communication link model, the optimization model is standardized as a four-tuple<S,A,R,P> , where A represents the action space of the drone delivery process, and the action a selected by the drone within time t t =[a x ,a y ,a z ]∈A; S represents the state space of the UAV, and the state of the UAV at time t in this space is represented by s t =(s t1 ,s t2 ,s t3 ); P is the transition probability matrix, including the drone in state s t When taking action a t Transition to next state s t+1 The probability of the drone choosing action a t Arriving at the next state s t+1 The profit information obtained in the process, the reward function r t For: t =r t1 -r t2 -r t3 ; The drone is based on the position w t The amount of data collected is r t1 , according to the movement of the drone, the power penalty r is obtained t2 , according to whether the drone hits an obstacle in the current state and the next state, a collision penalty r is obtained t3 , and finally get the total reward r t ;

[0064] S3.6: According to w t+1 , determine the next state of the drone t+1 ;

[0065] S3.7: Transfer results t+1 ​,a t ,s t ,r t >Save to experience pool J. When the number of data sets in the experience pool reaches the threshold, select sample sets from the experience pool to train the neural network. During the training process, the new data will replace the old data sets in the experience pool.

[0066] S3.8: Calculate the error function L(θ) = ‖y t- Q(a t ,s t ;θ)‖ 2 , which is the mean square error used to measure the difference between the current Q value and the target Q value, where y t is the target Q value, and

[0067]

[0068] θ is the current network parameter, θ n is the real-time parameter of the target network, r t is the reward value for time period t, argmaxQ() means finding the action corresponding to the maximum Q value in the current Q network;

[0069] S3.9: Repeat steps S3.2.2 to S3.2.8. When the number of repetitions reaches the network update frequency F, update the target network parameter θ' = θ;

[0070] S3.10: When the maximum number of training times is reached, output the optimal drone trajectory

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] By integrating the working characteristics of timely delivery and non-timely delivery, a new logistics UAV task allocation method is proposed, which can better adapt to different types of order requirements; by determining and optimizing the location of UAV airports and logistics distribution points, the distribution efficiency and accuracy are improved; a logistics UAV fleet size planning scheme is given, which helps to rationally allocate resources and reduce costs; it can better respond to instant orders and emergencies, and through the combination of static and dynamic planning, a more reasonable flight path can be planned to reduce air traffic congestion and conflicts; the status and location of the goods are monitored in real time to ensure the safety and integrity of delivery.

[0073] The above purpose is achieved through the following technical solutions: BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is the overall flow chart of the present invention;

[0075] Figure 2 is the clustering result of site selection points;

[0076] Figure 3 is the particle fitness in multiple iterations. DETAILED DESCRIPTION

[0077] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0078] The present invention provides a method for planning a dynamic and static urban logistics UAV mission, comprising the following steps:

[0079] (1) Based on the existing urban POI data, sample data such as airspace attributes, store locations, and customer locations are obtained, and the logistics drone airport location C is determined based on the k-means clustering algorithm. The location of the logistics distribution point is determined based on the k-means clustering algorithm considering the walking reachability constraint, and the drone airport site selection is optimized according to the distribution of distribution points to balance the tasks of each airport.

[0080] (2) Static task allocation is performed based on the HPSO (hybrid particle swarm optimization) algorithm to determine the task allocation results for non-instant delivery orders.

[0081] (3) The DDQN algorithm is improved by training the neural network based on the logistics order set, determining the task allocation results for instant delivery orders and determining the logistics drone fleet size planning plan.

[0082] In the specific embodiment, 10 data points are selected for application. These data points represent the latitude and longitude information of potential airport site selection points:

[0083] Table 1: Latitude and longitude information of potential airport sites

[0084] Serial number Site selection number longitude latitude 1 A 116.4075 39.9040 2 B 121.4737 31.2304 3 C 114.0579 22.5431 4 D 108.9480 34.2632 5 E 126.6361 45.7569 6 F 113.2644 23.1291 7 G 104.0658 30.5728 8 H 118.7969 32.0603 9 I 106.5516 29.5630 10 J 117.2149 39.1256

[0085] Furthermore, the step 1 comprises the following steps:

[0086] 1.1 Assume there are two data points p 1 (x 1 ,y 1 , t 1 ) and p 2 (x 2 ,y 2 , t 2 ), where (x, y) is the spatial coordinate and t is the numerical representation of the timestamp. The spatiotemporal constraint is a composite distance metric that considers both time and space. This metric can reflect the proximity of data points in time and space. Therefore, the spatiotemporal distance metric formula can be defined as:

[0087] d st (p 1 , p 2 )=αdspatial (p 1 , p 2 )+βd temporal (p 1 , p 2 )

[0088] where d spatial (p 1 , p 2 ) is a spatial distance metric, defined as the Euclidean distance:

[0089]

[0090] d temporal (p 1 , p 2 ) is a temporal distance metric, defined as:

[0091] d temporal (p 1 , p 2 )=|t 2 -t 1 |

[0092] α and β are weight coefficients between 0 and 1, which are used to balance the contribution of spatial distance and temporal distance in the total distance.

[0093] 1.2 Assume that there are n clusters in the i-th cluster i data points, whose spatial coordinates and timestamps are (x ij ,y ij , t ij )(where j = 1, 2, ..., ni). The updated cluster center ((x c ,yc,t c )) can be calculated by the following formula

[0094]

[0095] Using the above formula and Kmeans algorithm, we can calculate the following cluster centers and the clusters to which each point belongs: cluster center 1: (110.0, 30.0), cluster center 2: (120.0, 35.0), cluster center 3: (115.0, 25.0).

[0096] The cluster to which each point belongs (hypothesis):

[0097] A: Cluster 2 B: Cluster 2 C: Cluster 3 D: Cluster 1 E: Cluster 2

[0098] F: Cluster 3 G: Cluster 1 H: Cluster 2 I: Cluster 1 J: Cluster 2

[0099] Table 2: First clustering results

[0100]

[0101]

[0102] The scatter plot of the first clustering result is as shown in the instruction manual. Figure 2 :

[0103] Attached Figure 2 The first distribution of clustering data points based on time and space constraints in the time and space dimensions is shown. Different clusters are represented by points of different depths, showing the correlation between clusters in time and space. It can also be observed that there are scattered points that deviate from the clusters. This figure helps to analyze the relationship between clustering results under time and space constraints.

[0104] 1.3 According to the flight characteristics and restrictions of drones in urban environments, the key factors for selecting logistics drone take-off and landing sites are comprehensively analyzed from four aspects: airspace conditions, traffic conditions, construction conditions of take-off and landing facilities, and the size of demand. Many factors need to be considered in selecting drone take-off and landing sites: no-fly zones and flight altitude restrictions ensure flight safety, the density of surrounding high-rise buildings assesses obstacle risks, the adaptability of terrain and conditions affects take-off and landing conditions, traffic accessibility and the number of transfer stations improve logistics efficiency, and at the same time, the construction available land area determines the actual use space of the take-off and landing site. In order to ensure customer satisfaction, it is expected that the take-off and landing site can be as close to most of the demand points as possible. To this end, the Kmeans clustering algorithm based on Euclidean distance is used to classify and aggregate each demand point. First, the number of alternative take-off and landing sites k is determined, and then the Kmeans algorithm is used to cluster the demand points. Finally, the center points of these clustering results are selected as alternative take-off and landing sites for further evaluation.

[0105] 1.4 Kmeans comprehensive evaluation and analysis to determine the alternative addresses. In order to ensure the applicability of the alternative addresses for the construction of logistics drone landing and take-off sites, the factors affecting the site selection of the landing and take-off sites are used as evaluation indicators to evaluate the cluster centers using the Kmeans method. When using the Kmeans clustering algorithm to model the site selection of logistics drone landing and take-off sites, it is first necessary to determine the evaluation indicators. These indicators usually include key factors such as airspace conditions, traffic characteristics, and construction conditions. Then collect the data of each alternative address on the evaluation indicators to form a data set. According to actual needs or data analysis, determine the number of clusters k into which the data is to be clustered, that is, the number of alternative landing and take-off sites, randomly select k points from the data set as the initial cluster centers, and assign each point to the nearest cluster center to form n clusters. Recalculate the center point of each cluster as the new cluster center, and repeat the above steps until the cluster center no longer changes significantly or reaches the preset number of iterations. a iis the average distance from point i to other points in its cluster, reflecting the compactness within the cluster, b i It is the average distance from point i to the nearest point in other clusters, reflecting the degree of separation between clusters. In order to evaluate the clustering effect, the Silhouette Coefficient can be used as an evaluation indicator. For each point i in the data set, calculate its Silhouette Coefficient S i :

[0106]

[0107] The silhouette coefficient measures the closeness of points within the same cluster and the degree of separation between different clusters. Its value range is between -1 and 1. The larger the value, the better the clustering effect. Finally, the average value of the silhouette coefficient of all data points is calculated as the evaluation index of the entire clustering result. The closer this average value is to 1, the better the clustering effect is, and the higher the applicability of the alternative address is.

[0108] The following is an example application. Using the above formula and Kmeans algorithm, we can get Table 3.

[0109] Table 3: Potential airport site selection results

[0110]

[0111] The following example uses the particle swarm optimization algorithm (PSO) to solve the UAV static task allocation problem. Specifically, there are multiple tasks that need to be assigned to multiple UAVs, and the total distance and total time for each UAV to perform the task should be minimized. At the same time, constraints need to be met, such as the flight range of the UAV and the rationality of the task allocation.

[0112] Furthermore, the step 2 comprises the following steps:

[0113] 2.1 Initialization

[0114] 2.1.1 Defining Particles

[0115] Each particle represents a non-instant delivery task allocation scheme. Assuming there are 5 tasks and 3 drones, each particle is a vector containing 5 elements, such as [1,2,3,1,2], which means that the first task is performed by drone 1, the second task is performed by drone 2, and so on.

[0116] 2.1.2 Setting parameters

[0117] Subgroup size: 20 Maximum number of iterations: 100

[0118] Inertia weight: 0.7 Individual optimal weight: 1.5

[0119] Optimal group weight: 1.5

[0120] 2.2 Particle position and velocity initialization

[0121] 2.2.1 Generate initial position

[0122] The task allocation scheme for each particle is generated. For example, a particle may be generated as [2,1,3,1,2], ensuring that each task is assigned to a drone.

[0123] 2.2.2 Initialization speed

[0124] The velocity vector of each particle is randomly initialized. The velocity is a vector with the same dimension as the task allocation scheme and is used to update the position of the particle.

[0125] 2.3 Evaluating Fitness

[0126] 2.3.1 Calculation of the objective function

[0127] The objective function consists of two parts: minimizing the total distance and minimizing the time.

[0128] For example, the objective function can be expressed as:

[0129]

[0130] Among them, D i is the total distance that the i-th UAV performs its mission, T j is the total time that the jth UAV performs its mission.

[0131] 2.3.2 Check Constraints

[0132] Check whether each particle satisfies the constraints. For example, make sure all tasks are assigned and the total distance of each drone's tasks does not exceed its flight range. If the constraints are violated, they will be punished by the penalty function.

[0133] 2.4 Update individual optimal solution and group optimal solution

[0134] 2.4.1 Update individual and group optimal solutions

[0135] If the fitness of the current particle is better than its historical optimal fitness, the individual optimal solution is updated; if the fitness of the current particle is better than the historical optimal group fitness, the group optimal solution is updated.

[0136] 2.5 Update particle speed and position

[0137] 2.5.1 Speed ​​Update

[0138] Update particle velocity according to the formula:

[0139] v op(t+1)=Ω·v op (t)+c 1 ·r 1 ·(p op -x op (t))+c 2 ·r 2 ·(g op -x op (t))

[0140] Among them, v op is the velocity of particle o in dimension p, x op is the particle position, p op is the individual optimal solution, g op is the group optimal solution, Ω is the inertia weight, c 1 and c 2 is the learning factor, r 1 and r 2 is a random number.

[0141] 2.5.2 Update particle position according to speed:

[0142] x op (t+1)=x op (+v op (t+1)

[0143] 2.5.3 Position Adjustment

[0144] Make sure the updated particle positions satisfy the task allocation constraints.

[0145] 2.6 Termination Condition Check

[0146] Check whether the maximum number of iterations or convergence conditions have been reached.

[0147] 2.7 Result Output Optimal Task Allocation

[0148] Output the task allocation plan of particles with the best fitness and evaluate the performance of the task allocation plan, including the total distance, total time and whether all constraints are met.

[0149] Figure 3 This is an example chart of the particle swarm optimization algorithm, showing the fitness changes of different particles in multiple iterations. The Global Best curve represents the optimal fitness of the group in each iteration (i.e., the minimized objective function value). It can be observed in the figure that as the number of iterations increases, the optimal fitness of the group gradually decreases and tends to be stable, indicating that the algorithm is converging.

[0150] Furthermore, in step 3, based on specific spatial information, the performance of the drone and the weather conditions at the time are taken into consideration. Based on the static task planning results, the processing of instant delivery orders is optimized, and the dynamic task planning method of the DDQN algorithm is improved. The static task planning results and the dynamic task planning results are integrated, that is, the maximum number of fleets at each location of the two planning results is selected as the fleet size planning result to ensure that the delivery order requirements are met. Specifically, the following steps are included:

[0151] 3.1 Construct the communication scenario between logistics drones and communication terminals, IoT devices, and user devices, including a drone and obstacles of different heights. Within time t, the position of the drone is w;

[0152] 3.2: Based on the communication scenario, determine the UAV air-to-ground channel gain and establish the communication link model between the UAV and the ground user:

[0153] where α Los α QLos is the communication efficiency between the UAV and the terminal in the Los time interval and the average communication efficiency in I Los time intervals, β Los β QLos is the communication efficiency between the ground equipment and the terminal in the time interval of Los and the average communication efficiency in I time intervals; is the channel gain between the ground equipment and the UAV; η Los,ifLos is the basic communication efficiency in the time interval of Los; η TLos,ifILos is the basic communication efficiency in the total time of I Los time intervals.

[0154] 3.3: The drone collects data and establishes a drone trajectory optimization model under the data collection scenario;

[0155] 3.4: Based on the communication link model, the optimization model is standardized as a four-tuple<S,A,R,P> The Markov decision process is represented by A, where A represents the action space of the drone and the action a selected by the drone in time t t =[a x, a y ,a z ]∈A; S represents the state space of the UAV, and the state of the UAV at time t in this space is represented by s t =(s t1 ,s t2 ,s t3 ); P is the transition probability matrix, including the drone in state s t When taking action a t Transition to next state s t+1 The probability of the drone choosing action at Arriving at the next state s t+1 The profit information obtained in the process, the reward function r t For: t =r t1 -r t2 -r t3 ;

[0156] 3.5: Select action a according to the ε strategy t , that is, randomly select action a t The probability of selecting an action according to the Q value is ε, and the probability of selecting an action according to the Q value is 1-ε, to ensure that the drone has a certain degree of exploration;

[0157] 3.6: Change a n Enter the data collection environment and get the current state s n and the next position of the drone w t+1 , where w t+1 =w t +a t ;

[0158] 3.7: The drone is located at position w t The amount of data collected is r t1 , according to the movement of the drone, the power penalty r is obtained t2 , according to whether the drone hits an obstacle in the current state and the next state, a collision penalty r is obtained t3 , and finally get the total reward r t ;

[0159] 3.8: According to w t+1 , determine the next state of the drone n+1 ;

[0160] 3.9: Transfer results t+1 ,a t ,s t ,r t >Save to experience pool J. When the number of data sets in the experience pool reaches the threshold, select sample sets from the experience pool to train the neural network. During the training process, the new data will replace the old data sets in the experience pool.

[0161] 3.10: Calculate the error function L(θ) = ‖y t- Q(a t ,s t ;θ)‖ 2 , which is the mean square error used to measure the difference between the current Q value and the target Q value, where y t is the target Q value, and

[0162] ​

[0163] θ is the target network parameter, θ n is the real-time parameter of the target network, r t is the reward value for time period t, argmaxQ() means finding the action corresponding to the maximum Q value in the current Q network;

[0164] 3.11: Repeat steps 3.2 to 3.11. When the number of repetitions reaches the network update frequency F, update the target network parameter θ'=θ, where θ is the current network parameter;

[0165] 3.12: When the maximum number of training times is reached, output the optimal drone trajectory The results are shown in Table 4.

[0166] Table 4: Dynamic task planning DDQN algorithm parameter results table

[0167]

Claims

1. A dynamic and static combined urban logistics UAV mission planning method, characterized in that: The method comprises the following steps: S1. Input the urban logistics drone mission planning urban logistics order data of the combination of dynamic and static real-time and non-real-time delivery to obtain sample data, which includes airspace attributes, weather information, store location, and customer location. Based on the store and customer location data, the k-means clustering algorithm improved based on spatiotemporal constraints is used to preliminarily select the location of the logistics drone airport; S2. Use the static task planning method based on the HPSO (hybrid particle swarm optimization) algorithm to determine the static task planning results for non-instant delivery orders, use the DBSCAN clustering algorithm considering the walkability constraint to determine the location of the logistics delivery point, and optimize the location of the drone airport according to the distribution of logistics delivery points to balance the number of tasks at each airport; S3. Based on the location of drone airports and logistics distribution points, and taking into account spatial information, drone performance, and weather conditions, a dynamic task planning method based on the static task planning results to improve the DDQN algorithm is proposed to determine the dynamic task planning results of instant delivery orders, thereby ultimately determining the planning and scheduling plan for the logistics drone fleet.

2. According to the method of claim 1, the method is characterized in that: The step S1.1 considers the k-means clustering algorithm with spatiotemporal constraints, comprising the following steps: S1.1.1 Determination of alternative address: (1) The k-means clustering algorithm based on Euclidean distance is used to cluster the demand points, determine the number k of alternative logistics drone airports, perform k-means clustering on the demand points, and use the cluster centers in the clustering results as the alternative logistics drone airports to be evaluated; (2) Collect the data of each candidate logistics drone airport on the evaluation indicators to form a data set, wherein the evaluation indicators include airspace conditions, traffic characteristics, and construction conditions. Then, according to actual needs or data analysis, optimize and adjust the number of candidate take-off and landing sites, randomly select k points from the data set as the initial clustering center, assign each point in the data set to the nearest clustering center, form k clusters, recalculate the center point of each cluster as the new clustering center, and repeat the above steps until the clustering center no longer changes significantly or reaches the preset number of iterations; S1.1.2 Establishment of multi-objective site selection model: Use a distance-based function to characterize customer satisfaction: a i is the average distance from point i to other points in its cluster, b i is the average distance from point i to the nearest point in the other clusters. The silhouette coefficient evaluation index is used to calculate its silhouette coefficient S i : The silhouette coefficient measures the compactness of points within the same cluster and the degree of separation between different clusters. Its value range is between -1 and 1. The larger the value, the better the clustering effect. Finally, the average value of the silhouette coefficient of all data points is calculated as the evaluation indicator of the entire clustering result.

3. According to the method for urban logistics UAV task planning combining dynamic and static functions as described in claim 1 or 2, the step S1.2 is based on the k-means clustering algorithm of Euclidean distance, and the specific implementation steps are as follows: S1.2.1 Based on the obtained sample data, obtain two arbitrary data points p1(x1, y1, t1) and p2(x2, y2, t2), where (x, y) are spatial coordinates, t is the numerical representation of the timestamp, and subscripts 1 and 2 are used to distinguish different data points; S1.2.2 imposes spatiotemporal constraints on data points. The spatiotemporal constraint condition is a composite distance metric that considers both time and space, called the spatiotemporal distance metric. The spatiotemporal distance metric can reflect the proximity of data points in time and space. The spatiotemporal distance metric formula is defined as: d st (p1,p2)=αd spatial (p1,p2)+βd temporal (p1,p2) where d spatial (p1, p2) is a spatial distance metric, defined as the Euclidean distance: d temporal (p1, p2) is a temporal distance measure, defined as: d temporal (p1, p2) = |t2-t1|; α is the weight coefficient of spatial distance, β is the weight coefficient of temporal distance, and the values ​​of α and β are between 0 and 1, which are used to balance the contribution of spatial distance and temporal distance in the total distance respectively; S1.2.3 Assume that there are n i data points, whose spatial coordinates and timestamps are (x ij ,y ij , t ij ), where j = 1, 2, ..., n i ; Then the updated cluster center of the i-th cluster ((x c ,y c , t c )) is calculated by the following formula 4. The method for planning a dynamic and static urban logistics UAV mission according to claim 1 is characterized in that: The static task planning method based on the HPSO (hybrid particle swarm optimization) algorithm described in step 2 comprises the following steps: S2.1 Initialization: S2.1.1 Define particles: Each particle represents a possible non-immediate delivery task allocation scheme. The dimension of the particle is equal to the number of task points, and the value of each dimension indicates which drone the task point is assigned to. S2.1.2 Setting parameters: define the size of the particle swarm, the maximum number of iterations, the inertia weight, the individual optimal weight and the group optimal weight; S2.2 Initialize particle position and velocity: S2.2.1 Generate initial position: Randomly generate a task allocation plan for each particle to ensure that each task point is assigned to a drone; S2.2.2 Initialization speed: Randomly set the initial speed for each particle; S2.3 Evaluating fitness: S2.3.1 Calculate the objective function: For each particle, calculate its fitness value, that is, the objective function value, including minimizing the total distance and the minimum time. The objective function is expressed as: Among them, D x is the total distance that the x-th UAV performs its mission, T y is the total time for the yth UAV to perform the mission, N and M are the number of UAVs in each particle; S2.3.2 Check constraints: Evaluate whether each particle satisfies the constraints. Particles that do not meet the constraints will be punished by the penalty function. S2.4 Update individual optimal solution and group optimal solution: S2.4.1 Update individual optimal solution: If the fitness of the current particle is better than its historical optimal fitness, update the individual optimal solution; S2.4.2 Update the optimal solution of the group: If the fitness of the current particle is better than the historical optimal group fitness, then update the optimal solution of the group; S2.5 updates particle velocity and position: S2.5.1 Speed ​​update: According to the formula: v op (t+1)=Ω·v op (t)+c1·r1·(p op -x op (t))+c2·r2·(g op -x op (t)) Among them, v op is the velocity of particle o in dimension p, x op is the particle position, p op is the individual optimal solution, g op is the optimal solution of the group, Ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers; S2.5.2 Position update: Update particle position according to velocity: x op (t+1)=x op (t)+v op (t+1) S2.5.3 Position adjustment: ensure that the updated particle position meets the task allocation constraints; S2.6 Termination condition check: S2.6.1 Iteration number: Check whether the maximum number of iterations has been reached; S2.6.2 Convergence condition: Check whether the fitness reaches a predetermined threshold or converges to a stable value; S2.7 Result output: S2.7.1 Optimal task allocation: Output the task allocation plan of particles with optimal fitness; S2.7.2 Performance evaluation: Evaluate the performance of the task allocation scheme, including total distance, total time, and whether all constraints are met; S2.8 Actual deployment.

5. The method for planning urban logistics UAV missions combining dynamic and static methods according to claim 1 is characterized in that: In step 3, based on specific spatial information, considering the performance of the drone and the weather conditions at the time, according to the static task planning results, the processing of instant delivery orders is optimized, and the dynamic task planning method of the DDQN algorithm is improved. The static task planning results and the dynamic task planning results are integrated, that is, the maximum number of fleets at each location of the two planning results is selected as the fleet size planning result to ensure that the delivery order requirements are met. The specific steps include: S3.1: Construct a communication scenario between a drone and a communication terminal, IoT device, and user equipment, including a drone and obstacles of different heights. Within time η, the position of the drone is w; S3.2: Based on the communication scenario, determine the UAV air-to-ground channel gain and establish the communication link model between the UAV and the ground user: where α Los α QLos is the communication efficiency between the UAV and the terminal in the Los time interval and the average communication efficiency in I Los time intervals, β Los β QLos is the communication efficiency between the ground equipment and the terminal in the time interval of Los and the average communication efficiency in I time intervals; is the channel gain between the ground equipment and the UAV; η Los,ifLos is the basic communication efficiency in the time interval of Los; η TLos,ifILos is the basic communication efficiency in the total time of I Los time intervals; S3.3: Select action a according to the ε strategy t , that is, randomly select action a t The probability of selecting an action according to the position w of the UAV is ε, and the probability of selecting an action according to the position w of the UAV is 1-ε, to ensure that the UAV has a certain degree of exploration; S3.4: Add a t Enter the data collection environment and get the current state s t and the next position of the drone w t+1 , where w t+1 =w t +a t ; S3.5: Based on the communication link model, the optimization model is standardized as a four-tuple<S,A,R,P> , where A represents the action space of the drone delivery process, and the action a selected by the drone within time t t =[a x ,a y ,a z ]∈A; S represents the state space of the UAV, and the state of the UAV at time t in this space is represented by s t =(s t1 ,s t2 ,s t3 ); P is the transition probability matrix, including the drone in state s t When taking action a t Transition to next state s t+1 The probability of the drone choosing action a t Arriving at the next state s t+1 The profit information obtained in the process, the reward function r t For: t =r t1 -r t2 -r t3 ; The drone is based on the position w t The amount of data collected is r t1 , according to the movement of the drone, the power penalty r is obtained t2 , according to whether the drone hits an obstacle in the current state and the next state, a collision penalty r is obtained t3 , and finally get the total reward r t ; S3.6: According to w t+1 , determine the next state of the drone t+1 ; S3.7: Transfer results t+1 ,a t ,s t ,r t >Save to experience pool J. When the number of data sets in the experience pool reaches the threshold, select sample sets from the experience pool to train the neural network. During the training process, the new data will replace the old data sets in the experience pool.​ S3.8: Calculate the error function L(θ) = ‖y t- Q(a t ,s t ;θ)‖ 2 , which is the mean square error used to measure the difference between the current Q value and the target Q value, where y t is the target Q value, and θ is the current network parameter, θ n is the real-time parameter of the target network, r t is the reward value for time period t, argmaxQ() means finding the action corresponding to the maximum Q value in the current Q network; S3.9: Repeat steps S3.2.2 to S3.2.

8. When the number of repetitions reaches the network update frequency F, update the target network parameter θ' = θ; S3.10: When the maximum number of training times is reached, output the optimal UAV trajectory W = [w1, w2, w3…, w t ],

Images