Unmanned aerial vehicle path planning method and system for assisting mobile edge calculation

Abstract

The invention discloses an unmanned aerial vehicle path planning method and system for assisting mobile edge calculation. The method comprises the following steps: acquiring the position of each user in a service area; clustering all the users according to the positions of the users to obtain a plurality of hovering points; performing cluster division on all the hovering points to obtain a hovering point cluster of each unmanned aerial vehicle; for each unmanned aerial vehicle, determining a sequence that the unmanned aerial vehicle accesses each hovering point in the corresponding hovering point cluster by taking the maximum reciprocal of the distance that the unmanned aerial vehicle walks around all the hovering points in the corresponding hovering point cluster and then returns to the central base station as a target; and for each unmanned aerial vehicle, determining the optimal path of the unmanned aerial vehicle by taking the maximum energy efficiency of the unmanned aerial vehicle as a target according to the sequence of the unmanned aerial vehicle accessing each hovering point in the corresponding hovering point cluster. Therefore, the unmanned aerial vehicle has the highest working efficiency on the basis of having the shortest driving path.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technologies, and in particular to a method and system for unmanned aerial vehicle path planning assisted by mobile edge computing. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] For areas such as mountains, forests and deserts that lack cellular infrastructure and mobile edge servers, drones can be allowed to participate in mobile edge computing by setting up hovering points in the area they visit to communicate with ground infrastructure and edge servers, thereby meeting users' 5G wireless communication needs. The communication performance of drones in wireless communications can be improved by dynamically adjusting the position of the hovering points and the flight of the drone.

[0004] In the related art, when planning the trajectory of a drone, it is assumed that users in the service area are always within the communication coverage of the drone, and when planning the path of a drone, the energy consumption of the drone is mainly considered.

[0005] However, in actual scenarios, there is no guarantee that users in the area are always within the communication coverage of the drone, and when the low energy consumption of the drone is excessively pursued, it will also affect the communication performance of the entire area. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a UAV path planning method and system assisted by mobile edge computing, so that the UAV has the highest working efficiency on the basis of having the shortest driving path.

[0007] To achieve the above object, the present invention adopts the following technical solution:

[0008] First, a UAV path planning method assisted by mobile edge computing is proposed, including:

[0009] Get the location of each user in the service area;

[0010] All users are clustered according to their locations to obtain multiple hovering points;

[0011] All hovering points are clustered to obtain the hovering point cluster of each drone;

[0012] For each drone, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station;

[0013] For each UAV, the optimal path of the UAV is determined according to the order in which the UAV visits the hovering points in the corresponding hovering point cluster, with the goal of maximizing the energy efficiency of the UAV.

[0014] Furthermore, according to the location of each user, a density clustering algorithm is used to cluster all users and obtain multiple groups of users;

[0015] Through the AP algorithm, each group of users is clustered again to obtain multiple hovering points.

[0016] Furthermore, the number of hovering point clusters divided is the same as the number of drones;

[0017] And there is at least one hovering point in each hovering point cluster.

[0018] Furthermore, the same number of hovering points as the number of drones are randomly selected from all hovering points as the clustering centers of each cluster;

[0019] The hovering points closest to the cluster center are placed in the same cluster to obtain multiple hovering point clusters;

[0020] Calculate the class center of each hover point cluster;

[0021] The error between the class center and cluster center of each hovering point cluster is calculated. When the error is greater than the set value, the cluster center of each cluster is reselected from all hovering points, and the hovering point cluster is re-determined based on the re-selected cluster center until the error between the class center and cluster center of the determined hovering point cluster is less than or equal to the set value, and the final hovering point cluster is obtained.

[0022] Furthermore, a genetic algorithm is used to determine the order in which the UAV visits each hovering point in the corresponding hovering point cluster.

[0023] Furthermore, with the goal of maximizing the energy efficiency of the UAV, a Markov decision process is used to determine the optimal path of the UAV.

[0024] Secondly, a UAV path planning system assisted by mobile edge computing is proposed, including:

[0025] A data acquisition unit, used to acquire the location of each user in the service area;

[0026] A hovering point determination unit, used to cluster all users according to their positions to obtain multiple hovering points;

[0027] A hovering point clustering unit is used to cluster all hovering points to obtain a hovering point cluster for each drone;

[0028] A hovering point sequence determination unit is used to determine, for each drone, the sequence in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the reciprocal of the distance that the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station;

[0029] The drone optimal path determination unit is used to determine the optimal path of each drone according to the order in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the drone energy efficiency.

[0030] In a third aspect, a computer device is provided, the device comprising:

[0031] a processor adapted to execute a computer program;

[0032] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, a method for drone path planning for assisted mobile edge computing proposed in the first aspect is implemented.

[0033] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing a drone path planning method for assisted mobile edge computing proposed in the first aspect.

[0034] In a fifth aspect, a computer program product is proposed, which includes a computer program. When the computer program is executed by a processor, it implements a drone path planning method for assisted mobile edge computing proposed in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention proposes a method and system for drone path planning that assists mobile edge computing. The method clusters users in a service area to determine a set of hovering points, and then clusters the hovering points to ensure that the number of clusters divided is the same as that of drones, and that there are hovering points in each cluster, so that all users in the service area can be covered by the communication range of the corresponding drone. Then, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance of the drone traveling through all hovering points in the corresponding hovering point cluster and returning to the central base station; finally, according to the determined order, the optimal path of the drone is determined with the goal of maximizing the drone energy efficiency. This ensures that the drone has the highest working efficiency on the basis of having the shortest driving path.

[0037] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0039] Figure 1 A flow chart of a method for drone path planning assisted by mobile edge computing disclosed in an embodiment;

[0040] Figure 2 A schematic diagram of a multi-UAV assisted mobile edge computing system to which the disclosed method of the embodiment is applicable;

[0041] Figure 3 A flowchart for determining the optimal path of a drone in a drone path planning method for assisting mobile edge computing disclosed in an embodiment. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0044] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0045] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0046] Example 1

[0047] With the rapid development of 5G wireless communication and Internet of Things technology, a series of new applications have emerged, such as unmanned driving, environmental monitoring, etc. However, due to various restrictions on production costs, the computing resources of related terminal devices are often limited, which in turn affects the quality of user experience. Mobile edge computing (MEC) is a widely studied and promising technology for supporting various IoT devices. It can provide low-latency, high-reliability services, and can perform user tasks in a nearby edge cloud and transmit the results back to the user. Since the physical distance between the MEC server and the user is smaller, MEC consumes less energy than mobile cloud computing. By deploying MEC servers at the edge of the communication network, such as ground cellular infrastructure, ground users can improve the quality of user experience by offloading tasks to edge servers. However, since the location of the edge cloud is usually fixed and cannot be flexibly changed according to user requirements, it still has certain shortcomings in meeting user requirements. In the event of a natural disaster, it may not be able to complete the service work due to the interruption of the ground communication link.

[0048] To effectively deal with the above problems, unmanned aerial vehicles (UAVs) are considered to be one of the favorable technologies for achieving these ambitious goals. Compared with the communication system of traditional ground fixed base stations, the biggest advantage of UAVs lies in their unparalleled flexibility, maneuverability and low cost. UAVs have higher cost benefits and are more likely to achieve better service quality. In fact, with the help of UAVs, system performance can be significantly enhanced by establishing a line-of-sight link between the UAV and the user. In addition, by dynamically adjusting the hovering point position and the flight of the UAV, it is possible to improve the communication performance of the UAV in wireless communication.

[0049] Mobile edge computing (MEC) effectively improves the computing performance of users with limited computing resources. However, some IoT-related optimizations are deployed in remote areas, such as mountains, forests, and deserts. These scenarios lack cellular infrastructure and mobile edge servers and cannot provide services to users. Fortunately, drone-assisted mobile edge computing has attracted a lot of attention due to its mobility and flexibility. When drones assist in mobile edge computing, it is assumed that users in the service area are always within the communication coverage of the drone, but this is obviously inconsistent with the actual situation. In addition, current research on drone MEC systems mainly tends to focus on users themselves or the energy consumption of drones. For example, they mainly consider how to reduce the energy consumption of drones by designing the trajectory of drones. However, blindly pursuing the goal of drone energy saving will lead to bottlenecks in the computing performance of the entire system, such as the inability to improve the performance of the system in processing computing tasks).

[0050] In order to solve the above technical problems, in this embodiment, a method for UAV path planning assisted by mobile edge computing is disclosed, which is suitable for Figure 2 The drone-assisted mobile edge computing system shown in the figure includes multiple ground users, multiple drones and a central base station, and multiple drones provide services for multiple ground users. In order to improve the working efficiency of drones in the drone-assisted mobile edge computing system, this embodiment determines the optimal path of the drone with the maximum energy efficiency of the drone under the constraints of UAV unloading decision, frequency band bandwidth ratio variable, UAV grouping set, and UAV trajectory. And when determining the optimal path of the drone, the UAV hovering point, UAV scheduling decision, and the order in which the UAV visits the hovering point are particularly considered, which is a non-convex, nonlinear, NP-hard and mixed integer problem. In order to solve this problem, this embodiment proposes a drone path planning method for assisted mobile edge computing, which uses an algorithm that combines multi-cluster genetic trajectory planning and deep deterministic policy gradient (DDPG) with Dueling-DQN, namely MCGTP-PDDQN, to perform drone path planning. The algorithm is divided into two parts. The first part is a solution algorithm based on multi-clustering combined with genetic algorithm to determine the sequence of drone visits to hovering points; the second part is an algorithm based on the combination of DDPG for processing continuous actions and deep dual Q network (Dueling-DQN) for processing discrete action space to determine the optimal path of the drone when the drone has maximum energy efficiency. The two parts are integrated to form the MCGTP-PDDQN algorithm model of this embodiment to solve the original problem of optimizing the maximum working efficiency of drones, and the data verifies that the drone path planning method for assisting mobile edge computing disclosed in this embodiment is superior to other comparison algorithms in terms of UAV working efficiency when used for drone path planning.

[0051] A method for drone path planning assisted by mobile edge computing disclosed in this embodiment is described in detail.

[0052] like Figure 1 As shown, the present embodiment discloses a method for drone path planning for assisting mobile edge computing, including:

[0053] S1: Get the location of each user in the service area.

[0054] S2: Cluster all users according to their locations to obtain multiple hovering points;

[0055] S3: cluster all hovering points to obtain the hovering point cluster of each drone;

[0056] S4: For each drone, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station;

[0057] S5: For each UAV, the optimal path of the UAV is determined according to the order in which the UAV visits the hovering points in the corresponding hovering point cluster, with the goal of maximizing the energy efficiency of the UAV.

[0058] This embodiment plans the path of the drone based on the drone path planning model; wherein the drone path planning model includes constraints such as unloading decision constraints, UAV scheduling variable constraints, user bandwidth ratio variable constraints, grouping variable constraints served by the drone, drone trajectory constraints and computing resource constraints, and an objective function with the goal of maximizing the energy efficiency of the drone. Specifically, the drone path planning model is:

[0059] Objective Function

[0060] C2:α i +α j <2

[0061]

[0062]

[0063] Among them, C1 represents the limited energy constraint of UAVs, C2 represents the task offloading constraint, C3 represents that there is no intersection between the service groups of the two UAVs, C4 represents that the groups served by all UAVs should cover the entire cluster C, C5 represents the bandwidth share constraint, C6 represents the group priority constraint between the groups served by each UAV, represents the group set served by UAV j and sorted by priority, C7 ensures that the initial point and the end point are at the same position, C8 and C9 constrain the operation trajectory of UAV respectively, C10 divides all users (UEs) into different groups by minimizing the total distance between the user and its group center; it means that the UAV can cover all users (UE) corresponding to the hovering point at the hovering point, and d j Indicates the communication radius corresponding to each UAV. C11 limits the user's task volume, and C12 indicates that the UAV's service time for each UE must not exceed the maximum tolerance time of its service UE. Respectively represent the number of time slots, the number of users (UE) in the service area, and the number of drones; α i (t) represents the variable of partial offloading of UE tasks, α j (t) represents the unloading variable of the UAV part; represents the total energy consumption of the jth UAV in the tth time slot, Indicates the remaining energy of the drone, is a constant, indicating the upper limit of energy that the battery can provide within a time slot; C13 indicates that the movement of the UAV is constrained by the power of its own battery.

[0064] This embodiment uses the MCGTP algorithm to initialize the state variables and action variables in the drone path planning model, determine the user grouping, hovering point set, and the order in which the drone visits the hovering points, so as to improve the convergence speed and stability of the drone path planning model, specifically including:

[0065] S1: Get the location of each user in the service area.

[0066] S2: Cluster all users according to their locations to obtain multiple hovering points. This includes:

[0067] S21: Based on the location of each user, a density clustering algorithm is used to cluster all users to obtain multiple user groups.

[0068] In this embodiment, all users (UE) are divided into different groups according to the distribution of user density, C = {C 1 ,C 2 ,…,C k} represents the user groups of the k groups obtained.

[0069] User Density Clustering (UDC) is an algorithm based on DBSCAN (density clustering algorithm). The core idea of ​​the UDC algorithm is to first select the core points according to the set search radius ε and the domain sample number threshold MinPts, and then start from a core point and continuously expand to the density reachable area, so as to obtain a maximized area containing core points and boundary points. Any two points in this area are densely connected. Repeat the above operation until all core points are traversed. Finally, the generated noise points are processed.

[0070] This embodiment includes users in the nearest core point set or boundary point set, thereby forming a user group covering all users.

[0071] S22: Cluster each user group again using the AP algorithm to obtain multiple hovering points.

[0072] This embodiment uses the AP (Affinity Propagation) algorithm to generate hovering points. It is not necessary to determine the number of clusters in advance. Instead, all user positions u i ={x i ,y i,0} are regarded as potential cluster centers (exemplars), which is different from clustering algorithms such as K-means. In this embodiment, the AP algorithm is used to cluster each user group again, and the specific steps of obtaining multiple hovering points are as follows:

[0073] First, let u i The user position, i.e. the user's coordinates, is represented by the matrix S to represent the similarity between the positions of each user. In this embodiment, the Euclidean distance between the positions of each user is calculated to represent the similarity between the positions of each user. In this embodiment, the initial similarity matrix S is set (0) for:

[0074] S (0) =(S i,k ) I×I

[0075] The similarity is calculated as follows:

[0076] S i,k =s(i,k)=-d 2 (u i ,u k )=-‖u i -u k ‖ 2

[0077] Among them, S i,k Represents user u i With user u k The similarity between locations, expressed as Euclidean distance.

[0078] If S(i,k)>S(i,i′), then user u i To user u k The distance ratio u i to u i′ The distance between them is close. Where S(k,k) represents user u k As user u k The suitability of the cluster center can be understood as the user u k Become the cluster center fitness. At the beginning, this value is the value given during initialization and will affect the final number of clusters.

[0079] Afterwards, the attraction matrix and the belonging matrix are iterated.

[0080] At the beginning of the calculation, the attraction matrix R and the attribution matrix A are all initialized to 0 matrices. The process of iteratively updating the attraction matrix and the attribution matrix is:

[0081] Attraction matrix R = [r(i,k)] I×I :r(i,k) represents user uk Suitable for user u i The class represents the degree of representativeness of the point.

[0082] r(i,k)=s(i,k)-min k′≠k (a(i,k′)+s(i,k′))

[0083] Attribution matrix A = [a(i,k)] I×I :a(i,k) represents user u i Select user u k The degree of suitability as a representative of the class.

[0084]

[0085] For the attraction matrix R, suppose there is now a cluster center u k , we find another hypothetical cluster center u k′ , redefine u k′ The attraction matrix and belonging matrix of u are: a(i,k′)+s(i,k′), find the one with the largest sum of these two values, and then subtract the largest one from s(i,k), which means that u k The cluster center of user u i degree of attraction.

[0086] For the attribution matrix A, if user u k As another user i′ The fitness of the cluster center is very large, so user u k As another user i′ The fitness of the cluster center may also be large, so we can first calculate the user u k To other users ′′ The attractiveness of r(i′,k) is then summed to represent the user u k The attractiveness to other users is obtained as ∑max(0,r(i′,k)).

[0087] Again, after the attraction matrix R and the attribution matrix A are iteratively updated, the attraction matrix R and the attribution matrix A are attenuated respectively by the damping coefficient λ.

[0088]

[0089] Finally, by repeatedly updating the attraction matrix R and the attribution matrix A in sequence, and decaying the attraction matrix R and the attribution matrix A, until the maximum number of iterations MNI is reached or the attraction and attribution matrices are stable, the iterative update is completed and multiple hovering points are output.

[0090] S3: Cluster all hovering points to obtain the hovering point cluster of each drone.

[0091] In this embodiment, the number of hovering point clusters divided is limited to the number of drones; and there is at least one hovering point in each hovering point cluster, and all hovering points are clustered based on the K-means algorithm to obtain a hovering point cluster for each drone, and the process includes:

[0092] Randomly select the same number of hovering points as the number of drones from all hovering points as the cluster centers of each cluster, and use M′ to represent the number of drones to be dispatched.

[0093] The distance between each hovering point and each cluster center is calculated respectively, and the hovering point closest to the cluster center is placed in the same cluster to obtain multiple hovering point clusters;

[0094] Calculate the class center of each hover point cluster, which is the average value of all hover point positions in the hover point cluster;

[0095] Calculate the error between the class center of each hovering point cluster and its cluster center. When the error is greater than the set value, it indicates that the cluster center has changed. Reselect the cluster center of each cluster from all hovering points, and re-determine the hovering point cluster based on the reselected cluster center until the error between the class center of the determined hovering point cluster and its cluster center is less than or equal to the set value, and obtain the final hovering point cluster C. When the error between the class center of the hovering point cluster and its cluster center is less than or equal to the set value, the registered cluster center will no longer change.

[0096] All hovering points are clustered to ensure that one drone is responsible for one hovering point cluster. It should be noted that the number of hovering points in a cluster is non-zero. The drones follow a certain motion trajectory and successively arrive at each transmission circle with the hovering point as the center and the transmission distance as the radius to collect data.

[0097] S4: For each drone, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance the drone travels through all hovering points in the corresponding hovering point cluster and then returns to the central base station.

[0098] The flight planning problem of each drone can be solved as a traveling salesman (TSP) problem. This embodiment uses a genetic algorithm to determine the order in which the drone visits each hovering point in the corresponding hovering point cluster. The process includes:

[0099] S41. Perform chromosome encoding on all hovering points in the hovering point cluster.

[0100] This embodiment uses integer permutation coding to encode the hovering points. For a TSP problem with n hovering points, the chromosome is divided into n segments, where each segment is the number of the corresponding hovering point. For example, for a TSP problem with 10 hovering points {1,2,3,4,5,6,7,8,9,10}, |1|10|2|9|3|7|4|6|5 is a legal chromosome.

[0101] S42. Population initialization

[0102] After completing the chromosome encoding, an initial population must be generated as the starting solution, so the number of the initial population must be determined first. The number of the initial population is generally obtained based on experience. In general, the number of populations is determined by the size of the hovering point. The value fluctuates between 50-200.

[0103] S43. Set fitness function

[0104] Let k 1 |k 2 |…|k i |…|k n | is a chromosome encoded with integers, is the hovering point k i To the hovering point k j The distance between the individual and the target is:

[0105]

[0106] Where fitness is the value of the fitness function, which is the reciprocal of the distance the UAV travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station.

[0107] The goal of genetic algorithm optimization is to select chromosomes with the largest possible fitness function value. The larger the fitness function value, the better the quality of the chromosome, and vice versa.

[0108] S44. Select operation

[0109] The selection operation is to select individuals from the old group to the new group with a certain probability. The probability of an individual being selected is related to the fitness value. The larger the individual fitness value, the greater the probability of being selected.

[0110] S45, Cross Operation

[0111] Partial mapping crossover is used to determine the parent generation of the crossover operation, and the parent generation samples are grouped in pairs. The following process is repeated for each group (assuming that the number of hovering points that each UAV needs to visit is 10):

[0112] (1) Generate random integers r1 and r2 in the interval [1,10], determine two positions, and cross the intermediate data between the two positions.

[0113] (2) After the crossover, if there are repeated hovering point numbers in the same individual, the non-repeated numbers will be retained, and the conflicting numbers will be eliminated by partial mapping, that is, mapping will be performed using the corresponding relationship of the middle segment.

[0114] S46, mutation operation

[0115] The mutation strategy takes two randomly selected points and swaps them. Generate two random integers r1 and r2 in the range [1,10], determine two positions, and swap them.

[0116] S47, Evolutionary Reverse Operation

[0117] In order to improve the local search capability of the genetic algorithm, multiple continuous evolutionary reversal operations are introduced after selection, crossover, and mutation. The "evolution" here refers to the unidirectional nature of the reversal operator, that is, only those with improved fitness after reversal can be accepted, otherwise the reversal is invalid. Generate two random integers r1 and r2 in the interval [1,10], determine two positions, and swap them. Perform crossover and mutation on each individual, and then bring it into the fitness function for evaluation, and select individuals with large fitness values ​​for crossover and mutation and evolutionary reversal operations in the next generation. Loop operation: Determine whether the set maximum genetic generation is met. If not, continue to calculate the fitness value; otherwise, end the genetic operation.

[0118] S5: For each UAV, the optimal path of the UAV is determined according to the order in which the UAV visits the hovering points in the corresponding hovering point cluster, with the goal of maximizing the energy efficiency of the UAV.

[0119] After obtaining the order in which the drone visits the hovering points in the first part, this embodiment determines the optimal path of the drone for each drone according to the order in which the drone visits the hovering points in the corresponding hovering point cluster, with the goal of maximizing the energy efficiency of the drone.

[0120] This embodiment aims to maximize the energy efficiency of the drone and adopts a Markov decision process to determine the optimal path of the drone, wherein the Markov decision process includes a state space, an action space, and a reward function.

[0121] Preferably, this embodiment aims to maximize the energy efficiency of the drone, regards each drone as an intelligent agent, and processes the order in which the drone visits each hovering point in the corresponding hovering point cluster through a multi-agent deep reinforcement learning algorithm to determine the optimal path of the drone.

[0122] Specifically, the Markov decision process of I agents can be represented by the following tuple: Γ=(S,I,A,R). Where S represents the state, S={s(t)}, represents the state set of agent i. A={a[t]} represents the action set of agent i. Each drone acts as an agent, observes the environmental data, takes actions in the action set at the tth time slot, and obtains rewards. The environment updates the state and traverses to the new state. The state, action, and reward are defined as follows:

[0123] State space: The state space of the scene in this embodiment includes the position of the drone User location u i ={x i ,y i ,0}, user group set The set of grouping feature vectors c.

[0124]

[0125] Action space: The action a[t] of the drone includes computing resource allocation, partial offloading decision variable A, user bandwidth ratio variable μ, drone trajectory Q, number of drones used M′, and drone flight direction Θ. Therefore, the action space can be expressed as

[0126] a[t]={A,μ,Q,M′,Θ}

[0127] Reward function: In deep reinforcement learning, reward is the feedback given by the environment to the agent for performing an action. This embodiment converts the complex optimization objective function into maximum or minimum cumulative reward by designing a reward function. This embodiment aims to maximize the energy efficiency of the drone. Therefore, the reward function is defined as:

[0128]

[0129] According to the Markov decision process, the action-value network (actor-critic network) is configured to build the PDDQN algorithm model.

[0130] Using the Markov decision process, the process of determining the optimal path for the UAV is:

[0131] In the first step, it initializes the network parameters ω of the actor network μ and the network parameters ω of the critic network Q ω Q Including V and ω A .

[0132] The second step is to initialize the capacity of the experience replay pool and the remaining energy E of the drone. remain [t] and the total number of time slots. The remaining energy E of the droneremain [t] is initialized to the battery capacity per time slot.

[0133] The third step is to use the actions and states of the intelligent agent when the drone energy efficiency is maximum to train the PDDQN algorithm model through the stochastic gradient descent algorithm. After the training is completed, a trained drone optimal path determination model is obtained.

[0134]

[0135]

[0136] The PDDQN algorithm model processes the input data as follows: At the beginning of each time slot, the initial state s of the agent is obtained 0 , the initial state s[t] is first input into the actor network to obtain the continuous actions of the agent in each time slot; the initial state and continuous actions of the agent are input into the critic network, and the discrete actions j[t], j[t]∈J are selected from the continuous actions of the agent through the ∈-greedy exploration mechanism. First, the number Ψ is randomly generated based on ∈-greedy. Next, if Ψ>∈, through the formula Select a discrete action j[t]. Otherwise, randomly select a discrete action j[t]∈J. Based on the obtained a[t]=(j[t], a j [t]), determine the action function value of the agent to perform the action, where the action a[t] is used as a parameter. Then, based on the action function value, calculate and determine the reward r[t] that the drone can obtain and the latest state s[t+1] of the agent.

[0137] The agent's latest state s[t+1] interacts with the environment to obtain a tuple (s[t], a[t], r[t], s[t+1]), which is saved in the experience pool. A small batch of tuples (s[ζ], a[ζ], r[ζ], s[ζ+1]) is randomly selected from the experience pool. If the time slot ends in state [ζ+1], then y[ζ] = r[ζ]; otherwise, y[ζ] = r[ζ] + γmax j[ζ+1]∈J Q(s[ζ+1],j[ζ+1],μ(s[ζ+1];ω μ );ω Q ); until the time slot is in state [ζ+1], use the data {y[ζ], s[ζ], a[ζ+1]} to calculate and Gradient; according to

[0138] Gradient Pass and To update the weight value; if the remaining energy E of the drone renain[t]≤0, the algorithm enters a new time slot. As a constraint, satisfy This constraint.

[0139] The formula can be Solve, where is the total energy consumption of the jth UAV before the tth time slot, is a constant, indicating the upper limit of energy that the battery can provide within a time slot, which is consistent with the C13 constraint of the previous UAV path planning model.

[0140] Finally, the algorithm enters a new time slot after the previous time slot ends.

[0141] After the PDDQN algorithm model is trained, the trained UAV optimal path determination model is obtained; when actually determining the optimal path of the UAV, the state space of the Markov decision process is constructed according to the order in which the UAV visits each hovering point in the corresponding hovering point cluster; the state space is input into the trained UAV optimal path determination model to calculate the final UAV optimal path.

[0142] The present embodiment discloses a drone path planning for assisting mobile edge computing. The users in the service area are clustered to determine a set of hovering points. The hovering points are then clustered to ensure that the number of clusters divided is the same as that of drones, and that there are hovering points in each cluster, so that all users in the service area can be covered by the communication range of the corresponding drone. Then, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance of the drone traveling through all the hovering points in the corresponding hovering point cluster and returning to the central base station. Finally, according to the determined order, the optimal path of the drone is determined with the goal of maximizing the drone energy efficiency. This ensures that the drone has the highest working efficiency on the basis of having the shortest driving path.

[0143] Example 2

[0144] In this embodiment, a drone path planning system for assisting mobile edge computing is disclosed, including:

[0145] A data acquisition unit, used to acquire the location of each user in the service area;

[0146] A hovering point determination unit, used to cluster all users according to their positions to obtain multiple hovering points;

[0147] A hovering point clustering unit is used to cluster all hovering points to obtain a hovering point cluster for each drone;

[0148] A hovering point sequence determination unit is used to determine, for each drone, the sequence in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the reciprocal of the distance that the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station;

[0149] The drone optimal path determination unit is used to determine the optimal path of each drone according to the order in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the drone energy efficiency.

[0150] The present invention also discloses a computer device, which includes:

[0151] a processor adapted to execute a computer program;

[0152] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, a method for drone path planning for assisted mobile edge computing disclosed in Example 1 is implemented.

[0153] The present invention also discloses a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing a drone path planning method for assisted mobile edge computing disclosed in Example 1.

[0154] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a drone path planning method for assisted mobile edge computing disclosed in Example 1.

[0155] The method disclosed in Example 1 can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for UAV path planning assisted by mobile edge computing, characterized in that: include: Get the location of each user in the service area; All users are clustered according to their locations to obtain multiple hovering points; All hovering points are clustered to obtain the hovering point cluster of each drone; For each drone, the order in which the drone visits each hovering point in the corresponding hovering point cluster is determined with the goal of maximizing the reciprocal of the distance the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station; For each UAV, the optimal path of the UAV is determined according to the order in which the UAV visits the hovering points in the corresponding hovering point cluster, with the goal of maximizing the energy efficiency of the UAV.

2. A method for UAV path planning assisted by mobile edge computing as claimed in claim 1, characterized in that: According to the location of each user, a density clustering algorithm is used to cluster all users and obtain multiple groups of users; Through the AP algorithm, each group of users is clustered again to obtain multiple hovering points.

3. The method for UAV path planning assisted by mobile edge computing according to claim 1, characterized in that: The number of hovering point clusters divided is the same as the number of drones; And there is at least one hovering point in each hovering point cluster.

4. The method for UAV path planning assisted by mobile edge computing according to claim 1, characterized in that: Randomly select the same number of hovering points as the number of drones from all hovering points as the cluster centers of each cluster; The hovering points closest to the cluster center are placed in the same cluster to obtain multiple hovering point clusters; Calculate the class center of each hover point cluster; The error between the class center and cluster center of each hovering point cluster is calculated. When the error is greater than the set value, the cluster center of each cluster is reselected from all hovering points, and the hovering point cluster is re-determined based on the re-selected cluster center until the error between the class center and cluster center of the determined hovering point cluster is less than or equal to the set value, and the final hovering point cluster is obtained.

5. The method for UAV path planning assisted by mobile edge computing according to claim 1, characterized in that: A genetic algorithm is used to determine the order in which the UAV visits each hovering point in the corresponding hovering point cluster.

6. The method for UAV path planning assisted by mobile edge computing according to claim 1, characterized in that: With the goal of maximizing the energy efficiency of the UAV, the Markov decision process is used to determine the optimal path of the UAV.

7. A UAV path planning system assisted by mobile edge computing, characterized in that: include: A data acquisition unit, used to acquire the location of each user in the service area; A hovering point determination unit, used to cluster all users according to their positions to obtain multiple hovering points; A hovering point clustering unit is used to cluster all hovering points to obtain a hovering point cluster for each drone; A hovering point sequence determination unit is used to determine, for each drone, the sequence in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the reciprocal of the distance that the drone travels through all the hovering points in the corresponding hovering point cluster and then returns to the central base station; The drone optimal path determination unit is used to determine the optimal path of each drone according to the order in which the drone visits each hovering point in the corresponding hovering point cluster, with the goal of maximizing the drone energy efficiency.

8. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the method for drone path planning for assisted mobile edge computing according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing a drone path planning method for assisted mobile edge computing as described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the drone path planning method for assisted mobile edge computing as described in any one of claims 1-6.

Images