Unmanned aerial vehicle task unloading and trajectory planning method and system

Abstract

The invention relates to the field of mobile edge computing, in particular to an unmanned aerial vehicle task unloading and trajectory planning method and system. According to the method, a system model, an unmanned aerial vehicle moving model, a task model and an unmanned aerial vehicle energy consumption model of a target area are constructed, and an optimization function and constraint conditions of an unmanned aerial vehicle task unloading and path planning strategy are constructed based on the system model, the unmanned aerial vehicle moving model, the task model and the unmanned aerial vehicle energy consumption model of the target area; describing a process of solving an unmanned aerial vehicle task unloading and path planning strategy as a Markov decision process, and establishing a state space, an action space and a reward function to obtain a deep reinforcement learning framework; and solving the Markov decision process by adopting a dual deep Q network to obtain an optimal unmanned aerial vehicle task unloading and path planning strategy meeting constraint conditions. According to the invention, the task completion rate of the system is obviously improved, and the stability and effectiveness of the unmanned aerial vehicle in a complex environment are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing, and in particular to a method and system for unloading unmanned aerial vehicle tasks and trajectory planning. Background Art

[0002] In recent years, with the popularization of mobile edge computing (MEC) technology, unmanned aerial vehicles (UAVs) have gradually become a research hotspot as a flexible computing resource provider. In mobile edge computing scenarios, drones can be used as mobile edge servers to provide computing services for local devices on the ground. For example, in some remote areas or temporary activity venues where the ground infrastructure is not perfect, drones can quickly fly to designated locations, build temporary edge computing environments, and assist ground devices in data processing, thereby expanding the coverage of mobile edge computing and enhancing the flexibility and adaptability of the system.

[0003] At present, in the existing UAV-assisted mobile edge computing system task offloading and trajectory planning, for task offloading, factors such as data volume, transmission rate, and server processing capacity are usually considered. According to the computing resources of the UAV and the ground base station, the communication link status and other information, a mathematical model is established to decide the location of task offloading, which can consider the allocation of all tasks and computing resources as a whole. For the situation where the task relationship is relatively simple and the computing resources are relatively sufficient, it can achieve better results. For trajectory planning, most of them focus on finding the shortest path, or planning based on static strategies; among them, the method of finding the shortest path gradually determines the shortest path by continuously updating the distance value of the node, and the planning based on static strategies usually formulates the task offloading and trajectory planning of the UAV according to pre-set rules and conditions.

[0004] However, there are still some technical problems that need to be solved for the application of UAVs in task assistance. First, for task offloading, the existing research on task offloading focuses on independent tasks, ignoring the complex dependencies between many tasks in actual scenarios, which may lead to unreasonable scheduling. For example, some dependent predecessor tasks are not completed before the subsequent tasks are executed, resulting in low efficiency of task execution, increasing the delay of overall task processing, and failing to meet the needs of some application scenarios with high real-time requirements. In addition, for the energy consumption modeling of UAVs, although the existing methods take into account some energy consumption factors, the energy consumption modeling is relatively simple and fails to fully reflect the dynamic changes of energy consumption of UAVs during flight, task offloading and data transmission. Such assumptions may be effective in experimental environments, but in real applications, they are prone to decision-making errors, making it impossible for UAVs to complete the scheduled tasks or use energy efficiently. In addition, for trajectory planning problems, the existing methods focus on finding the shortest path or planning based on static strategies, and do not fully combine the task offloading requirements with energy consumption constraints, resulting in the failure of UAVs to achieve a comprehensive balance between tasks and energy during movement. Especially in complex task scenarios, this strategy is difficult to adapt to the dynamically changing environmental requirements, affecting the overall performance of UAVs. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defects of the existing methods that ignore the dependencies between tasks in actual scenarios, energy consumption modeling fails to dynamically reflect energy consumption changes, and trajectory planning does not fully combine task offloading requirements and energy consumption constraints, resulting in a decrease in the scheduling efficiency of UAV computing resources.

[0006] In order to solve the above technical problems, the present invention provides a method for unloading tasks and planning trajectories of unmanned aerial vehicles, comprising the following steps:

[0007] The target area is modeled as an M×M hexagonal grid, and each area is numbered in turn according to the order in which the drone passes through each area, to obtain the system model of the target area; where M is the set number of rows and columns;

[0008] Based on the system model of the target area, a UAV movement model is constructed, wherein the UAV movement model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables;

[0009] Constructing a task model, the task model comprising:

[0010] Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained;

[0011] Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained;

[0012] Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained;

[0013] Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained;

[0014] Based on the unloading decision variables of each task node in the current area and the amount of data calculated by each task node in the current area, the amount of data for auxiliary calculation by the UAV in the current area is obtained;

[0015] Constructing a UAV energy consumption model, the UAV energy consumption model includes:

[0016] Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area;

[0017] Based on the total energy consumption of the drone in the previous area and the energy when the drone entered the previous area,

[0018] Get the energy of the drone when it enters the current area;

[0019] Based on the system model of the target area, the UAV movement model, the task model and the UAV energy consumption model, the optimization function and constraint conditions of the UAV task offloading and path planning strategy are constructed, and the process of solving the UAV task offloading and path planning strategy is described as a Markov decision process. The state space, action space and reward function are established to obtain a deep reinforcement learning framework.

[0020] A dual deep Q-network is used to solve the Markov decision process and obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

[0021] Preferably, the position of the drone in each area includes: the area numbered N for the last time the drone entered during flight, the area number set of the target area is {1, 2, ..., n, ..., N}, then the starting position q of the drone is s The coordinates are (x1, y1, 0), and the end point position is q f The coordinates of (x N ,y N ,0), the position q[n] in the nth region is (x n ,y n ,H); where x n is the horizontal coordinate of the center point of the nth region, y n is the ordinate of the center point of the nth region, and H is the flying height of the UAV.

[0022] Preferably, constructing a directed acyclic graph of the current area includes:

[0023] Take each task in the current area as a task node. Each task node includes the amount of data that needs to be transmitted from the local device to the drone and the amount of data during calculation, and build a set of task nodes in the current area.

[0024] If the i-th task node in the current area is calculated, the amount of data generated by the j-th task node is required ji , then construct a directed edge E from the jth task node to the ith task node ji (d ji ), based on all directed edges in the current region, build a directed edge set of the current region;

[0025] Based on the task node set and directed edge set of the current area, a directed acyclic graph of the current area is constructed.

[0026] Preferably, after the UAV enters the nth area, according to the regional decision variable of the nth area Determines whether to hover to assist in calculating tasks in the current area. When it is 0, the drone jumps to the adjacent area. When it is 1, the drone hovers to assist in calculating the tasks in the current area;

[0027] When the unloading decision variable S of the i-th task node in the n-th region n,i When it is 0, the i-th task node in the n-th region is offloaded to the local device. When the offloading decision variable S of the i-th task node in the n-th region is n,i = 1, the i-th task node in the n-th area is unloaded to the drone, S n,i ∈{0,1}.

[0028] Preferably, the time consumed by executing each task node in the current area is calculated as follows:

[0029]

[0030] in, The time consumed to execute the i-th task node in the n-th region, is the time for the i-th task node in the n-th region to be calculated on the local device, c n,i The amount of data when calculating the i-th task node in the n-th region, is the time for the i-th task node in the n-th area to be calculated by the drone, f uav The computing power of the drone, is the transmission time of the i-th task node in the n-th region from the local device to the drone, w n,i is the amount of data that needs to be transferred from the local device to the UAV when the i-th task node in the n-th region is unloaded, R trans The rate at which the drone receives and sends data in the current area;

[0031] The calculation formula for the execution start time of each task node in the current area is:

[0032]

[0033] in, is the execution start time of the ith task node in the nth region, max(.) is the maximum value, is the set of predecessor tasks of the ith task node in the nth region, is the completion time of the jth task node in the nth region, is the transmission time caused by the jth task node in the nth region for the predecessor task of the ith task node, d ji To calculate the amount of data generated by the jth task node when calculating the i-th task node in the n-th region, S n,j is the offloading decision variable of the jth task node in the nth region, S n,i is the unloading decision variable of the i-th task node in the n-th region;

[0034] The calculation formula for the completion time of each task node in the current area is:

[0035]

[0036] in, is the completion time of the i-th task node in the n-th area;

[0037] The time calculation formula to complete the directed acyclic graph of the current area is:

[0038]

[0039] Among them, t total [n] is the time to complete the directed acyclic graph of the nth region, is the completion time of the L[n]th task node in the nth region, L[n] is the total number of task nodes in the nth region, is the execution start time of the first task node in the nth region;

[0040] The calculation formula for the amount of data assisted by the drone in the current area is:

[0041] C[n]=∑ i∈L[n] S n,i c n,i ,

[0042] Among them, C[n] is the amount of data assisted by the UAV in the nth area.

[0043] Preferably, the total energy consumption of the drone in the current area is calculated as:

[0044]

[0045] in, is the total energy consumption of the UAV in the nth area, is the regional decision variable of the nth region, is the energy consumption of the drone hovering in the nth area, Calculate the energy consumption of the directed acyclic graph of the nth region for the drone, The flight energy consumption of the drone flying from the previous area to the next area;

[0046] The formula for the energy of the drone when it enters the current area is:

[0047]

[0048] Among them, E[n] is the energy of the drone when it enters the nth area, and E[n-1] is the energy of the drone when it enters the n-1th area. is the total energy consumption of the UAV in the n-1th area;

[0049] The energy consumption of the drone hovering in the nth area for:

[0050]

[0051] Among them, p stay The power of the drone hovering;

[0052] The drone calculates the energy consumption of the directed acyclic graph of the nth region for:

[0053]

[0054] in, The energy consumption of the UAV processing the i-th task node in the n-th area, p exec Calculate the power of the mission for the drone, is the energy consumption of the UAV receiving the transmission of the i-th task node in the n-th area, p rece The power transmitted to the drone is received, The consumption of drones caused by the j-th task node in the n-th area being the predecessor task of the i-th task node

[0055] p send The power transmitted to the drone;

[0056] The flight energy consumption of the drone flying from the previous area to the next area for:

[0057]

[0058]

[0059] Among them, p fly is the flight power of the UAV, t fly is the flight time of the UAV between two adjacent areas, P0 is the power of the UAV blade, P1 is the induced power of the UAV hovering, v0 is the average speed of the UAV rotor, U tip is the tip speed of the drone blade, d0 is the drag ratio of the drone fuselage, s is the solidity of the drone rotor, A is the rotor area of ​​the drone, ρ is the air density, and v fly is the flight speed of the drone.

[0060] Preferably, the optimization function and constraint conditions for constructing the drone task offloading and path planning strategy include:

[0061] The optimization function of the UAV task offloading and path planning strategy is:

[0062]

[0063] The constraints include:

[0064]

[0065] Among them, α1 is the first weight coefficient. If the drone finally reaches the end position, α1 is 1. If the drone does not reach the end position, α1 is 0. is the reward for the drone to finally reach the destination, α is the second weight coefficient, R p As a punishment for drones crossing the border, R p is a negative value. If the drone exceeds the regional boundary during flight, α2 is 1. If the drone does not exceed the regional boundary during flight, α2 is 0. N is the number of the area that the drone entered last time during flight. α3 is the third weight coefficient. R t [n] is the reward for the drone to calculate the unloading task in the nth area, R t [n] = ξ × C[n], ξ is the first coefficient, R c [n] is the reward calculated by the drone in the nth auxiliary area, q U [1] is the first area the drone enters, q S is the starting position of the UAV, q U [N] is the location of the last area entered by the drone, N is the number of the area the drone entered last during flight, and q F is the final position of the UAV, and E[1] is the initial energy of the UAV in the first area.

[0066] Preferably, the process of solving the drone task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain a deep reinforcement learning framework, including:

[0067] The state space includes: the coordinate representation of the current position of the drone in the hexagonal grid, the energy and the characteristics of the directed acyclic graph task in the current area Among them, the characteristics of the directed acyclic graph task in the current area in, is the normalized adjacency matrix of the directed acyclic graph of the current region, is the normalized task node feature matrix of the directed acyclic graph of the current region, and I is the unit matrix;

[0068] The action space includes: hovering in the current area, moving to the upper adjacent area of ​​the current area, moving to the upper right adjacent area of ​​the current area, moving to the lower right adjacent area of ​​the current area, moving to the lower adjacent area of ​​the current area, moving to the lower left adjacent area of ​​the current area, and moving to the upper left adjacent area of ​​the current area;

[0069] The reward function includes: the reward for the drone to finally reach the destination Penalty for Drones Crossing Borders p , negative reward for the drone consuming energy in the nth area The reward R calculated by the drone in the nth region auxiliary area c [n], the drone calculates the reward R of the unloading task in the nth area t [n] = ξ × C[n], the drone distance change reward R d =η×Δ d ;

[0070] Among them, R t [n], R c The coefficients of [n] include E[n] / E[1], R d The coefficients include (E[1]-E[n]) / E[1], ξ is the first coefficient, ∈ is the second coefficient, η is the third coefficient, Δ d is the change in the distance between the UAV and the end point.

[0071] Preferably, the use of a dual deep Q network to solve the Markov decision process includes:

[0072] Step S601: Initialize the parameters θ of the Q network and the parameters θ of the target network ′ =θ, noise network parameters, capacity of experience replay pool I, discount factor γ∈[0,1), priority strength parameter α, weight correction parameter β, constant ∈, preset batch size m, number of training steps T, current training times k, maximum training times K, initial state s0 of the drone;

[0073] Step S602: Initialize the time step index t to 1;

[0074] Step S603: Set the current state of the drone t Input the Q network to obtain the Q values ​​corresponding to all actions, select the action with the largest Q value as the preliminary action, use the noise network to generate noise, and obtain the execution action a based on the preliminary action and noise. t ;

[0075] Step S604: The drone performs the selected action a t , according to the system model of the target area, the UAV movement model, the mission model, the UAV energy consumption model and the state transition probability, the state transition is performed to obtain the reward function value r t and the next state of the drone t+1 , the generated experience pool samples (s t ,a t ,r t ,s t+1 ) is stored in the experience pool;

[0076] Step S605: determine whether the number of samples in the experience pool is less than I. If it is less than I, then t=t+1, and return to execute step S603 until the number of samples in the experience pool is greater than or equal to I;

[0077] Step S606: When the number of experience pool samples is greater than or equal to I, the TD error of each experience pool sample is calculated based on the reward function value of each experience pool sample, the discount factor γ, the Q value corresponding to the action, and the maximum Q value of all actions obtained through the target network in the next state of the drone;

[0078] Step S607: Based on the TD error, priority strength parameter α, and constant ∈ of each experience pool sample, the priority of each experience pool sample is calculated, and according to the priority of each experience pool sample, experience pool samples of a preset batch size m are taken out, and the sampling frequency of each experience pool sample taken out is calculated;

[0079] Step S608: Calculate the importance sampling weight of each sample taken out from the experience pool based on the sampling frequency, weight correction parameter β, and the capacity I of the experience playback pool; and correct the TD error of each sample in the experience pool according to the importance sampling weight;

[0080] Step S609: Based on the corrected TD error of each experience pool sample, the parameter θ of the Q network is updated using the gradient descent method;

[0081] Step S610: Update the parameters θ of the target network every T steps ′ ;

[0082] Step S611: Determine whether k is less than K. If so, set k=k+1 and return to step S602. If not, obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

[0083] The present invention also provides a UAV task offloading and trajectory planning system, which is characterized by comprising:

[0084] The system model building module is used to model the target area as an M×M hexagonal grid, and number each area in turn according to the order in which the drone passes through each area to obtain the system model of the target area; where M is the set number of rows and columns;

[0085] A UAV mobility model building module is used to build a UAV mobility model based on the system model of the target area. The UAV mobility model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables;

[0086] The task model building module is used to build a task model, and the task model includes:

[0087] Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained;

[0088] Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained;

[0089] Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained;

[0090] Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained;

[0091] Based on the unloading decision variables of each task node in the current area and the amount of data calculated by each task node in the current area, the amount of data assisted by the drone in the current area is obtained;

[0092] The UAV energy consumption model building module is used to build the UAV energy consumption model. The UAV energy consumption model includes:

[0093] Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area;

[0094] Based on the total energy consumption of the drone in the previous area and the energy when the drone entered the previous area,

[0095] Get the energy of the drone when it enters the current area;

[0096] The deep reinforcement learning framework building module is used to build the optimization function and constraint conditions of the drone task offloading and path planning strategy based on the system model of the target area, the drone movement model, the task model and the drone energy consumption model. The process of solving the drone task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain the deep reinforcement learning framework.

[0097] The solution module is used to solve the Markov decision process using a dual deep Q network to obtain the optimal UAV task offloading and path planning strategy that meets the constraints. The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0098] The method and system for unloading and trajectory planning of unmanned aerial vehicle (UAV) tasks described in the present invention model the target area as an M×M area, and construct a directed acyclic graph task model for each area, wherein each task node covers the information of the amount of data required to be transmitted from the local device to the UAV and the amount of data during calculation, and each directed edge contains the amount of data required to be transmitted by two task nodes with a dependency relationship, thereby fully describing the task nodes and their relationships, accurately depicting the sequence and data flow between the task nodes, and realizing accurate expression of complex task topology, effectively avoiding scheduling confusion caused by improper handling of task dependencies, and improving the scheduling efficiency of UAV computing resources.

[0099] By building an accurate energy consumption model, we comprehensively and meticulously analyze the energy consumption factors of drones in various links such as flight, task offloading, and data transmission, quantify the energy consumption of drones under different operations, optimize the energy allocation strategy, ensure the task completion rate and extend the operation time of drones.

[0100] The present invention unifies the decision-making offloading scheme and path planning through deep reinforcement learning, and can dynamically adjust the behavior of the UAV in a complex environment, thereby improving the system's ability to complete tasks under energy-constrained conditions. Specifically, the present invention comprehensively considers the delay requirements of the task and the remaining energy of the UAV through deep reinforcement learning, and optimizes the offloading decision and the flight path at the same time. Under the premise of meeting the task delay constraints, it reasonably allocates energy for task processing and movement, ensuring that the UAV can complete more auxiliary computing tasks and successfully reach the target location. Compared with the prior art, the present invention avoids the inefficiency caused by the separation of offloading decisions and path planning in traditional methods, and overcomes the defects that uneven energy distribution may lead to mission failure or energy exhaustion in the middle. Through this comprehensive optimization strategy, the present invention significantly improves the system's task completion rate and ensures the stability and effectiveness of the UAV in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0102] Figure 1 It is a flow chart of a method for unloading missions and planning trajectories of a UAV according to the present invention.

[0103] Figure 2It is a flowchart of solving the Markov decision process using a dual deep Q network. DETAILED DESCRIPTION

[0104] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0105] like Figure 1 As shown, Figure 1 The present invention is a flowchart of a method for unloading tasks and planning trajectories of unmanned aerial vehicles.

[0106] This embodiment provides a method for unloading UAV tasks and planning trajectories, including the following steps:

[0107] Step S1: Model the target area as an M×M hexagonal grid, number each area in turn according to the order in which the drone passes through each area, and obtain a system model of the target area; where M is the set number of rows and columns;

[0108] There are different numbers of users distributed in each area. At the same time, the drone can only assist in calculating one hexagonal area, and jump to the adjacent area after the calculation is completed; each area is numbered in sequence according to the order in which the drone passes through each area, n represents the nth area passed by the drone, and the area where the drone starts is recorded as 1.

[0109] Step S2: Based on the system model of the target area, a UAV movement model is constructed, wherein the UAV movement model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables;

[0110] In this embodiment, preferably, the position of the drone in each area includes: the area numbered N for the last time the drone entered during the flight, the area number set of the target area is {1, 2, ..., n, ..., N}, then the starting position q of the drone is s The coordinates are (x1, y1, 0), and the end point position is q f The coordinates of (x N ,y N ,0), the position q[n] in the nth region is (x n ,y n ,H); where x n is the horizontal coordinate of the center point of the nth region, y n is the ordinate of the center point of the nth region, and H is the flying height of the UAV.

[0111] In this embodiment, specifically, it is assumed that the drone moves at a uniform speed. Since each area is modeled as a hexagon, the distance of each jump of the drone is fixed. The flight distance between two adjacent areas of the drone is D, and the flight speed of the drone is v. fly , the flight time of the drone between two adjacent areas is t fly .

[0112] Step S3: constructing a task model, wherein the task model includes:

[0113] Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained;

[0114] In this embodiment, specifically, the calculation formula for the time consumed by executing each task node in the current area is:

[0115]

[0116] in, The time consumed to execute the i-th task node in the n-th region, is the time for the i-th task node in the n-th region to be calculated on the local device, c n,i The amount of data when calculating the i-th task node in the n-th region, is the time for the i-th task node in the n-th area to be calculated by the drone, f uav The computing power of the drone, is the transmission time of the i-th task node in the n-th region from the local device to the drone, w n,i is the amount of data that needs to be transferred from the local device to the UAV when the i-th task node in the n-th region is unloaded, R trans The rate at which the drone receives and sends data in the current area;

[0117] Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained;

[0118] In this embodiment, specifically, the calculation formula for the execution start time of each task node in the current area is:

[0119]

[0120] in, is the execution start time of the ith task node in the nth region, max(.) is the maximum value, is the set of predecessor tasks of the ith task node in the nth region, is the completion time of the jth task node in the nth region, is the transmission time caused by the jth task node in the nth region for the predecessor task of the ith task node, d ji To calculate the amount of data generated by the jth task node when calculating the i-th task node in the n-th region, S n,j is the offloading decision variable of the jth task node in the nth region, S n,i is the unloading decision variable of the i-th task node in the n-th region;

[0121] Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained;

[0122] In this embodiment, specifically, the calculation formula for the completion time of each task node in the current area is:

[0123]

[0124] in, is the completion time of the i-th task node in the n-th area;

[0125] Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained;

[0126] In this embodiment, specifically, the time calculation formula for completing the directed acyclic graph of the current area is:

[0127]

[0128] Among them, t total [n] is the time to complete the directed acyclic graph of the nth region, is the completion time of the L[n]th task node in the nth region, L[n] is the total number of task nodes in the nth region, is the execution start time of the first task node in the nth region;

[0129] Based on the unloading decision variables of each task node in the current area and the amount of data calculated by each task node in the current area, the amount of data assisted by the drone in the current area is obtained;

[0130] In this embodiment, specifically, the calculation formula for the amount of data assisted by the drone in the current area is:

[0131] C[n]=∑ i∈L[n] S n,i c n,i ,

[0132] Among them, C[n] is the amount of data assisted by the UAV in the nth area.

[0133] In this embodiment, preferably, constructing a directed acyclic graph (DAG) of the current area includes:

[0134] Take each task in the current area as a task node. Each task node includes the amount of data that needs to be transmitted from the local device to the drone and the amount of data during calculation, and build a set of task nodes in the current area.

[0135] Among them, the task node set of the nth area is:

[0136]

[0137] Among them, V j (w j ,c j ) is the jth task node in the nth region, V i (w i ,c i ) is the i-th task node in the n-th region, is the Lth region of the nth region [n] task nodes, w i is the amount of data that needs to be transferred from the local device to the drone when the i-th task node in the n-th region is unloaded, c i The amount of data to be calculated for the i-th task node in the n-th region.

[0138] If the i-th task node in the current area is calculated, the amount of data generated by the j-th task node is required ji , then construct a directed edge E from the jth task node to the ith task node ji (d ji ), based on all directed edges in the current region, build a directed edge set of the current region;

[0139] Based on the task node set and directed edge set of the current area, a directed acyclic graph of the current area is constructed.

[0140] Step S4: constructing a drone energy consumption model, wherein the drone energy consumption model includes:

[0141] Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area;

[0142] In this embodiment, specifically, the total energy consumption of the drone in the current area is:

[0143]

[0144] in, is the total energy consumption of the UAV in the nth area, is the regional decision variable of the nth region, is the energy consumption of the drone hovering in the nth area, Calculate the energy consumption of the directed acyclic graph of the nth region for the drone, The flight energy consumption of the drone flying from the previous area to the next area;

[0145] Based on the total energy consumption of the drone in the previous area and the energy of the drone when it entered the previous area, the energy of the drone when it enters the current area is obtained;

[0146] In this embodiment, specifically, the formula for the energy when the drone enters the current area is:

[0147]

[0148] Among them, E[n] is the energy of the drone when it enters the nth area, and E[n-1] is the energy of the drone when it enters the n-1th area. is the total energy consumption of the UAV in the n-1th area;

[0149] The drone calculates the energy consumption of the directed acyclic graph of the nth region for:

[0150]

[0151] in, The energy consumption of the UAV processing the i-th task node in the n-th area, p exec Calculate the power of the mission for the drone, is the energy consumption of the UAV receiving the transmission of the i-th task node in the n-th area, p rece The power transmitted to the drone is received, The consumption of drones caused by the j-th task node in the n-th area being the predecessor task of the i-th task node

[0152] p send The power transmitted to the drone;

[0153] The flight energy consumption of the drone flying from the previous area to the next area for:

[0154]

[0155] Among them, p fly is the flight power of the UAV, t fly is the flight time of the UAV between two adjacent areas, P0 is the power of the UAV blade, P1 is the induced power of the UAV hovering, v0 is the average speed of the UAV rotor, U tip is the tip speed of the drone blade, d0 is the drag ratio of the drone fuselage, s is the solidity of the drone rotor, A is the rotor area of ​​the drone, ρ is the air density, and v fly is the flight speed of the drone.

[0156] In this embodiment, specifically, after the drone enters the nth area, according to the regional decision variable of the nth area, Determines whether to hover to assist in calculating tasks in the current area. When it is 0, the drone jumps to the adjacent area. When it is 1, the drone hovers to assist in calculating the tasks in the current area;

[0157] When the unloading decision variable S of the i-th task node in the n-th region n,i When it is 0, the i-th task node in the n-th region is offloaded to the local device. When the offloading decision variable S of the i-th task node in the n-th region is n,i = 1, the i-th task node in the n-th area is unloaded to the drone, S n,i ∈{0,1}.

[0158] Step S5: Based on the system model of the target area, the UAV movement model, the task model and the UAV energy consumption model, the optimization function and constraint conditions of the UAV task offloading and path planning strategy are constructed, and the process of solving the UAV task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain a deep reinforcement learning framework;

[0159] In this embodiment, the drone uses limited energy to assist in calculations. Every time the drone arrives at an area, it starts making decisions. Only when the drone decides to stay in the current area will it perform auxiliary calculations on the current area. Once the drone's energy is exhausted, this process optimization problem is stopped.

[0160] In this embodiment, preferably, the optimization function and constraint conditions for constructing the drone task offloading and path planning strategy include:

[0161] The optimization function of the UAV task offloading and path planning strategy is:

[0162]

[0163] The constraints include:

[0164]

[0165] Among them, α1 is the first weight coefficient. If the drone finally reaches the end position, α1 is 1. If the drone does not reach the end position, α1 is 0. is the reward for the drone to finally reach the destination, α is the second weight coefficient, R p As a punishment for drones crossing the border, R p is a negative value. If the drone exceeds the regional boundary during flight, α2 is 1. If the drone does not exceed the regional boundary during flight, α2 is 0. N is the number of the area that the drone entered last time during flight. α3 is the third weight coefficient. R t [n] is the reward for the drone to calculate the unloading task in the nth area, R t [n] = ξ × C[n], ξ is the first coefficient, R c [n] is the reward calculated by the drone in the nth auxiliary area, q U [1] is the first area the drone enters, q S is the starting position of the UAV, q U [N] is the location of the last area entered by the drone, N is the number of the area the drone entered last during flight, and q F is the final position of the UAV, and E[1] is the initial energy of the UAV in the first area.

[0166] In reinforcement learning, the agent continuously learns how to choose actions to maximize the cumulative reward by interacting with the environment. A reinforcement learning problem is usually described by a Markov decision process (MDP), which usually consists of the following five tuples:

[0167] S: state space, representing the set of all possible states

[0168] A: Action space, representing the set of all possible actions

[0169] P(s ′ |s,a): state transition probability, indicating that after taking action a in state s, the state is transferred to state s. ′ Probability

[0170] R(s,a): Reward function, which represents the reward obtained after taking action a in state s

[0171] γ∈[0,1): Discount factor used to weigh the importance of current rewards and future rewards.

[0172] like Figure 2 As shown, Figure 2 Flowchart for solving the Markov decision process using a dual deep Q network. In this embodiment, the drone is regarded as an agent interacting with the environment, so that the agent makes decisions on drone trajectory planning and task offloading. Every time the drone enters an area, it makes a decision based on the current environmental state.

[0173] In this embodiment, specifically, the process of solving the drone task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain a deep reinforcement learning framework, including:

[0174] The state space includes: the coordinate representation of the current position of the drone in the hexagonal grid, the energy and the characteristics of the directed acyclic graph task in the current area Among them, the characteristics of the directed acyclic graph task in the current area in, is the normalized adjacency matrix of the directed acyclic graph of the current region, is the normalized task node feature matrix of the directed acyclic graph of the current region, and I is the unit matrix;

[0175] The coordinates of the current position of the drone in the hexagonal grid are represented as a two-dimensional vector. For example, (0, 0) represents the grid area in the upper left corner of the hexagonal grid. In order to address the problem of different matrix dimensions caused by different numbers of directed acyclic graph (DAG) nodes, this embodiment sets the maximum number of nodes, and low-dimensional matrices are padded with zeros.

[0176] The action space includes: hovering in the current area, moving to the upper adjacent area of ​​the current area, moving to the upper right adjacent area of ​​the current area, moving to the lower right adjacent area of ​​the current area, moving to the lower adjacent area of ​​the current area, moving to the lower left adjacent area of ​​the current area, and moving to the upper left adjacent area of ​​the current area;

[0177] The reward function includes: the reward for the drone to finally reach the destination Penalty for Drones Crossing Borders p , negative reward for the drone consuming energy in the nth area The reward R calculated by the drone in the nth region auxiliary area c[n], the drone calculates the reward R of the unloading task in the nth area t [n] = ξ × C[n], the drone distance change reward R d =η×Δ d ;

[0178] In order to make the drone assist in calculation as much as possible when it has sufficient energy and move closer to the end point when it has insufficient energy, a coefficient setting is added when designing the reward function, where R t [n], R c The coefficients of [n] include E[n] / E[1], R d The coefficients include (E[1]-E[n]) / E[1], ξ is the first coefficient, ∈ is the second coefficient, μ is the third coefficient, Δ d is the change in the distance between the UAV and the end point.

[0179] The reward R calculated by the drone in the nth region auxiliary region c [n], specifically: if the calculation of the nth area is successfully assisted, a fixed positive reward is returned; otherwise, if no time is saved, indicating an inappropriate decision, a fixed negative reward is returned;

[0180] Step S6: Use a dual deep Q network to solve the Markov decision process and obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

[0181] In the task offloading and trajectory planning decision-making, the drone perceives the grid environment in real time, generates the optimal drone task offloading and path planning strategy that meets the constraints based on the deep reinforcement learning framework, and determines the next movement direction and the calculation method of the task node (local calculation or offloading to the drone). In the action execution stage, the drone moves according to the decision results and completes the corresponding task, while updating the status information in real time, including the position, remaining energy, and remaining task information in the environment. These feedbacks are used as inputs to the reinforcement learning algorithm. In network training, the parameters of the policy network are continuously optimized through priority experience sampling and loss calculation, and the quality of decision-making is gradually improved. In addition, by setting the penalty for drone crossing the boundary, the reward for the drone finally reaching the end position, and the reward for the drone distance change, it is ensured that the drone can efficiently complete the auxiliary calculation and reach the target position.

[0182] The standard deep Q network uses the same network for action selection and Q value estimation, which is prone to overestimation bias. This is because each time the max operation is estimated, the largest Q value in the current estimate is selected. However, since the network's prediction is erroneous, these errors may sometimes cause the Q value of non-optimal actions to be overestimated. Therefore, the present invention adopts a double deep Q network (DoubleDQN), which introduces a new neural network as the target network relative to the deep Q network, as follows:

[0183] Preferably, the use of a dual deep Q network to solve the Markov decision process includes:

[0184] Step S601: Initialize the parameters θ of the Q network and the parameters θ of the target network ′ =θ, noise network parameters, capacity of experience replay pool I, discount factor γ∈[0,1), priority strength parameter α, weight correction parameter β, constant ∈, preset batch size m, number of training steps T, current training times k, maximum training times K, initial state s0 of the drone;

[0185] Step S602: Initialize the time step index t to 1;

[0186] Step S603: Set the current state of the drone t Input the Q network to obtain the Q values ​​corresponding to all actions, select the action with the largest Q value as the preliminary action, use the noise network to generate noise, and obtain the execution action a based on the preliminary action and noise. t ;

[0187] The traditional ∈-greedy strategy controls the degree of exploration through a pre-set exploration value. Due to the random selection of actions, the direction of strategy learning may fluctuate greatly. Especially in some critical states, random exploration may cause the agent to deviate from the learning goal. The controllable noise exploration of the noise network can better balance the stability of learning and the need for exploration. In addition, the noise network introduces noise in the weights of the neural network, allowing the network to dynamically adapt to the exploration behavior in different environments. Add a learnable noise term to the weight w and the bias b:

[0188]

[0189] Among them, w μ is the mean part of weight w, w σ is the standard deviation part of weight w, ⊙ is element-wise multiplication, ∈ w is the noise sample applied to weight w, b μ is the mean part of the bias b, b σ is the standard deviation part of the bias b, ∈ b is the noise sample applied to bias b.

[0190] Step S604: The drone performs the selected action a t , according to the system model of the target area, the UAV movement model, the mission model, the UAV energy consumption model and the state transition probability, the state transition is performed to obtain the reward function value r t and the next state of the drone t+1 , the generated experience pool samples (s t ,at ,r t ,s t+1 ) is stored in the experience pool;

[0191] Step S605: determine whether the number of samples in the experience pool is less than I. If it is less than I, then t=t+1, and return to execute step S603 until the number of samples in the experience pool is greater than or equal to I;

[0192] Step S606: When the number of experience pool samples is greater than or equal to I, the TD error of each experience pool sample is calculated based on the reward function value of each experience pool sample, the discount factor γ, the Q value corresponding to the action, and the maximum Q value of all actions obtained through the target network in the next state of the drone;

[0193] The TD error of each experience pool sample is calculated using the formula:

[0194]

[0195] Among them, δ t is the TD error of the experience pool sample corresponding to time step t, Q(s t ,a t ; θ) is the predicted Q value of the experience pool sample corresponding to time step t, is the target Q value of the experience pool sample corresponding to time step t, The maximum Q value of all actions is obtained through the target network in the next state of the drone.

[0196] Step S607: Based on the TD error, priority strength parameter α, and constant ∈ of each experience pool sample, the priority of each experience pool sample is calculated, and according to the priority of each experience pool sample, experience pool samples of a preset batch size m are taken out, and the sampling frequency of each experience pool sample taken out is calculated;

[0197] The priority p of each experience pool sample t and its TD error δ t Related, the priority p of each experience pool sample t The calculation formula is: t =(δ t +∈) α , the calculation formula for calculating the sampling frequency P(t) for each sample taken out of the experience pool is: P(t) = p t / ∑ e p e , p t is the experience pool sample corresponding to time step t, pe is the experience pool sample corresponding to time step e;

[0198] Step S608: Based on the sampling frequency of each sample taken from the experience pool, the weight correction parameter β, and the capacity I of the experience playback pool, the importance sampling weight of each sample taken from the experience pool is calculated. The importance sampling weight formula is: ω t is the importance sampling weight of the experience pool sample corresponding to time step t; according to the importance sampling weight, the TD error of each experience pool sample is corrected, and the formula is: Loss t =ω t (δ t ) 2 , Loss t is the TD error of the experience pool sample corresponding to the corrected time step t;

[0199] Since priority experience replay is non-uniform sampling, experience pool samples with large TD errors will be sampled more frequently, resulting in excessive amplification of the contribution of these experience pool samples to the Q value update. In order to correct the sampling bias and ensure that the algorithm can still estimate the performance of the strategy unbiasedly, this paper introduces importance sampling correction. The core idea is to adjust the sample weight according to the sampling probability of each experience pool sample, thereby rebalancing the contribution of different samples to learning.

[0200] Step S609: Based on the corrected TD error of each experience pool sample, the parameter θ of the Q network is updated using the gradient descent method; the updating method is: Among them, θ new are the updated parameters of the Q network, Q(s t ,a t ; θ) is the gradient of θ, σ is the learning rate;

[0201] Step S610: Update the parameter θ′ of the target network every T steps; the updating method is: θ′ new ←τ·θ new +(1-τ)·θ′, where, θ′ new is the updated parameter of the target network, and τ is the coefficient that controls the update degree of the target network parameters;

[0202] Step S611: Determine whether k is less than K. If so, set k=k+1 and return to step S602. If not, obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

[0203] In this embodiment, in actual application, based on Example 1, a method for unmanned aerial vehicle task offloading and trajectory planning proposed in accordance with the present invention can be used to construct a deep reinforcement learning model, and a lightweight network structure can be used to adapt to scenarios with limited computing resources. By carrying an environmental perception module (such as a camera and a computing module), the unmanned aerial vehicle can perceive the mission environment in real time. During the actual flight process, the environmental perception module captures mission information (such as task volume and distribution) in real time, combines the energy and location of the unmanned aerial vehicle itself, and inputs the reinforcement learning model to generate a decision. According to the action instructions output by the model, the unmanned aerial vehicle adjusts the flight path and performs task offloading.

[0204] In this paper, a deep reinforcement learning framework ETOU-DQN is proposed for path planning and task offloading optimization of UAVs in energy-constrained scenarios. The framework uses a combination of double deep Q networks (Double DQN), Prioritized Experience Replay (PER) and Noisy Networks to optimize the flight trajectory of the UAV in a reinforcement learning manner, while dynamically deciding to offload computing tasks to the UAV or execute them locally. The framework senses the remaining energy of the UAV and the distribution of computing tasks around it, and plans the optimal flight path and makes the best task offloading decision while ensuring the reasonable energy consumption of the UAV.

[0205] The deep reinforcement learning framework ETOU-DQN (Energy-aware Trajectory and Offloading with DQN) is designed to maximize the amount of tasks and data processing that drones can complete under limited energy conditions. It not only optimizes the energy utilization of drones, but also improves the execution efficiency of complex task scheduling. It is suitable for dynamic task scheduling in drone-assisted edge computing and IoT scenarios. Specifically, at each decision moment, the state of the drone, including the coordinate representation of the drone's current position coordinates in the hexagonal grid, the energy and the characteristics of the directed acyclic graph task in the current area is input into the policy network with a noisy network. In this state, the drone selects an action based on the output of the deep reinforcement learning framework, which can be to move to the surrounding grid or hover in the current grid to perform a directed acyclic graph (DAG) task. When the drone chooses to fly, it can move to an adjacent grid, update its own state and restart the next round of decision-making. If the drone chooses to hover, it will perform the directed acyclic graph (DAG) task submitted by the user within the current grid.

[0206] Specifically, the drone decides whether each node in the directed acyclic graph (DAG) task is executed by the drone or the user locally according to the deep reinforcement learning framework. Through this dynamic task scheduling, the drone can optimize the efficiency of task execution under limited energy. After the task is completed, the environment returns the reward function value based on factors such as task completion and energy consumption, and the drone stores the current experience as experience in the experience replay pool. Prioritized experience replay improves sampling efficiency by introducing a priority mechanism, that is, according to the degree of influence of the experience pool samples on the policy network, different priority weights are assigned, so that the experience with greater influence on the policy is sampled first, thereby accelerating the update of the policy. At the same time, in order to avoid sampling bias, importance sampling correction is used to reweight the experience to ensure the convergence of the algorithm. During the training process, the drone samples experience from the priority experience replay pool to update the network parameters. The target network in the dual deep Q network is periodically updated to calculate a stable target Q value and used to update the parameters θ of the Q network. Through this process, the drone optimizes its flight path and task offloading strategy by continuously learning from experience while performing the task.

[0207] The present invention combines dual DQN, noisy network and priority experience replay, so that the UAV can optimize task scheduling and trajectory planning in a complex directed acyclic graph (DAG) task environment, and effectively balance exploration and utilization. The present invention effectively solves the shortcomings of the prior art in task scheduling, path planning and energy management, has a good task completion rate and destination arrival rate, and enhances the adaptability of the system in complex environments.

[0208] This second embodiment provides a UAV task offloading and trajectory planning system, including:

[0209] The system model building module is used to model the target area as an M×M hexagonal grid, and number each area in turn according to the order in which the drone passes through each area to obtain the system model of the target area; where M is the set number of rows and columns;

[0210] A UAV mobility model building module is used to build a UAV mobility model based on the system model of the target area. The UAV mobility model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables;

[0211] The task model building module is used to build a task model, and the task model includes:

[0212] Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained;

[0213] Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained;

[0214] Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained;

[0215] Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained;

[0216] Based on the unloading decision variables of each task node in the current area and the amount of data calculated by each task node in the current area, the amount of data for auxiliary calculation by the UAV in the current area is obtained;

[0217] The UAV energy consumption model building module is used to build the UAV energy consumption model. The UAV energy consumption model includes:

[0218] Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area;

[0219] Based on the total energy consumption of the drone in the previous area and the energy when the drone entered the previous area,

[0220] Get the energy of the drone when it enters the current area;

[0221] The deep reinforcement learning framework building module is used to build the optimization function and constraint conditions of the drone task offloading and path planning strategy based on the system model of the target area, the drone movement model, the task model and the drone energy consumption model. The process of solving the drone task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain the deep reinforcement learning framework.

[0222] The solution module is used to solve the Markov decision process using a dual deep Q network to obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

[0223] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0224] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0225] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0227] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A method for unloading UAV tasks and trajectory planning, characterized in that: The following steps are involved: The target area is modeled as an M×M hexagonal grid, and each area is numbered in turn according to the order in which the drone passes through each area, to obtain the system model of the target area; where M is the set number of rows and columns; Based on the system model of the target area, a UAV movement model is constructed, wherein the UAV movement model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables; Constructing a task model, the task model comprising: Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained; Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained; Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained; Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained; Based on the unloading decision variables of each task node in the current area and the amount of data when each task node in the current area is calculated, the amount of data for auxiliary calculation of the drone in the current area is obtained; and a drone energy consumption model is constructed, which includes: Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area; Based on the total energy consumption of the drone in the previous area and the energy of the drone when it entered the previous area, the energy of the drone when it enters the current area is obtained; Based on the system model of the target area, the UAV movement model, the task model and the UAV energy consumption model, the optimization function and constraint conditions of the UAV task offloading and path planning strategy are constructed, and the process of solving the UAV task offloading and path planning strategy is described as a Markov decision process. The state space, action space and reward function are established to obtain a deep reinforcement learning framework. A dual deep Q-network is used to solve the Markov decision process and obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

2. A method for unloading UAV tasks and trajectory planning according to claim 1, characterized in that: The position of the drone in each area includes: the area numbered N for the last time the drone entered during flight, and the area number set of the target area is {1,2,…,n,…,N}, then the starting position of the drone is q s The coordinates are (x1, y1, 0), and the end point position is q f The coordinates of (x N ,y N ,0), the position q[n] in the nth region is (x n ,y n ,H); where x n is the horizontal coordinate of the center point of the nth region, y n is the ordinate of the center point of the nth region, and H is the flying height of the UAV.

3. The method for unloading and trajectory planning of a UAV according to claim 1, characterized in that: Construct a directed acyclic graph of the current region, including: Take each task in the current area as a task node. Each task node includes the amount of data that needs to be transmitted from the local device to the drone and the amount of data during calculation, and build a set of task nodes in the current area. If the i-th task node in the current area is calculated, the amount of data generated by the j-th task node is required ji , then construct a directed edge E from the jth task node to the ith task node ji (d ji ), based on all directed edges in the current region, build a directed edge set of the current region; Based on the task node set and directed edge set of the current area, a directed acyclic graph of the current area is constructed.

4. The method for unloading UAV tasks and planning trajectories according to claim 1, characterized in that: After the drone enters the nth area, according to the regional decision variables of the nth area Determines whether to hover to assist in calculating tasks in the current area. When it is 0, the drone jumps to the adjacent area. When it is 1, the drone hovers to assist in calculating the tasks in the current area; When the unloading decision variable S of the i-th task node in the n-th region n,i When it is 0, the i-th task node in the n-th region is offloaded to the local device. When the offloading decision variable S of the i-th task node in the n-th region is n,i = 1, the i-th task node in the n-th area is unloaded to the drone, S n,i ∈{0,1}.

5. A method for unloading UAV tasks and planning trajectories according to claim 4, characterized in that: The calculation formula for the time consumed in executing each task node in the current area is: in, The time consumed to execute the i-th task node in the n-th region, is the time for the i-th task node in the n-th region to be calculated on the local device, c n,i The amount of data when calculating the i-th task node in the n-th region, is the time for the i-th task node in the n-th area to be calculated by the drone, f uav The computing power of the drone, is the transmission time of the i-th task node in the n-th region from the local device to the drone, w n,i is the amount of data that needs to be transferred from the local device to the UAV when the i-th task node in the n-th region is unloaded, R trans The rate at which the drone receives and sends data in the current area; The calculation formula for the execution start time of each task node in the current area is: in, is the execution start time of the ith task node in the nth region, max(.) is the maximum value, is the set of predecessor tasks of the ith task node in the nth region, is the completion time of the jth task node in the nth region, is the transmission time caused by the jth task node in the nth region for the predecessor task of the ith task node, d ji To calculate the amount of data generated by the jth task node when calculating the i-th task node in the n-th region, S n,j is the offloading decision variable of the jth task node in the nth region, S n,i is the unloading decision variable of the i-th task node in the n-th region; The calculation formula for the completion time of each task node in the current area is: in, is the completion time of the i-th task node in the n-th area; The time calculation formula to complete the directed acyclic graph of the current area is: Among them, t total [n] is the time to complete the directed acyclic graph of the nth region, is the completion time of the L[n]th task node in the nth region, L[n] is the total number of task nodes in the nth region, is the execution start time of the first task node in the nth region; The calculation formula for the amount of data assisted by the drone in the current area is: C[n]=∑ i∈L[n] S n,i c n,i , Among them, C[n] is the amount of data assisted by the UAV in the nth area.

6. A method for unloading UAV tasks and planning trajectories according to claim 5, characterized in that: The formula for the total energy consumption of the drone in the current area is: in, is the total energy consumption of the UAV in the nth area, is the regional decision variable of the nth region, is the energy consumption of the drone hovering in the nth area, Calculate the energy consumption of the directed acyclic graph of the nth region for the drone, The flight energy consumption of the drone flying from the previous area to the next area; The formula for the energy of the drone when it enters the current area is: Among them, E[n] is the energy of the drone when it enters the nth area, and E[n-1] is the energy of the drone when it enters the n-1th area. is the total energy consumption of the UAV in the n-1th area; The energy consumption of the drone hovering in the nth area for: Among them, p stay The power of the drone hovering; The drone calculates the energy consumption of the directed acyclic graph of the nth region for: in, The energy consumption of the UAV processing the i-th task node in the n-th area, p exec Calculate the power of the mission for the drone, is the energy consumption of the UAV receiving the transmission of the i-th task node in the n-th area, p rece The power transmitted to the drone is received, The consumption of drones caused by the j-th task node in the n-th area being the predecessor task of the i-th task node p send The power transmitted to the drone; The flight energy consumption of the drone flying from the previous area to the next area for: Among them, p fly is the flight power of the UAV, t fly is the flight time of the UAV between two adjacent areas, P0 is the power of the UAV blade, P1 is the induced power of the UAV hovering, v0 is the average speed of the UAV rotor, U tip is the tip speed of the drone blade, d0 is the drag ratio of the drone fuselage, s is the solidity of the drone rotor, A is the rotor area of ​​the drone, ρ is the air density, and v fly is the flight speed of the drone.

7. A method for unloading UAV tasks and planning trajectories according to claim 6, characterized in that: The optimization function and constraint conditions for constructing the UAV task offloading and path planning strategy include: the optimization function of the UAV task offloading and path planning strategy is: The constraints include: Among them, α1 is the first weight coefficient. If the drone finally reaches the end position, α1 is 1. If the drone does not reach the end position, α1 is 0. qF is the reward for the drone to finally reach the destination, α is the second weight coefficient, R p As a punishment for drones crossing the border, R p is a negative value. If the drone exceeds the regional boundary during flight, α2 is 1. If the drone does not exceed the regional boundary during flight, α2 is 0. N is the number of the area that the drone entered last time during flight. α3 is the third weight coefficient. R t [n] is the reward for the drone to calculate the unloading task in the nth area, R t [n] = ξ × C[n], ξ is the first coefficient, R c [n] is the reward calculated by the drone in the nth auxiliary area, q U [1] is the first area the drone enters, q S is the starting position of the UAV, q U [N] is the location of the last area entered by the drone, N is the number of the area the drone entered last during flight, and q F is the final position of the UAV, and E[1] is the initial energy of the UAV in the first area.

8. A method for unloading UAV tasks and planning trajectories according to claim 7, characterized in that: The process of solving the drone task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain a deep reinforcement learning framework, including: The state space includes: the coordinate representation of the current position of the drone in the hexagonal grid, the energy and the characteristics of the directed acyclic graph task in the current area Among them, the characteristics of the directed acyclic graph task in the current area in, is the normalized adjacency matrix of the directed acyclic graph of the current region, is the normalized task node feature matrix of the directed acyclic graph of the current region, and I is the unit matrix; The action space includes: hovering in the current area, moving to the upper adjacent area of ​​the current area, moving to the upper right adjacent area of ​​the current area, moving to the lower right adjacent area of ​​the current area, moving to the lower adjacent area of ​​the current area, moving to the lower left adjacent area of ​​the current area, and moving to the upper left adjacent area of ​​the current area; The reward function includes: the reward for the drone to finally reach the destination Penalty for Drones Crossing Borders p , negative reward for the drone consuming energy in the nth area The reward R calculated by the drone in the nth region auxiliary area c [n], the drone calculates the reward R of the unloading task in the nth area t [n] = ξ × C[n], the drone distance change reward R d =η×Δ d ; Among them, R t [n], R c The coefficients of [n] include E[n] / E[1], R d The coefficients include (E[1]-E[n]) / E[1], ξ is the first coefficient, ∈ is the second coefficient, η is the third coefficient, Δ d is the change in the distance between the UAV and the end point.

9. A method for unloading UAV tasks and planning trajectories according to claim 8, characterized in that: The dual deep Q network is used to solve the Markov decision process, including: Step S601: Initialize the parameters θ of the Q network, the parameters θ′=θ of the target network, the noise network parameters, the capacity I of the experience replay pool, the discount factor γ∈[0,1), the priority intensity parameter α, the weight correction parameter β, the constant ∈, the preset batch size m, the number of training steps T, the current number of training times k, the maximum number of training times K, and the initial state s0 of the drone; Step S602: Initialize the time step index t to 1; Step S603: Set the current state of the drone t Input the Q network to obtain the Q values ​​corresponding to all actions, select the action with the largest Q value as the preliminary action, use the noise network to generate noise, and obtain the execution action a based on the preliminary action and noise. t ; Step S604: The drone performs the selected action a t , according to the system model of the target area, the UAV movement model, the mission model, the UAV energy consumption model and the state transition probability, the state transition is performed to obtain the reward function value r t and the next state of the drone t+1 , the generated experience pool samples (s t ,a t ,r t ,s t+1 ) is stored in the experience pool; Step S605: determine whether the number of samples in the experience pool is less than I. If it is less than I, then t=t+1, and return to execute step S603 until the number of samples in the experience pool is greater than or equal to I; Step S606: When the number of experience pool samples is greater than or equal to I, the TD error of each experience pool sample is calculated based on the reward function value of each experience pool sample, the discount factor γ, the Q value corresponding to the action, and the maximum Q value of all actions obtained through the target network in the next state of the drone; Step S607: Based on the TD error, priority strength parameter α, and constant ∈ of each experience pool sample, the priority of each experience pool sample is calculated, and according to the priority of each experience pool sample, experience pool samples of a preset batch size m are taken out, and the sampling frequency of each experience pool sample taken out is calculated; Step S608: Calculate the importance sampling weight of each sample taken out from the experience pool based on the sampling frequency, weight correction parameter β, and the capacity I of the experience playback pool; and correct the TD error of each sample in the experience pool according to the importance sampling weight; Step S609: Based on the corrected TD error of each experience pool sample, the parameter θ of the Q network is updated using the gradient descent method; Step S610: Update the parameters θ of the target network every T steps ′ ; Step S611: Determine whether k is less than K. If so, set k=k+1 and return to step S602. If not, obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

10. A UAV task offloading and trajectory planning system, characterized in that: include: The system model building module is used to model the target area as an M×M hexagonal grid, and number each area in turn according to the order in which the drone passes through each area to obtain the system model of the target area; where M is the set number of rows and columns; A UAV mobility model building module is used to build a UAV mobility model based on the system model of the target area. The UAV mobility model includes: the flight speed of the UAV, the flight distance and flight time of the UAV between two adjacent areas, the position representation of the UAV in each area, and the regional decision variables; The task model building module is used to build a task model, and the task model includes: Based on the offloading decision variables of each task node in the current area, the time for each node in the current area to be calculated on the local device, the time for each node in the current area to be calculated on the drone, and the transmission time for each node to be offloaded from the local device to the drone, the time consumed for executing each task node in the current area is obtained; Based on the directed acyclic graph of the current area and the transmission time caused by the dependency relationship between the nodes in the current area, the execution start time of each task node in the current area is obtained; Based on the time consumed for executing each task node in the current area and the execution start time of each task node in the current area, the completion time of each task node in the current area is obtained; Based on the completion time of each task node in the current area and the execution start time of the first task node in the current area, the time to complete the directed acyclic graph of the current area is obtained; Based on the unloading decision variables of each task node in the current area and the amount of data calculated by each task node in the current area, the amount of data for auxiliary calculation of the drone in the current area is obtained; a drone energy consumption model construction module is used to construct a drone energy consumption model, and the drone energy consumption model includes: Based on the regional decision variables of the current area, the drone calculates the energy consumption of the directed acyclic graph of the current area, the energy consumption of the drone hovering in the current area, and the flight energy consumption of the drone flying from the previous area to the next area, and obtains the total energy consumption of the drone in the current area; Based on the total energy consumption of the drone in the previous area and the energy of the drone when it entered the previous area, the energy of the drone when it enters the current area is obtained; The deep reinforcement learning framework building module is used to build the optimization function and constraint conditions of the UAV task offloading and path planning strategy based on the system model of the target area, the UAV movement model, the task model and the UAV energy consumption model. The process of solving the UAV task offloading and path planning strategy is described as a Markov decision process, and the state space, action space and reward function are established to obtain the deep reinforcement learning framework. The solution module is used to solve the Markov decision process using a dual deep Q network to obtain the optimal UAV task offloading and path planning strategy that meets the constraints.

Images