Unmanned aerial vehicle assisted mobile edge computing task unloading method
By optimizing UAV trajectory and task offloading decisions using the LDDQN algorithm, the problems of energy consumption and latency in UAV-assisted mobile edge computing are solved, and efficient computational offloading is achieved under complex constraints.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-11
- Publication Date
- 2026-05-15
AI Technical Summary
In existing drone-assisted mobile edge computing, the problems of energy consumption and latency optimization have not been effectively solved, and the drone's endurance problem has not been fully considered, resulting in complex and difficult-to-handle solutions under complex constraints.
The LDDQN algorithm based on deep reinforcement learning is adopted, combined with LSTM, DDPG and Dueling DQN algorithms to optimize UAV trajectory, mission offloading decision and solar energy collection. The algorithm is solved jointly by MDP modeling to reduce energy consumption and latency.
With limited battery capacity and computing resources, the LDDQN algorithm effectively reduces system energy consumption and latency, optimizes UAV trajectory and task offloading strategy, and achieves lower energy consumption and latency.
Smart Images
Figure CN122054232A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of IoT computing offloading technology, specifically a computing offloading method for mobile edge computing in IoT scenarios. Background Technology
[0002] Natural disasters such as tsunamis, floods, and earthquakes cause numerous casualties and property losses every year. While deploying pre-disaster prevention facilities can reduce the occurrence of disasters, some natural disasters are unavoidable. Therefore, timely execution of rescue missions after a disaster is crucial. Restoring existing communication infrastructure as quickly as possible facilitates rescue operations, saving many lives and minimizing property damage; however, this inevitably takes a long time. Furthermore, the harsh conditions in disaster areas make this process even more unpredictable.
[0003] Due to their high mobility, flexible deployment, line-of-sight communication, and low cost, drones are a practical and feasible option for disaster relief missions, including disaster monitoring and aerial search and rescue. Furthermore, mobile edge computing (MEC) has emerged as a promising technological solution that can overcome the challenges faced by mobile devices performing computationally intensive tasks. Therefore, combining both approaches is an effective means of assisting in disaster relief scenarios. On one hand, drones act as in-flight servers, flexibly adjusting their three-dimensional position to easily establish line-of-sight links, providing computing and communication services to ground equipment. On the other hand, MEC provides computing and storage resources closer to the drone's operational location. This proximity minimizes latency and enhances data processing capabilities, significantly improving drone performance.
[0004] In drone-assisted MEC, energy consumption, latency, and drone trajectory remain optimization objectives for task offloading. Currently, there is considerable research on the optimization of energy consumption, latency, and drone trajectory in drone-assisted MEC. However, most existing studies do not directly consider the drone's endurance problem, and they typically represent the optimization problem as a mixed-integer nonlinear programming problem, then decompose it into multiple sub-problems and use convex optimization methods to solve it. However, under the dynamic changes of various constraints such as drone trajectory, drone storage energy variations, and network resources, the solution becomes complex and difficult to handle. Summary of the Invention
[0005] Technical Problem: To address the above issues, this paper presents a method for unmanned aerial vehicle (UAV)-assisted mobile edge computing task offloading. Under limited battery capacity and computing resources, an LDDQN algorithm based on deep reinforcement learning (DRL) is employed to reduce energy consumption and latency during task offloading. The LDDQN algorithm combines a long short-term memory (LSTM) network, the deep deterministic policy gradient (DDPG) algorithm, and the Dueling DQN algorithm. Considering system computing resources, power resources, and latency constraints, it optimizes solar energy collection, task offloading decisions, and UAV trajectory, ultimately achieving lower energy consumption and latency. Simulation results show that the LDDQN algorithm outperforms similar algorithms.
[0006] Technical solution: To achieve the above objectives, the present invention is implemented through the following methods;
[0007] The present invention provides a method for offloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks, comprising the following steps:
[0008] Step 1: Establish a drone-assisted mobile edge computing offloading system. This system includes a group of mobile devices and a solar-powered drone, where the mobile devices generate computing tasks and the drone provides computing and communication services to the mobile devices.
[0009] Step 2: Initialize the relevant parameters of the mobile edge computing offloading system model;
[0010] Step 3: Establish communication, computation, and energy models for mobile devices and drones. Through these models, obtain the optimization problem and objective function related to offloading decisions, drone trajectories, and solar energy collection. Then, model the optimization problem and objective function using Markov Decision Process (MDP) modeling to obtain the MDP model, which includes three parts: state, action, and reward.
[0011] Step 4: Use the Deep Deterministic Duel Network (LDDQN) algorithm with Long Short-Term Memory to solve the problem jointly: Take the relative position of the mobile device, the position of the drone, and its battery state value as inputs. In the training phase, train the LDDQN neural network to obtain the trained neural network model. In the testing phase, the optimal unloading strategy is finally output through the trained LDDQN neural network model.
[0012] Step 2 specifically includes the following steps:
[0013] Step 2-1, Collection of mobile devices express, ={1,2,3...M}, where M represents the number of mobile devices; and the drone is represented by U. The horizontal flight area of the drone and the movement position of the mobile devices are divided into identical grid segments of 10 meters in length. The vertical flight altitude of the drone is also divided into line segments of equal length of 2 meters. Both the drone and the mobile devices can only move from one end of the grid to the other at a time in the horizontal direction, and the drone can only ascend or descend 2 meters at a time in the vertical direction. At any given moment... The coordinates of the mobile device on the ground are The coordinates of the drone are ; m represents a mobile device, and the distance from the mobile device to the drone is:
[0014] ;time ={1, 2,...T} is considered as a specific segment containing a series of T time slots, each representing a duration of 0.1 seconds, using... This indicates that the drone is starting from its initial position. It begins and reaches its position at the end of time T. .
[0015] Step 3 specifically includes the following steps:
[0016] Step 3-1: Calculate the line-of-sight (LoS) probability of the mobile device at time slot t. 'm' represents a specific mobile device, and 'Los' represents the line-of-sight distance.
[0017]
[0018] Where a and b are environmental parameters, and exp is an exponential function of e. The elevation angle is satisfied.
[0019]
[0020] in Let t be the altitude of the drone. This is the distance from the mobile device to the drone at this point, and the non-line-of-sight (NLoS) channel probability is equal to... The path loss expression at time t is as follows:
[0021]
[0022] in Represents carrier frequency, Represents the speed of light. and These are the relevant parameters for LoS and NLoS communication;
[0023] Step 3-2: Based on step 3-1 above, the transmission rate between the mobile device and the drone at this time is expressed as...
[0024]
[0025] in The uplink transmission power representing MD. B represents the noise power spectral density, and B represents the transmission bandwidth.
[0026] Step 3-3: The computational model includes a local computational model and a drone offloading computational model. This means that when a mobile device receives a task in any time slot, it offloads the task to the drone for processing, or processes it locally. The offloading decision is made using... This indicates that when a task is processed locally by the mobile device, =0, the required number of time slots is 'm' represents a specific mobile device, and 'E' represents a processing identifier.
[0027]
[0028] in, Let be the size of the computational task of the mobile device at time slot t; The computing power of a mobile device is expressed in cycles per second; the processing density of a task is expressed as... It represents the number of CPU cycles required to complete the task execution; when the task is offloaded to the drone for processing, the number of time slots required to transfer the task to the drone is... T represents the transfer task to the drone.
[0029]
[0030] When drones handle tasks =1, the required number of time slots is Simplify to
[0031]
[0032] in, Let t be the size of the computational task of the mobile device. The computing power of the drone is represented by (number of cycles per second); the total latency of the mobile device within time slot t is represented by the following formula:
[0033]
[0034] The total delay of the system within time slot t is:
[0035]
[0036] Steps 3-4: The energy model includes a solar energy collection model and an energy consumption model. At time slot t, the UAV's altitude is z(t), and the attenuation of sunlight can be modeled as follows: , The absorption coefficient represents the optical properties of the simulated cloud. The average output power of a solar panel can be expressed as the distance sunlight travels through the cloud.
[0037]
[0038] in S, These represent energy conversion efficiency, solar panel area, and average solar radiation on Earth, respectively. , These are the heights of the upper and lower boundaries of the cloud, respectively. Let z(t) be a variable, expressed by the following formula.
[0039]
[0040] in, This is expressed as the maximum achievable value of atmospheric transmittance. Expressed as atmospheric extinction coefficient, Expressed as height on a global scale; at time slot t, if the task is executed locally, the corresponding execution energy consumption is... Represented as
[0041]
[0042] This represents the local processing task identifier, where 'm' represents a specific mobile device. Let t be the size of the computational task of the mobile device. This indicates the processing power consumption of mobile devices. This represents a duration of 0.1 seconds; if the task is transmitted to a drone for execution, the corresponding transmission energy consumption is...
[0043]
[0044] This represents the transmission power consumption of a mobile device; while the computational power consumption of a drone in time slot t can be expressed as... C represents the local processing energy consumption indicator.
[0045]
[0046] in, This represents the processing power consumption of the drone; assuming the drone's horizontal and vertical speeds are respectively... At time t, it can be quantized as And based on the available propulsion power as
[0047]
[0048] in, W is the tip speed of the drone's rotor blades, and W is the weight of the drone. , , and It is a constant related to the weight and aerodynamic parameters of the drone; therefore, the energy consumption of the drone represents...
[0049]
[0050] in Calculate the energy consumption of the UAV in time slot t. The average output power of the solar panel at time slot t. For the propulsion power of the drone, Representing a duration of 0.1 seconds, the total energy consumption of the system within time slot t is ultimately:
[0051] .
[0052] In step 3, the objective function of the optimization problem is established as follows:
[0053]
[0054] in For the unloading decision at time slot t, Let t be the position of the UAV. The average output power of the solar panel at time slot t. This represents the total energy consumption of the system within time slot t. Let be the total delay of the system within time slot t. Represented as a weighted value, it is a constant; constraint (C1) represents the offloading decision of the mobile device; constraint (C2) represents the horizontal flight speed of the UAV at time slot t. Greater than or equal to minimum horizontal flight speed And less than or equal to the maximum horizontal flight speed Constraint (C3) indicates that the UAV's vertical flight speed at time slot t is greater than or equal to the minimum vertical flight speed. And less than or equal to the maximum vertical flight speed Constraint (C4) indicates that the energy consumption of the UAV in time slot t cannot exceed the UAV's battery capacity. Constraint (C5) represents the distance between the mobile device and the drone at time slot t. Less than or equal to the maximum coverage radius of the drone Constraint (C6) represents the processing delay for each task. Greater than or equal to 0 and less than or equal to the maximum tolerable delay .
[0055] In step 3, the resulting MDP model comprises three parts: state, action, and reward. Specifically, the state... Including mobile device location drone location and the drone's battery power ;action This can be simply represented as the following action tuple:
[0056]
[0057] That is, the change in the horizontal and vertical position of the drone. and task unloading decisions The drone's position changes include stationary, ascending, descending, backward, forward, right, and left movements; while the unloading decision... This indicates that the task will be unloaded and processed locally. This indicates that the task will be offloaded to the drone for processing;
[0058] award Defined as
[0059]
[0060] The reward value is the system overhead. This represents the total energy consumption of the system within time slot t. The total delay of the system within time slot t, Represented as a weighted value, it is a constant, all The value is negative, where R>0 represents a constant penalty value, which indicates the penalty the system will face when its behavior does not meet the constraints; for example, a drone flying out of the boundary, excessive latency between the mobile device and the drone, and insufficient battery power of the drone.
[0061] In step 4, during the training phase, the neural network used by LDDQN is trained to obtain the trained LDDQN neural network model, specifically as follows:
[0062] Step 6-1: Initialize user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery level E, experience buffer size, drone starting position coordinates Network parameters ;
[0063] Step 6-2: Parameters of the target network and Perform a hard update;
[0064] Step 6-3: Initialize the drone status ;
[0065] Step 6-4: Randomly generate computational tasks and obtain actions through the actor and critic networks. This includes the horizontal and vertical movement of the drone and the unloading decisions of the mobile device;
[0066] Step 6-5: The drone performs the corresponding actions to update its coordinates and battery level, the mobile device performs the corresponding unloading actions, a reward value is obtained, and the state at the next moment is observed. ;
[0067] Step 6-6, That is, the state of time slot t. ,action ,award and the state in the next time slot t+1 Stored in the experience buffer;
[0068] Steps 6-7: Update Status ;
[0069] Steps 6-8: Uniformly and randomly sample from the experience buffer, use the temporal difference method to update the Critic network parameters, and update the Critic network parameters through gradient descent.
[0070] Steps 6-9: Update the Actor target network and Critic target network using a soft update method;
[0071] Steps 6-10: When the number of training rounds is greater than 200, repeat steps 6-8 to 6-9 every 10 rounds;
[0072] Step 6-11: Repeat steps 6-2 to 6-10 until the maximum number of iterations is reached;
[0073] Steps 6-12: Output the trained LDDQN neural network model.
[0074] In step 4, during the testing phase, the optimal unloading strategy is finally output through the trained LDDQN neural network model, specifically as follows:
[0075] Step 7-1: Load the trained LDDQN neural network model saved during the training phase;
[0076] Step 7-2: Initialize input parameters: user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery capacity E;
[0077] Step 7-3: Initialize network parameters: Neural network learning rate ( , ), soft update factor Reward Discount Factor ;
[0078] Step 7-4: Initialize the drone status ;
[0079] Step 7-5: Randomly generate computational tasks and obtain actions through the Actor network and Critic network. ;
[0080] Step 7-6: Execute the corresponding actions to update its coordinates and battery level. The mobile device performs the corresponding uninstallation action, obtains the reward value, and then observes the state at the next moment. ;
[0081] Step 7-7, Storage Action ;
[0082] Steps 7-8: Repeat steps 7-4 to 7-7 until the maximum number of iterations is reached, and output the optimal task unloading strategy.
[0083] Beneficial Effects: This invention's algorithm takes mobile device location information, UAV location, and battery power as inputs. Considering system computing resources, power resources, and latency constraints, it minimizes the weighted system overhead of energy consumption and latency by jointly optimizing solar energy collection, task offloading decisions, and UAV trajectory. Experimental results show that the proposed LDDQN algorithm outperforms existing algorithms in reducing system energy consumption and latency. Under limited battery capacity and computing resources, an LDDQN algorithm based on deep reinforcement learning (DRL) is used to reduce energy consumption and latency during system task offloading. The LDDQN algorithm combines a long short-term memory (LSTM) network, the deep deterministic policy gradient (DDPG) algorithm, and the dueling deep Q network (dueling DQN) Dueling DQN algorithm. By comprehensively considering system computing resources, power resources, and latency constraints, it optimizes solar energy collection, task offloading decisions, and UAV trajectory, ultimately achieving lower energy consumption and latency. Simulation results show that the LDDQN algorithm outperforms similar algorithms. Attached Figure Description
[0084] Figure 1 This is a flowchart illustrating a method for unloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks according to the present invention.
[0085] Figure 2 This is a schematic diagram of the UAV-assisted mobile edge computing system model in this invention.
[0086] Figure 3 The diagram illustrates the training process of four algorithms.
[0087] Figure 4 This diagram illustrates the comparison of task arrival rates and system overhead for four algorithms.
[0088] Figure 5 This diagram illustrates the comparison of the number of mobile devices and system overhead for the four algorithms. Detailed Implementation
[0089] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.
[0090] like Figure 1-5 As shown, the present invention is a method for offloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks, comprising the following steps:
[0091] Step 1: Establish a drone-assisted mobile edge computing offloading system. This system includes a group of mobile devices and a solar-powered drone, where the mobile devices generate computing tasks and the drone provides computing and communication services to the mobile devices.
[0092] Step 2: Initialize the relevant parameters of the mobile edge computing offloading system model;
[0093] Step 21, Collection of mobile devices express, ={1,2,3...M}, where M represents the number of mobile devices; and the drone is represented by U. The horizontal flight area of the drone and the movement position of the mobile devices are divided into identical grid segments of 10 meters in length. The vertical flight altitude of the drone is also divided into line segments of equal length of 2 meters. Both the drone and the mobile devices can only move from one end of the grid to the other at a time in the horizontal direction, and the drone can only ascend or descend 2 meters at a time in the vertical direction. At any given moment... The coordinates of the mobile device on the ground are The coordinates of the drone are The distance from the mobile device to the drone is 'm' represents a specific mobile device. Time ={1, 2,...T} is considered as a specific segment containing a series of T time slots, each representing a duration of 0.1 seconds, using... This indicates that the drone is starting from its initial position. It begins and reaches its position at the end of time T. .
[0094] Step 3: Establish communication, computation, and energy models for mobile devices and drones. Through these models, obtain the optimization problem and objective function related to offloading decisions, drone trajectories, and solar energy collection. Then, model the optimization problem and objective function using Markov Decision Process (MDP) modeling to obtain the MDP model, which includes three parts: state, action, and reward.
[0095] Step 31: Calculate the line-of-sight (LoS) probability of the mobile device at time slot t. 'm' represents a specific mobile device, and 'Los' represents the line-of-sight distance.
[0096]
[0097] Where a and b are environmental parameters, and exp is an exponential function of e. The elevation angle is satisfied.
[0098]
[0099] in Let t be the drone altitude, and the non-line-of-sight (NLoS) channel probability be equal to... The path loss expression at time t is as follows:
[0100]
[0101] in Represents carrier frequency, Represents the speed of light. and These are the relevant parameters for LoS and NLoS communication;
[0102] Step 32: Based on step 31 above, the transmission rate between the mobile device and the drone at this time can be expressed as:
[0103]
[0104] in The uplink transmission power representing MD. B represents the noise power spectral density, and B represents the transmission bandwidth.
[0105] Step 33: The computational model includes a local computational model and a drone offloading computational model. This means that when a mobile device receives a task in any time slot, it can offload the task to the drone for processing, or process it locally. The offloading decision is made using... This indicates that when a task is processed locally by the mobile device, =0, the required number of time slots is 'm' represents a specific mobile device, and 'E' represents a processing identifier.
[0106]
[0107] in, Let t be the size of the computational task of the mobile device. (In terms of loops per second) represents the computing power of a mobile device; task processing density is expressed as... It represents the number of CPU cycles required to complete the task execution; when the task is offloaded to the drone for processing, the number of time slots required to transfer the task to the drone is... T represents the transfer task to the drone.
[0108]
[0109] When drones handle tasks =1, the required number of time slots is Simplify to
[0110]
[0111] in, Let t be the size of the computational task of the mobile device. The computing power of the drone is represented by (number of cycles per second); the total latency of the mobile device within time slot t is represented by the following formula:
[0112]
[0113] The total delay of the system within time slot t is:
[0114]
[0115] Step 34: The energy model includes a solar energy collection model and an energy consumption model. At time slot t, the UAV's altitude is z(t), and the attenuation of sunlight can be modeled as follows: , The absorption coefficient represents the optical properties of the simulated cloud. The average output power of a solar panel can be expressed as the distance sunlight travels through the cloud.
[0116]
[0117] in S, These represent energy conversion efficiency, solar panel area, and average solar radiation on Earth, respectively. , These are the heights of the upper and lower boundaries of the cloud, respectively. Let z(t) be a variable, expressed by the following formula.
[0118]
[0119] in, This is expressed as the maximum achievable value of atmospheric transmittance. Expressed as atmospheric extinction coefficient, Expressed as height on a global scale; at time slot t, if the task is executed locally, the corresponding execution energy consumption is... Represented as
[0120]
[0121] Represents the local processing identifier, where Let m represent the computational task size of the mobile device at time slot t, where m represents a specific mobile device. This indicates the processing power consumption of mobile devices. This represents a duration of 0.1 seconds; if the task is transmitted to a drone for execution, the corresponding transmission energy consumption is...
[0122]
[0123] This represents the transmission power consumption of a mobile device; while the computational power consumption of a drone in time slot t can be expressed as... C represents the local processing energy consumption indicator.
[0124]
[0125] in, This represents the processing power consumption of the drone; assuming the drone's horizontal and vertical speeds are respectively... At time t, it can be quantized as And based on the available propulsion power as
[0126]
[0127] in, W is the tip speed of the drone's rotor blades, and W is the weight of the drone. , , and It is a constant related to the weight and aerodynamic parameters of the drone; therefore, the energy consumption of the drone represents...
[0128]
[0129] in Calculate the energy consumption of the UAV in time slot t. The average output power of the solar panel at time slot t. For the propulsion power of the drone, Representing a duration of 0.1 seconds, the total energy consumption of the system within time slot t is ultimately:
[0130] )
[0131] In steps 35 and 3, the objective function of the optimization problem is established as follows:
[0132]
[0133] in For the unloading decision at time slot t, Let t be the position of the UAV. The average output power of the solar panel at time slot t. The total energy consumption of the system in time slot t. Let be the total delay of the system within time slot t. Represented as a weighted value, it is a constant; constraint (C1) represents the offloading decision of the mobile device; constraint (C2) represents the horizontal flight speed of the UAV at time slot t. Greater than or equal to minimum horizontal flight speed And less than or equal to the maximum horizontal flight speed Constraint (C3) indicates that the UAV's vertical flight speed at time slot t is greater than or equal to the minimum vertical flight speed. And less than or equal to the maximum vertical flight speed Constraint (C4) indicates that the energy consumption of the UAV in time slot t cannot exceed the UAV's battery capacity. Constraint (C5) represents the distance between the mobile device and the drone at time slot t. Less than or equal to the maximum coverage radius of the drone Constraint (C6) represents the processing delay for each task. Greater than or equal to 0 and less than or equal to the maximum tolerable delay ;
[0134] In steps 36 and 37, the resulting MDP model comprises three parts: state, action, and reward. Specifically, the state (using...) (This includes the location of the mobile device) drone location and the drone's battery power Action (using) The action can be concisely represented as the following action tuple:
[0135]
[0136] That is, the change in the horizontal and vertical position of the drone. and task unloading decisions The drone's position changes include stationary, ascending, descending, backward, forward, right, and left movements; while the unloading decision... This indicates that the task will be unloaded and processed locally. This indicates that the task will be unloaded onto the drone for processing. Rewards (using...) (representation) is defined as
[0137]
[0138] Because the reward value is a system overhead. This represents the total energy consumption of the system within time slot t. The total delay of the system within time slot t, Represented as a weighted value, it is a constant, all The value is negative, where R>0 represents a constant penalty value, indicating the punishment the system will face when its behavior does not meet the constraints. Examples include a drone flying out of bounds, excessive latency between the mobile device and the drone, and the drone's low battery power.
[0139] Step 4: Use the LDDQN algorithm for joint solution: Take the relative position of the mobile device, the position of the drone, and its battery status value as inputs. In the training phase, train the LDDQN neural network to obtain the trained neural network model. In the testing phase, the optimal unloading strategy is finally output through the trained LDDQN neural network model.
[0140] Step 41: Initialize user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery level E, experience buffer size, drone starting position coordinates Network parameters ;
[0141] Step 42: Parameterize the target network and Perform a hard update;
[0142] Step 43: Initialize the drone status ;
[0143] Step 44: Randomly generate computational tasks and obtain actions through the Actor network and Critic network. This includes the horizontal and vertical movement of the drone and the unloading decisions of the mobile device;
[0144] Step 45: The drone performs the corresponding actions to update its coordinates and battery level, the mobile device performs the corresponding unloading actions, a reward value is obtained, and the state at the next moment is observed. ;
[0145] Step 46, That is, the state at time slot t. ,action ,award and the state in the next time slot t+1 Stored in the experience buffer;
[0146] Step 47: Update Status ;
[0147] Step 48: Uniformly and randomly sample from the experience buffer, use the temporal difference method to update the Critic network parameters, and update the Critic network parameters through gradient descent.
[0148] Step 49: Update the Actor target network and Critic target network using a soft update method;
[0149] Step 50: When the number of training rounds is greater than 200, repeat steps 48-49 every 10 rounds;
[0150] Step 51: Repeat steps 42 through 50 until the maximum number of iterations is reached;
[0151] Step 52: Output the trained LDDQN neural network model.
[0152] In steps 4 and 5, during the testing phase, the optimal unloading strategy is finally output through the trained LDDQN neural network model. Specifically...
[0153] Step 53: Load the trained LDDQN neural network model saved during the training phase;
[0154] Step 54: Initialize input parameters: user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery capacity E;
[0155] Step 55: Initialize network parameters: neural network learning rate ( , ), soft update factor Reward Discount Factor ;
[0156] Step 56: Initialize the drone status ;
[0157] Step 57: Randomly generate computational tasks and obtain actions through the Actor network and Critic network. ;
[0158] Step 58: Execute the corresponding actions to update its coordinates and battery level, the mobile device performs the corresponding uninstallation action, obtains a reward value, and then observes the state at the next moment. ;
[0159] Step 59, Storage Action ;
[0160] Step 60: Repeat steps 56-59 until the maximum number of iterations is reached, and output the optimal task unloading strategy.
[0161] Experimental simulation was conducted on a method for unmanned aerial vehicle-assisted mobile edge computing task offloading according to the present invention:
[0162] The simulation software environment consisted of Python 3.8 and the PyTorch framework, while the hardware platform was a personal computer equipped with an Intel Core i5-13500HX 2.50 GHz processor, an NVIDIA RTX 4060 graphics card, and 16 GB of RAM. In the experiment, six mobile devices were randomly distributed within a 1000m × 1000m square area. The drone started from its initial position... Flight commences, with a maximum altitude of 200m and a minimum altitude of 100m. The horizontal flight zone is divided into equal-length grids, each 10m long. Similarly, the drone's flight altitude is divided into equal segments, each 3m long. The drone's weight... It weighs 2 kg. The parameters related to the communication channel are set as follows: , , , , , The maximum coverage area of the drone is Drone battery capacity =80000J, drones must meet energy consumption constraints.
[0163] Simulation results show that the LDDQN algorithm has better performance in reducing system energy consumption and system latency compared with existing algorithms.
[0164] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.
Claims
1. A method for unloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks, characterized in that, Includes the following steps: Step 1: Establish a drone-assisted mobile edge computing offloading system. This system includes a group of mobile devices and a solar-powered drone, where the mobile devices generate computing tasks and the drone provides computing and communication services to the mobile devices. Step 2: Initialize the relevant parameters of the mobile edge computing offloading system model; Step 3: Establish communication, computation, and energy models for mobile devices and drones. Through these models, obtain the optimization problem and objective function related to offloading decisions, drone trajectories, and solar energy collection. Then, model the optimization problem and objective function using Markov Decision Process (MDP) modeling to obtain the MDP model, which includes three parts: state, action, and reward. Step 4: Use the Deep Deterministic Duel Network (LDDQN) algorithm with Long Short-Term Memory to solve the problem jointly: Take the relative position of the mobile device, the position of the drone, and its battery state value as inputs, and train the LDDQN neural network during the training phase to obtain the trained neural network model. During the testing phase, the optimal unloading strategy is ultimately output through the trained LDDQN neural network model.
2. The method for unloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 2-1, Collection of mobile devices express, ={1,2,3...M}, where The number of mobile devices is represented by 'U'; drones are represented by 'U'. The horizontal flight area of the drone and the movement position of the mobile devices are divided into identical grid segments of 10 meters in length. The vertical flight altitude of the drone is also divided into line segments of equal length of 2 meters. Both the drone and the mobile device can only move from one end of the grid to the other at a time in the horizontal direction, and the drone can only ascend or descend 2 meters at a time in the vertical direction. At any given moment... The coordinates of the mobile device on the ground are The coordinates of the drone are ; m represents a mobile device, and the distance from the mobile device to the drone is: ;time ={1, 2,...T} is considered as a specific segment containing a series of T time slots, each representing a duration of 0.1 seconds, using... This indicates that the drone is starting from its initial position. It begins and reaches its position at the end of time T. .
3. The method for unloading unmanned aerial vehicle-assisted mobile edge computing tasks according to claim 2, characterized in that, Step 3 specifically includes the following steps: Step 3-1: Calculate the line-of-sight (LoS) probability of the mobile device at time slot t. 'm' represents a specific mobile device, and 'Los' represents the line-of-sight distance. ； Where a and b are environmental parameters, and exp is an exponential function of e. The elevation angle is satisfied. ； in Let t be the altitude of the drone. This is the distance from the mobile device to the drone at this point, and the non-line-of-sight (NLoS) channel probability is equal to... The path loss expression at time t is as follows: ； in Represents carrier frequency, Represents the speed of light. and These are the relevant parameters for LoS and NLoS communication; Step 3-2: Based on step 3-1 above, the transmission rate between the mobile device and the drone at this time is expressed as... ； in The uplink transmission power representing MD. B represents the noise power spectral density, and B represents the transmission bandwidth. Step 3-3: The computational model includes a local computational model and a drone offloading computational model. This means that when a mobile device receives a task in any time slot, it processes the task locally or offloads it to the drone for processing. The offloading decision is made using... This indicates that when a task is processed locally by the mobile device, =0, the required number of time slots is 'm' represents a specific mobile device, and 'E' represents a processing identifier. ； in, Let be the size of the computational task of the mobile device at time slot t; The computing power of a mobile device is expressed in cycles per second; the processing density of a task is expressed as... It represents the number of CPU cycles required to complete the task execution; when the task is offloaded to the drone for processing, the number of time slots required to transfer the task to the drone is... T represents the transfer task to the drone. ； When drones handle tasks =1, the required number of time slots is Simplify to ； in, Let t be the size of the computational task of the mobile device. The computing power of the drone is expressed in cycles per second; the total latency of the mobile device within time slot t is expressed by the following formula: ； The total delay of the system within time slot t is: ； Steps 3-4: The energy model includes a solar energy collection model and an energy consumption model. At time slot t, the UAV's altitude is z(t), and the attenuation of sunlight can be modeled as follows: , The absorption coefficient represents the optical properties of the simulated cloud. The average output power of a solar panel can be expressed as the distance sunlight travels through the cloud. ； in S, These represent energy conversion efficiency, solar panel area, and average solar radiation on Earth, respectively. , These are the heights of the upper and lower boundaries of the cloud, respectively. Let z(t) be a variable, expressed by the following formula. ； in, This is expressed as the maximum achievable value of atmospheric transmittance. Expressed as atmospheric extinction coefficient, Expressed as height on a global scale; at time slot t, if the task is executed locally, the corresponding execution energy consumption is... Represented as ； This represents the local processing task identifier, where 'm' represents a specific mobile device. Let t be the size of the computational task of the mobile device. This indicates the processing power consumption of mobile devices. Represents a duration of 0.1 seconds; if the task is transmitted to a drone for execution, the corresponding transmission energy consumption is... ； This represents the transmission power consumption of a mobile device; while the computational power consumption of a drone in time slot t can be expressed as... C represents the local processing energy consumption indicator. ； in, This represents the processing power consumption of the drone; assuming the drone's horizontal and vertical speeds are respectively... At time t, it can be quantized as And based on the available propulsion power as ； in, W is the tip speed of the drone's rotor blades, and W is the weight of the drone. , , and It is a constant related to the weight and aerodynamic parameters of the drone; therefore, the energy consumption of the drone represents... ； in Calculate the energy consumption of the UAV in time slot t. The average output power of the solar panel at time slot t. For the propulsion power of the drone, Representing a duration of 0.1 seconds, the total energy consumption of the system within time slot t is ultimately: 。 4. The method for unloading unmanned aerial vehicle-assisted mobile edge computing tasks according to claim 3, characterized in that, In step 3, the objective function of the optimization problem is established as follows: ； in For the unloading decision at time slot t, Let t be the position of the UAV. The average output power of the solar panel at time slot t. This represents the total energy consumption of the system within time slot t. Let be the total delay of the system within time slot t. Represented as a weighted value, it is a constant; constraint (C1) represents the offloading decision of the mobile device; constraint (C2) represents the horizontal flight speed of the UAV at time slot t. Greater than or equal to minimum horizontal flight speed And less than or equal to the maximum horizontal flight speed Constraint (C3) indicates that the UAV's vertical flight speed at time slot t is greater than or equal to the minimum vertical flight speed. And less than or equal to the maximum vertical flight speed Constraint (C4) indicates that the energy consumption of the UAV in time slot t cannot exceed the UAV's battery capacity. Constraint (C5) represents the distance between the mobile device and the drone at time slot t. Less than or equal to the maximum coverage radius of the drone ; Constraint (C6) represents the processing latency for each task. Greater than or equal to 0 and less than or equal to the maximum tolerable delay .
5. The method for unloading unmanned aerial vehicle (UAV)-assisted mobile edge computing tasks as described in claim 4, characterized in that: In step 3, the resulting MDP model comprises three parts: state, action, and reward. Specifically, the state... Including mobile device location drone location and the drone's battery power ;action This can be simply represented as the following action tuple: ； That is, the change in the horizontal and vertical position of the drone. and task unloading decisions The drone's position changes include stationary, ascending, descending, backward, forward, right, and left movements; while the unloading decision... This indicates that the task will be unloaded and processed locally. This indicates that the task will be offloaded to the drone for processing; award Defined as ； The reward value is the system overhead. This represents the total energy consumption of the system within time slot t. The total delay of the system within time slot t, Represented as a weighted value, it is a constant, all The value is negative, where R > 0 represents a constant penalty value, which indicates the penalty the system will face when its behavior does not meet the constraints; for example, a drone flying out of the boundary, excessive latency between the mobile device and the drone, and insufficient battery power of the drone.
6. The method for unloading unmanned aerial vehicle-assisted mobile edge computing tasks as described in claim 5, characterized in that: In step 4, during the training phase, the neural network used by LDDQN is trained to obtain the trained LDDQN neural network model, specifically as follows: Step 6-1: Initialize user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery level E, experience buffer size, drone starting position coordinates Network parameters ; Step 6-2: Parameters of the target network and Perform a hard update; Step 6-3: Initialize the drone status ; Step 6-4: Randomly generate computational tasks and obtain actions through the actor and critic networks. This includes the horizontal and vertical movement of the drone and the unloading decisions of the mobile device; Step 6-5: The drone performs the corresponding actions to update its coordinates and battery level, the mobile device performs the corresponding unloading actions, a reward value is obtained, and the state at the next moment is observed. ; Step 6-6, That is, the state of time slot t. ,action ,award and the state in the next time slot t+1 Stored in the experience buffer; Steps 6-7: Update Status ; Steps 6-8: Uniformly and randomly sample from the experience buffer, use the temporal difference method to update the Critic network parameters, and update the Critic network parameters through gradient descent. Steps 6-9: Update the Actor target network and Critic target network using a soft update method; Steps 6-10: When the number of training rounds is greater than 200, repeat steps 6-8 to 6-9 every 10 rounds; Step 6-11: Repeat steps 6-2 to 6-10 until the maximum number of iterations is reached; Steps 6-12: Output the trained LDDQN neural network model.
7. The method for unloading unmanned aerial vehicle-assisted mobile edge computing tasks as described in claim 6, characterized in that: In step 4, during the testing phase, the optimal unloading strategy is finally output through the trained LDDQN neural network model, specifically as follows: Step 7-1: Load the trained LDDQN neural network model saved during the training phase; Step 7-2: Initialize input parameters: user coordinates Number of mobile devices M, number of time slots T, number of iterations P, drone battery capacity E; Step 7-3: Initialize network parameters: Neural network learning rate ( , ), soft update factor Reward Discount Factor ; Step 7-4: Initialize the drone status ; Step 7-5: Randomly generate computational tasks and obtain actions through the Actor network and Critic network. ; Step 7-6: Execute the corresponding actions to update its coordinates and battery level. The mobile device performs the corresponding uninstallation action, obtains the reward value, and then observes the state at the next moment. ; Step 7-7, Storage Action ; Steps 7-8: Repeat steps 7-4 to 7-7 until the maximum number of iterations is reached, and output the optimal task unloading strategy.