A multi-UAV assisted mobile edge computing method, device and storage medium based on SAC

Abstract

The present invention discloses a multi-UAV assisted mobile edge computing method, device and storage medium based on SAC, which belongs to the field of mobile edge computing technology. The method includes: obtaining basic element information in the edge computing system; establishing an optimization model for UAV path planning and resource allocation based on the information; taking each UAV as a decision maker, the observation of the UAV as a state, and the path planning and resource allocation strategy selected by the UAV as an action, and converting the optimization model into a Markov decision process based on a preset reward function and discount factor; each UAV uses a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain an optimized UAV flight trajectory and resource allocation strategy. The present invention assists ground user edge computing through multi-UAV joint base station, and optimizes the strategy to achieve improved task processing efficiency, dispersed energy consumption and saved computing resources.

Description

Technical Field

[0001] The present invention relates to a multi-UAV assisted mobile edge computing method, device and storage medium based on SAC (Soft Actor-Critic), belonging to the field of mobile edge computing technology. Background Art

[0002] Mobile edge computing can provide computing services to user devices at the edge of wireless networks with low transmission latency and light access load. Drones play an important role in mobile edge computing due to their flexibility, mobility, and low cost.

[0003] However, due to the dynamic wireless channel environment, battery capacity limitations, and computational resource constraints of drones, traditional methods are inefficient. The decision-making process during a drone mission is essentially a constrained hybrid optimization problem. Traditional methods not only face the challenges of high computational complexity and long latency, but also require recalculation and solution at every step in complex and changing environments, making them unable to meet real-time requirements.

[0004] The rapid development of reinforcement learning (RL) offers an efficient and feasible solution to these problems. RL, an algorithm that learns through trial and error, enables real-time resource allocation, trajectory planning, and intelligent decision-making, making it particularly well-suited for latency-sensitive tasks. RL also significantly reduces computational complexity, exhibits strong robustness, and can effectively cope with complex and changing environments.

[0005] However, these current algorithms typically only consider computation offloading and task allocation between a single drone and a ground user. In practical applications, drone edge computing scenarios may involve multiple drones simultaneously providing computing services to ground users. In this case, if each drone only performs direct computation offloading with the ground user, it may lead to wasted computing resources and low efficiency. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-UAV assisted mobile edge computing method based on SAC, which aims to optimize the path planning, resource allocation and task offloading strategies of UAVs when multiple UAVs jointly assist ground users in edge computing by using deep reinforcement learning algorithms, improve the processing efficiency of user tasks, and avoid concentrated energy consumption and resource waste.

[0007] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0008] In one aspect, the present invention provides a multi-UAV assisted mobile edge computing method, comprising:

[0009] Obtain basic element information in a system where multiple drones work together with base stations to assist ground users in edge computing;

[0010] Based on the basic element information, an optimization model for UAV path planning, channel resource allocation, and computing resource allocation is established;

[0011] Taking each UAV as a decision maker, the observation of the UAV as the state, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions, the optimization model is converted into a Markov decision process based on a preset reward function and discount factor.

[0012] Each UAV uses a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain the optimized UAV flight trajectory, channel resource allocation and computing resource allocation strategy.

[0013] In order to avoid the disadvantages of direct computing offloading between drones and ground users in the existing technology, which leads to waste of computing resources and low efficiency, the present invention, based on the deep reinforcement learning algorithm, uses the base station as a computing resource to participate in the computing offloading and task allocation between drones and ground users. Through the coordinated cooperation between the edge computing server and the ground server, it is ensured that the drone can offload some intensive computing tasks to the ground base station server, thereby optimizing the drone flight trajectory, channel resource allocation and computing resource allocation strategy, and ultimately achieving a significant improvement in user computing experience quality and overall computing efficiency.

[0014] Optionally, the system in which multiple drones jointly with base stations assist ground users in performing edge computing includes several drones, several ground users and base stations, wherein the drones serve as edge computing servers and the base stations serve as ground servers;

[0015] Specifically, the basic element information includes: the number of ground users N, the number of drones M, the location of the ground users x = (x1, ..., x M ), the position of the drone y=(y1,...,y M ), the movement of the drone move=(move1,...,move M ), the channel resource allocation of UAV to ground users is α=(α1,...,α M ), the computing resource allocation of the UAV to the ground user is β=(β1,...,β M ), the mission data volume, computational workload and location of the i-th ground user (D i ,fi ,x i ), energy consumption E, the number of ground users served by the drone (I1,...,I M ), fairness coefficient I, maximum delay T, user information transmission power P user , the maximum computing resource F of the UAV uav , the maximum bandwidth of the drone B0, the transmission power of the drone P uav , the communication time T for the i-th ground user to offload the task to the j-th UAV 0i j , the computing time T for the jth UAV to locally calculate the task of the i-th ground user 1i j , the communication time T for the jth UAV to offload the task of the i-th ground user to the base station 2i j , power gain g0 at a reference distance of 1 meter, effective switching capacitance k, noise power δ 2 , the maximum single moving distance of the drone V max , the UAV flight range boundary, the base station location U, and the maximum communication distance R between the UAV and the ground user.

[0016] Optionally, the optimization model for UAV path planning, channel resource allocation, and computing resource allocation is expressed as:

[0017]

[0018] Among them, the intermediate parameter k i,j ,θ i,j The following relations are satisfied:

[0019]

[0020]

[0021] Optionally, converting the optimization model into a Markov decision process includes:

[0022] Let the observation of the j-th UAV be (D1, ..., D N ,f1,...,f N ,x1,...,x N ,y1,...,y M );

[0023] Let the jth UAV action be (α i ,β j ,move j );

[0024] The reward function is set to or

[0025] If the drone exceeds the speed limit or moves out of bounds, the reward will be reduced by 0.3;

[0026] If the task of the ground user is not uninstalled, the reward is reduced by (0.1*the number of mobile terminal users whose tasks are not uninstalled).

[0027] By converting the optimization model into a Markov decision process, the decision maker can periodically or continuously observe the random dynamic system with Markov properties and make corresponding decisions sequentially.

[0028] Optionally, the pre-trained deep reinforcement neural network is trained using a SAC algorithm, and the training process includes:

[0029] Build and initialize the Target V network, Critic V network, Critic Q1 network, and Critic Q2 network of the SAC algorithm architecture;

[0030] The parameters of the Target V network and the Critic V network are initialized the same, and the parameters of the Critic Q1 network and the Critic Q2 network are initialized the same;

[0031] The input of the Target V network and the Critic V network is the observation of any drone, and their output is the state value under the observation. The input of the Critic Q1 network and the Critic Q2 network is the observation of any drone and the actions of all drones, and their output is the value of taking the action under the observation.

[0032] Furthermore, the training process further includes:

[0033] Construct and initialize the Actor network of the SAC algorithm architecture. Each Actor network inputs the observation of the corresponding drone, generates a mean and variance for each element in the action, samples using a normal distribution based on the mean and variance, and processes the obtained data to obtain the action of the drone.

[0034] The observations of different drones are the same at the same moment;

[0035] During training, the Actor network generates an action for each drone in the current observation, then integrates the actions into an action group, and jumps to the next observation after receiving the reward, thereby forming a four-tuple.

[0036] Specifically, the four-tuple is expressed as: (obs z ,a z ,r z,next_obs z ), where the four elements represent observation, action group, reward and next observation respectively, and the four-tuple is stored in the replay buffer;

[0037] If the capacity of the replay buffer reaches a certain value, the latest arriving quadruple replaces the latest arriving quadruple, and Z quadruples are randomly sampled from the replay buffer for training.

[0038] Furthermore, the quadruplets Z randomly sampled from the replay buffer are used to optimize the UAV path planning, channel resource allocation, and computing resource allocation strategies. The optimization process includes:

[0039] The quadruple obtained by sampling (obs z ,a z ,r z ,next_obs z ) updating the Critic V network, performing gradient descent using the loss function of the Actor network, and updating the Critic V network;

[0040] Using the loss functions of the Critic Q1 network and the Critic Q2 network, respectively, the Critic Q1 network and the Critic Q2 network are updated;

[0041] Use the gradient ascent maximization function to update the M Actor networks separately;

[0042] Soft-updating the Target V network using the Critic V network;

[0043] Repeat the above steps several times to reset the environment until the optimal strategy is reached.

[0044] Among them, the loss function of the Actor network is:

[0045]

[0046] The loss functions of the Critic Q1 network and the Critic Q2 network are:

[0047]

[0048] The gradient ascent maximization function is:

[0049]

[0050] Ground base stations are fixed servers with fixed coverage areas. However, terminal devices are often mobile, so they cannot effectively handle computing task offloading while on the move. Meanwhile, drones can move to cover offloading areas, but in addition to the energy consumed by the drones themselves during flight and hovering, task offloading and computing consume a significant amount of energy.

[0051] Therefore, the present invention regards the ground base station as a computing resource and the drone as a relay node to selectively offload the ground user's high-computational complexity tasks to the ground base station. At the same time, the drone uses a pre-trained deep reinforcement neural network to timely optimize the drone's flight trajectory, channel resource allocation and computing resource allocation strategies based on the observed real-time information, thereby minimizing the energy consumed by the drone and ensuring the fairness of the drone's offloaded tasks.

[0052] On the other hand, the present invention also provides a device for multi-UAV assisted mobile edge computing, comprising:

[0053] An information acquisition module is configured to obtain basic element information in a system where multiple UAVs and base stations assist ground users in edge computing;

[0054] An optimization model building module is configured to establish an optimization model for UAV path planning, channel resource allocation, and computing resource allocation based on the basic element information;

[0055] An optimization model conversion module is configured to convert the optimization model into a Markov decision process based on a preset reward function and discount factor, using each UAV as a decision maker, the UAV's observations as states, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions;

[0056] In addition, the training optimization module is configured to use each drone to use a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain optimized drone flight trajectory, channel resource allocation and computing resource allocation strategy.

[0057] The multi-UAV assisted mobile edge computing device can execute the above-mentioned multi-UAV assisted mobile edge computing method to achieve the technical effect corresponding to the multi-UAV assisted mobile edge computing method.

[0058] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) Innovatively using the base station as a computing resource to participate in the computation offloading and task allocation between UAVs and ground users, by effectively utilizing the computing resources and network bandwidth of the base station, it ensures that the tasks can be completed and returned to the users in a timely manner. The coordinated cooperation between the two not only improves the task processing efficiency, but also avoids the disadvantages of concentrated energy consumption and resource waste.

[0061] (2) Based on the SAC algorithm, the present invention not only realizes real-time resource allocation, trajectory design, and intelligent decision-making in the cloud-edge-end architecture, but also effectively copes with complex and changing computing environments. In addition, the algorithm can also be adaptively adjusted dynamically according to the current environment and task requirements to achieve adaptive computing resource allocation and task offloading.

[0062] (3) Through collaborative computing among multiple drones, while assigning tasks to different drones, it is ensured that each drone can obtain reasonable computing tasks; at the same time, by introducing a fairness coefficient, the problem of large differences in the number of users directly served by drones when drones act as relay nodes to offload tasks to base stations is solved, so as to achieve a better balance in the allocation of computing resources among drones and improve the overall task completion efficiency to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the fairness coefficient I curve obtained by training 5000 episodes (100 rounds per episode) when N=24 and M=3;

[0064] Figure 2 Schematic diagram of the logE curve obtained when N=24, M=3 and training for 5000 episodes (100 rounds per episode);

[0065] Figure 3 Schematic diagram of the reward curve obtained when N=24, M=3, and training for 5000 episodes (100 rounds per episode);

[0066] Figure 4 Schematic diagram of the algorithm structure when N=24 and M=3. DETAILED DESCRIPTION

[0067] The technical concept of the present invention is to use the base station as a computing resource to participate in the computing offloading and task allocation between drones and ground users, and to perform computing resource allocation and task offloading between multiple drones and between drones and base stations, thereby improving overall computing efficiency.

[0068] The implementation of the above technical concept requires consideration of the following issues:

[0069] (1) Collaborative computing among multiple drones: When multiple drones participate in computation offloading at the same time, they need to work together to avoid wasting computing resources and conflicts. The algorithm should consider how to assign tasks to different drones and ensure that each drone can get a reasonable computing task.

[0070] (2) Computation offloading between UAVs and base stations: When considering the participation of base stations, the algorithm should consider how to offload computing tasks to the base stations. This requires considering how to effectively utilize the computing resources and network bandwidth of the base stations and ensure that the tasks can be completed and returned to the users in a timely manner.

[0071] (3) Dynamic allocation of computing resources: When the computing resources of the UAV and base station change over time, the algorithm should be able to allocate computing resources and offload tasks in real time. This requires the algorithm to have the ability to dynamically adjust and be able to make adaptive adjustments based on the current environment and task requirements.

[0072] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0073] Example 1

[0074] Combine Figures 1 to 4 This embodiment provides a multi-UAV assisted mobile edge computing method, the specific steps are as follows:

[0075] Obtain basic element information in a system where multiple drones work together with base stations to assist ground users in edge computing;

[0076] Based on the basic element information, an optimization model for UAV path planning, channel resource allocation, and computing resource allocation is established;

[0077] Taking each UAV as a decision maker, the observation of the UAV as the state, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions, the optimization model is converted into a Markov decision process based on a preset reward function and discount factor.

[0078] Each UAV uses a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain the optimized UAV flight trajectory, channel resource allocation and computing resource allocation strategy.

[0079] When applied, this embodiment is optimized based on the SAC algorithm in deep reinforcement learning. A centralized algorithm is first proposed, which achieves good results. On this basis, a centralized training and distributed execution strategy is adopted to further improve the optimization effect. At the same time, the base station is crucially involved in resource allocation as a computing resource, so that the drone and the base station can collaborate with each other to process computing resources, which not only greatly improves efficiency but also avoids concentrated energy consumption and waste of resources.

[0080] Example 2

[0081] Combine Figures 1 to 4 Based on Example 1, this embodiment also has the following design:

[0082] The system for multiple drones and base stations to assist ground users in edge computing includes several drones, several ground users, and base stations, wherein the drones serve as edge computing servers and the base stations serve as ground servers;

[0083] Specifically, the UAV flight area and the distribution area of ​​ground users are limited to a 300×300 square area. The number of ground users is 24, the number of UAVs is 3, and x = (x1,...,x 24 ) represents the position of the ground user, using y=(y1,...,y 24 ) to represent the position coordinates of the drone, move=(move1,...,move 24 ) represents the movement of the UAV, α=(α1,α2,α3) represents the channel resource allocation of the UAV to the ground user, β=(β1,β2,β3) represents the computing resource allocation of the UAV to the ground user, (D i ,f i ,x i ) represent the task data volume, computational complexity and location coordinates of the i-th ground user, E represents energy consumption, (I1, I2, I3) represent the number of ground users served by the UAV, I represents the fairness coefficient, T represents the maximum delay, which is 1, and F uav Indicates the maximum computing resources of the drone, with a value of 3e9, B0 indicates the maximum bandwidth of the drone, with a value of 1e7, P user Indicates the user's information transmission power, with a value of 0.5, P uav Indicates the transmission power of the drone, with a value of 5, T 0i j T represents the communication time for the i-th ground user to offload the task to the j-th UAV. 1i j T represents the computation time of the jth UAV to locally calculate the task of the ith ground user, 2i jrepresents the communication time when the j-th UAV offloads the task of the i-th ground user to the base station, g0 represents the power gain when the reference distance is 1 meter, and its value is 1e-5, k is the capacitance coefficient of the UAV CPU, and its value is 1e-28, δ 2 is the noise power, which is 1e-10, V max The maximum single movement distance of the drone is 20;

[0084] In addition, the users are assumed to be evenly distributed on the diagonal of the square area. The initial positions of the three drones are (0, 0), (120, 120), and (240, 240), respectively. The mission data size of each ground user is {1e5, 2e5, 3e5}, the computational size ranges from {100, 200, 300}, and the maximum communication distance between the drone and the ground user is 200. To simplify the problem, it is assumed that the distance between the drone and the base station is a fixed value of 1000.

[0085] Each UAV is regarded as a decision maker, the observation of the UAV is regarded as the state, the path planning, channel resource allocation and computing resource allocation strategy selected by the UAV are regarded as the action, and based on the preset reward function and discount factor, the optimization model is converted into a Markov decision process.

[0086] Specifically, the optimization model of UAV path planning, channel resource allocation and computing resource allocation is expressed as:

[0087]

[0088] in:

[0089]

[0090]

[0091] Specifically, let the observation of the j-th UAV be (D1, ..., D N ,f1,...,f N ,x1,...,x N ,y1,...,y M ), the j-th UAV action is (α j , β j , move j ), the reward function reward is set to If a drone exceeds the speed limit or moves out of bounds, the reward is reduced by 0.3. If a ground user's tasks are not offloaded, the reward is reduced by 0.1 * the number of ground users whose tasks are not offloaded. The discount factor γ = 0.99.

[0092] Each UAV uses a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain the optimized UAV flight trajectory, channel resource allocation and computing resource allocation strategy.

[0093] Specifically, the four neural networks are initialized as Target V, Critic V, Critic Q1, and Critic Q2, and Target V and Critic V have the same structure and parameter initialization, both are three-layer fully connected networks with 256 neurons; Critic Q1 and Critic Q2 have the same structure and parameter initialization, both are three-layer fully connected networks with 256 neurons.

[0094] Among them, the two V networks input the observation of any drone and output the state value under the observation. The two Q networks input the observation of any drone and the actions of all drones and output the value of taking the action under the observation.

[0095] Initialize an Actor network for each drone, a three-layer fully connected network with 256 neurons and a sigmoid function for output.

[0096] Each Actor network input corresponds to the state of the drone, generates a mean and variance for each element in the action, and then uses the normal distribution to sample based on the mean and variance. The sampled position parameter is first subtracted by 0.5 and then multiplied by 20. All communication resource allocation parameters and all computing resource allocation parameters obtained are processed using the softmax function respectively, and finally the action of the drone in this state is obtained.

[0097] It is worth noting that since the states of different drones are always the same at the same time, the states of different drones will no longer be distinguished below.

[0098] At the same time, the above neural network optimizer is set to Adam and the parameters are all 3e-4, and the soft update coefficient of the Target V network is 0.001.

[0099] Combine Figure 4 During training, the Actor network first generates an action for each drone in the current state, which is then integrated into an action group, rewarded, and jumped to the next observation; we store (observation, action group, reward, next observation) in the replay buffer. When the capacity of the replay buffer reaches 50,000, the latest arriving quadruple (observation, action group, reward, next observation) will replace the latest arriving quadruple, and 128 quadruples will be randomly sampled from the replay buffer for training.

[0100] Among them, Z quadruples are randomly sampled from the replay buffer to optimize the UAV path planning, channel resource allocation, and computing resource allocation strategies. The specific optimization process is as follows:

[0101] First, the quadruple obtained by sampling (obs z , a z , r z , next_obs z ) Update the Critic V network, use the loss function, perform gradient descent, and update the Critic V network;

[0102] Secondly, the two Q networks are updated separately using the loss function;

[0103] Then, the M Actor networks are updated separately using the gradient ascent maximization function;

[0104] Finally, the Target V network is soft-updated using the Critic V network;

[0105] Repeat this 100 times, reset the environment, and repeat the above steps 5000 times.

[0106] At this point, the present invention has obtained a strategy for path planning, communication resource allocation, and computing resource allocation for multi-UAV mobile edge computing in the above environment.

[0107] Among them, the loss function of the Actor network is:

[0108]

[0109] The loss functions of the Critic Q1 network and the Critic Q2 network are:

[0110]

[0111] The gradient ascent maximization function is:

[0112]

[0113] The present invention requires that within a certain period of time, multiple drones and ground base stations are used to achieve task offloading for all ground users within a certain range while meeting the delay requirements; and it is required to ensure fairness when drones serve ground users and minimize energy consumption.

[0114] To solve the above problems, this embodiment optimizes the flight trajectories, wireless channel resource allocation, and computing resource allocation of multiple drones based on the centralized training and distributed execution algorithms of SAC.

[0115] Specifically, the base station is added as a computing resource to the task processing process. At the same time, the drone only needs to obtain the current ground user's location information, task information, and the drone's location information to quickly make path planning and resource allocation decisions, thereby optimizing the final decision-making purpose.

[0116] Example 3

[0117] This embodiment provides a device for multi-UAV assisted mobile edge computing, which can be used to implement the method described in the above embodiment 2, including:

[0118] An information acquisition module is configured to obtain basic element information in a system where multiple UAVs and base stations assist ground users in edge computing;

[0119] An optimization model building module is configured to establish an optimization model for UAV path planning, channel resource allocation, and computing resource allocation based on the basic element information;

[0120] An optimization model conversion module is configured to convert the optimization model into a Markov decision process based on a preset reward function and discount factor, using each UAV as a decision maker, the UAV's observations as states, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions;

[0121] In addition, the training optimization module is configured to use each drone to use a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain optimized drone flight trajectory, channel resource allocation and computing resource allocation strategy.

[0122] Combine Figures 1 to 4 The multi-UAV assisted mobile edge computing device provided in this embodiment can execute the method provided in the above embodiment 2, and at the same time can achieve the corresponding beneficial effects of the multi-UAV assisted mobile edge computing method described in the above embodiment 2.

[0123] Example 4

[0124] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in Example 1 or Example 2 are performed.

[0125] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-UAV assisted mobile edge computing method, characterized in that: include: Obtain basic element information in a system where multiple drones work together with base stations to assist ground users in edge computing; Based on the basic element information, an optimization model for UAV path planning, channel resource allocation, and computing resource allocation is established; Taking each UAV as a decision maker, the observation of the UAV as the state, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions, the optimization model is converted into a Markov decision process based on a preset reward function and discount factor. Each UAV uses a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain the optimized UAV flight trajectory, channel resource allocation, and computing resource allocation strategy; in: The system for multiple drones and base stations to assist ground users in edge computing includes several drones, several ground users, and base stations, wherein the drones serve as edge computing servers and the base stations serve as ground servers. The basic element information includes: the number of ground users N, the number of drones M, the location of the ground users x = (x1, ..., x M ), the position of the drone y=(y1,...,y M ), the movement of the drone move=(move1,...,move M ), the channel resource allocation of UAV to ground users is α=(α1,...,α M ), the computing resource allocation of the UAV to the ground user is β=(β1,...,β M ), the mission data volume, computational workload and location of the i-th ground user (D i ,f i ,x i ), energy consumption E, the number of ground users served by the drone (I1,...,I M ), fairness coefficient I, maximum delay T, user information transmission power P user , the maximum computing resource F of the UAV uav , the maximum bandwidth of the drone B0, the transmission power of the drone P uav , the communication time T for the i-th ground user to offload the task to the j-th UAV 0i j , the computing time T for the jth UAV to locally calculate the task of the i-th ground user 1i j , the communication time T for the jth UAV to offload the task of the i-th ground user to the base station 2i j , power gain g0 at a reference distance of 1 meter, effective switching capacitance k, noise power δ 2 , the maximum single moving distance of the drone V max , the UAV flight range boundary, the base station location U, the maximum communication distance R between the UAV and the ground user; The optimization model of UAV path planning, channel resource allocation and computing resource allocation is expressed as: Among them, the intermediate parameter k i,j ,θ i,j The following relations are satisfied:

2. The multi-UAV assisted mobile edge computing method according to claim 1, characterized in that: The optimization model is converted into a Markov decision process, including: Let the observation of the j-th UAV be (D1, ..., D N ,f1,...,f N , x1, ..., x N ,y1,...,y M ); Let the jth UAV action be (α j , β j , move j ); The reward function is set to or If any of the drones exceeds the speed limit or moves out of bounds, the reward will be reduced by 0.3; If the task of any of the ground users is not offloaded, the reward will be reduced by (0.1*the number of mobile terminal users whose tasks are not offloaded).

3. The multi-UAV assisted mobile edge computing method according to claim 1, characterized in that: The pre-trained deep reinforcement neural network is trained using the SAC algorithm. The training process includes: Build and initialize the Target V network, Critic V network, Critic Q1 network, and Critic Q2 network of the SAC algorithm architecture; The parameters of the Target V network and the Critic V network are initialized the same, and the parameters of the Critic Q1 network and the Critic Q2 network are initialized the same; The input of the Target V network and the Critic V network is the observation of any drone, and its output is the state value under the observation. The input of the Critic Q1 network and the Critic Q2 network is the observation of any drone and the actions of all drones, and its output is the value of taking the action under the observation.

4. The multi-UAV assisted mobile edge computing method according to claim 3, characterized in that: The training process further includes: Construct and initialize the Actor network of the SAC algorithm architecture. Each Actor network inputs the observation of the corresponding drone, generates a mean and variance for each element in the action, samples using a normal distribution based on the mean and variance, and processes the obtained data to obtain the action of the drone. The observations of different drones are the same at the same moment; During training, the Actor network generates an action for each drone in the current observation, then integrates the actions into an action group, and jumps to the next observation after receiving the reward, thereby forming a four-tuple.

5. The multi-UAV assisted mobile edge computing method according to claim 4, characterized in that: The four-tuple is expressed as: (obs z , a z , r z , next_obs z ), where the four elements represent the observation, action group, reward, and next observation, respectively, and the four-tuple is stored in the replay buffer; If the capacity of the replay buffer reaches a certain value, the latest arriving quadruple replaces the latest arriving quadruple, and Z quadruples are randomly sampled from the replay buffer for training.

6. The multi-UAV assisted mobile edge computing method according to claim 5, characterized in that: The Z quadruplets randomly sampled from the replay buffer are used to optimize the UAV path planning, channel resource allocation, and computing resource allocation strategies. The optimization process includes: The quadruple obtained by sampling (obs z , a z , r z , next_obs z ) updating the Critic V network by performing gradient descent using the loss function of the Actor network to update the Critic V network; Using the loss functions of the Critic Q1 network and the Critic Q2 network, respectively, the Critic Q1 network and the Critic Q2 network are updated; Use the gradient ascent maximization function to update the M Actor networks separately; Soft-updating the Target V network using the Critic V network; Repeat the above steps several times to reset the environment until the optimal strategy is reached. Among them, the loss function of the Actor network is: The loss functions of the Critic Q1 network and the Critic Q2 network are: The gradient ascent maximization function is:

7. A device for multi-UAV assisted mobile edge computing using the method according to any one of claims 1 to 6, characterized in that: include: An information acquisition module is configured to obtain basic element information in a system where multiple UAVs and base stations assist ground users in edge computing; An optimization model building module is configured to establish an optimization model for UAV path planning, channel resource allocation, and computing resource allocation based on the basic element information; An optimization model conversion module is configured to convert the optimization model into a Markov decision process based on a preset reward function and discount factor, using each UAV as a decision maker, the UAV's observations as states, and the path planning, channel resource allocation, and computing resource allocation strategies selected by the UAV as actions; In addition, the training optimization module is configured to use each drone to use a pre-trained deep reinforcement neural network to solve the Markov decision process based on real-time observation information to obtain optimized drone flight trajectory, channel resource allocation and computing resource allocation strategy.

8. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are performed.

Images