Resource allocation and task unloading method and system for NOMA-assisted multi-unmanned aerial vehicle edge computing

Abstract

The invention relates to the technical field of unmanned aerial vehicle edge computing, in particular to a resource allocation and task unloading method and system for NOMA-assisted multi-unmanned aerial vehicle edge computing. The method comprises the following steps: deploying an unloading architecture in a NOMA-assisted multi-unmanned aerial vehicle edge calculation scene, defining related parameters of unmanned aerial vehicles and user equipment, clustering the user equipment, and establishing a network communication model; based on a network communication model, establishing a calculation model, determining the trajectory of the unmanned aerial vehicle, and calculating a joint optimization strategy of resource allocation and task unloading proportion; determining an objective function and a constraint condition of an optimization problem according to the system model; an optimization problem is converted into a Markov decision process, states are normalized, an MSAC algorithm is adopted, multiple unmanned aerial vehicles share a neural network, the Markov decision process is solved, and joint optimization of a task unloading proportion, resource allocation and multiple unmanned aerial vehicle trajectories is realized according to a solving result. According to the method, the minimization of the total maximum time delay of all the user equipment in the NOMA-assisted multi-unmanned aerial vehicle edge computing system is realized.

Description

Technical Field

[0001] The present invention relates to the field of drone edge computing technology, and specifically to a NOMA-assisted multi-drone edge computing resource allocation and task offloading method and system. Background Art

[0002] In recent years, the rapid development of IoT applications has created a contradiction between the limited computing power of user devices and the surging demand. In order to solve the challenges brought by computing-intensive tasks, combining drones with mobile edge computing can achieve timely task processing, better adapt to complex task requirements, and provide user devices with more flexible and higher-quality services.

[0003] Edge computing has the advantages of low latency, decentralization, high security and reliability. Combining drones with edge computing technology can be used in smart transportation, environmental monitoring and other fields. The services in these scenarios will generate a large amount of data. Through edge computing technology, the huge computing tasks of user devices can be offloaded to drones equipped with edge servers, alleviating the computing pressure of user devices and reducing latency.

[0004] Most of the existing research on drone edge computing scenarios has adopted the traditional orthogonal multiple access technology (OMA). However, in the face of scarce communication resources, spectrum efficiency remains a challenge. Due to the precious and scarce communication resources, traditional OMA faces the problems of low spectrum efficiency and limited user access. Non-orthogonal multiple access technology (NOMA) achieves efficient resource utilization and improves spectrum efficiency by allowing users to share the same time-frequency and other resource blocks. It can be effectively applied to the massive data access of the future Internet of Things. In addition, in the existing research on multi-drone edge computing scenarios, there is no joint optimization of offloading strategies, resource allocation, and drone 3D flight trajectories, which fails to fully utilize the flexibility of drones. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a resource allocation and task offloading method and system for NOMA-assisted multi-UAV edge computing, which achieves the minimization of the total maximum delay of all user devices in the NOMA-assisted multi-UAV edge computing system.

[0006] The technical solution adopted in the embodiment of the present invention is: a NOMA-assisted multi-UAV edge computing resource allocation and task offloading method, comprising the following steps:

[0007] S10. Deploy the offloading architecture in the NOMA-assisted multi-UAV edge computing scenario, define the parameters related to the UAVs and user devices, cluster the user devices, and establish a network communication model;

[0008] S20. Based on the network communication model, a computing model is established to determine the joint optimization strategy of the UAV trajectory, computing resource allocation and task offloading ratio;

[0009] S30, determining the objective function and constraint conditions of the optimization problem according to the calculation model and the joint optimization strategy;

[0010] S40, converting the optimization problem into a Markov decision process, determining the environment, state, action space and reward of the Markov decision process; normalizing the state, using the MSAC algorithm to solve the Markov decision process, and realizing the joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory according to the solution result;

[0011] The MSAC algorithm is a multi-agent SAC based on the single-agent SAC algorithm; in the MSAC algorithm, before the state is input into the neural network, the state information of each drone is extracted and formed into a standard array, so that multiple drones can share the neural network; the location of the drone that is serving and connected to the neural network and the channel gain of the user equipment it serves, the task size, and the remaining power of the drone are input into the designated neuron.

[0012] The technical solution adopted in another embodiment of the present invention is: a NOMA-assisted multi-UAV edge computing resource allocation and task offloading system, comprising:

[0013] The architecture deployment unit deploys the offload architecture in the NOMA-assisted multi-UAV edge computing scenario, defines the parameters related to the UAVs and user devices, clusters the user devices, and establishes a network communication model;

[0014] A strategy determination unit is used to establish a computing model based on a network communication model to determine a joint optimization strategy for UAV trajectory, computing resource allocation, and task offloading ratio;

[0015] An optimization problem determination unit, used to determine the objective function and constraint conditions of the optimization problem according to the calculation model and the joint optimization strategy;

[0016] The conversion and solution unit is used to convert the optimization problem into a Markov decision process, determine the environment, state, action space and reward of the Markov decision process; normalize the state, use the MSAC algorithm to solve the Markov decision process, and realize the joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory according to the solution result;

[0017] The MSAC algorithm is a multi-agent SAC based on the single-agent SAC algorithm; in the MSAC algorithm, before the state is input into the neural network, the state information of each drone is extracted and formed into a standard array, so that multiple drones can share the neural network; the location of the drone that is serving and connected to the neural network and the channel gain of the user equipment it serves, the task size, and the remaining power of the drone are input into the designated neuron.

[0018] Compared with the prior art, the beneficial effects of the present invention mainly include:

[0019] 1. In the present invention, each drone is equipped with an edge server to provide communication and computing services for user devices; NOMA technology is combined with a multi-drone edge computing system to achieve reasonable clustering of users by considering the load capacity of each drone, and jointly optimize the offloading strategy, computing resource allocation and drone 3D flight trajectory; it provides better support for computing-intensive and delay-sensitive services, making up for the shortcomings of limited communication resources, limited computing power of user devices, or the inability to complete communication and computing services due to damage to ground base stations in natural disasters in traditional OMA scenarios, thereby reducing the total maximum delay of all user tasks in the system.

[0020] 2. The present invention constructs a NOMA-assisted multi-UAV communication computing network system model, defines the relevant parameters of UAVs and user devices, clusters user devices, and establishes a network communication model; based on the network communication model, a computing model is established to determine the joint optimization strategy of UAV trajectory, computing resource allocation, and task offloading; according to the system model, the objective function and constraints of the optimization problem are determined; the optimization problem is converted into a Markov decision process and the MSAC algorithm in multi-agent reinforcement learning is used to jointly optimize the task offloading ratio, resource allocation, and UAV 3D trajectory for solution. By comparing the performance of the MSAC algorithm and the benchmark algorithm, it is concluded that the present invention can effectively reduce the total maximum delay, obtain a suitable offloading strategy, and reasonable resource allocation.

[0021] 3. The present invention is effectively applicable to computing-intensive and delay-sensitive services, making up for the shortcomings of limited communication resources and limited computing power of user equipment in traditional OMA scenarios, or the inability to complete communication computing services due to damage to ground base stations caused by natural disasters. It has practical significance for reducing the total maximum delay of the system in multi-UAV communication scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flowchart of a NOMA-assisted multi-UAV edge computing resource allocation and offloading method provided by an embodiment of the present invention;

[0023] Figure 2It is a schematic diagram of the NOMA-assisted multi-UAV edge computing system architecture environment provided by an embodiment of the present invention;

[0024] Figure 3 It is a schematic diagram of the principle of the proposed MSAC algorithm provided in an embodiment of the present invention and its difference from the SAC algorithm;

[0025] Figure 4 It is a schematic diagram of the structure of the MSAC algorithm for implementing multivariable optimization in a NOMA-assisted multi-UAV edge computing system provided by an embodiment of the present invention;

[0026] Figure 5 is a convergence diagram of the MSAC algorithm provided by an embodiment of the present invention;

[0027] Figure 6 is an example diagram of optimized flight trajectories of multiple UAVs provided in an embodiment of the present invention;

[0028] Figure 7 is a relationship diagram between total delay and computing capability of each user equipment provided by an embodiment of the present invention;

[0029] Figure 8 is a relationship diagram between the total delay and the total computing task scale of each user equipment provided by an embodiment of the present invention;

[0030] Fig. 9 It is a structural diagram of the NOMA-assisted multi-UAV edge computing resource allocation and offloading strategy system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Example 1

[0033] The NOMA-assisted multi-UAV edge computing resource allocation and task offloading method of this embodiment minimizes the total maximum delay of the multi-UAV edge computing network by combining the NOMA technology, provides better support for computing-intensive user services, and makes up for the shortcomings of limited communication resources, limited user equipment computing power, or ground base stations being damaged and unable to complete communication computing services in traditional OMA scenarios. By defining relevant parameters of UAVs and user equipment, user equipment is clustered to establish a network communication model; based on the network communication model, a computing model is established to determine the joint optimization strategy of UAV trajectory, computing resource allocation and task offloading; according to the system model and the optimization strategy, the objective function and constraints of the optimization problem are determined; the optimization problem is converted into a Markov decision process, and according to state normalization, the Multi-agent Soft Actor-Critic (MSAC) algorithm is used to solve the Markov decision process, and the task offloading ratio, resource allocation and multi-UAV trajectory are jointly optimized according to the solution result.

[0034] See also Figure 1 , this embodiment specifically includes the following steps:

[0035] S10. Deploy the offloading architecture in the NOMA-assisted multi-UAV edge computing scenario, define the parameters related to UAVs and user devices, cluster the user devices, and establish a network communication model.

[0036] In this embodiment, the parameters related to the drone and the user equipment include location, mission parameters, computing power, and task offloading ratio.

[0037] The deployed offload architecture is as follows Figure 2 As shown in the figure, there are M drones serving K user devices, and each drone can Indicates that the user device is available The position of UAV m is represented by q m =(x m ,y m ,h m ), the position of user equipment k is denoted as q k =(x k ,y k ,0),x m is the horizontal coordinate of UAV m, y m is the ordinate of the drone m, h m is the flight height of the UAV m, x k is the horizontal coordinate of user device k, y k is the vertical coordinate of user equipment k.

[0038] In this embodiment, the continuous time is divided into N equal-length and small time slots. In the nth time slot, the task size of the user equipment k is D k (n), the ratio of tasks offloaded to the UAV for computation (i.e., offloading ratio) is defined as R k (n), R k (n)∈[0,1].

[0039] Furthermore, when the UAV serves the ground user equipment, the user equipment is clustered according to the number of UAVs and the initial positions of the user equipment. Each cluster includes a UAV and several user equipments. The service indicator β is defined as m,k (n)∈{0,1}, specifically:

[0040]

[0041] In this step, NOMA technology improves channel resource utilization by allowing user devices to share the same time-frequency and other resource blocks. According to the NOMA protocol criteria, the successive interference cancellation (SIC) technology is used at the receiving end to detect the signal. k,l (n)∈{0,1} represents the SIC decoding order of user equipment k and user equipment l, specifically:

[0042]

[0043] Among them, g m,k (n), g m,l (n) denotes the channel gains of user device k and user device l served by drone m, respectively.

[0044] Therefore, at time slot n, the transmission rate available for user device k to the drone m providing computing services to it is r m,k (n) means, specifically:

[0045]

[0046] Where B is the bandwidth allocated to each drone, N0 represents the power spectral density of the additive Gaussian white noise during reception, and P k (n) is the transmission power of user equipment k in time slot n, P l (n) is the transmit power of user equipment l in time slot n.

[0047] S20. Based on the network communication model, a computing model is established to determine the joint optimization strategy of UAV trajectory, computing resource allocation and task offloading ratio.

[0048] In the joint optimization strategy determined in this step, the delay calculated locally by user equipment k in time slot n can be Indicates, specifically:

[0049]

[0050] Among them, f UE is the CPU frequency of each user device, C is the number of CPU cycles required to process each unit bit, and D k (n) is the task size of user equipment k in time slot n, R k (n) is the offloading ratio of user equipment k in time slot n.

[0051] At time slot n, the transmission time required for user device k to upload the offloaded proportion task to drone m is To represent, the computational delay of drone m in processing this part of the upload task can be Indicates, specifically:

[0052]

[0053] Among them, f m,k (n) is the computing resource allocated by drone m to user device k in time slot n. Since the computation result of the edge server is usually very small, the transmission delay back to the user device is negligible.

[0054] During the N time slots of the entire service time, the total energy consumption of UAV m in processing the tasks of the user equipment it serves is It means that the energy consumption of UAV flight is Indicates, specifically:

[0055]

[0056] Among them, k is the CPU capacitance coefficient of each drone, M is the cube of the computing resources allocated by drone m to user device k. UAV is the mass of the drone, v(n) is the average speed of the drone, t fly The flight time of the drone.

[0057] S30. Determine the objective function and constraints of the optimization problem based on the calculation model and the joint optimization strategy.

[0058] The objective function is:

[0059]

[0060] The constraints are:

[0061]

[0062] Among them, q m(n) is the position of UAV m at time slot n, X size is the length of the service area, Y size is the width of the service area, H range is the flight altitude range of the drone; α l,k (n) represents the SIC decoding order of user equipment l and user equipment k; θ(n) is the elevation angle of the UAV flight, is the azimuth of the UAV flight; f UAV is the total computing resources of the UAV; E UAV is the total energy of the drone battery.

[0063] S40. Convert the optimization problem into a Markov decision process and determine the environment, state, action space and reward of the Markov decision process.

[0064] The environment includes drones and user equipment; the state includes the location information of the drones in the environment, the channel gain between the drones and the user equipment, the size of the computing tasks generated by each user equipment, and the remaining power of each drone;

[0065] The action space is determined by the task offloading ratio strategy of each user in each cluster, the computing resource allocation of the edge server carried by the drone, and the flight trajectory of the drone;

[0066] The reward includes reward and penalty terms for performing actions in the target direction.

[0067] The SAC algorithm is an advanced single-agent off-policy algorithm designed for continuous action spaces. It extends the standard actor-critic architecture with soft value functions and entropy maximization, where entropy represents the uncertainty state to promote efficient exploration and stable learning through random action selection.

[0068] In this embodiment, a multi-agent SAC (Multi-agent SoftActor-Critic) algorithm, namely the MSAC algorithm, is formed based on the single-agent SAC algorithm. In order for multiple drones to share a neural network, the state information of each drone must be extracted and formed into a standard array before the state is input into the neural network; the specific differences between the MSAC algorithm principle and the SAC algorithm are as follows: Figure 3As shown, the drone currently connected to the neural network, that is, the drone in service, needs to latch its input neuron. For example, in the MSAC algorithm, when drone 1 is in service and connected to the neural network, the location information of drone 1 needs to be input into the first neuron; when drone 2 is in service, the location information of drone 2 must also be input into the same neuron. In other words, the location of the drone that is in service and connected to the neural network, the channel gain of the user equipment it serves, the task size, and the remaining battery power of the drone must be input into the specified neuron. In the SAC algorithm, it does not need to be input into the specified neuron.

[0069] In this step, the state is normalized, and the Markov decision process is solved using the MSAC algorithm. The joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory is achieved according to the solution results.

[0070] See also Figure 4 In this embodiment, a process of joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory based on MSAC algorithm is provided, including: converting the optimization problem into Markov decision process (MDP), and proposing a method based on MSAC algorithm to achieve joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory to solve the MDP problem; simulating the system environment and verifying the convergence to prove the feasibility and practicality of the algorithm and model.

[0071] Specifically, the MSAC algorithm is used to solve the Markov decision process as follows:

[0072] 1) The Markov decision process MDP is defined as a five-tuple (S, A, P, R, γ), where S is a finite set of system states, A is a finite set of actions, P is the state transition probability, R is the reward function, and γ is a discount factor used to calculate the cumulative reward, γ∈(0,1].

[0073] 2) Deep reinforcement learning is the interaction process between the agent (i.e., drone) and the environment. When the agent performs a task, it interacts with the environment, generates a new state, and receives rewards from the environment. Deep reinforcement learning continuously corrects the action strategy of the agent based on the data generated by the interaction. After multiple iterations, the agent continuously performs actions in the direction of the maximum reward until the task is completed.

[0074] The state includes the location information of the drone in the environment, the channel gain between the drone and the user equipment, the task volume of the user equipment and the remaining power of the drone. The state is defined as:

[0075] s n = {q m (n),qi (n),g m,k (n),g i,l (n),D k (n),D l (n),E m (n),E i (n)},

[0076]

[0077] Among them, m represents the drone m being served, i represents other drones, k represents the user equipment associated with the drone m being served, and l represents the remaining user equipment.

[0078] 3) The action space is mainly determined by the user task offloading ratio, the UAV computing resource allocation, and the UAV flight trajectory. n Defined as:

[0079]

[0080] Among them, R k (n) is the unloading ratio of user equipment k in time slot n, f m,k (n) is the computing resource allocated by drone m to user device k at time slot n, θ(n) represents the elevation angle of the drone flight, Indicates the azimuth of the drone’s flight, 0≤θ(n)≤π,

[0081] 4) In the process of interacting with the environment, the agent will continue to execute the action strategy towards the path with the maximum cumulative reward. In this embodiment, the opposite of the total maximum delay is set as the reward. When the task in this time slot is not completed, a penalty item needs to be set, so the reward function is set as follows:

[0082]

[0083] Among them, λ is the penalty coefficient. When the action performed by the agent cannot complete the calculation task within the current time slot, λ increases from 0, and the reward becomes smaller, which is regarded as a penalty. is the delay calculated locally by user equipment k in time slot n; is the transmission time required for user device k to upload the offloaded ratio task to drone m at time slot n, is the computational delay of UAV m processing the upload task of user device k in time slot n.

[0084] 5) Based on the MSAC algorithm, the task offloading ratio, resource allocation and multi-UAV trajectory are jointly optimized to solve the MDP problem.

[0085] When the intelligent agent (i.e., the drone) starts to serve the user device, it seeks the optimal action based on the current state and takes optimized user device task offloading ratio, computing resource allocation, and drone flight trajectory to provide services to the user device. After the action is executed, the intelligent agent will receive the reward for executing the action and update the state of the drone after executing the action.

[0086] Set the relevant parameters, allocate bandwidth B to each drone = 4MHz, user equipment maximum transmit power P = 20dBm, user equipment and drone CPU frequencies are f UE = 0.8 GHz and f UAV =10GHz, number of time slots N = 60, C = 1000 cycles / bit, CPU capacitance coefficient of drone κ = 10 -28 , UAV mass M UAV =9.65kg, total energy of drone battery E UAV =100kJ. The algorithm was simulated with data, and the experimental environment was implemented by python3.9 and PyTorch 2.0.

[0087] Figure 5 The convergence of the proposed algorithm is demonstrated under the conditions of learning rate lr = 0.000009, discount factor γ = 0.95, batch_size = 128, and soft update coefficient τ = 0.005. It can be seen that in the first 300 trainings, JOORT always maintains a delay of more than 300 seconds, and then gradually decreases. After about 1100 trainings, the delay stabilizes at around 140 seconds, indicating that the algorithm has reached convergence.

[0088] Figure 6 An example of the optimized multi-UAV flight trajectory is given. It can be observed that when the user device moves linearly, each UAV shows a tendency to move upward and approaches the user device it serves in the horizontal direction. After reaching the upper limit of the flight altitude, each UAV adjusts its flight trajectory in the horizontal direction to get closer to the user device it serves.

[0089] Figure 7 The relationship between total delay and the computing power of each user device is shown. As the computing power of the user device increases, the total delay shows a significant downward trend. The proposed MSAC algorithm is also significantly better than other algorithms in terms of delay performance. It can be seen from the figure that even if other variables are optimized but the flight trajectory of the drone is not optimized, that is, a circular trajectory (CircularTrajectory), the performance of the entire system is poor, which is not conducive to delay-sensitive tasks. Therefore, trajectory optimization is an important optimization variable that cannot be ignored.

[0090] Figure 8The relationship between the total latency and the total computational task size for each user device is shown. As the number of tasks increases, the total latency increases accordingly, and the gap between MSAC and other schemes becomes larger and larger. It is clear that MSAC outperforms other schemes and the performance also improves with the increase of optimization variables, especially the multi-UAV 3D flight trajectory as a key optimization factor. When the task size is small, the shortcomings of random user clustering are not obvious. However, as the task size increases, the latency of random user clustering will increase significantly because user devices are assigned to inappropriate UAVs, thereby expanding the transmission and overall latency.

[0091] In summary, the present invention carries the edge server on the drone, and assists the multi-drone edge computing network through NOMA technology. Under this system architecture, user clustering is realized, a computing model is established according to the network model, the joint optimization strategy of drone trajectory, computing resource allocation and task offloading ratio is determined, the objective function and constraints of the optimization problem are determined, and the optimization problem is converted into a Markov decision process. The MSAC algorithm adopted by the present invention is a multi-agent SAC (Multi-agent Soft Actor-Critic) algorithm formed on the basis of the single-agent SAC algorithm, allowing multiple drones to share a neural network. At the same time, the present invention adopts NOMA technology instead of OMA technology to improve spectrum efficiency, and reduces the total maximum delay of all user devices in the multi-drone edge computing network by combining NOMA technology, providing better support for computing-intensive and delay-sensitive services, making up for the shortcomings of limited communication resources, limited computing power of user devices, or ground base stations being damaged and unable to complete communication computing services under natural disasters in traditional OMA scenarios, and has practical significance for reducing the total maximum delay of the system in multi-drone communication scenarios.

[0092] Example 2

[0093] See also Fig. 9 Based on the same inventive concept as Example 1, this embodiment provides a NOMA-assisted multi-UAV edge computing resource allocation and task offloading system, including an architecture deployment unit 01, a strategy determination unit 02, an optimization problem determination unit 03 and a conversion and solution unit 04.

[0094] The architecture deployment unit is used to define the relevant parameters of drones and user devices, cluster user devices, and establish a network communication model in the NOMA-assisted multi-drone edge computing scenario;

[0095] The NOMA-assisted multi-UAV edge computing network architecture includes M UAVs serving K user devices, and each UAV can Indicates that the user device is available The position of UAV m is represented by qm =(x m ,y m ,h m ), the position of user equipment k is denoted as q k =(x k ,y k ,0),x m is the horizontal coordinate of the UAV m, y m is the ordinate of the drone m, h m is the flight height of the UAV m, x k is the horizontal coordinate of user device k, y k is the ordinate of user equipment k. In this embodiment, the continuous time is divided into N equal and small time slots. In the nth time slot, the task size of user equipment k is D k (n), the ratio of tasks offloaded to the UAV for computation (i.e., offloading ratio) is defined as R k (n), R k (n)∈[0,1].

[0096] The user equipment is clustered according to the number of drones and the initial location of the user equipment. Each cluster includes a drone and several user equipments. The service indicator β is defined. m,k (n)∈{0,1}, specifically:

[0097]

[0098] NOMA technology improves channel resource utilization by allowing users to share the same time domain and other resource blocks. According to the NOMA protocol guidelines, the receiver uses the successive interference cancellation (SIC) technology to detect the signal. k,l (n)∈{0,1} represents the SIC decoding order of user equipment k and user equipment l, specifically:

[0099]

[0100] Among them, g m,k (n), g m,l (n) denotes the channel gains of user device k and user device l served by drone m, respectively.

[0101] The strategy determination unit is used to establish a computing model based on the network communication model to determine the joint optimization strategy of UAV trajectory, computing resource allocation and task offloading ratio.

[0102] At time slot n, the transmission rate available for user device k to the drone m providing computing services is r m,k (n) means, specifically:

[0103]

[0104] Where B is the bandwidth allocated to each drone, N0 represents the power spectral density of the additive Gaussian white noise during reception, and P k (n) is the transmission power of user equipment k in time slot n, P l (n) is the transmit power of user equipment l in time slot n.

[0105] In time slot n, the task delay calculated locally by user device k is available Indicates, specifically:

[0106]

[0107] Among them, f UE is the CPU frequency of each, C is the number of CPU cycles required to process each unit byte, and D k (n) is the total task size of user equipment Uk in time slot n, R k (n) is the unloading ratio of user equipment Uk in time slot n.

[0108] At time slot n, user device k uploads the offloaded proportion task to drone m for processing. The transmission time required to upload to drone m is The computational delay required for UAV m to process this part of the task is During the entire service time N time slots, the total energy consumption of UAV m in processing the user equipment tasks it serves and its flight energy consumption can be respectively expressed as and express.

[0109] The optimization problem determination unit is used to determine the objective function and constraint conditions of the optimization problem according to the calculation model and the joint optimization strategy.

[0110] The conversion and solution unit is used to convert the optimization problem into a Markov decision process, normalize the state, solve the Markov decision process using the MSAC algorithm, and generate a strategy method based on the solution result.

[0111] It can be understood that the system provided in this embodiment is used to execute the method provided in the above embodiment and achieve the same technical effect, which will not be further described here.

[0112] The present invention combines NOMA technology to reduce the total maximum delay of all user devices in a multi-UAV edge computing network, providing better support for computing-intensive and delay-sensitive services. It makes up for the shortcomings of limited communication resources and limited computing power of user devices in traditional OMA scenarios, or the inability to complete communication computing services due to damage to ground base stations caused by natural disasters. It has practical significance for reducing the total maximum delay of the system in multi-UAV communication scenarios.

[0113] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A NOMA-assisted multi-UAV edge computing resource allocation and task offloading method, characterized in that: The steps include: S1 0. Deploy the offloading architecture in the NOMA-assisted multi-UAV edge computing scenario, define the parameters related to UAVs and user devices, cluster the user devices, and establish a network communication model; S20. Based on the network communication model, a computing model is established to determine the joint optimization strategy of the UAV trajectory, computing resource allocation and task offloading ratio; S30, determining the objective function and constraint conditions of the optimization problem according to the calculation model and the joint optimization strategy; S40, converting the optimization problem into a Markov decision process, determining the environment, state, action space and reward of the Markov decision process; normalizing the state, using the MSAC algorithm to solve the Markov decision process, and realizing the joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory according to the solution result; The MSAC algorithm is a multi-agent SAC based on the single-agent SAC algorithm. In the MSAC algorithm, before the state is input into the neural network, the state information of each drone is extracted and formed into a standard array, so that multiple drones can share the neural network. The location of the drone being served and connected to the neural network, the channel gain of the user device it serves, the mission size, and the remaining battery power of the drone are input to the specified neuron.

2. The method according to claim 1, characterized in that: In step S10, the parameters related to the drone and the user equipment include location, mission parameters, computing power, and task offloading ratio; The deployed offloading architecture consists of M drones serving K user devices. Indicates that the user device uses The position of UAV m is represented by q m =(x m ,y m ,h m ), the position of user equipment k is denoted as q k =(x k ,y k ,0),x m is the horizontal coordinate of UAV m, y m is the ordinate of the drone m, h m is the flight height of the UAV m, x k is the horizontal coordinate of user device k, y k is the vertical coordinate of user equipment k.

3. The method according to claim 1, characterized in that: In step S10, the continuous time is divided into N time slots of equal length. When the UAV serves the ground user equipment, the user equipment is clustered according to the number of UAVs and the initial positions of the user equipment. Each cluster includes a UAV and several user equipments. The service indicator β is defined as m，k (n)∈{0,1}, specifically: At time slot n, the transmission rate of user device k to drone m providing computing services is r m , k(n) is expressed as: Where B is the bandwidth allocated to each drone, N0 represents the power spectral density of the additive Gaussian white noise during reception, and P k (n) is the transmission power of user equipment k in time slot n; α k,l (n) represents the SIC decoding order of user equipment k and user equipment l, α k,l (n)∈{0,1}; g m,k (n), g m,l (n) represent the channel gains of user equipment k and user equipment l served by drone m respectively; and:

4. The method according to claim 3, characterized in that: In the joint optimization strategy determined in step S20, the delay calculated locally by user equipment k in time slot n is for: Among them, f UE is the CPU frequency of each user device, C is the number of CPU cycles required to process each unit bit, and D k (n) is the task size of user equipment k in time slot n, R k (n) is the offloading ratio of user equipment k in time slot n; At time slot n, the transmission time required for user device k to upload the offloaded ratio task to drone m is The computational delay of drone m processing the upload task of user device k is Among them, f m，k (n) is the computing resource allocated by drone m to user device k at time slot n; In the N time slots of the entire service time, the total energy consumption of UAV m in processing the tasks of the user equipment it serves is The flight energy consumption of UAV m is Among them, K is the CPU capacitance coefficient of each drone, M is the cube of the computing resources allocated by drone m to user device k. UAV is the mass of the drone, v(n) is the average speed of the drone, t fly The flight time of the drone.

5. The method according to claim 1, characterized in that: The objective function determined in step S30 is: The constraints are: in, is the delay calculated locally by user equipment k in time slot n; is the transmission time required for user device k to upload the offloaded ratio task to drone m at time slot n, is the computational delay of drone m processing the upload task of user device k in time slot n; q m (n) is the position of UAV m at time slot n, X size is the length of the service area, Y size is the width of the service area, H range is the flight altitude range of the drone; α k,l (n) represents the SIC decoding order of user equipment k and user equipment l, α l,k (n) represents the SIC decoding order of user equipment l and user equipment k, g m ,k(n),g m , l(n) respectively represent the channel gains of user equipment k and user equipment l served by drone m; θ(n) is the elevation angle of the drone flight, is the azimuth of the UAV flight, R k (n) is the unloading ratio of user equipment k in time slot n; f m，k (n) is the computing resource allocated by drone m to user device k in time slot n, f UAV is the total computing resource of the drone; in the N time slots of the entire service time, the total energy consumption of drone m to process the tasks of the user equipment it serves is The flight energy consumption of UAV m is E UAV is the total energy of the drone battery.

6. The method according to claim 1, characterized in that: In step S40, the environment includes drones and user equipment; the state includes the location information of the drones in the environment, the channel gain between the drones and the user equipment, the size of the computing tasks generated by each user equipment, and the remaining power of each drone; The action space is determined by the task offloading ratio of each user in each cluster, the computing resource allocation of the edge server carried by the drone, and the flight trajectory of the drone; The reward includes reward and penalty terms for performing actions in the target direction.

7. The method according to claim 6, characterized in that: The steps of using the MSAC algorithm to solve the Markov decision process in step S40 include: The Markov decision process MDP is a five-tuple (S, A, P, R, Y), where S is a finite set of states, A is a finite set of actions, P is the state transition probability, and R is the reward function; Y is a discount factor used to calculate the cumulative reward, Y∈(0,1]; When the drone performs a task, it interacts with the environment, generates a new state, and receives rewards from the environment. Deep reinforcement learning continuously corrects the action strategy of the drone based on the data generated by the interaction. After multiple iterations, the drone continues to perform actions in the direction with the maximum reward until the task is completed. The action space a n Defined as: Among them, R k (n) is the task offloading ratio of user equipment k in time slot n, f m，k (n) is the computing resource allocated by drone m to user device k at time slot n, θ(n) represents the elevation angle of the drone flight, Indicates the azimuth of the drone’s flight, 0≤θ(n)≤π, Set the opposite of the total maximum delay as a reward, and set a penalty term when the task in this time slot is not completed; set the reward function to: Among them, λ is the penalty coefficient. When the action performed by the drone cannot complete the calculation task within the current time slot, λ increases from 0, and the reward becomes smaller, which is regarded as a penalty; is the delay calculated locally by user equipment k in time slot n; is the transmission time required for user device k to upload the offloaded ratio task to drone m at time slot n, is the computational delay of drone m processing the upload task of user device k in time slot n; Based on the MSAC algorithm, the task offloading ratio, resource allocation and multi-UAV trajectory are jointly optimized to solve the Markov decision problem. When the UAV starts to serve the user device, it seeks the optimal action according to the current state, and takes the optimized user device task offloading ratio, computing resource allocation and UAV flight trajectory to provide services to the user device. After the action is executed, the UAV will receive the reward for executing the action and update the state of the UAV after executing the action.

8. A NOMA-assisted multi-UAV edge computing resource allocation and task offloading system, characterized in that: include: The architecture deployment unit deploys the offload architecture in the NOMA-assisted multi-UAV edge computing scenario, defines the parameters related to the UAVs and user devices, clusters the user devices, and establishes a network communication model; A strategy determination unit is used to establish a computing model based on a network communication model to determine a joint optimization strategy for UAV trajectory, computing resource allocation, and task offloading ratio; An optimization problem determination unit, used to determine the objective function and constraint conditions of the optimization problem according to the calculation model and the joint optimization strategy; The conversion and solution unit is used to convert the optimization problem into a Markov decision process, determine the environment, state, action space and reward of the Markov decision process; normalize the state, use the MSAC algorithm to solve the Markov decision process, and realize the joint optimization of task offloading ratio, resource allocation and multi-UAV trajectory according to the solution result; The MSAC algorithm is a multi-agent SAC based on the single-agent SAC algorithm. In the MSAC algorithm, before the state is input into the neural network, the state information of each drone is extracted and formed into a standard array, so that multiple drones can share the neural network. The location of the drone being served and connected to the neural network, the channel gain of the user device it serves, the mission size, and the remaining battery power of the drone are input to the specified neuron.

9. The system according to claim 8, characterized in that: The objective function determined by the optimization problem determination unit is: The constraints are: in, is the delay calculated locally by user equipment k in time slot n; is the transmission time required for user device k to upload the offloaded ratio task to drone m at time slot n, is the computational delay of drone m processing the upload task of user device k in time slot n; q m (n) is the position of UAV m at time slot n, X size is the length of the service area, Y size is the width of the service area, H range is the flight altitude range of the drone; α k,l (n) represents the SIC decoding order of user equipment k and user equipment l, α l,k (n) represents the SIC decoding order of user equipment l and user equipment k, g m ,k(n),g m , l(n) respectively represent the channel gains of user equipment k and user equipment l served by drone m; θ(n) is the elevation angle of the drone flight, is the azimuth of the UAV flight, R k (n) is the unloading ratio of user equipment k in time slot n; f m，k (n) is the computing resource allocated by drone m to user device k in time slot n, f UAV is the total computing resource of the drone; in the N time slots of the entire service time, the total energy consumption of drone m to process the tasks of the user equipment it serves is The flight energy consumption of UAV m is E UAV is the total energy of the drone battery.

10. The system according to claim 8, characterized in that: In the conversion and solution unit, the environment includes drones and user equipment; the state includes the location information of the drones in the environment, the channel gain between the drones and the user equipment, the size of the computing tasks generated by each user equipment, and the remaining power of each drone; The action space is determined by the task offloading ratio strategy of each user in each cluster, the computing resource allocation of the edge server carried by the drone, and the flight trajectory of the drone; The reward includes reward and penalty terms for performing actions in the target direction.

Images