Dynamic task unloading method based on non-orthogonal multiple access and unmanned aerial vehicle assistance

Abstract

The invention belongs to the technical field of resource optimization allocation, and particularly relates to a dynamic task unloading method based on non-orthogonal multiple access and unmanned aerial vehicle assistance, which specifically comprises the following steps: S1, constructing an unmanned aerial vehicle assisted edge computing framework combined with non-orthogonal multiple access, the framework comprising an unmanned aerial vehicle base station, a ground base station and intelligent mobile equipment; in the edge calculation framework, the intelligent mobile device unloads a task to be executed on a local side and an unmanned aerial vehicle side, each unmanned aerial vehicle dynamically adjusts the trajectory according to the moving position of the device, and an unmanned aerial vehicle auxiliary edge calculation mathematical model combined with non-orthogonal multiple access is established; and S2, constructing a dynamic task unloading problem based on a non-orthogonal multiple access unmanned aerial vehicle auxiliary edge calculation framework and a mathematical model. The method can be applied to an Internet of Things system, the problem of computing resource shortage in an intensive task scene is effectively relieved, and the defect that a traditional ground cellular network is difficult to meet real-time response and wide-area coverage of a user is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of resource optimization allocation, and in particular to a dynamic task offloading method based on non-orthogonal multiple access and unmanned aerial vehicle assistance. Background Art

[0002] With the rapid development of the Internet of Things and B5G / 6G wireless technologies, the number and complexity of smart applications are increasing rapidly. Faced with these applications with high computing demands and low latency requirements, traditional smart mobile devices are difficult to meet user expectations due to the limitations of computing power and battery life. To meet this challenge, mobile edge computing has become an effective solution. By offloading computing tasks to edge servers, the energy consumption and latency of smart devices can be significantly reduced. At the same time, drones, as flexible nodes of mobile edge computing, can be quickly deployed to the required areas to provide a wide range of computing services. Therefore, we propose a dynamic task offloading method based on non-orthogonal multiple access and drone assistance to optimize resource utilization and improve computing efficiency and service quality.

[0003] The Chinese patent publication number is "CN114070451B", and the name is "UAV mobile edge computing secure communication method based on non-orthogonal multiple access". The method provides a UAV mobile edge computing secure communication method based on non-orthogonal multiple access that can effectively resist UAV eavesdropping on user unloading information. Through the continuous convex optimization method and the block coordinate descent method, the above problem is converted into two sub-problems. First, the relationship between the UAV US and the user's changing channel, the CPU computing frequency assigned to the user, the user's transmission power, and the user's local computing data volume are optimized, and then the UAV US flight trajectory is optimized. The convex problem is converted into a convex optimization problem through the Taylor first-order expansion method, and then iterative solution is performed to obtain the optimal solution of the problem. However, the designed UAV mobile edge computing model based on non-orthogonal multiple access is too simple, and does not take into account the movement of UAVs and ground users in a real environment, resulting in a significant reduction in the system's computing efficiency and resource allocation effect in a complex dynamic environment. Therefore, designing a complete computing model that can simulate the movement of UAVs and ground users in a real environment to improve the system performance in multi-user and multi-task scenarios is an important problem that the present invention needs to solve. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the shortcomings of the prior art, the present invention provides a dynamic task offloading method based on non-orthogonal multiple access and UAV assistance, which solves the problem that the existing UAV mobile edge computing model is too single and does not consider the movement of UAVs and ground users in the real environment, resulting in low computing efficiency and poor resource allocation in the complex dynamic environment. By optimizing the delay, the overall performance of the system in multi-user and multi-task scenarios is improved.

[0006] (II) Technical solution

[0007] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0008] The dynamic task offloading method based on non-orthogonal multiple access and drone assistance specifically includes the following steps:

[0009] S1, construct a drone-assisted edge computing framework combined with non-orthogonal multiple access, which includes a drone base station, a ground base station and an intelligent mobile device. In this edge computing framework, the intelligent mobile device offloads the tasks to be executed on the local side and the drone side respectively, and each drone dynamically adjusts the trajectory according to the mobile position of the device, and establishes a mathematical model of drone-assisted edge computing combined with non-orthogonal multiple access;

[0010] S2, a UAV-assisted edge computing framework and mathematical model based on non-orthogonal multiple access is constructed to solve the dynamic task offloading problem;

[0011] S3, based on the dynamic task offloading problem, is transformed into the first optimization of the offloading strategy, the second optimization of the selection of drones for smart mobile devices, and the third optimization of the flight trajectory and power control of drones. A dynamic task offloading method is proposed, and the steps are as follows:

[0012] S31, solving the first optimization using the increase and decrease characteristics;

[0013] S32, using the improved K-Means algorithm to solve the second optimization;

[0014] S33, use the multi-agent dual-delay deep deterministic policy gradient algorithm to solve the third optimization.

[0015] Furthermore, in S1, the drone-assisted edge computing framework combined with non-orthogonal multiple access includes a dynamic drone base station, a ground base station and an intelligent mobile device, and the established mathematical model is:

[0016] S11, the data transmission rate and offloading strategy of the smart mobile device k connected by drone u are and

[0017] S12, the transmission delay and energy consumption of the smart mobile device k connected by drone u are and When the smart mobile device k partially offloads the computing task to the drone u, the local computing delay and energy consumption of the smart mobile device k are and The computational delay and energy consumption of the computing task of the smart mobile device at the drone u are and

[0018] S13, the time T required for the u-th drone to complete the tasks of all connected smart mobile devices in time slot t u,fly (t), let P u,fly (t) is the UAV propulsion power. Therefore, the flight energy consumption of UAV u at time slot t is E u,fly (t).

[0019] Furthermore, in S2, based on the non-orthogonal multiple access UAV-assisted edge computing framework and mathematical model, the dynamic task offloading problem is constructed as follows:

[0020] S21, the total system delay of drone u and smart mobile device k in time slot t is given by formula (1):

[0021]

[0022] Therefore, the dynamic task offloading optimization problem is expressed as (2):

[0023]

[0024]

[0025] Among them, (3)-(5) represent the constraints on the three-dimensional position of the UAV, that is, the UAV must be located in the airspace above the service area within the achievable altitude range to avoid collisions between UAVs or interference with other communication devices outside the offloading cell; constraint (6) is the optional range of the offloading strategy; (7) and (8) ensure that the smart mobile device can only be associated with one of the UAV servers; constraint (9) represents the transmission power constraint to ensure that the power consumption of each UAV does not exceed the upper limit of the transmission power; constraint (10) represents the decoding order for successful serial interference cancellation; constraint (11) indicates that the computing and flight energy consumption of the UAV does not exceed the total energy of the battery.

[0026] Furthermore, in S3, based on the dynamic task offloading problem, it is converted into a first optimization of the offloading strategy, a second optimization of the intelligent mobile device selecting the drone, and a third optimization of the flight trajectory and power control of the drone, to construct a dynamic task offloading method.

[0027] In the above S31, the step of solving the first optimization using the increase and decrease characteristics of the function is:

[0028] S311,H and P in a given time slot t,selectivity of fixed smart mobile device to UAV,T π(k) (t) can achieve the minimum value.

[0029] In the above S32, the steps of solving the second optimization using the improved K-Means algorithm are:

[0030] S321, initialize the location information of the drone and the smart mobile device, the maximum number of iterations N, and the maximum flight load η of the drone.

[0031] S322, initializing the location set of the smart mobile device Randomly select an initial center point (μ1...μ U ).

[0032] S323, calculate the distance d between the center point and the position ku =‖l k -μ u ‖.

[0033] S324, take the minimum d ku The corresponding l k Assign the corresponding cluster partition C u .

[0034] S325, according to Update μ u .

[0035] S326, repeat steps S323-S325 until μ is satisfied u (n) = μ u (n-1) or the maximum number of iterations N, all cluster centers are no longer changing or the maximum number of iterations is reached.

[0036] S327, for each cluster group C i , check whether its capacity exceeds the threshold value n. If it exceeds the threshold value, repeat steps S328-S3210 until the capacity of all cluster groups does not exceed the threshold value n. If it does not exceed the threshold value, execute step S3211.

[0037] S328, from group C i Select the smart mobile device location l that is farthest from the cluster center k .

[0038] S329, the selected smart mobile device location l k From Group C i Removed.

[0039] S3210, the location of the smart mobile device to be removed k Reclassify to the nearest new center point to ensure that the cluster group capacity is adjusted.

[0040] S3211, output the final clustering result (C1, C2...C U ).

[0041] In the above S33, the step of solving the third optimization using the multi-agent double-delay deep deterministic policy gradient algorithm is:

[0042] S331, establishing the third optimization problem as a Markov process.

[0043] S3311, establish state space: Since each drone agent can only observe a part of the entire environment and there is no information exchange between them, the observation results of drone u can be described by formula (12):

[0044] s u (t) = {u u (t),f u (t),z u ,D π(k) (t),δ π(k) (t),id π(k) ,pos π(k) ,E r (t)} (12)

[0045] Among them, u u (t) represents the position of the UAV u, f u (t) represents the available computing power of UAV u. In addition, D π(k) (t),δ π(k) (t) represents the detailed mission information generated by the smart mobile device k serving the drone u, id π(k) and pos π(k) denote the index and position of the smart mobile device k, respectively, and E r (t) represents the remaining battery power of the drone.

[0046] S3312, establish action space: each agent determines its trajectory and resource allocation strategy based on the current observation results. The action of drone u in time slot t can be expressed by formula (13):

[0047]

[0048] in, represents the computing resources allocated by drone u to smart mobile device k, x u (t),y u (t),h u(t) represents the location information of the drone.

[0049] S3313, establish reward value: drones that do not meet (3)-(5) will be subject to a constant penalty η1, drones that do not meet (7) and (8) will be subject to a penalty constant η2, and drones that do not meet (10) will be subject to a penalty of η3. The reward of drone u is given by formula (14):

[0050]

[0051] Wherein, Λ(*) means that if the condition (*) is met, then Λ(*)=1, otherwise it is 0.

[0052] S332, solving the third optimization based on the multi-agent double-delay deep deterministic policy gradient algorithm.

[0053] S3321, initialize the parameters of the original action network and the evaluation network, set the initial parameters of the original action network and the evaluation network, including the optimal unloading strategy Ψ. Initialize the experience replay pool e, which is used to store the experience data during the training process.

[0054] S3322, the original action network parameter u m Copy to the target network, and copy the evaluation network parameters w to the target network.

[0055] S3324, set the initial position of the drone and smart mobile device, calculate the channel gain of each drone, and observe the current state o m , execute the joint strategy of flight trajectory and power control, and observe the next state o' m and the instantaneous reward r of environmental feedback.

[0056] S3325, the drone will collect the experience tuple e = {o1,...,o M ,a1,...,a M ,r,o′1,...,o′ M}Stored in the experience pool. And check whether the experience replay pool meets the update threshold. If it does not exceed the threshold, repeat step S3325 until all experience replay pools exceed the threshold. If it exceeds the threshold, execute steps S3326-S3328.

[0057] S3326, randomly sample a small batch of experiences from the experience pool e.

[0058] S3327, update the original network parameters, update the parameters of the evaluation network and the action network according to the stochastic gradient descent rule, and soft-update the target network parameters.

[0059] S3328, the UAV and the intelligent mobile device perform corresponding movement operations according to the calculated trajectory and power strategy until the optimal trajectory and power strategy are obtained.

[0060] (III) Beneficial effects

[0061] Compared with the prior art, the present invention provides a dynamic task offloading method based on non-orthogonal multiple access and drone assistance, which has the following beneficial effects:

[0062] (1) Improve computing resource utilization and system performance: This invention adopts a UAV-assisted edge computing framework based on non-orthogonal multiple access, which can effectively alleviate the problem of tight computing resources in intensive task scenarios. The collaborative work of the UAV base station and the ground base station improves the flexibility and efficiency of task offloading, thereby optimizing the configuration and utilization of computing resources and significantly improving the overall performance of the system.

[0063] (2) Enhanced dynamic adaptability and coverage capability: The dynamic task offloading method proposed in the present invention can adjust the flight trajectory and power control strategy of the drone in real time according to the location and task requirements of the smart mobile device, so that the system can dynamically adapt to different mission scenarios. This method effectively makes up for the shortcomings of traditional ground cellular networks in wide-area coverage and real-time response capabilities.

[0064] (3) Improving the decision accuracy of task offloading: The present invention improves the accuracy of task offloading decisions by converting the dynamic task offloading problem into three optimization decisions, namely, offloading strategy optimization, intelligent mobile device selection optimization, and UAV trajectory and power control optimization. This method helps to achieve optimal collaboration between UAVs and ground base stations and ensures efficient processing of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flowchart of a dynamic task offloading method based on non-orthogonal multiple access and UAV assistance;

[0066] Figure 2 This is a schematic diagram of the structure of drone-assisted edge computing based on non-orthogonal multiple access of the present invention;

[0067] Figure 3 This is a schematic diagram of the multi-agent dual-delay deep deterministic policy gradient method of the present invention;

[0068] Figure 4 This is a qualitative comparison effect diagram between the dynamic task offloading method of the present invention and the existing method. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] Example

[0071] In one embodiment, Figure 1 As shown, the dynamic task offloading method based on non-orthogonal multiple access and drone assistance specifically includes the following steps:

[0072] S1, builds a drone-assisted edge computing framework combined with non-orthogonal multiple access, which includes drone base stations, ground base stations and smart mobile devices. The structure of the framework is as follows Figure 2 In this edge computing framework, the intelligent mobile device offloads the tasks to be executed on the local side and the drone side respectively. Each drone dynamically adjusts its trajectory according to the mobile position of the device, and establishes a mathematical model of drone-assisted edge computing combined with non-orthogonal multiple access:

[0073] The set of intelligent mobile devices served by ground base stations is represented by m∈M={1,2,3...M}, and the set of intelligent mobile devices served by drones is represented by Indicates that the same device can only select one base station, so there is The smart mobile devices served by drones are divided into U cells, where The total service time T of the drone is divided into T / T r re-clustering time periods, denoted as {1,...,t,...,T}.

[0074] Smart mobile devices use random roaming models and directional walking models. The maximum moving speed of the user is V max , represents the position of the smart mobile device at time slot t; for the drone mobility model, {x u (t),y u (t),h u (t)} is the position of the UAV at time slot t, and the three-dimensional distance between the UAV u and the smart mobile device k at time t is expressed as

[0075] is whether the drone u serves the smart mobile device k, Indicates that drone u is serving smart mobile device k, otherwise Combined with non-orthogonal multiple access technology, for K uA smart mobile device calculates the decoding order according to the equivalent channel gain: Where π(k) represents the decoding order of smart mobile device k.

[0076] make represents the power allocated by drone u to smart mobile device k, is the channel gain from UAV u to smart mobile device k in time slot t, represents additive white Gaussian noise, and the signal to interference noise ratio of the kth decoding user is obtained by (1).

[0077]

[0078] in, represents the channel gain between the drone s≠u and the smart mobile device k, P s (t) represents the total power consumption of drone s≠u. Therefore, the data transmission rate of the smart mobile device k connected to drone u can be calculated as:

[0079]

[0080] Where B represents the bandwidth of UAV u.

[0081] The offloading strategy of smart mobile device k for drone u service is expressed by formula (3).

[0082]

[0083] The task available for smart mobile device k in time slot t (D π(k) (t),M) represents. Among them, D π(k) (t) represents the computing task size of the smart mobile device (unit: bit), M represents the task computing intensity, representing the computing resources required for one bit. Then the transmission delay and energy consumption of the smart mobile device k connected to the drone u are:

[0084]

[0085] When the smart mobile device k partially offloads the computing task to the drone u, the local computing delay of the smart mobile device k is:

[0086]

[0087] where f π(k) is the local computing resource of the smart mobile device k. The computing delay of the computing task of the smart mobile device at the drone u is:

[0088]

[0089] in, The computing resources allocated by drone u to the kth smart mobile device.

[0090] Let K a and K b are the impact factors of the chip structures of smart mobile devices and drones on CPU processing, and the computing energy consumption of smart mobile devices and drones are expressed by equations (8) and (9), respectively.

[0091]

[0092] To avoid wasting resources, assume that tasks are queued on the drone side. That is, when the first task is offloaded to the u-th drone, the drone uses all computing resources to process the task until it is completed, and then processes the next task. Finally, the larger value between the maximum time for processing tasks on the smart mobile device side and the maximum time for transmitting tasks plus the total time for processing tasks on the drone side is taken as the time T required for the u-th drone to complete all connected smart mobile device tasks in time slot t. u ,fly (t), expressed by formula (10).

[0093]

[0094] Let P u,fly (t) is the UAV propulsion power, so the flight energy consumption of UAV u in time slot t is:

[0095] E u,fly (t) = P u,fly (t)T u,fly (t) (11)

[0096] S2, a UAV-assisted edge computing framework and mathematical model based on non-orthogonal multiple access is constructed to solve the dynamic task offloading problem;

[0097] The unloading strategy, the selection of UAVs by smart mobile devices, the UAV trajectories, and the power allocation strategy are optimized under the constraints of maximum power, unloading, space, and maximum battery total energy. The total system delay of UAV u and smart mobile device k in time slot t is given by equation (12):

[0098]

[0099] The transmission power of each UAV is P u Indicates that the power allocation strategy is express. Indicates the position of the drone within the service time 0≤t≤T. Selection of drones by smart mobile devices Used to represent user association. Finally, the uninstall policy is used Therefore, the optimization problem is expressed as (13):

[0100]

[0101]

[0102] Among them, (14)-(16) represent the constraints on the three-dimensional position of the UAV, that is, the UAV must be located in the airspace above the service area within the achievable altitude range to avoid collisions between UAVs or interference with other communication devices outside the offloading cell; constraint (17) is the optional range of the offloading strategy; (18) and (19) ensure that the smart mobile device can only be associated with one of the UAV servers; constraint (20) represents the transmission power constraint to ensure that the power consumption of each UAV does not exceed the upper limit of the transmission power; constraint (21) represents the decoding order for successful serial interference elimination; constraint (22) indicates that the computing and flight energy consumption of the UAV does not exceed the total energy of the battery.

[0103] S3, based on the dynamic task offloading problem, is transformed into the first optimization of the offloading strategy, the second optimization of the selection of drones for smart mobile devices, and the third optimization of the flight trajectory and power control of drones. A dynamic task offloading method is proposed, and the steps are as follows:

[0104] S31, solving the first optimization using the increase and decrease characteristics;

[0105] In a given time slot t, H and P are fixed, and the selectivity of the smart mobile device to the drone is set. The delay problem (13) is simplified to:

[0106]

[0107] in, It is the transmission delay, the delay on the smart mobile device side, and the delay coefficient on the drone side.

[0108] T π(k) (t) is a linear function of Ψ, where and T π(k) (t) is an increasing function and a decreasing function. Therefore, in order to make T π(k) (t) is the smallest, we only need to satisfy equation (24).

[0109]

[0110] when When T π(k) (t) can obtain the minimum value. Therefore, at time slot t, the optimal unloading strategy of the smart mobile device k connected to the drone u is expressed by formula (25):

[0111]

[0112] S32, using the improved K-Means algorithm to solve the second optimization;

[0113] Initialize the location information of the drone and smart mobile device, the maximum number of iterations N, the maximum flight load of the drone η, and the location set of the smart mobile device And randomly select the initial center point (μ1...μ U ).

[0114] Repeatedly calculate the distance d between the center point and the position ku =‖l k -μ u ‖, take the minimum d ku The corresponding l k Assign the corresponding cluster partition C u , and according to Update μ u Until μ is satisfied u (n) = μ u (n-1) or the maximum number of iterations N, all cluster centers are no longer changing or the maximum number of iterations is reached.

[0115] For each cluster group C i , check whether its capacity exceeds the threshold η. If it exceeds the threshold, i Select the smart mobile device location l that is farthest from the cluster center k , the selected smart mobile device location l k From Group C i Remove it and set the location of the removed smart mobile device to l k Reclassify to the nearest new center point until the capacity of all cluster groups does not exceed the threshold η. If it does not exceed the threshold, output the final clustering result (C1, C2...C U ).

[0116] S33, using a multi-agent double-delay deep deterministic policy gradient algorithm to solve the third optimization problem. The third optimization problem is established as a Markov process.

[0117] First, establish the state space: Since each drone agent can only observe a part of the entire environment and there is no information exchange between them, the observation results of drone u can be described by formula (26):

[0118] s u (t) = {u u (t),f u (t),z u ,D π(k) (t),δ π(k) (t),idπ(k) ,pos π(k) ,E r (t)} (26)

[0119] Among them, u u (t) represents the position of the UAV u, f u (t) represents the available computing power of UAV u. In addition, D π(k) (t),δ π(k) (t) represents the detailed mission information generated by the smart mobile device k serving the drone u, id π(k) and pos π(k) denote the index and position of the smart mobile device k, respectively, and E r (t) represents the remaining battery power of the drone.

[0120] Next, we establish the action space: each agent determines its trajectory and resource allocation strategy based on the current observation results. The action of drone u at time slot t can be expressed by formula (27):

[0121]

[0122] in, represents the computing resources allocated by drone u to smart mobile device k, x u (t),y u (t),h u (t) represents the location information of the drone.

[0123] Finally, the reward value is established: In order to minimize the average processing delay, the reward function enables the multi-UAV server to provide services to the smart mobile device under certain conditions. If the constraints are violated, the corresponding UAV will be punished by the environment. Specifically, if the constraints (14)-(16) are not met, the corresponding UAV will be subject to a constant penalty η1. It should be noted that if two UAVs provide associated services to the same smart mobile device at the same time or a smart mobile device is not served by any UAV, that is, the constraints (18) and (19) are not met, the relevant UAV will be subject to a penalty constant η2. In addition, if serial interference cancellation is not successfully achieved, the UAV covering the smart mobile device will be penalized by η3. If the total energy consumption does not exceed the total energy of the battery, the task is considered to have been successfully completed. Therefore, based on the above factors, the reward of UAV u can be given by formula (28):

[0124]

[0125] Wherein, Λ(*) means that if the condition (*) is met, then Λ(*)=1, otherwise it is 0.

[0126] like Figure 3As shown, the third optimization is solved based on the multi-agent double-delay deep deterministic policy gradient algorithm.

[0127] Initialize the parameters of the original action network and the evaluation network, set the initial parameters of the original action network and the evaluation network, including the optimal unloading strategy Ψ. Initialize the experience replay pool e to store the experience data during the training process.

[0128] The original action network parameters u m Copy to the target network, and copy the evaluation network parameters w to the target network.

[0129] Set the initial positions of the drone and smart mobile device, calculate the channel gain of each drone, and observe the current state o m , execute the joint strategy of flight trajectory and power control, and observe the next state o' m and the instantaneous reward r of environmental feedback.

[0130] The drone will collect the experience tuple e = {o1,...,o M ,a1,...,a M ,r,o′1,...,o′ M}Stored in the experience pool. And check whether the experience replay pool meets the update threshold. If it does not exceed the threshold, continue to collect experience until all experience replay pools exceed the threshold. If it exceeds the threshold, randomly sample a small batch of experience in the experience pool e. Update the original network parameters, update the parameters of the evaluation network and action network according to the stochastic gradient descent rule, and soft-update the target network parameters. Drones and smart mobile devices perform corresponding mobile operations according to the calculated trajectory and power strategy until the optimal trajectory and power strategy are obtained.

[0131] Figure 4 The comparison of the offloading performance of the prior art and the method proposed in the present invention is shown. All local computing method: This method enables each smart mobile device to perform tasks locally; All offloading computing method: This method completely offloads the entire computing task to the edge layer, and all tasks of all smart mobile devices are offloaded to their corresponding associated base stations for execution; Deep deterministic policy gradient method: A deep deterministic policy gradient is used to solve the third optimization of the present invention.

[0132] Compared with other existing methods, the method proposed in the present invention significantly reduces system latency. Under the condition that the task size is 60MB, the method proposed in the present invention reduces the latency by about 7% compared with the deep deterministic policy gradient method. This significant performance improvement not only proves the advantage of the method of the present invention in reducing system latency, but also means that the invention can manage resources more effectively, optimize task execution strategies, thereby accelerating the task processing process and improving overall system efficiency.

[0133] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dynamic task offloading method based on non-orthogonal multiple access and drone assistance, characterized in that: The method specifically comprises the following steps: S1, construct a drone-assisted edge computing framework combined with non-orthogonal multiple access, which includes a drone base station, a ground base station and an intelligent mobile device; in the edge computing framework, the intelligent mobile device unloads the tasks to be executed on the local side and the drone side respectively, and each drone dynamically adjusts the trajectory according to the mobile position of the device, and establishes a drone-assisted edge computing mathematical model combined with non-orthogonal multiple access; S2, a UAV-assisted edge computing framework and mathematical model based on non-orthogonal multiple access is constructed to solve the dynamic task offloading problem; S3, based on the dynamic task offloading problem, is transformed into the first optimization of the offloading strategy, the second optimization of the selection of drones for smart mobile devices, and the third optimization of the flight trajectory and power control of drones. A dynamic task offloading method is proposed, and the steps are as follows: S31, solving the first optimization using the increase and decrease characteristics; S32, using the improved K-Means algorithm to solve the second optimization; S33, use the multi-agent dual-delay deep deterministic policy gradient algorithm to solve the third optimization.

2. The method for dynamic task offloading based on non-orthogonal multiple access and drone assistance according to claim 1 is characterized in that: In S1, the drone-assisted edge computing framework combined with non-orthogonal multiple access includes a dynamic drone base station, a ground base station and an intelligent mobile device, and the established mathematical model is: S11, the data transmission rate and offloading strategy of the smart mobile device k connected by drone u are and S12, the transmission delay and energy consumption of the smart mobile device k connected by drone u are and When the smart mobile device k partially offloads the computing task to the drone u, the local computing delay and energy consumption of the smart mobile device k are and The computational delay and energy consumption of the computing task of the smart mobile device at the drone u are and S13, the time T required for the u-th drone to complete the tasks of all connected smart mobile devices in time slot t u,fly (t), let P u,fly (t) is the propulsion power of the UAV. Therefore, the flight energy consumption of UAV u at time slot t is E u,fly (t).

3. The method for dynamic task offloading based on non-orthogonal multiple access and drone assistance according to claim 1 is characterized in that: In S2, based on the non-orthogonal multiple access UAV-assisted edge computing framework and mathematical model, the dynamic task offloading problem is constructed as follows: S21, the total system delay of drone u and smart mobile device k in time slot t is given by formula (1): Therefore, the dynamic task offloading optimization problem is expressed as (2): Among them, (3)-(5) represent the constraints on the three-dimensional position of the UAV, that is, the UAV must be located in the airspace above the service area within the achievable altitude range to avoid collisions between UAVs or interference with other communication devices outside the offloading cell; constraint (6) is the optional range of the offloading strategy; (7) and (8) ensure that the smart mobile device can only be associated with one of the UAV servers; constraint (9) represents the transmission power constraint to ensure that the power consumption of each UAV does not exceed the upper limit of the transmission power; constraint (10) represents the decoding order for successful serial interference cancellation; constraint (11) indicates that the computing and flight energy consumption of the UAV does not exceed the total energy of the battery.

4. The method for dynamic task offloading based on non-orthogonal multiple access and drone assistance according to claim 1 is characterized in that: In S3, based on the dynamic task offloading problem, the problem is converted into a first optimization of the offloading strategy, a second optimization of the intelligent mobile device selecting the drone, and a third optimization of the flight trajectory and power control of the drone, thereby constructing a dynamic task offloading method; In the above S31, the step of solving the first optimization using the increase and decrease characteristics of the function is: S311,H and P in a given time slot t,selectivity of fixed smart mobile device to UAV,T π(k) (t) can obtain the minimum value; In the above S32, the steps of solving the second optimization using the improved K-Means algorithm are: S321, initializing the location information of the drone and the smart mobile device, the maximum number of iterations N, and the maximum flight load η of the drone; S322, initializing the location set of the smart mobile device Randomly select an initial center point (μ1...μ U ); S323, calculate the distance d between the center point and the position ku =‖l k -μ u ‖; S324, take the minimum d ku The corresponding l k Assign the corresponding cluster partition C u ; S325, according to Update μ u ; S326, repeat steps S323-S325 until μ is satisfied u (n) = μ u (n-1) or the maximum number of iterations N, all cluster centers are no longer changing or the maximum number of iterations is reached; S327, for each cluster group C i , check whether its capacity exceeds the threshold value η. If it exceeds the threshold value, repeat steps S328-S3210 until the capacity of all cluster groups does not exceed the threshold value η. If it does not exceed the threshold value, execute step S3211; S328, from group C i Select the smart mobile device location l that is farthest from the cluster center k ; S329, the selected smart mobile device location l k From Group C i Removed; S3210, the location of the smart mobile device to be removed k Reclassify to the nearest new center point to ensure that the cluster group capacity is adjusted; S3211, output the final clustering result (C1, C2...C U ); In S33, the step of using the multi-agent double-delay deep deterministic policy gradient algorithm to solve the third optimization is: S331, establishing the third optimization problem as a Markov process; S3311, establish state space: Since each drone agent can only observe a part of the entire environment and there is no information exchange between them, the observation results of drone u can be described by formula (12): s u (t)={u u (t),f u (t),z u ,D π(k) (t),δ π(k) (t),id π(k) ,pos π(k) ,E r (t)} (12) Among them, u u (t) represents the position of the UAV u, f u (t) represents the available computing power of UAV u. In addition, D π(k) (t),δ π(k) (t) represents the detailed mission information generated by the smart mobile device k serving the drone u, id π(k) and pos π(k) denote the index and position of the smart mobile device k, respectively, and E r (t) indicates the remaining battery power of the drone; S3312, establish action space: Each agent determines its trajectory and resource allocation strategy based on the current observation results. The action of drone u in time slot t can be expressed by formula (13): in, represents the computing resources allocated by drone u to smart mobile device k, x u (t),y u (t),h u (t) represents the location information of the drone; S3313, establish reward value: drones that do not meet (3)-(5) will be subject to a constant penalty η1, drones that do not meet (7) and (8) will be subject to a penalty constant η2, and drones that do not meet (10) will be subject to a penalty of η3. The reward of drone u is given by formula (14): Wherein, Λ(*) means that if the condition (*) is met, then Λ(*) = 1, otherwise it is 0; S332, solving the third optimization based on a multi-agent double-delay deep deterministic policy gradient algorithm; S3321, initializing the parameters of the original action network and the evaluation network, setting the initial parameters of the original action network and the evaluation network, including the optimal unloading strategy Ψ, and initializing the experience replay pool e for storing the experience data in the training process; S3322, the original action network parameter u m Copy to the target network, and copy the evaluation network parameter w to the target network; S3324, set the initial position of the drone and smart mobile device, calculate the channel gain of each drone, and observe the current state o m , execute the joint strategy of flight trajectory and power control, and observe the next state o' m and the instantaneous reward r of environmental feedback; S3325, the drone collects the experience tuple e={o1,...,o M ,a1,...,a M ,r,o′1,...,o′ M } Store in the experience pool, and check whether the experience replay pool meets the update threshold. If it does not exceed the threshold, repeat step S3325 until all experience replay pools exceed the threshold. If it exceeds the threshold, execute steps S3326-S3328; S3326, randomly sample a small batch of experiences from the experience pool e; S3327, update the original network parameters, update the parameters of the evaluation network and the action network according to the stochastic gradient descent rule, and soft-update the target network parameters; S3328, the UAV and the intelligent mobile device perform corresponding movement operations according to the calculated trajectory and power strategy until the optimal trajectory and power strategy are obtained.

Images