Task joint unloading and migration method for air-ground collaborative mobile edge computing

Abstract

The invention belongs to the technical field of mobile group computing. The invention provides a task joint unloading and migration method for air-ground collaborative mobile edge computing. According to the embodiment of the invention, the scenes of resource heterogeneous, resource limited, communication range limited and network dynamic fluctuation under the calculation of the edge calculation of the unmanned aerial vehicle-vehicle networking are considered; the method comprises the following steps: modeling a joint unloading and migration problem of a DAG task, and expressing the problem as a user DAG task average delay minimization problem; according to the method, different characteristics of the UAV and the vehicle are considered, and the task migration triggering conditions are different; according to the method, the situation that the same DAG task may be migrated for multiple times is considered, and the shortest transmission path of task migration, task uploading and calculation result downloading is considered.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of mobile group computing technology, and in particular to a task joint offloading and migration method for air-ground collaborative mobile edge computing. Background Art

[0002] With the transition to the sixth generation of mobile communication technology (6G), the rapid expansion of smart devices and the explosive growth of new applications have brought about the demand for computing tasks, such as video stream analysis and extraction of smart home monitoring systems, sensor data processing of IoT systems, etc. The processing capacity of smart devices is limited, and computing tasks need to be offloaded to the edge for processing. Computational tasks refer to tasks submitted to the edge server. The edge server only needs to return the processing results of the computing tasks to the user within the delivery period. The tasks are submitted once and delivered once. There is no secondary interaction between the intermediate user and the server. The latency sensitivity of computing tasks is relatively low.

[0003] In the traditional edge computing field, BS is generally used as an edge server, but BS resources are limited, so BS is generally used to handle highly delay-sensitive services such as VR and AR that require continuous interaction. Many existing studies consider using drones and vehicles as edge servers to assist BS in providing services to users. In order to reduce the pressure on BS, considering that mobile servers are the trend of future development, and mobile servers also have good processing capabilities and can handle computing tasks with relatively low latency sensitivity, this application proposes a solution that is completely independent of BS, that is, dispatching a group of drones equipped with lightweight CPUs and GPUs, and roadside parked vehicles as self-organizing networks as mobile MEC servers to handle computing tasks.

[0004] In the related technologies, none of them considers both the characteristics of mobile MEC servers with heterogeneous resources and the characteristics that certain complex computing tasks can be processed at the granularity of subtasks. Generally, considering the interdependence of tasks, complex computing tasks can be modeled as some typical task flows. For example, the video navigation task involves graphics, face detection, camera preview, and video processing, which can be represented as a dependency graph of 14 tasks. If migration decisions can be made at the granularity of subtasks rather than the entire task, the performance improvement brought by offloading to edge server processing can be amplified.

[0005] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the invention

[0007] In order to avoid the shortcomings of the prior art, the present application provides a task joint offloading and migration method for air-ground collaborative mobile edge computing to solve XX problem existing in the prior art.

[0008] According to an embodiment of the present disclosure, a task joint offloading and migration method for air-ground collaborative mobile edge computing is provided, the method comprising:

[0009] Construct an MEC scenario, where the MEC scenario includes M edge servers, K users, and 1 SDN controller, and the edge servers include drones or vehicles.

[0010] According to the joint offloading and migration problem of the user's DAG tasks in the MEC scenario, the MDP problem model is constructed;

[0011] The MDP problem model is transformed into a Markov decision process, and the four-tuple<S,A,P,R> Represents; where S is the state set of the Markov decision process, A is the action set of the Markov decision process, P is the state transition probability of the Markov decision process, and R is the reward of the Markov decision process;

[0012] The DJOMD-PPO algorithm is used to solve the Markov decision process to obtain the minimized average delay of user DAG tasks.

[0013] Furthermore, in the MEC scenario, each user corresponds to a DAG task, and each subtask of the DAG task is processed by a different resource type. The user's DAG task arrives in a Poisson distribution over time. After the DAG task arrives, its predecessor subtask is offloaded to the edge server with corresponding resources in topological order for processing; after the predecessor subtask is processed, the subsequent subtask begins to be processed. Each edge server processes a task, and the SDN controller makes an offloading decision at the beginning of each moment; when the edge server cannot process the corresponding subtask, the SDN controller makes a migration decision to migrate the subtask to other edge servers with corresponding resources.

[0014] Furthermore, in the MEC scenario, the task scheduling plan, the calculation process of the response time of the user DAG request, the task start time, the task execution time, the migration trigger condition and the task migration time, the task download time and the communication model are set.

[0015] Furthermore, the objective function of the MDP problem model is:

[0016]

[0017] Among them, k represents the kth user, T resp The response time for user requests.

[0018] Furthermore, the state set Among them, S1 represents the user basic information domain, including the coordinates of all users and the CPU cycles required for all user tasks. S2 represents the user DAG task information domain, including the input data size, output data size and required resource type of each subtask in all user DAG requests. S3 represents the status of the edge server composed of drones and vehicles, including the geographic locations of all drones and vehicles, the computing power of all drones and vehicles, the current power of all drones, the CPU clock frequency of all drones and vehicles, and the resource types owned by all drones and vehicles. S4 represents the network bandwidth size of each link of the network link of the drone-vehicle edge computing network.

[0019] Each action a in the action set A t is an M×N two-dimensional decision matrix The value of each element in the two-dimensional decision matrix is ​​0 or 1. i∈M,j∈K means that the SDN controller does not make a decision at time t. i∈M,j∈K means that the decision made by the SDN controller for user j at time t is served by edge server i, and at the same time, a user's DAG task request is served by an edge server;

[0020] The state transition probability P: S×A×S→[0,1] represents the state transition probability distribution, p0:S→[0,1] is the distribution of the initial state s0, and the initial state s0 is changed by taking action a t Based on the probability P(s t+1 |s t ,a t ) from s t Transfer to t+1 ;

[0021] Reward R: r t : represents the immediate reward received by the SDN controller after making a decision for the user request at time t; the goal of the MDP problem model is to find the optimal action in each step so that the expected cumulative reward R episode Maximum, the reward function is set as follows:

[0022]

[0023] When the round ends, that is, when all users’ DAG requests are executed, the average delay of all users’ DAG requests is calculated, and the reward setting is modeled based on the average delay; in other cases, the reward is set to 0.

[0024] Furthermore, the DJOMD-PPO algorithm is used to solve the Markov decision process to obtain the step of minimizing the average delay of the user DAG task, including:

[0025] The hybrid graph neural network model in the DJOMD-PPO algorithm is used to extract features from S3 and S4 to obtain the first feature; wherein the hybrid graph neural network model includes a graph convolutional neural network GCN and a graph Gaussian mixture neural network GMM;

[0026] Using the hybrid graph neural network model to extract features from S1, so as to obtain the second feature;

[0027] Using the hybrid graph neural network model to extract features from S2, so as to obtain the third feature;

[0028] The first, second, and third features are fused to output the minimized average delay of the user DAG task.

[0029] Furthermore, the DJOMD-PPO algorithm includes:

[0030] Initialize network parameters and experience replay buffer, and reset environment state;

[0031] Collect sample data from multiple stem groups according to the strategy and calculate the advantage function

[0032] Optimize and update the first network parameter θ based on the reward function e and the second network parameter ψ e ;

[0033] For a set of sample data, perform P updates;

[0034] The first network parameter θ of the updated strategy e Copy to The second network parameter ψ of the updated value network e Copy to

[0035] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0036] In the embodiments of the present disclosure, through the above-mentioned task joint offloading and migration method for air-ground collaborative mobile edge computing, the method takes into account the scenarios of resource heterogeneity, limited resources, limited communication range and dynamic network fluctuations under edge computing composed entirely of drone-vehicle networking; the method models the joint offloading and migration problem of DAG tasks and expresses it as the problem of minimizing the average delay of user DAG tasks; the method takes into account the different characteristics of UAVs and vehicles, and the different conditions for triggering task migration; the method takes into account the situation where the same DAG task may be migrated multiple times, and considers the shortest transmission path for task migration, task upload, and download of calculation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0038] Figure 1 A step diagram showing a method for joint task offloading and migration for air-ground collaborative mobile edge computing in an exemplary embodiment of the present disclosure;

[0039] Figure 2 A scene diagram in an exemplary embodiment of the present disclosure is shown;

[0040] Figure 3 Another scene diagram in an exemplary embodiment of the present disclosure is shown;

[0041] Figure 4 A framework diagram of the DJOMD-PPO algorithm in an exemplary embodiment of the present disclosure is shown;

[0042] Figure 5 A performance graph of the DJOMD-PPO algorithm is shown in the case where 20 users receive services from the edge server in an exemplary embodiment of the present disclosure;

[0043] Figure 6 A performance graph of the DJOMD-PPO algorithm is shown in the case where 15 users receive services from the edge server in an exemplary embodiment of the present disclosure;

[0044] Figure 7 A performance graph of the DJOMD-PPO algorithm is shown in the case where 10 users receive services from the edge server in an exemplary embodiment of the present disclosure;

[0045] Figure 8 A performance graph of the DJOMD-PPO algorithm is shown in the case where 5 users receive services provided by the edge server in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0047] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0048] This example embodiment provides a task joint offloading and migration method for air-ground collaborative mobile edge computing. Figure 1 As shown in , the task joint offloading and migration method for air-ground collaborative mobile edge computing may include: steps S101 to S104.

[0049] Step S101: constructing an MEC scenario; wherein the MEC scenario includes M edge servers, K users, and 1 SDN controller, and the edge server includes a drone or a vehicle;

[0050] Step S102: construct an MDP problem model according to the joint offloading and migration problem of the user's DAG tasks in the MEC scenario;

[0051] Step S103: Convert the MDP problem model into a Markov decision process and<S,A,P,R> Represents; where S is the state set of the Markov decision process, A is the action set of the Markov decision process, P is the state transition probability of the Markov decision process, and R is the reward of the Markov decision process;

[0052] Step S104: using the DJOMD-PPO algorithm to solve the Markov decision process to obtain the minimized average delay of the user DAG tasks.

[0053] Through the above-mentioned task joint offloading and migration method for air-ground collaborative mobile edge computing, the method takes into account the scenarios of resource heterogeneity, limited resources, limited communication range and dynamic network fluctuations under edge computing composed entirely of drone-vehicle networks; the method models the joint offloading and migration problem of DAG tasks and expresses it as the problem of minimizing the average delay of user DAG tasks; the method takes into account the different characteristics of UAVs and vehicles, and the conditions for triggering task migration are different; the method takes into account the situation where the same DAG task may be migrated multiple times, and considers the shortest transmission path for task migration, task upload, and download of calculation results.

[0054] Next, we will refer to Figures 1 to 8 The various steps of the above-mentioned task joint offloading and migration method for air-ground collaborative mobile edge computing in this example implementation are described in more detail.

[0055] In step S101 and step S102, in the constructed drone-vehicle MEC environment, M mobile servers provide services for the task requests of K users. The M servers in the Mobile Edge Computing (MEC) system are represented as U = {U1,…,U M},U m = {r m ,f m ,kind m ,loc m ,h m ,p m ,θ m ,rad m}, where r m ∈{0,1,…,X} means that each server has different heterogeneous resources, and different values ​​represent different resource types; f m Indicates the CPU clock frequency of the server; kind m is a binary variable, kind m 0 means it is a drone, kind m 1 represents a vehicle; loc m Represents the horizontal coordinate of the server; h m Represents the vertical height of the server; p m Represents the transmission power of the server; θ m Indicates the computing power of the server, that is, the number of CPU cycles required to process each bit of data; rad m Indicates the radius of the server coverage. Each user can be represented by a tuple as <loc k ,p k ,G k >, where loc kIndicates the two-dimensional coordinates of the user, p k represents the transmission power of user k, G k =(N k ,E k ) represents the user's DAG task request, where N = {n1, n2, ..., n j} is a collection of subtasks, each subtask n j ∈N consists of triples <input j ,resource j ,output j > indicates that input j Represents subtask n j The input data size, resource j Represents subtask n j The resource type required for execution, output j Represents subtask n j The processing result data size is , E is the set of data flow edges. i,j (e i,j ∈E), task i is the predecessor of task j, and task j is the successor of task i. Due to task dependency, a task can only be executed after receiving the processing results of all its predecessor tasks, where pre(i) represents the set of predecessor tasks of task i. Figure 2 Taking request 2 of DAG as an example, the set of predecessor tasks of task 5 is pre(5)={3, 4}, so task 5 must wait for data from tasks 3 and 4 before it can be executed.

[0056] The arrival interval of user task requests follows a Poisson distribution with an arrival interval of λ. When a user request is generated, the SDN controller makes a decision to assign the first subtask of the request to a specific mobile MEC server that has the resources required for the subtask for processing. Each time a subtask is completed, the SDN controller also determines the target MEC server to execute the next subtask. The scenarios considered are defined in more detail below.

[0057] Task scheduling plan:

[0058] For a user request G with n subtasks k , the scheduling plan is defined as in represents the scheduling decision at time step t. Specifically, Indicates scheduling task i to computing node m. Figure 1 As shown, the scheduling decision at time step 0 is as follows: (1) Task 1 of user request 1 is scheduled to ES1, and (2) Task 1 of user request 2 is scheduled to ES9. To better illustrate, user request Gk The task schedule at time step t is defined as in Represents the scheduling plan of task i, using a one-dimensional vector Indicates that y m represents the mth server, y m ∈{0,1},y m =1 means task i is executed on the mth server, It means that task i has not started to execute at time t, or it is just executed on a certain server. According to the scheduling decisions at different time steps, the final scheduling plan can be obtained. In addition, T t represents the delay of the user request at time step t, where T t The detailed calculation process of will be given below. Based on the above, the response time of user request is defined as T resp , which represents the time interval from task submission to completion of the last subtask.

[0059] The calculation process of the response time of the user DAG request is as follows:

[0060] As mentioned above, T t represents the delay of a user request at time step t. Since subtasks may be executed in parallel on different mobile servers, T t represents the maximum end time of all completed tasks at time step t. Therefore, T t It can be defined as

[0061]

[0062] Where D t is the set of completed tasks at time step t. end (i) is the end time of task i, defined as

[0063] t end (i) = t start (i)+t exe (i)+t mig (i)+t down (i)

[0064] where t exe (i), t start (i), t mig (i), t down (i) are the execution time, start time, migration time, and download time of task i, respectively.

[0065] Task start time:

[0066] The start time of task i depends on two aspects. First, task i should receive the execution results of all its predecessor tasks. Second, the server that executes task i should be available. Therefore, the start time of task i is defined as

[0067]

[0068] Among them, t avl (m) is the time when node m becomes available. tran (i, j) represents the time required to transfer the intermediate data of task i’s predecessor task j to the execution server of task i, which is similar to the offloading delay of traditional service migration. At time t, it is defined as

[0069]

[0070] Set Path l,m (t) represents the data transmission path from server l to server m. Task i and its predecessor task j are scheduled to nodes m and l respectively. If points m and l are the same node, there is no data transmission time because data transmission can be achieved through shared memory. When i is the first task requested by the user, l represents the user who initiated the request. Among them, d i,j is the data transferred from task i to j, that is, the data size after task i is executed. TR s,s+1 Represents the transmission rate of the channel between nodes s and s+1. The calculation formula of the transmission rate can be found in the communication model.

[0071] Task execution time:

[0072] The execution time of task i depends on the computing power of server m and the amount of computation of task i. However, considering that the server may not be able to continue processing the task due to its own reasons, which will trigger task migration, this application considers that after the task migration, it can resume operation on site. The data migrated is the intermediate data, that is, task i may be executed on multiple nodes. For the convenience of calculation, it is defined as

[0073]

[0074] Among them, c m Represents the amount of task data processed on server m.

[0075] Migration trigger conditions and task migration time:

[0076] In this application, the conditions that trigger migration can be mainly divided into two categories.

[0077] The first category, considering the endurance of drones, is that the drone is processing the task. Because the drone is about to run out of power (the power is lower than the threshold α), it needs to migrate the task it is currently processing to other servers with corresponding resources. The power prediction formula of drone m is as follows:

[0078]

[0079] Among them, E m (t) is the remaining power of drone m at time t, is the initial power of drone m, is the hovering power consumption of the drone, It is the CPU power consumption of the drone, which is a function of the CPU frequency.

[0080] The challenge for this application when the drone triggers migration due to battery issues is: what algorithm should be used to select the appropriate target server based on geographic location, network channel, and load conditions. According to research, existing work selects migration targets by considering a partial combination of geographic location, network channel, and load conditions. It is challenging for this application to consider all of these factors.

[0081] Figure 3 This is an example diagram of the first migration scenario. The drone is about to run out of power, and it needs to dynamically find a new server with corresponding resources, and then migrate the intermediate data and the remaining unprocessed data to the new server for breakpoint processing. When selecting a new target server, you need to consider the location, computing power, and network bandwidth.

[0082] The second category is vehicles that are processing tasks. Since the vehicles are temporarily parked on the roadside, they may move away at any time. At this time, considering that the coverage of the SDN central controller is limited and the vehicle moves quickly, once it moves out of the coverage of the SDN central controller, the calculation results cannot be returned to the user. Therefore, it is necessary to predict the vehicle's movement trajectory and determine whether to make a migration decision based on whether the predicted vehicle position exceeds the coverage of the SDN central controller.

[0083] The challenge that the roadside parking vehicle triggers migration due to movement is: how to make a reasonable trade-off between relay backhaul and migration. This application considers that the vehicle can complete the task before moving out of the coverage of the SDN central controller. Considering that migration requires additional overhead, there is no need to migrate at this time; otherwise, the task should be migrated in advance before the vehicle leaves the SDN central controller. A key difficulty is how to determine how long the vehicle moves to trigger migration, because the vehicle movement trajectory is complex. In order to simplify the problem, and this application focuses on the unloading / migration decision, it is assumed that the vehicle movement trajectory is known.

[0084] Task i may trigger multiple migrations. The migration time of task i at time step t is defined as follows:

[0085]

[0086] Here, set represents the set of servers that have hosted task i.

[0087] Task download time:

[0088] When the last task requested by the user is completed, the calculation result needs to be sent to the user, so the task download time is defined as follows:

[0089]

[0090] Among them, res(i) represents the size of the calculation result of task i, v represents the server node hosting the task, and u represents the user.

[0091] Communication model:

[0092] In this application, only large-scale fading, i.e., communication path loss, is considered, and small-scale fading or fast fading is not considered, because the time scale of decision making is not small. Every time a decision is made, it remains valid until the next decision time. Without loss of generality, this application assumes that the decision will not be changed within a single time slot, so no decision will be made within the time scale of fast fading. For air-to-ground (A2G) communications, the path loss between the drone and the ground node is given by:

[0093]

[0094] Where h and r represent the altitude of the drone and the horizontal distance between the drone and the ground terminal device, respectively. f (in Hz) is the carrier frequency, and c (in m / s) is the speed of light. LoS represents the line-of-sight transmission probability of the A2G link, which is given by:

[0095]

[0096] Where (a, b, η LoS ,η NLoS ) is a parameter related to the environment, which is equal to (4.88, 0.43, 0.1, 21) in the suburban environment.

[0097] Based on the expression of path loss, the Ground-UAV data rate and UAV-Ground data rate are given by the following two equations:

[0098]

[0099] Among them, B represents the bandwidth of WiFi, EB and E U Represent the transmission power of ground equipment and UAV respectively, σ 2 represents the power of Gaussian noise.

[0100] Air-to-air (A2A) communication is mainly based on line-of-sight transmission, so the free space path loss model is used, as shown below:

[0101] PL A2A (d) = 32.45 + 20logf c +20logd

[0102] where f c is the carrier frequency (MHz), and d is the distance between the UAVs (km). Therefore, the UAV-UAV data transmission rate is given by:

[0103]

[0104] Ground-to-ground (G2G) communication is mainly based on line-of-sight transmission, so the free space path loss model is used. The path loss of ground-to-ground communication is as follows:

[0105]

[0106] where f G is the carrier frequency (Hz), and d is the distance between ground devices (m). Therefore, the GROUND-GROUND data transmission rate is given by:

[0107]

[0108] Based on the above description, considering that the transmission rate involves different types of channels, the summary is as follows: Let TR a,b represents the actual transmission rate of the channel between the source node a and the target node b, which is a nonlinear function as follows:

[0109]

[0110] Where E a , σ 2 , B, PL represent the transmission power, noise power, channel bandwidth, and composite fading vector of source node a, respectively. The fading vector PL is determined based on source node a and target node b. If both the source node and the target node are drones, PL = PL_A2A; if both the source node and the target node are ground equipment, PL = PL G2G Otherwise PL=PL A2G .

[0111] Problem definition:

[0112] In order to improve user QoS, the optimization goal of this application is to minimize the response time of all user DAG task requests. The objective function is defined as:

[0113]

[0114] Considering the volatility of the network and the complexity of DAG task scheduling, the selection of task offloading / migration decision solutions will become a combinatorial explosion problem. Using some traditional heuristic algorithms takes a long time to find a suitable migration solution, which cannot meet user requirements well. To address this problem, deep reinforcement learning (DRL) will be used to implement adaptive offloading / migration decisions for user DAG tasks in a dynamic network environment. That is, the deep reinforcement learning stored on the SDN controller at each moment will make corresponding decisions.

[0115] In step S103, the problem is transformed into a Markov decision process:

[0116] First, this application transforms the average delay minimization problem defined above into a Markov decision process consisting of a four-tuple<S,A,P,R> Indicates that each element is explained as follows:

[0117] a) State set: S is the state set of the Markov decision process. The design of the state is crucial. In order to reduce the difficulty of the intelligent agent to understand the environment, it is better to extract and classify the environmental information and use different neural network modules for feature extraction, rather than directly inputting all environmental information into the neural network. To this end, this application defines the state set S as It includes three elements: S1 represents the user basic information domain, including the coordinates of all users and the CPU cycles required for all user tasks; S2 represents the user DAG task information domain, including the input and output data size and the required resource type of each subtask in all user DAG requests; S3 represents the status of the edge server composed of drones and roadside parking vehicles, including the geographic location of all drones / vehicles, the computing power of all drones / vehicles, the current power of all drones, the computing power of all drones / vehicles, the CPU clock frequency of all drones / vehicles, and the resource types owned by all drones / vehicles; S4 represents the network bandwidth size of each link of the network link of the drone-vehicle edge computing network.

[0118] b) Action set: A, each action a in the action set (space) t is an M×N two-dimensional decision matrix The value of each element in this two-dimensional decision matrix can only be 0 and 1, for example i∈M,j∈K, which means that the decision made by the central controller for user j at time t is provided by server i. Similarly, it ensures that a user's DAG request is provided by at most one server at the same time.

[0119] c) State transition probability: P:S×A×S→[0,1] represents the state transition probability distribution, and p0:S→[0,1] is the distribution of the initial state s0. State s0 is changed by taking action a. t Based on the probability P(s t+1 |s t ,a t ) from s t Transfer to t+1 .

[0120] d) Reward: r t : represents the immediate reward received by the central controller after making a decision for the user request at time t. The goal of the central controller is to minimize the average delay of all users' DAG requests by solving the constructed MDP problem. That is, the goal of solving the MDP is to find the optimal action in each step so that the expected cumulative reward R episode Maximum, so the reward function is set as follows:

[0121]

[0122] The reward function is considered in two cases: 1. At the end of the round, that is, when all users' DAG requests are executed, the average delay of all users' DAG requests can be calculated, so the reward setting is modeled based on the average delay; 2. In other cases, the reward is set to 0.

[0123] In step S104, the characteristics of the problem raised in this application are analyzed and solved using the PPO algorithm.

[0124] The PPO algorithm is a policy-based reinforcement learning algorithm. It is an improvement on the policy gradient (PG) algorithm and retains the advantages of excellent performance in continuous state space and continuous action space.

[0125] In PPO, for a network structure where the policy network and the value network do not share parameters, the objective function is usually optimized for the policy and value networks respectively. The following formula is the objective function of the policy network, θ e are the parameters of the policy network:

[0126]

[0127] The objective function of the value network is as follows:

[0128]

[0129] where ψ e Represents the parameters of the value network. is the state value estimated by the value network. V target is the actual return, usually estimated by sampling trajectories. is the mean square error of the value function.

[0130] For gym environments or some simple scenarios, the Actor network or Critic network of the traditional PPO algorithm is generally built based on a multi-layer perceptron (MLP), and the neural network model is relatively simple. In the joint DAG task offloading / migration scenario considered in this application, based on the state space and scenario characteristics defined in step 1, we proposed a set of neural network models suitable for computing task migration, such as Figure 4 As shown. First, considering that the MEC network composed of drones / vehicles has a dynamic network and each drone / vehicle has computing resources and other characteristic attributes, the data is naturally presented as a graph structure. The points in the graph represent each drone / vehicle, and each edge in the graph represents the bandwidth value of the wireless communication network of the two servers. Based on this feature, it is reasonable to use a graph neural network to extract edge server features. This application proposes a hybrid graph neural network model that combines a graph convolutional neural network GCN and a graph Gaussian mixed neural network GMM. Feature extraction is performed for S3 and S4 defined in the state of the Markov decision process. By combining different neural network structures, this application can better capture the complex characteristics of the network, thereby improving decision-making performance; then, the user's basic information S1 is represented by a low-dimensional vector, and feature extraction can be performed through a simple MLP. The user's DAG task information S2 is represented by a low-dimensional vector, and feature extraction can be performed through a simple MLP. Then, the extracted features are feature fused to output the corresponding action or value function. This application names the PPO algorithm DJOMD-PPO (PPO-Based DAGTaskJointOffloadingandMigrationDecisionAlgorithm).

[0131] The pseudo code of the DJOMD-PPO algorithm is shown in Table 1. It describes the training process of the DJOMD-PPO algorithm in detail. First, the network parameters and experience replay buffer are initialized, and the environment state is reset. After collecting enough samples according to the strategy, the advantage function is calculated. Then optimize and update the network parameters θ based on the reward function e , ψ e For a set of sample data, P updates are required. Finally, the network parameters θ of the updated strategy are e Copy to The parameter ψ of the value network eCopy to

[0132] Table 1 DJOMD-PPO algorithm pseudo code

[0133]

[0134] In a specific embodiment, the present application has conducted a large number of simulation experiments. The experimental settings and numerical results will be introduced below to demonstrate the effectiveness of the proposed solution.

[0135] Experimental setup

[0136] The experiments in this article are conducted on an Intel(R) Xeon(R) Gold 6148CPU@2.40GHz, 128G memory, and GeForce RTX3090 device based on Python 3.8 and Pytorch1.10.

[0137] This paper builds a virtual simulation environment and deploys 25 drones and vehicle servers in a square area of ​​2500m*2500m. The server can completely cover the area and users can enjoy the services provided by the server in the area. At each time t, it detects which users' DAG tasks need to be offloaded or migrated.

[0138] Drone server settings: Assume that the CPU frequency of the drone is 5Ghz, the communication radius of the drone is 500m, the computing power of the drone is randomly generated in the range [1500CPU Circles / byte, 2000CPU Circles / byte] according to a uniform distribution, the transmission power of the drone is 20mW, and the flight altitude of the drone is 70 meters.

[0139] Vehicle server settings: Assume that the vehicle's CPU frequency is 5 GHz, the communication radius is 500 m, the computing power of the vehicle server is randomly generated in the range [1500 CPU Circles / byte, 2000 CPU Circles / byte] according to a uniform distribution, and the transmission power of the vehicle server is 20 mW.

[0140] Edge server networking settings: The network rate unit of the communication link of the edge computing network composed of drones and vehicle servers is MHZ, which is randomly generated according to the normal distribution with mean μ=μ1 and standard deviation σ=σ1. μ1 and σ1 are expected to be set according to specific experiments. By default, μ1=80 and σ1=25.

[0141] User settings: The CPU cycles requested by the user's DAG task are randomly generated in the range [1.5e8, 2e8], the input data size of the task is randomly generated in the range [3MB, 5MB], the calculation result size is randomly generated in the range [1.5MB, 2.5MB], and the computing power of the user's mobile device is 1.5mW.

[0142] For the designed DJOMD-PPO algorithm, we consider different user scales, namely, 5 users, 10 users, 15 users, and 20 users receiving services from edge servers. Figures 5 to 8 As shown in the figure, it is the performance diagram of DJOMD-PPO algorithm under different user scales; Figure 5 Performance graph of the DJOMD-PPO algorithm for 20 users receiving services from edge servers; Figure 6 Performance graph of the DJOMD-PPO algorithm for 15 users receiving services from edge servers; Figure 7 Performance graph of the DJOMD-PPO algorithm for 10 users receiving services from the edge server; Figure 8 Figure 2 shows the performance graph of the DJOMD-PPO algorithm when 5 users are served by the edge server.

[0143] Through the above-mentioned task joint offloading and migration method for air-ground collaborative mobile edge computing, the method takes into account the scenarios of resource heterogeneity, limited resources, limited communication range and dynamic network fluctuations under edge computing composed entirely of drone-vehicle networks; the method models the joint offloading and migration problem of DAG tasks and expresses it as the problem of minimizing the average delay of user DAG tasks; the method takes into account the different characteristics of UAVs and vehicles, and the conditions for triggering task migration are different; the method takes into account the situation where the same DAG task may be migrated multiple times, and considers the shortest transmission path for task migration, task upload, and download of calculation results.

[0144] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0146] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A task joint offloading and migration method for air-ground collaborative mobile edge computing, characterized in that: The method includes: Construct an MEC scenario, where the MEC scenario includes M edge servers, K users, and 1 SDN controller, and the edge servers include drones or vehicles. According to the joint offloading and migration problem of the user's DAG tasks in the MEC scenario, the MDP problem model is constructed; The MDP problem model is transformed into a Markov decision process, and the four-tuple<S,A,P,R> Represents; where S is the state set of the Markov decision process, A is the action set of the Markov decision process, P is the state transition probability of the Markov decision process, and R is the reward of the Markov decision process; The DJOMD-PPO algorithm is used to solve the Markov decision process to obtain the minimized average delay of user DAG tasks.

2. According to claim 1, the task joint offloading and migration method for air-ground collaborative mobile edge computing is characterized in that: In the MEC scenario, each user corresponds to a DAG task. Each subtask of the DAG task is processed by different resource types. The user's DAG tasks arrive in a Poisson distribution over time. After the DAG task arrives, its preceding subtasks are offloaded to the edge server with corresponding resources in topological order for processing. After the preceding subtask is processed, the subsequent subtask begins to be processed. Each edge server processes one task, and the SDN controller makes an offloading decision at the beginning of each moment. When the edge server cannot process the corresponding subtask, the SDN controller makes a migration decision to migrate the subtask to other edge servers with corresponding resources.

3. According to claim 2, the task joint offloading and migration method for air-ground collaborative mobile edge computing is characterized in that: In the MEC scenario, set the task scheduling plan, the calculation process of the response time of the user DAG request, the task start time, the task execution time, the migration trigger condition and the task migration time, the task download time and the communication model.

4. According to claim 3, the task joint offloading and migration method for air-ground collaborative mobile edge computing is characterized in that: The objective function of the MDP problem model is: Among them, k represents the kth user, T resp The response time for user requests.

5. The task joint offloading and migration method for air-ground collaborative mobile edge computing according to claim 4 is characterized in that: State Collection Among them, S1 represents the user basic information domain, including the coordinates of all users and the CPU cycles required for all user tasks. S2 represents the user DAG task information domain, including the input data size, output data size and required resource type of each subtask in all user DAG requests. S3 represents the status of the edge server composed of drones and vehicles, including the geographic locations of all drones and vehicles, the computing power of all drones and vehicles, the current power of all drones, the CPU clock frequency of all drones and vehicles, and the resource types owned by all drones and vehicles. S4 represents the network bandwidth size of each link of the network link of the drone-vehicle edge computing network. Each action a in the action set A t is an M×N two-dimensional decision matrix The value of each element in the two-dimensional decision matrix is ​​0 or 1. It means that the SDN controller does not make a decision at time t. It means that the decision made by the SDN controller for user j at time t is served by edge server i, and at the same time, a user's DAG task request is served by an edge server; The state transition probability P: S×A×S→[0,1] represents the state transition probability distribution, p0:S→[0,1] is the distribution of the initial state s0, and the initial state s0 is changed by taking action a t Based on the probability P(s t+1 |s t ,a t ) from s t Transfer to t+1 ; Reward R: r t : represents the immediate reward received by the SDN controller after making a decision for the user request at time t; the goal of the MDP problem model is to find the optimal action in each step so that the expected cumulative reward R episode Maximum, the reward function is set as follows: When the round ends, that is, when all users’ DAG requests are executed, the average delay of all users’ DAG requests is calculated, and the reward setting is modeled based on the average delay; in other cases, the reward is set to 0.

6. The task joint offloading and migration method for air-ground collaborative mobile edge computing according to claim 5 is characterized in that: The steps of solving the Markov decision process using the DJOMD-PPO algorithm to obtain the minimized average delay of the user DAG tasks include: The hybrid graph neural network model in the DJOMD-PPO algorithm is used to extract features from S3 and S4 to obtain the first feature; wherein the hybrid graph neural network model includes a graph convolutional neural network GCN and a graph Gaussian mixture neural network GMM; Using the hybrid graph neural network model to extract features from S1, so as to obtain the second feature; Using the hybrid graph neural network model to extract features from S2, so as to obtain the third feature; The first, second, and third features are fused to output the minimized average delay of the user DAG task.

7. The task joint offloading and migration method for air-ground collaborative mobile edge computing according to claim 6 is characterized in that: The DJOMD-PPO algorithm includes: Initialize network parameters and experience replay buffer, and reset environment state; Collect sample data from multiple stem groups according to the strategy and calculate the advantage function Optimize and update the first network parameter θ based on the reward function e and the second network parameter ψ e ; For a set of sample data, perform P updates; The first network parameter θ of the updated strategy e Copy to The second network parameter ψ of the updated value network e Copy to

Images