Multi-user time series dependent service computing offloading method based on graph neural network

Abstract

The application discloses a time series dependent service offloading method based on a graph neural network and deep reinforcement learning, models and studies a multi-user multi-server MEC system, application generated by a user is dynamically arrived, and the size and shape of the application are arbitrary, time series dependent tasks constituting the application can be offloaded to a MEC server where a service base station is located, or can be offloaded to a MEC server of other adjacent base stations. We use a graph neural network to more effectively capture the feature state of time series dependent services and applications. The fine-grained offloading problem is modeled as a Markov process, and the policy gradient algorithm of deep reinforcement learning is used for decision-making, and the order of task selection and the position of offloading are optimized in a complex and high-dimensional state space to reduce the scheduling time of the application in the system. Simulation proves that the algorithm has good convergence in reducing the scheduling time of the application, is superior to other benchmark algorithms, and thus improves the efficiency of the MEC system.

Description

Technical Field

[0001] This invention relates to a fine-grained task scheduling and offloading method based on graph neural networks and deep reinforcement learning algorithms. In a multi-user and multi-server MEC scenario, a decision scheme is designed to reduce the application scheduling time in the system. The method selects the task to be scheduled in a complex and high-dimensional state space and selects a server for the task to be offloaded, which optimizes both the offloading timing and the offloading location. This invention belongs to the field of wireless communication and computing offloading. Background Technology

[0002] With the rapid development of the Internet of Things (IoT) and mobile devices, applications are becoming increasingly sensitive to computational load and latency, posing a severe challenge to resource-constrained devices. Mobile edge computing (MEC) provides computing and storage infrastructure closer to data sources or users, and offers cloud services and IT environment services for edge applications, solving the problems of excessive latency and excessive traffic aggregation. Compute offloading, as a key technology in MEC, provides better support for real-time and bandwidth-intensive business applications. Future networks need to establish a more granular and intelligent task offloading mechanism tailored to the characteristics and diverse performance requirements of intelligent products and applications, comprehensively managing network resources, improving network service efficiency, and enhancing user experience.

[0003] In the study of offloading time-dependent services, the time-dependent characteristics of services are usually modeled as a Directed Acyclic Graph (DAG) model. Due to the complexity and diversity of DAG structures, the computational offloading problem of DAG-based applications is difficult. References [1-3] adopted heuristic algorithms to solve the scheduling and offloading problem of time-dependent services. However, heuristic algorithms require a large number of iterations to obtain the optimal solution and are not suitable for high real-time MEC systems. Deep reinforcement learning can find the optimal solution in complex and large states. Reference [4] proposed a priority-based time-dependent service scheduling scheme, which ensures the time-dependent nature of tasks while improving scheduling efficiency. Through offline training and online deployment, efficient offloading of computational tasks was achieved. Reference [5] proposes an S2S neural network to capture the temporal dependencies of DAG, encodes the nodes of DAG into a series of embeddings with fixed length, trains the S2S neural network with the policy gradient algorithm, and realizes the computation offloading in the single-user single-server scenario. However, in this method, the embedding vector of each task is fixed length, and the length of the embedding vector input to the neural network cannot be too long, which will lead to the inaccurate extraction of the structural information of DAG. Reference [6] proposes a Directed Acyclic Graph Neural Network (DAGNN) to solve the problem of losing the structural information of DAG when inputting the DAG graph into the neural network of the DRL algorithm. It combines GNN with the DRL algorithm to solve the offloading scheduling problem in the single-user scenario.

[0004] The scenarios mentioned above that utilize deep reinforcement learning algorithms to solve the offloading problem are mostly limited to static scenarios with a single user or a single server. They lack research on the offloading of dynamically arriving applications in MEC scenarios with multiple users and multiple servers. Furthermore, the feature extraction for time-dependent services is limited to information about the task itself, lacking global information about the relationships between tasks and the application.

[0005] Unlike existing work, (1) this invention considers a more complex MEC scenario and models a multi-user, multi-server MEC system. Each user's time-dependent services can be offloaded to a directly connected server or to other servers. User-generated applications arrive dynamically, and the size and shape of the applications are arbitrary. (2) This invention utilizes two-level embeddings (task-level and global-level embeddings) of graph neural networks to more effectively capture the state of time-dependent services and applications of arbitrary size and shape. (3) This invention models the fine-grained offloading problem using Markov processes and uses the policy gradient algorithm of deep reinforcement learning to make decisions, selecting the tasks to be scheduled and the servers to be offloaded for the tasks, thus optimizing both the offloading timing and the offloading location.

[0006] [1].W.He,L.Gao and J.Luo,"A Multi-Layer Offloading Framework forDependency-Aware Tasks in MEC,"ICC 2021-IEEE International Conference onCommunications,Montreal,QC,Canada,2021,pp.1-6.

[0007] [2].H.Liao,X.Li,D.Guo,W.Kang and J.Li,"Dependency-Aware ApplicationAssigning and Scheduling in Edge Computing,"in IEEE Internet of ThingsJournal,vol.9,no.6,pp.4451-4463.

[0008] [3].L.Chen,J.Wu,J.Zhang,H.-N.Dai,X.Long and M.Yao,"Dependency-AwareComputation Offloading for Mobile Edge Computing With Edge-CloudCooperation,"in IEEE Transactions on Cloud Computing,vol.10,no.4,pp.2451-2468.

[0009] [4].L.Chen,J.Wu,J.Zhang,H.-N.Dai,X.Long and M.Yao,"Dependency-AwareComputation Offloading for Mobile Edge Computing With Edge-CloudCooperation,"in IEEE Transactions on Cloud Computing,vol.10,no.4,pp.2451-2468.

[0010] [5].N.Mao, Y.Chen, M.Guizani and GMLee, "Graph Mapping OffloadingModel Based On Deep Reinforcement Learning With Dependent Task," 2021International Wireless Communications and Mobile Computing (IWCMC), 2021, pp.21-28.

[0011] [6]. Cui Shuo, Qin Shaohua, Xie Zhibin, Zhang Jiahao, Bian Shengqiang. A computational offloading method for dependency-based tasks based on graph neural networks [J]. Computer Measurement & Control, 2021, 29(11):189-195. Summary of the Invention

[0012] In a multi-user, multi-server MEC scenario, this invention proposes a fine-grained task scheduling and offloading method based on graph neural networks and deep reinforcement learning algorithms. Each user generates a series of dynamically arriving applications. The graph neural network with scalable state is used to better capture the characteristics of each time-dependent service. At the same time, the global information between multiple tasks and applications is fully considered. The parameters of the graph neural network and the policy network are jointly trained by the policy gradient algorithm. In the complex and high-dimensional state space, the optimal task to be scheduled and the optimal server for the task are selected to reduce the scheduling time of all applications in the system.

[0013] The key problem this invention aims to solve is to offload and schedule time-dependent services that dynamically arrive from different users in a multi-user, multi-server network scenario. The technical solution adopted is as follows:

[0014] A multi-user dependent business computation offloading optimization algorithm based on graph neural networks includes the following steps:

[0015] Step 1: Establish a system model based on the network scenario.

[0016] The network scenario is a multi-user, multi-server MEC system. In the MEC system, each user generates a series of applications that arrive in a streaming manner, and each application consists of time-dependent tasks.

[0017] Step 1.1 Based on server and user information, establish a dynamic model for application streaming arrival in a multi-user, multi-server scenario.

[0018] Figure 1This illustrates a MEC system composed of multiple heterogeneous edge servers, each with varying computing capabilities. The user devices and server sets within the MEC system are respectively... and Let u represent any user device in the set, m represent any server in the server set, and f represent the computing power of any server. m A group of user equipment is connected to the same base station. During the offloading process, the user communicates with the directly connected server. The user task can be offloaded to the MEC server where its serving base station is located, or it can be offloaded to the MEC server of other adjacent base stations.

[0019] like Figure 2 As shown, the application is modeled as a directed acyclic graph (DAG), and a DAG is described as... E and E represent the vertex set and edge set of the graph, respectively. In the MEC system, any user... The resulting set of applications can be represented as G un On behalf of users Any application n generated. N u On behalf of users The total number of applications generated.

[0020] Wherein, any application G generated by user u (u∈U) un The number of subtasks included is G un The set of all tasks is represented as: Representing G un Total number of tasks Representative application G un The i-th subtask This represents the dependencies between tasks. Represents and There is a dependency relationship between them, and It must be ensured that You can only execute it after it has been completed. Called The father's task, yes Subtasks, any one of the subtasks The set of all forward tasks is represented as The set of backward tasks (subtasks) is represented as The amount of data uploaded by the task is The required number of CPU cycles is

[0021] Each user generates a series of applications that arrive in a streaming manner. According to the assumptions of queuing theory, the time intervals between applications arriving at a user within a certain period of time follow an exponential distribution. The time interval between a user arriving at the first application from time 0 is λ0, and the time interval between the nth application and the (n+1)th application is λ. n :

[0022] λ n ~Exp(λ)

[0023] Where λ is the arrival rate parameter, representing the average number of applications generated by a user per unit time. The relationship between the arrival times of two adjacent applications is as follows:

[0024]

[0025] Step 1.2 Model the multi-user, multi-server system. Users can communicate with directly connected base stations. When a user unloads a task generated in an application, they can choose to unload it from either a directly connected server or another server. The system latency model can be obtained based on the impact of the unloading decision on latency.

[0026] Users and edge servers transmit wirelessly via base stations; B is the channel bandwidth, p u,m It is the user's upload power, g u,m σ represents the channel gain between user u and the connected server m. 2 Representing noise power, the transmission rate can be obtained using Shannon's formula:

[0027]

[0028] Data transmission between any two servers is conducted via a wired connection at a fixed rate of r. m,m′ If the task If you choose to uninstall on another server, the transfer time will need to be supplemented by the wired transfer time between servers m and m'. Transmission time can be expressed as follows:

[0029]

[0030] After the task is uploaded to the MEC server, the latency of the task's computation on the MEC server is related to the number of CPU cycles required by the task and the selected server f. selected The calculation frequency is related to the calculation formula, which is as follows:

[0031]

[0032] Task start time The start time of a task is equal to the maximum sum of the completion times of all its preceding tasks and the waiting time. The sum of .

[0033]

[0034] Mission End Time The task's end time is equal to the sum of the task's start time, transmission time, and computation time.

[0035]

[0036] Application end time The application's end time is equal to the maximum of the completion times of all tasks within that application.

[0037]

[0038] Let the start time of the job be the arrival time of the application. The application's scheduling time is It can be represented by the application's start time and end time:

[0039]

[0040] Step 2: Establish the calculation unloading optimization problem in the MEC system.

[0041] Optimization goal: Minimize the scheduling time of all applications in the system.

[0042]

[0043]

[0044]

[0045] Constraint 1: A subtask can only be scheduled after all preceding tasks have been completed.

[0046]

[0047] Constraint 2: Subtasks can only select one of all servers to uninstall;

[0048]

[0049] in, To unload decision variables, if Representative subtask Uninstall on the m-th server, otherwise...

[0050] Step 3: Use graph neural networks to solve the embedding problem of time-dependent business states.

[0051] When using DRL to solve the computational offloading problem, it is necessary to transform the state information into features and pass them to the policy network. However, most neural networks are only suitable for processing Euclidean space data and are not suitable for processing the offloading problem of DAG type applications. In order to better input DAG type graphs of any number and size into the neural network of DRL, a graph neural network is used to process scalable states. This network encodes the state information into a set of embedding vectors.

[0052] Step 3.1 Obtain the task-level embedding;

[0053] like Figure 3 As shown in the Graph Neural Network section, the Graph Neural Network takes a DAG-type application as input. Each task in the application has a set of features (task CPU cycles, data upload size, average transfer time on the server, average computation time on the server, etc.), and outputs two levels of embeddings: task-level embeddings and global-level embeddings. The task-level embeddings capture information about the task and its subtasks, while the global-level embeddings contain embedding information from all DAG-type applications.

[0054] Task-level embedding: The i-th subtask of the n-th application generated by the u-th user. The original characteristics are In this method, the task The original characteristics include the amount of data uploaded by the task. Task Number of CPU cycles required Task Transmission time with the server where the serving base station is located Average computation time of the task on servers with different computing capabilities

[0055]

[0056] By using two layers of nonlinear functions f1(·) and f2(·), the nth application G generated by the uth user is obtained. un any subtask Embedded features Features are passed sequentially from the leaf nodes of the Directed Acyclic Graph (DAG) to their predecessor nodes. Any subtask... The embedded calculation method is as follows:

[0057]

[0058] Here, f1(·) and f2(·) are nonlinear functions composed of small neural networks, with f1(·) aggregating the tasks. All forward tasks Embedded.

[0059] Step 3.2 Obtain the global-level embedding.

[0060] Global-level embeddings contain embedding information from all DAG-type applications (e.g., the number of applications). Graph neural networks first compute the sum of embeddings for all nodes of each DAG. To compute the global-level embedding, a summary node is added to each DAG graph. This node has all nodes in the DAG as its children, thus obtaining the global-level summary.

[0061] Step 4: Use deep reinforcement learning algorithms to make unloading decisions

[0062] To balance the size of the action space and the number of operations, the scheduling decision is decomposed into a series of two-dimensional actions, outputting the next scheduled task and the server to which the task should be unloaded. The scheduling agent observes the system's state to determine each scheduling operation, receives rewards based on the optimization objective, and uses a graph neural network to transform the application DAG into a vector of the policy network. The rewards guide the policy network to output the action for each decision.

[0063] Step 4.1 Model the Markov (MDP) process based on the unloading problem;

[0064] In the framework of reinforcement learning, the machine that learns and makes decisions is called an agent, and the things that the agent interacts with are called the environment. At each discrete time t, the agent interacts with the environment and observes a certain characteristic of the state of the environment. And based on this, choose an action. Receive a reward in the next moment As a result of choosing this action, a new state S is entered. t+1 This process is called a Markov decision process, and is usually described as a quadruple. Let these represent the state space, action space, state transition probability, and reward, respectively. State space The states input into the graph neural network include the application state S. Gun and the status S of the task Gun,i Two parts.

[0065]

[0066] Any task Status: The status of a task includes the amount of data uploaded. CPU cycles required for the task Transmission time between the task and the server to which the user is connected Average computation time of the task on servers with different computing capabilities

[0067]

[0068] The application's state includes: the index I of the server that the user u of the application is connected to. u The channel gain g between the user u where the application is located and the connected server m u,m .

[0069]

[0070] Action space The decision-making action is a two-dimensional action, namely, outputting the next scheduled task V. next And the server s to which the task needs to be performed for uninstallation. next This optimizes both the uninstallation location and the uninstallation sequence.

[0071] A t = <V next ,s next >

[0072] Scheduling event: When a task completes, the release of the corresponding executor or the arrival of a new application is called a scheduling event at time t.

[0073] Task selection: When the scheduling event occurs at time t, the corresponding state is S. t Any subtask of the nth application generated by the uth user Calculate a fraction Transforming the vectors embedded in the graph neural network into a scalar, to some extent, allows the fraction to represent schedulable tasks. The priority is determined, and then the probability of selecting a task is calculated based on the priority score using a softmax operation.

[0074]

[0075] in, G(x) is the set of tasks that can be scheduled at time t, and G(x) is the application DAG where task x is located.

[0076] Server selection: For each server, the policy network also computes a score g(·), and uses another score function g(·) to assign an unloading location for the task. Similar to selecting schedulable tasks, the system applies a softmax operation to these scores to calculate the probability of selecting a server.

[0077] Reward R: Let T be the total number of actions in one epoch, t k J is the clock count corresponding to the k-th action. k Representative in (t) k-1 ,t k Applications existing within a time period.

[0078] R k =-(t) k -t k-1 )J k

[0079] Step 4.2 Use the policy gradient algorithm to make the unloading decision;

[0080] Policy gradient methods are a class of reinforcement learning methods that directly optimize policies using gradient descent based on the expected return. These methods avoid some of the difficulties faced by other traditional reinforcement learning methods, such as the lack of an accurate value function, or the intractability caused by continuous state and action spaces and the uncertainty of state information.

[0081] Using the reference function b(S) t′ This paper presents a policy gradient algorithm that significantly improves the performance of policy gradient methods. All operations in this system, from the graph neural network to the policy network, are differentiable. For simplicity, all parameters in these operations are jointly represented as θ, and the scheduling policy π is used. θ (A t |S t ) is defined as being in state S t The probability of taking action at each step t. Consider an event of length T, where the agent collects (state, action, reward) at each step t, i.e., (S... t A t ,R t The agent updates its policy π using the REINFORCE policy gradient algorithm. θ (A t |S t The parameter θ of ) is given by the benchmark function b(S). t′ The gradient of the objective function Written as:

[0082]

[0083] The parameters are updated as follows, where 'a' is the learning rate:

[0084]

[0085] Technical Summary:

[0086] Compared to existing technologies, this invention considers a more complex MEC system, conducting fine-grained offloading research on dynamically arriving time-dependent services in a multi-user, multi-server network scenario. Subtasks of applications generated by each user can be offloaded to directly connected servers or other servers. This invention utilizes graph neural networks to more effectively capture the characteristic states of time-dependent services and applications of arbitrary size and shape, comprehensively considering both task-specific and global information. Finally, a Markov process model is applied to the offloading problem, and a policy gradient algorithm based on deep reinforcement learning is used for offloading decisions, selecting tasks to be scheduled and choosing servers to offload to, thus optimizing both the offloading timing and location. Attached Figure Description

[0087] Figure 1 The system model includes a schematic diagram of the structure of the base station, the user, and the application that generates streaming arrivals for the user.

[0088] Figure 2 The application model graph represents the dependencies of time-dependent business processes in the form of a directed acyclic graph.

[0089] Figure 3 The graph neural network and deep reinforcement learning framework demonstrate the correlation between state, action, and reward. In the graph neural network section of the diagram, the squares represent task-level feature summaries, and the triangles represent summaries of state information at all application levels.

[0090] Figure 4 The graph represents the convergence of the algorithm when the learning rate lr is equal to 0.001. The horizontal axis represents the number of epochs in the training process, and the vertical axis represents the total discounted reward during training.

[0091] Figure 5 The figure shows a comparison of the average application scheduling delay (s) in this system with other algorithms, including the first-in-first-out algorithm and the random scheduling method. Detailed Implementation

[0092] The following section provides a further explanation of the technical solution for the time-dependent service offloading method based on graph neural networks, in conjunction with accompanying figures and examples.

[0093] The system model diagram involved in this invention is shown below. Figure 1 .

[0094] The application model diagram involved in this invention is shown below. Figure 2 .

[0095] This invention relates to graph neural networks and deep reinforcement learning algorithm frameworks. Figure 3 .

[0096] The convergence graph of the algorithm involved in this invention is shown in the figure. Figure 4 .

[0097] The comparison chart of the average completion time of the system application involved in this invention with other algorithms is shown below. Figure 5 .

[0098] 1. System initialization: Generate the number of users and servers in the small system, the location relationship between users and servers, the arrival rate of applications generated by users within a certain period, the adjacency matrix read by applications, the number of tasks contained in each application, system bandwidth, user transmit power and noise power, and the computing power of different servers.

[0099] 2. Based on problem modeling, the time delay under different unloading decisions is obtained;

[0100] 3. Establish a Markov decision process (state, action, reward) based on the unloading problem;

[0101] 4. The task-level and application-level feature embeddings are obtained through the message passing and embedding process of the graph neural network, and the DAG is transformed into a vector of the policy network.

[0102] 5. Perform iterative learning based on policy gradient algorithm, select actions (schedulable tasks and selected servers) to update the state and obtain rewards.

[0103] 6. Receive rewards based on the optimization goal, and use the rewards to guide the policy network in outputting actions for each decision.

[0104] System main parameter settings:

[0105]

[0106]

[0107] Algorithm 1 shows the flowchart of the policy gradient algorithm with baseline:

[0108]

[0109] Algorithm 2 illustrates the overall flowchart of a multi-user time-dependent service offloading method based on graph neural networks.

[0110]

[0111]

[0112] Figure 4 This is the convergence graph of the algorithm when the learning rate is 1e-3, derived from... Figure 4It can be seen that the total discount reward of the system gradually increases with the number of epochs until it remains basically constant, that is, the algorithm is convergent.

[0113] Figure 5 This is a comparison chart of the average application completion time under the present invention's scheme and the application of a first-in-first-out (FIFO) and random server and task uninstallation scheme. The number of users is 6, and the total number of applications generated by all users is 27. Each application contains 4 to 6 tasks, and all applications arrive within 10 seconds. Figure 5 It can be seen that the average application scheduling time of the present invention is lower and the effect is better than other solutions.

Claims

1. A multi-user dependent business computation offloading optimization method based on graph neural networks, characterized in that, Includes the following steps: Step 1: Establish a system model based on the network scenario; The network scenario is a multi-user, multi-server MEC system. In the MEC system, each user generates a series of streaming applications, and each application consists of time-dependent tasks. Step 1.1 Based on server and user information, establish a dynamic model for application streaming arrival in a multi-user, multi-server scenario; In a MEC system composed of multiple heterogeneous edge servers, the user equipment and server sets in the MEC system are respectively composed of and Let u represent any user device in the set, m represent any server in the server set, and f represent the computing power of any server. m ; The application is modeled as a directed acyclic graph (DAG), and a DAG is described as follows: E and E represent the vertex set and edge set of the graph, respectively. In the MEC system, any user... The resulting set of applications can be represented as G. u ={G u1 G u2 ,...,G un ,...,G uNu },G un On behalf of users Any application n; N generated u On behalf of users The total number of applications generated; Wherein, any application G generated by user u (u∈U) un The number of subtasks included is G un The set of all tasks is represented as: Representing G un Total number of tasks Representative application G un The i-th subtask This represents the dependencies between tasks. Represents and There is a dependency relationship between them, and It must be ensured that You can only execute it after it has been completed. Called The father's task, yes Subtasks, any one of the subtasks The set of all forward tasks is represented as The set of backward task subtasks is represented as The amount of data uploaded by the task is The required number of CPU cycles is Each user generates a series of applications that arrive in a streaming manner. According to the assumptions of queuing theory, the time intervals between applications arriving at a user within a certain period of time follow an exponential distribution. The time interval between a user arriving at the first application from time 0 is λ0, and the time interval between the nth application and the (n+1)th application is λ. n : l n ~Exp(λ) Where λ is the arrival rate parameter, representing the average number of applications generated by a user per unit time. The relationship between the arrival times of two adjacent applications is as follows: Step 1.2 Model the multi-user multi-server system. Users communicate with directly connected base stations. When a subtask generated by a user in an application is offloaded, it is offloaded to the MEC server where its serving base station is located, or to the MEC server of other adjacent base stations. The system latency model can be obtained based on the impact of the offloading decision on latency. Users and edge servers transmit wirelessly via base stations; B is the channel bandwidth, p u,m It is the user's upload power, g u,m The channel gain between user u and the connected server m is used to calculate the transmission rate using Shannon's formula: Data transmission between any two servers is conducted via a wired connection at a fixed rate of r. m,m′ If the task If you choose to uninstall on another server, the transfer time will need to be supplemented by the wired transfer time between servers m and m'. Transmission time can be written as follows: After the task is uploaded to the MEC server, the latency of the task's computation on the MEC server is related to the number of CPU cycles required by the task and the selected server f. selected The calculation frequency is related to the calculation formula, which is as follows: Task start time The start time of a task is equal to the maximum sum of the completion times of all its preceding tasks and the waiting time. The sum of; Mission End Time The end time of a task is equal to the sum of its start time, transmission time, and computation time. Application end time The application's end time is equal to the maximum of the completion times of all tasks in that application; Let the start time of the job be the arrival time of the application. The application's scheduling time is It can be represented by the application's start time and end time: Step 2: Establish the calculation and unloading optimization problem in the MEC system; Optimization goal: Minimize the scheduling time of all applications in the system. Constraint 1: A subtask can only be scheduled after all preceding tasks have been completed. Constraint 2: Subtasks can only select one of all servers to uninstall; in, To unload decision variables, if Representative subtask Uninstall on the m-th server, otherwise... Step 3: Use graph neural networks to solve the embedding problem of time-dependent business states; Graph neural networks are used to process scalable states, which encode state information into a set of embedding vectors; Step 3.1 Obtain the task-level embedding; Graph neural networks take DAG-type applications as input, with each task of the application having a set of features, and output two levels of embeddings: task-level embeddings and global-level embeddings. Task-level embeddings capture information about the task and its subtasks, while global-level embeddings contain embedding information from all DAG-type applications. Task-level embedding: The i-th subtask of the n-th application generated by the u-th user. The original characteristics are In this method, the task The original characteristics include the amount of data uploaded by the task. Task Number of CPU cycles required Task Transmission time with the server where the serving base station is located Average computation time of the task on servers with different computing capabilities By using two layers of nonlinear functions f1(·) and f2(·), the nth application G generated by the uth user is obtained. un any subtask Embedded features Features are passed sequentially from the leaf nodes of the DAG to the preceding nodes; any subtask The embedded calculation method is as follows: Here, f1(·) and f2(·) are nonlinear functions composed of small neural networks, with f1(·) aggregating the tasks. All forward tasks Embedding; Step 3.2 Obtain the global-level embedding. Global-level embeddings contain embedding information from all DAG-type applications; graph neural networks first compute the sum of embeddings for all nodes of each DAG. To compute the global-level embedding, a summary node is added to each DAG graph. This node has all nodes in the DAG as its children, thus obtaining the global-level summary. Step 4: Use deep reinforcement learning algorithms to make unloading decisions To balance the size of the action space and the number of operations, the scheduling decision is decomposed into a series of two-dimensional actions, outputting the next scheduled task and the server to which the task should be unloaded. The scheduling agent observes the state of the system to determine each scheduling operation, receives rewards according to the optimization objective, and uses a graph neural network to transform the application DAG into a vector of the policy network. The rewards guide the policy network to output the action for each decision. Step 4.1 Model the Markov MDP process based on the unloading problem; In the framework of reinforcement learning, at each discrete time t, the agent interacts with the environment and observes a certain feature representation of the state of the environment. And based on this, choose an action. Receive a reward in the next moment As a result of choosing this action, a new state S is entered. t+1 It is described as a quadruple. Let these represent the state space, action space, state transition probability, and reward, respectively. state space The states input into the graph neural network include the application state. and the status of the task Two parts; Any task Status: The status of a task includes the amount of data uploaded. CPU cycles required for the task Transmission time between the task and the server to which the user is connected Average computation time of the task on servers with different computing capabilities The application's state includes: the index I of the server that the user u of the application is connected to. u The channel gain g between the user u where the application is located and the connected server m u,m ; Action space The decision-making action is a two-dimensional action, namely, outputting the next scheduled task V. next And the server s to which the task is to perform the uninstallation. next This optimizes both the uninstallation location and the uninstallation timing. A t =<V next ,s next > Scheduling event: When a task completes, the release of the corresponding executor or the arrival of a new application is called a scheduling event at time t; Task selection: When the scheduling event occurs at time t, the corresponding state is S. t Any subtask of the nth application generated by the uth user Calculate a fraction Transforming the vectors embedded in the graph neural network into a scalar, to some extent, allows the fraction to represent schedulable tasks. The priority is determined, and then the probability of selecting a task is calculated based on the priority score using a softmax operation. in, G(x) is the set of tasks that can be scheduled at time t, and G(x) is the application DAG where task x is located. Server selection: For each server, the policy network also calculates a score g(·), and uses another score function g(·) to assign an unloading location for the task; similar to the selection of schedulable tasks, the system applies a softmax operation to these scores to calculate the probability of selecting a server; award Let T be the total number of actions in one epoch, t k J is the clock count corresponding to the k-th action. k Representative in (t) k-1 ,t k Applications existing within a certain time period; R k =-(t k -t k-1 )J k Step 4.2 Use the policy gradient algorithm to make the unloading decision; Using the reference function b(S) t′ The policy gradient algorithm represents all parameters in the operation as θ, and the scheduling policy π is used to represent the policy gradient. θ (A t |S t ) is defined as being in state S t The probability of taking action at each step t; consider an event of length T, where the agent collects (state, action, reward) at each step t, i.e., (S t A t ,R t The agent updates its policy π using the REINFORCE policy gradient algorithm. θ (A t |S t The parameter θ of the reference function b(S) is given; t′ The gradient of the objective function Written as: The parameters are updated as follows, where 'a' is the learning rate:

2. The multi-user dependent service computation offloading optimization method based on graph neural networks according to claim 1, characterized in that, A group of user equipment is connected to the same base station. During the offloading process, the user communicates with the directly connected server. The user task is offloaded to the directly connected server or to other servers via wired transmission.

Images