Graph convolutional network enhanced edge computing task unloading method

Abstract

The invention discloses a graph convolutional network enhanced edge computing task unloading method, relates to the technical field of wireless communication, and solves the problem of dependency on task unloading in a mobile edge computing system with heterogeneous resources. A GCN is a machine learning model, can embed information of adjacent nodes, and can be used for realizing task unloading. According to the basic principle of the downloading method, the task dependency relationship is described by using the GCN, and a dependency task unloading decision problem is converted into a node classification problem; as task node features can be embedded together, the GCN can perform node classification according to an embedding result; then, the superiority of an unloading decision is determined by calculating the total time required for completing execution of the whole application program, so that the scheduling and resource allocation efficiency is improved, the system can deal with dynamic changes and resource limitations more effectively, and more intelligent task management is realized.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular to a graph convolutional network enhanced edge computing task offloading method. Background Art

[0002] Applications running on mobile devices may have low latency and computationally intensive requirements (such as AR / VR, autonomous driving, smart robots, etc.), and the computing power of local devices may not be able to meet these requirements. For this reason, tasks can be offloaded to nearby wireless edge nodes. The edge node here refers to a device with computing and storage resources located near the mobile device running the application. Compared with remote cloud systems, edge nodes have limited computing resources. Therefore, it is very important to make offloading decisions effectively to better utilize the resources of edge nodes.

[0003] Typically, an application that needs to be offloaded can be represented as a set of interdependent tasks, and the relationships between these tasks can form a directed acyclic graph (DAG). In a DAG, each vertex is a task, and each edge represents the dependency between tasks. Specifically, the dependency indicates that these tasks should be executed in a certain order. In a mobile edge computing system, how to make effective offloading decisions while considering task dependencies is a difficult problem. The challenge lies in how to minimize the completion time of the entire application while satisfying the dependencies between tasks in a resource-constrained mobile edge computing system. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a graph convolutional network enhanced edge computing task offloading method, comprising the following steps:

[0005] S1. Build an edge computing model and initialize user nodes and server nodes in the edge computing environment. The server node is the edge node.

[0006] S2. Determine the optimization goal, describe the optimization problem as a Markov decision process, and determine the objective function and constraints;

[0007] S3. Build a deep reinforcement learning model GRL based on graph convolutional network to determine the state, action and reward function of the agent;

[0008] S4. Train the deep reinforcement learning model GRL based on graph convolutional network. The intelligent agent optimizes and updates its own network and optimization strategy by continuously interacting with the environment, and finally obtains the optimal task offloading strategy.

[0009] The technical solution further defined in the present invention is:

[0010] Furthermore, in step S1, the number of edge nodes is initialized to m, denoted as N = {N1, N2, ..., N m}, using a discrete time model; the edge node N e The resources are recorded as (c e , b e ), where c e Represents edge node N e The number of CPU cycles processed per second, b e Represents edge node N e The rate at which data is transmitted per second.

[0011] As described above, in a graph convolutional network enhanced edge computing task offloading method, in step S2, each application consists of multiple tasks, and the relationship between the tasks is represented as a directed acyclic graph, that is, the vertices of the directed acyclic graph are tasks, and the edges represent the computational dependencies between the tasks.

[0012] As described above, in the edge computing task offloading method enhanced by graph convolutional network, in step S2, each application A is represented as G A =(V A , E A ), where V A ={k1, k2, ..., k n} represents the task set, E A represents the set of edges connecting tasks with execution dependencies, i.e., if task k i In task k j If it is executed before <k i , k j >∈E A ; For each task k i Further expressed as k i =(p i , r i ), p i Indicates the number of CPU cycles required to execute the task, r i represents the data size of the execution result; and the task set V A The tasks in have an execution order that satisfies the topological order in the directed acyclic graph of application A.

[0013] As described above, in the edge computing task offloading method enhanced by a graph convolutional network, in step S2, each task is regarded as a basic unit size, that is, each task can only be offloaded to one edge node; suppose Represents task k i Whether to offload to the edge node decision variable, if so, then otherwise therefore:

[0014]

[0015] make Represents task k i The start time of the execution, so Not less than zero, that is Assume that task k i Offloaded to edge node N e ,but Calculation time Expressed as

[0016] Communication time occurs only when a task and its subsequent tasks in the directed acyclic graph are executed on two different edge nodes. Assume that task k j It is task k i The successor node of task k i The execution result of i Transmit from edge node to adjacent edge node to perform task k j The communication time required is:

[0017]

[0018] For tasks offloaded to the same edge node, the processing order follows the principle that the smaller the topological order of the task, the earlier it is processed. That is, if task k i and task k j are offloaded to the same edge node, and i < j, then the edge node will first execute task k i , and then execute task k j ; Each edge node fully dedicates its computing resources to the task it is executing, and once it is completed, it will move on to the next task, as shown in the following equation:

[0019]

[0020] The following conditions should also be met:

[0021]

[0022] For two interdependent tasks k i and task k j , if they are all offloaded to the same edge node, then task k i The calculation result is transmitted to the subsequent task k j The time taken to execute is zero.

[0023] As described above, in a graph convolutional network enhanced edge computing task offloading method, in step S2, the minimum completion time of each application does not exceed the sum of the computation time of all tasks plus the necessary transmission time between tasks, and the optimization goal is to find the application with the minimum completion time T A Therefore, it should satisfy the following conditions:

[0024]

[0025] Therefore, the optimization goal of the entire unloading process is set as:

[0026] P:minT A .

[0027] As described above, in a graph convolution network enhanced edge computing task offloading method, in step S3, the state of the agent includes edge node features, application task features, and current offloading results; the edge node features include computing resources and communication resources of each edge node; the application task features include task features and task dependencies, the task features include CPU cycles and data size, and the task features and task dependencies are represented as an adjacency matrix of application A in the form of a directed acyclic graph;

[0028] Construct the following four matrices respectively:

[0029]

[0030] Where X is the task feature matrix; adj(A) is the application adjacency matrix; N is the edge node feature matrix; c i represents the computing resources of edge node i; b i represents the communication resources of edge node i; t is the offloading result matrix, where each element records the offloading result of each task; 1j Indicates the offloading result of each task; for tasks whose offloading decision has not yet been determined, 0 is used as their offloading result;

[0031] Using the above four matrices as input to construct the state, first construct A = adj(A) + I A and D, where I A is an identity matrix, D is a diagonal matrix, D ii =∑ j＝0 A ij is the element on the diagonal of the diagonal matrix, which is the sum of the elements in the i-th row of the matrix A; then calculate the aggregated directed acyclic graph feature matrix where θ f is the network parameter of the graph convolutional network; for the edge node feature matrix N and the unloading result matrix O t, use linear transformation to derive the dimension of the directed acyclic graph features that need to be further connected; then the matrix X′, the edge node feature matrix N after linear transformation, and the unloading result matrix O t concatenate together to generate state s t .

[0032] As described above, in the edge computing task offloading method enhanced by a graph convolutional network, in step S3, for the agent that makes the offloading decision, three convolutional layers and one graph convolutional network layer are used to construct the agent, and the graph convolutional network layer is used as the last layer of the agent; the agent takes the state as input, and then outputs the offloading result of the first task whose offloading decision is 0, and t Update to O t+1 , O t+1 Used to construct the next state.

[0033] As described above, in the edge computing task offloading method enhanced by graph convolutional network, in step S3, task k j Offload to edge node N e Rewards j Set to where k i In k j In front, and k i and k j All are offloaded to edge node N e ; The computation and communication time is set as negative reward, and the time saved due to two dependent tasks being offloaded to the same edge node is set as positive reward.

[0034] As described above, a graph convolutional network enhanced edge computing task offloading method, in step S4, the deep reinforcement learning model GRL based on the graph convolutional network includes a dual deep Q network structure, the dual deep Q network structure includes a main network and a target network, the main network is used to select actions, and the target network is used to evaluate actions and calculate target Q values; the graph convolutional network is used to extract features and represent nodes in a directed acyclic graph, and these graph features are then used as state inputs of the dual deep Q network structure, and the nodes in the directed acyclic graph include user nodes and server nodes.

[0035] The beneficial effects of the present invention are:

[0036] (1) In the present invention, for edge computing scenarios with time-varying properties, most traditional algorithms can no longer meet the high-demand task offloading solutions, while the deep reinforcement learning algorithm used in the present invention can effectively cope with the time-varying nature of information transmission, which is more practical;

[0037] (2) In the present invention, in edge computing task offloading, the GCN-enhanced deep reinforcement learning algorithm can improve the efficiency and accuracy of task allocation; GCN can effectively capture the complex relationships and dependencies between tasks, while deep reinforcement learning dynamically adjusts task offloading decisions through optimization strategies; this combination helps to make more intelligent decisions, thereby improving the overall performance and resource utilization of the system;

[0038] (3) In the present invention, task offloading is described as the dependency relationship between nodes and edges, which can more clearly show the complex interdependence and data flow between tasks. This representation method helps to accurately capture the impact between tasks and improve the efficiency of scheduling and resource allocation;

[0039] (4) In the present invention, by utilizing graph algorithms to optimize these dependencies, the system can more effectively cope with dynamic changes and resource constraints, thereby achieving more intelligent task management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0041] Figure 2 A schematic diagram of an application in DAG format in an embodiment of the present invention;

[0042] Figure 3 It is a training flow chart of the deep reinforcement learning model GRL based on graph convolutional network in an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of the network framework of the deep reinforcement learning model GRL based on graph convolutional network in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] This embodiment provides a graph convolutional network enhanced edge computing task offloading method, such as Figure 1 As shown, the following steps are included:

[0045] S1. Build an edge computing model, initialize user nodes and server nodes in the edge computing environment, where the server nodes are edge nodes; initialize the number of edge nodes to m, denoted by N = {N1, N2, ..., N m}, using a discrete time model; the edge node N e The resources are recorded as (c e , b e ), where c e Represents edge node N e The number of CPU cycles processed per second, b e Represents edge node N eThe data rate transmitted per second (in bps); since only nearby edge nodes are considered, the propagation delay can be ignored and the communication time only accounts for the transmission time between edge nodes.

[0046] Assume that the computing resources of each mobile device running an application are limited and cannot meet the needs of the application to execute tasks. Therefore, the mobile device must offload all its tasks to the adjacent wireless edge nodes. There are m edge nodes equipped with computing resources in the system, denoted as N = {N1, N2, ..., N m}, which can be servers, laptops, etc., so their computing and communication resources may be heterogeneous; in order to make an offloading decision for each task, the set of edge nodes considered is the set of nodes accessible to the mobile device.

[0047] S2. Determine the optimization goal, describe the optimization problem as a Markov decision process, determine the objective function and constraints; each application consists of multiple tasks, and the relationship between the tasks is represented as a directed acyclic graph (DAG), that is, the vertices of the DAG are tasks, and the edges represent the computational dependencies between tasks. Each task requires a certain number of CPU cycles to execute, and the results are obtained after execution.

[0048] Denote each application A as G A =(V A , E A ), where V A ={k1, k2, ..., k n} represents the task set, E A represents the set of edges connecting tasks with execution dependencies, i.e., if task k i In task k j Before execution, then <k i , k j >∈E A ; For each task k i Further expressed as k i =(p i , r i ), p i Indicates the number of CPU cycles required to execute the task, r i represents the data size of the execution result; and the task set V A The tasks in have an execution order that satisfies the topological order in the directed acyclic graph of application A; for example, Figure 2 As shown, an application in the form of a DAG with topological order is shown, that is, k2 and k3 can be executed in parallel, and they must run after k1 and before k4.

[0049] Consider each task as a basic unit size, that is, it cannot be divided into smaller parts, so each task can only be offloaded to one edge node; Represents task k i Whether to offload to the edge node decision variable, if so, then otherwise therefore:

[0050]

[0051] make Represents task k i The start time of the execution, so Not less than zero, that is Assume that task k i Offloaded to edge node N e ,but Calculation time Expressed as

[0052] Communication time occurs only when a task and its subsequent tasks in the directed acyclic graph are executed on two different edge nodes. Assume that task k j It is task k i The successor node of task k i The execution result of i Transmit from edge node to adjacent edge node to perform task k j The communication time required is:

[0053]

[0054] For tasks offloaded to the same edge node, the processing order follows the principle that the smaller the topological order of the task, the earlier it is processed. In other words, if task k i and task k j are offloaded to the same edge node, and i < j, then the edge node will first execute task k i , and then execute task k j , which ensures that the dependency order of tasks is not violated.

[0055] Each edge node fully dedicates its computing resources to the task it is executing, and once it is completed, it will move on to the next task, as shown in the following equation:

[0056]

[0057] The following conditions should also be met:

[0058]

[0059] For two interdependent tasks ki and task k j , if they are all offloaded to the same edge node, then task k i The calculation result is transmitted to the subsequent task k j The time taken to execute is zero.

[0060] The minimum completion time of each application does not exceed the sum of the computation time of all tasks plus the necessary transmission time between tasks. The optimization goal is to find the application with the minimum completion time T. A Therefore, it should satisfy the following conditions:

[0061]

[0062] Therefore, the optimization goal of the entire unloading process is set as:

[0063] P:minT A .

[0064] S3. Build a deep reinforcement learning (DRL) model GRL based on the graph convolution network (GCN) to determine the state, action and reward function of the agent.

[0065] Machine learning has the ability to handle complex problems, but these models are often not suitable for processing non-Euclidean data such as graphs. In order to solve problems related to graphs, several methods have been proposed, such as graph recursive neural networks (Graph RNNs), graph convolutional networks (Graph Convolutiona]Network, GCN), and graph autoencoders. In this embodiment, GCN is used because GCN defines how to apply convolution on graphs, which can capture the structure of graphs, achieve the purpose of content dissemination, and find high-level expressions of graphs.

[0066] For the method of this embodiment, the dependency between tasks makes the offloading problem a challenge. Specifically, it is very difficult to make effective offloading decisions on resource-constrained edge nodes by processing the DAG structure (i.e., graph-related features) through machine learning models. For this reason, this embodiment adopts GCN because it has an excellent ability to capture the features of adjacent nodes in the graph for classification. Technically, each GCN can be expressed as

[0067]

[0068] Among them, X is the graph input feature, D and A are derived from the adjacency matrix of X, v is the GCN model parameter, and Z is the new feature obtained; the goal is to find a suitable Θ that outputs the correct Z to extract useful features for subsequent classification.

[0069] In order to solve the problem of dependent task offloading, this embodiment designs a new deep reinforcement learning model GRL based on graph convolutional network; the main idea of ​​the GRL model is to combine DAG features and edge node features together for offloading, and the result is used as the environment (i.e., state), which is the input of DRL; then, the DRL agent outputs the offloading result of each task separately as an action output to further change the environment state.

[0070] The states, actions, and rewards of the GRL model are described as follows:

[0071] State: First, the state of the environment is constructed, which consists of three items: edge node characteristics, application task characteristics, and current offloading results; edge node characteristics include the computing resources and communication resources of each edge node; application task characteristics include task characteristics (i.e., CPU cycles and data size) and task dependencies, which are represented as an adjacency matrix of application A in the form of a directed acyclic graph, where if<i,j> ∈E A , then a ij =1, otherwise it is zero; the offloading result matrix is ​​a 1×n matrix, each element of which records the offloading result of each task; for those tasks whose offloading decisions have not been determined, 0 is used as their offloading results.

[0072] Therefore, the four matrices can be expressed as:

[0073]

[0074] Where X is the task feature matrix; adj(A) is the application adjacency matrix; N is the edge node feature matrix; c i represents the computing resources of edge node i; b i represents the communication resources of edge node i; t is the unloading result matrix; o 1j Indicates the offloading result of each task.

[0075] Using the above four matrices as input to construct the state, first construct A = adj(A) + I A and D, where I A is an identity matrix, D is a diagonal matrix, D ii =∑ j＝0 A ij is the element on the diagonal of the diagonal matrix, which is the sum of the elements in the i-th row of the matrix A; then calculate the aggregated directed acyclic graph feature matrix where θ f is the network parameter of the graph convolutional network; for the edge node feature matrix N and the unloading result matrix O t , use linear transformation to derive the dimension of the directed acyclic graph features that need to be further connected; then the matrix X′, the edge node feature matrix N after linear transformation, and the unloading result matrix O t concatenate together to generate state s t .

[0076] Action: For the agent that makes the offloading decision, three convolutional layers and a graph convolutional network layer are used to build the agent. The graph convolutional network layer is used as the last layer of the agent because this problem is regarded as a node classification problem. The agent takes the state as input and then outputs the offloading result of the first task with an offloading decision of 0. t Update to O t+1 , O t+1 Used to construct the next state; in addition, due to the limited number of actions, that is, a task can only be offloaded to one of the m edge nodes, a double deep Q-network (DDQN) structure is adopted.

[0077] Reward: Set task k j Offload to edge node N e Rewards j Set to where k i In k j In front, and k i and k j All are offloaded to edge node N e ; The reward function means that we want to minimize the task completion time, so the computation and communication time is a negative reward, while the time saved due to two dependent tasks being offloaded to the same edge node is set as a positive reward.

[0078] S4. Train the deep reinforcement learning model GRL based on graph convolutional network. The training process is as follows: Figure 3 As shown in the figure, the agent optimizes and updates its own network and optimization strategy by continuously interacting with the environment, and finally obtains the optimal task offloading strategy.

[0079] The algorithm designed in this embodiment is used to optimize task offloading. The traditional DQN uses the same network for action selection and Q-value estimation, which may lead to overestimation of the Q value. The deep reinforcement learning model GRL based on graph convolutional network in this embodiment uses a double deep Q network (Double Deep Q-Network, DDQN) structure. The double deep Q network structure includes a main network and a target network. The main network is used to select actions, and the target network is used to evaluate actions, thereby reducing the over-estimation problem and improving the stability and accuracy of learning. The separate selection and evaluation mechanism of DDQN makes the action selection process more stable.

[0080] The main network selects actions, while the target network is used to calculate the target Q value, which avoids the influence of overestimation of the main network when selecting actions; for further optimization, the algorithm combines GCN and uses GCN to process the graph data in the task offloading system. This graph data specifically refers to the data related to the nodes (including users and servers) in the task offloading system; specifically, GCN extracts features and learns representations of nodes in the directed acyclic graph, and these graph features are then used as the state input of the dual deep Q network structure. The nodes in the directed acyclic graph include user nodes and server nodes; GCN provides rich state representations by aggregating information from neighboring nodes, and DDQN uses these representations for action selection and Q value estimation; in this way, GCN provides DDQN with more accurate and comprehensive environmental state information, thereby improving the effect of task offloading optimization. The network architecture of GRL is as follows: Figure 4 shown.

[0081] The objective function is solved by GRL to obtain the optimal task offloading strategy, which includes the following steps:

[0082] S4.1. The task offloading system is modeled as a graph, with users and servers as nodes, and edges between nodes representing relationships. The node features are processed using a graph convolutional network (GCN) or a graph attention network (GAT) to generate hidden features of the nodes.

[0083] S4.2. Initialize the policy network (Actor) and Q-value network (Critic) parameters, initialize the target network parameters, create and initialize the experience replay buffer to store the agent's experience samples.

[0084] S4.3. The agent interacts with the environment, the policy network generates actions, and the environment returns rewards and the next state. These experience samples (current state, action, reward, next state) are stored in the experience replay buffer. The experience samples include the current state, action, reward, and next state.

[0085] S4.4. Sample batch data from the experience replay buffer, calculate the target Q value, minimize the difference between the actual Q value and the target Q value, and update the Q-value network parameters; calculate the policy gradient and update the policy network to generate actions with higher Q values; use the soft update method to update the parameters of the target network so that it gradually approaches the current network parameters.

[0086] S4.5. Iterate the sampling, learning and optimization process until the strategy converges, ensuring that the strategy maximizes the total reward of the task offloading system, thereby finding the optimal task offloading strategy.

[0087] In order to solve the problem of dependent task offloading in a mobile edge computing system with heterogeneous resources, this embodiment proposes a GCN enhanced DRL model (GRL). GCN is a machine learning model that can embed information of adjacent nodes and effectively combine the characteristics of different nodes for further classification. The basic principle of the download method of the present invention is to use GCN to describe task dependencies and transform the dependent task offloading decision problem into a node classification problem; since task node features can be embedded together, GCN can classify nodes according to the embedding results; then, by calculating the total time required to complete the execution of the entire application, the superiority of the offloading decision is determined, thereby improving the efficiency of scheduling and resource allocation, and the system can more effectively cope with dynamic changes and resource constraints, thereby achieving more intelligent task management.

[0088] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.

Claims

1. A graph convolutional network enhanced edge computing task offloading method, characterized by: The following steps are involved: S1. Build an edge computing model and initialize user nodes and server nodes in the edge computing environment. The server node is the edge node. S2. Determine the optimization goal, describe the optimization problem as a Markov decision process, and determine the objective function and constraints; S3. Build a deep reinforcement learning model GRL based on graph convolutional network to determine the state, action and reward function of the agent; S4. Train the deep reinforcement learning model GRL based on graph convolutional network. The intelligent agent optimizes and updates its own network and optimization strategy by continuously interacting with the environment, and finally obtains the optimal task offloading strategy.

2. According to claim 1, a graph convolution network enhanced edge computing task offloading method is characterized by: In step S1, the number of edge nodes is initialized to m, denoted as N={N1, N2, ..., N m }, using a discrete time model; the edge node N e The resources are recorded as (c e , b e ), where c e Represents edge node N e The number of CPU cycles processed per second, b e Represents edge node N e The rate at which data is transmitted per second.

3. According to claim 1, a graph convolution network enhanced edge computing task offloading method is characterized in that: In step S2, each application is composed of multiple tasks, and the relationship between the tasks is represented as a directed acyclic graph, that is, the vertices of the directed acyclic graph are tasks, and the edges represent the computational dependencies between the tasks.

4. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 3 is characterized in that: In step S2, each application A is represented as G A =(V A , E A ), where V A ={k1, k2, ..., k n } represents the task set, E A represents the set of edges connecting tasks with execution dependencies, i.e., if task k i In task k j If it is executed before <k i , k j >∈E A ; For each task k i Further expressed as k i =(p i , r i ), p i Indicates the number of CPU cycles required to execute the task, r i represents the data size of the execution result; and the task set V A The tasks in have an execution order that satisfies the topological order in the directed acyclic graph of application A.

5. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 4 is characterized in that: In step S2, each task is regarded as a basic unit size, that is, each task can only be unloaded to one edge node; Represents task k i Whether to offload to the edge node decision variable, if so, then otherwise therefore: make Represents task k i The start time of the execution, so Not less than zero, that is Assume that task k i Offloaded to edge node N e ,but Calculation time Expressed as Communication time occurs only when a task and its subsequent tasks in the directed acyclic graph are executed on two different edge nodes. Assume that task k j It is task k i The successor node of task k i The execution result of i Transmit from edge node to adjacent edge node to perform task k j The communication time required is: For tasks offloaded to the same edge node, the processing order follows the principle that the smaller the topological order of the task, the earlier it is processed. That is, if task k i and task k j are offloaded to the same edge node, and i < j, then the edge node will first execute task k i , and then execute task k j ; each edge node dedicates all its computing resources to the task being executed. After completion, it will continue to execute the next task, as shown in the following formula: The following conditions should also be met: For two interdependent tasks k i and task k j , if they are all offloaded to the same edge node, then task k i The calculation result is transmitted to the subsequent task k j The time taken to execute is zero.

6. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 5, characterized in that: In step S2, the minimum completion time of each application does not exceed the sum of the computation time of all tasks plus the necessary transmission time between tasks. The optimization goal is to find the application with the minimum completion time T A Therefore, it should satisfy the following conditions: Therefore, the optimization goal of the entire unloading process is set as: P:minT A 。 7. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 1, characterized in that: In step S3, the state of the agent includes edge node characteristics, application task characteristics, and current unloading results; the edge node characteristics include computing resources and communication resources of each edge node; the application task characteristics include task characteristics and task dependencies, the task characteristics include CPU cycles and data size, and the task characteristics and task dependencies are represented as an adjacency matrix of application A in the form of a directed acyclic graph; Construct the following four matrices respectively: Where X is the task feature matrix; adj(A) is the application adjacency matrix; N is the edge node feature matrix; c i represents the computing resources of edge node i; b i represents the communication resources of edge node i; t is the offloading result matrix, where each element records the offloading result of each task; 1j Indicates the offloading result of each task; for tasks whose offloading decision has not yet been determined, 0 is used as their offloading result; Using the above four matrices as input to construct the state, first construct A = adj(A) + I A and D, where I A is an identity matrix, D is a diagonal matrix, D ii =∑ j＝0 A ij is the element on the diagonal of the diagonal matrix, which is the sum of the elements in the i-th row of the matrix A; then calculate the aggregated directed acyclic graph feature matrix where θ f is the network parameter of the graph convolutional network; for the edge node feature matrix N and the unloading result matrix O t , use linear transformation to derive the dimension of the directed acyclic graph features that need to be further connected; then the matrix X′, the edge node feature matrix N after linear transformation, and the unloading result matrix O t concatenate together to generate state s t .

8. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 7, characterized in that: In step S3, for the agent that makes the offloading decision, three convolutional layers and one graph convolutional network layer are used to construct the agent, and the graph convolutional network layer is used as the last layer of the agent; the agent takes the state as input, and then outputs the offloading result of the first task whose offloading decision is 0, and t Update to O t+1 , O t+1 Used to construct the next state.

9. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 8, characterized in that: In step S3, task k j Offload to edge node N e Rewards j Set to where k i In k j In front, and k i and k j All are offloaded to edge node N e ; The computation and communication time is set as negative reward, and the time saved due to two dependent tasks being offloaded to the same edge node is set as positive reward.

10. The method for offloading edge computing tasks enhanced by graph convolutional networks according to claim 1, characterized in that: In the step S4, the deep reinforcement learning model GRL based on graph convolutional network includes a dual deep Q network structure, which includes a main network and a target network. The main network is used to select actions, and the target network is used to evaluate actions and calculate target Q values. The graph convolutional network is used to extract features and represent nodes in a directed acyclic graph, and these graph features are then used as state inputs of the dual deep Q network structure. The nodes in the directed acyclic graph include user nodes and server nodes.

Images