An edge computing offloading method and system for graph neural network-assisted reinforcement learning

Abstract

This invention discloses an edge computing offloading method and system using graph neural network-assisted reinforcement learning, comprising: acquiring the decomposable tasks required for mobile edge node computing services and decomposing them into multiple subtasks; modeling the graph as a directed acyclic graph based on the dependencies between subtasks; a scheduler modeling a weighted graph based on the communication and computing conditions between all edge nodes; the scheduler modeling the node features and edge features of the directed acyclic graph and the weighted graph as tensors, respectively, and processing them based on a graph convolutional neural network; the scheduler feeding the final state feature tensor into a trained deep reinforcement learning agent for processing, and outputting the optimal subtask-node allocation; each mobile edge node performing computation of each subtask according to the subtask-node allocation and the subtask dependencies. This invention significantly reduces the complexity of task allocation decisions and is easy to deploy at base stations and edge computing devices.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, specifically relating to a mobile edge multi-task offloading and collaborative computing method and system based on graph neural networks and reinforcement learning. Background Technology

[0002] With the development of the Internet of Things (IoT) and communication technologies, mobile edge computing has become an effective solution for mobile devices to handle complex computing tasks. Mobile devices can offload computationally difficult tasks to nearby edge nodes via wireless channels, thereby achieving low-latency and low-power computing.

[0003] In future mobile edge computing systems, offloading large-scale tasks to a single edge node makes real-time computation difficult. Therefore, offloading multiple tasks to multiple edge nodes for collaborative computation has become an effective solution. However, when complex dependencies exist between tasks, this offloading method needs to address the task-to-node allocation problem (i.e., deciding which task to assign to which node) and the allocation of communication and computing resources among nodes. Especially when the number of tasks or edge nodes is large, task and resource allocation faces exponential complexity challenges. Therefore, an efficient multi-task offloading and collaborative computation method needs to be designed to meet the demands of future mobile communication services. Summary of the Invention

[0004] To address the challenges of offloading and collaborative computing in future mobile edge computing networks, this invention provides a graph neural network-assisted reinforcement learning-based edge computing offloading method and system, thereby enabling low-latency mobile edge computing services.

[0005] To achieve the above objectives, the present invention adopts the following solution:

[0006] An edge computing offloading method for graph neural network-assisted reinforcement learning includes the following steps:

[0007] Step S1: Based on the mobile edge multi-task offloading and collaboration scenario, obtain the divisible tasks required for the mobile edge node computing service and split them into multiple sub-tasks. Then, based on the dependencies between the sub-tasks, perform graph modeling as a directed acyclic graph.

[0008] Step S2: Estimate the wireless channel information between each mobile edge node and other mobile edge nodes, and upload it to the scheduler. The scheduler models a weighted graph based on the communication and computation conditions between all edge nodes.

[0009] Step S3: The scheduler will respectively... and The node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.

[0010] Step S4: The scheduler sends the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node.

[0011] Step S5: Each mobile edge node performs calculations for each subtask based on the obtained task-node allocation and the subtask dependencies.

[0012] Furthermore, the mobile edge multi-task offloading collaboration scenario in step 1 includes:

[0013] K mobile edge node wireless cellular cells are deployed;

[0014] A base station, wherein a scheduler is deployed, and the scheduler broadcasts optimal task allocation and resource allocation strategies to mobile edge nodes.

[0015] Furthermore, step S1 includes the following sub-steps:

[0016] Step S11. The scheduler, based on the computing power and number of mobile edge nodes, as well as the divisibility of the divisible task, splits the divisible task into several subtasks until it becomes indivisible.

[0017] Step S12. The scheduler denotes the number of all subtasks obtained from the split as J, and the set of all tasks as . The scheduler models the dependencies between subtasks as a graph, representing all dependencies as a directed acyclic graph. in A collection of task indexes ε = {0, 1} J×J A matrix representing the dependencies between tasks;

[0018] Step S13. The scheduler normalizes or embeds the subtask information to generate feature vectors for subtask graph nodes.

[0019] Furthermore, the subtask information in step S13 includes: subtask computational load C. k The size of the subtask calculation result R k The index of all preceding tasks of this subtask. k Subsequent task index Suc kAnd the graph node indices (Nodes) to which the subtask and its predecessor tasks are currently assigned. k , where k is the subtask index.

[0020] Further, step S2 specifically includes:

[0021] Each mobile edge node estimates the wireless channel between itself and other nodes. After obtaining the estimated channel information, it uploads it to the scheduler. The communication and computation conditions between all edge nodes are then modeled as a weighted graph. in Each graph node represents a mobile edge node and its computational conditions. The edges between graph nodes represent the communication conditions between them. If the index of a mobile edge node is m, then the computational conditions between the mobile edge nodes include the processor computing frequency f of the mobile edge node. m The communication condition is the channel state vector h between all other edge nodes and the mobile edge node m. m .

[0022] Furthermore, step S3 includes the following sub-steps:

[0023] S31. Will The node features and edge features are modeled as subtask node feature tensors and subtask edge feature tensors, respectively. The node features and edge features are modeled as the moving edge node tensor and the moving edge node edge feature tensor, respectively.

[0024] S32. Construct a graph convolutional neural network based on communication and computational features. Input the subtask node feature tensor, subtask edge feature tensor, moving edge node tensor, and moving edge node edge feature tensor from step S31 into the graph convolutional neural network for processing, and output the final state feature tensor v. State .

[0025] Furthermore, the graph convolutional neural network based on communication and computational features includes several graph node information aggregation layers and several feedforward network layers;

[0026] The graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer. The task graph node information aggregation layer aggregates graph information for each sub-task graph node based on its characteristics and the characteristics of other task graph nodes. The edge node graph node information aggregation layer aggregates graph information for each mobile edge node graph node based on its computing power and the passage conditions between it and other edge nodes.

[0027] Furthermore, the task graph node information aggregation layer aggregates graph information for each subtask graph node based on its characteristics and the characteristics of other task graph nodes, including:

[0028] If task l is not a preceding task of task k, then its weight with respect to task k is 0; if it is a preceding task of task k, then the information weight α based on communication computation subtask k is calculated for all other subtasks l. l,k :

[0029]

[0030] Where I(k,l) is a function to determine whether two tasks are assigned to the same mobile edge node. If the two tasks are on the same server, its value is 1, otherwise it is -1.

[0031] The edge node graph node information aggregation layer performs graph information aggregation for each mobile edge node based on its computing power and its accessibility with other edge nodes. This includes: for mobile edge node m, calculating the weight β between it and another edge node n. m,n :

[0032]

[0033] Where p is the transmission power, σ 2 h represents the channel background noise power. m,n This represents the channel gain between the two edge nodes.

[0034] Furthermore, the deep reinforcement learning agent training based on the PPO algorithm in step S4 includes the following sub-steps:

[0035] 1) Set the number of training loops N, and randomly initialize the subtask node allocation;

[0036] 2) The agent will generate the final state feature tensor of the current time step. The data is fed into a deep neural network, which outputs the current task assignment matrix A. t ;

[0037] 3) The scheduler uses the allocation matrix A t The allocation information is broadcast to all mobile edge nodes. After receiving the information, the mobile edge nodes perform unloading and collaborative computation of all tasks according to the dependencies between subtasks, and record the time cost τ from the start of unloading to the completion of the last task. t The reward obtained by the agent is set to r. t =-τ t And the graph state of this step. Recorded as According to A t The graph state after reallocation is denoted as

[0038] 4) The agent performs the state transition for this step. Save to the experience pool;

[0039] 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence;

[0040] 6) Extract the network parameters of the trained agent as the parameters used for deploying the agent.

[0041] On the other hand, the present invention provides an edge computing offloading system for graph neural network-assisted reinforcement learning, comprising:

[0042] Module 1: This module is used in mobile edge multi-task offloading and collaboration scenarios to obtain the divisible tasks required for mobile edge node computing services, break them down into multiple subtasks, and model them into a directed acyclic graph based on the dependencies between the subtasks.

[0043] Module 2: This module estimates the wireless channel information between each mobile edge node and other mobile edge nodes, and uploads it to the scheduler. The scheduler then models this information as a weighted graph based on the communication and computational conditions between all edge nodes.

[0044] Module 3: It is used by the scheduler to respectively... and The node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.

[0045] Module 4: It is used by the scheduler to feed the final state feature tensor as the observation state into the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node.

[0046] Module 5: It is used by each mobile edge node to perform calculations of each subtask according to the obtained task-node allocation and the subtask dependency relationship;

[0047] The mobile edge multi-task offloading collaborative computing system based on graph neural networks and reinforcement learning is used to execute the steps in the mobile edge multi-task offloading collaborative computing method based on graph neural networks and reinforcement learning.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. In mobile edge computing scenarios, it provides an effective and low-latency offloading and computing method for a large number of mobile edge tasks with complex dependencies.

[0050] 2. The proposed method effectively utilizes the tasks, communication conditions, and computing power of edge nodes in mobile edge computing scenarios to encode graph nodes. Its performance is superior to traditional graph neural networks in the target scenario, and it can fully meet the needs of mobile edge computing.

[0051] 3. Compared with traditional convex optimization and combinatorial optimization methods, this method significantly reduces the complexity of task allocation decisions and is easy to deploy at base stations and edge computing devices.

[0052] 4. The proposed optimization method has high scalability and can be extended to other complex scenarios with graph structures in mobile edge computing. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of a system scenario for a mobile edge computing multi-task offloading and collaborative computing method based on graph neural networks and reinforcement learning, provided in an embodiment of this application.

[0055] Figure 2 This is a flowchart of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural networks and reinforcement learning, provided in an embodiment of this application.

[0056] Figure 3 This is a system delay simulation result diagram of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural networks and reinforcement learning provided in an embodiment of this application.

[0057] Figure 4 This is a simulation diagram of the normalized reward curve obtained by the agent during the reinforcement learning training process of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural networks and reinforcement learning provided in this application embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0059] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0060] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0061] <Example 1>

[0062] Figure 1 This is a schematic diagram of a system scenario for an edge computing offloading method for graph neural network-assisted reinforcement learning provided in an embodiment of this application. Figure 1 As shown, within a wireless cellular cell, a scheduler is deployed at the base station to broadcast optimal task allocation and resource allocation strategies to edge nodes.

[0063] like Figure 2 As shown, in step S1, the scheduler acquires and preprocesses information on multiple interdependent sub-tasks required for the mobile edge computing service. The scheduler obtains the dependencies and task information between all sub-tasks and performs graph modeling.

[0064] In step S2, all mobile edge nodes estimate the wireless channels between themselves and other nodes, and then upload the estimated channel information to the scheduler. The communication and computation conditions between all edge nodes are then modeled as a weighted graph. in Each graph node represents a mobile edge node and its computational conditions. The edges between graph nodes represent the communication conditions between them. If the index of a mobile edge node is m, then the computational conditions between the mobile edge nodes include the processor computing frequency f of the mobile edge node. mThe communication condition is the channel state vector h between all other edge nodes and the mobile edge node m. m .

[0065] Step S3: The scheduler will respectively... and The node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.

[0066] In step S4, the scheduler feeds the feature tensor extracted in step S3 as the observation state into the trained deep reinforcement learning agent based on the PPO (Proximal Policy Optimization) algorithm for processing. The agent outputs the optimal task-node allocation. The scheduler then broadcasts this allocation information to each edge node.

[0067] Step S5: Each edge node performs computation on each task according to the obtained task-node allocation and the task dependencies. Based on the allocation results, if a node completes the computation of a task and its successor is assigned to another node, that node forwards the task data via Device-to-Device (D2D) communication to the target node for computation of the subsequent task. This process continues until all tasks are completed.

[0068] Furthermore, step S1 includes the following sub-steps:

[0069] Step S11. The scheduler obtains the information of the divisible task (hereinafter referred to as the original task) to be computed based on the device requirements. This information includes the task's computational load, the data packet size of each stage of the task, the task's input data, and the task's output data. Subsequently, the scheduler splits the original task into several subtasks (hereinafter referred to as tasks). Specifically, the scheduler splits the original task into several tasks based on the specific device's computing power, the number of available edge nodes in the mobile edge computing network, the parallel computing requirements of each stage of the original task, and the divisibility of the original task, until it can no longer be split.

[0070] Step S12. The scheduler denotes the total number of tasks that need to be computed as J, and the set of all tasks as J. The scheduler then models the dependencies between these tasks using a graph model, that is, models all dependencies as a directed acyclic graph. Where V is the set of task indices. ε = {0, 1} J×J Let ε be a matrix representing the dependencies between tasks. The element e in the i-th row and j-th column is... ij=1 indicates that there is a sequential dependency between task i and task j, meaning that task j can only begin its calculation after task i has been completed. In this case, task i is called the predecessor task of task j, and task j is called the successor task of task i; e ij =0 indicates that there is no dependency between the two tasks.

[0071] Step S13. The scheduler performs feature encoding on each task based on the task information. Considering task index k, the task information includes: task computational load C. k (Unit: bits), Task calculation result size R k (Unit: bits) Index of all preceding tasks for this task. k Subsequent task index Suc k And the graph node indices (Nodes) to which this task and its predecessor tasks are currently assigned. k Subsequently, the scheduler normalizes or embeds the above task information to generate a graph node feature vector for each task.

[0072] Further, step S2 specifically includes:

[0073] Each edge node estimates the wireless channel between itself and other nodes, and then uploads the estimated channel information to the scheduler. The server models the communication and computation conditions between all edge nodes as a weighted graph. Each graph node represents an edge node and its computational conditions, and the edges between graph nodes represent the communication conditions between them. Considering an edge node index m, the communication and computational characteristics between edge nodes include the server's processor computation frequency f. m The channel state h between all other edge nodes and edge node m m The task index (Tasks) assigned to edge node m m .

[0074] Furthermore, step S3 includes the following sub-steps:

[0075] S31. Using PyTorch and other related tools, and Each node feature and edge feature is modeled as a tensor.

[0076] S32. Construct a graph convolutional neural network based on communication and computational features, and integrate the features from step S31. The input is fed into a graph convolutional neural network for processing. The graph convolutional neural network based on communication and computational features includes several graph node information aggregation layers and several feedforward network layers.

[0077] To effectively extract communication and computation-related features from graph structure data, the graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer. The task graph node encoding layer aggregates graph information for each task graph node based on its features and those of other task graph nodes. Specifically, a weight α based on communication and computation information is calculated for all other tasks. If task l is not a preceding task of task k, its weight for task k is 0; if it is a preceding task of task k, its weight for task k is calculated according to the following formula:

[0078]

[0079] Where I(k,l) is a function that determines whether two tasks are assigned to the same server; if the two tasks are on the same server, its value is 1, otherwise it is -1. After calculating and normalizing all weights, this layer aggregates the information of its neighboring graph nodes according to the weights for each task graph node to obtain the feature representation of each graph node.

[0080] The edge node graph node encoding layer aggregates graph information for each edge node graph node based on its computational power and its connectivity with other edge nodes. Specifically, for edge node m, the weight β between it and another edge node n is calculated according to the following formula. m,n :

[0081]

[0082] Where p is the transmission power, σ 2 h represents the channel background noise power. m,n This represents the channel gain between two edge nodes. Finally, after calculating and normalizing all weights, this layer aggregates the graph node information of its neighboring edge nodes according to the weights for each edge node to obtain the feature representation of each edge node graph node.

[0083] Subsequently, the graph convolutional neural network performs layer-by-layer operations on the extracted features based on graph convolution. The propagation rules for each layer of the graph convolutional neural network are as follows:

[0084]

[0085] in D is the degree matrix of the input graph, σ is the activation function, and H... q W is the hidden representation of the nodes at level q. q A is the parameter matrix of the trainable model. *= A + I, where A is the adjacency matrix of the input graph, and I is an identity matrix of the same size as A. During the layer-by-layer propagation of the graph convolutional neural network, each node aggregates its own features and those of its neighboring nodes according to the aforementioned weights, and feeds them into a non-linear activation function and a feedforward layer. After several layers of propagation, the final state feature tensor v is output through fully connected layers and softmax layers. State .

[0086] Furthermore, step S4, agent training, specifically involves:

[0087] 1) Set the number of training cycles N, and randomly initialize the task node allocation;

[0088] 2) The agent outputs the current time step. The data is fed into a deep neural network, which outputs the current task assignment matrix A. t ;

[0089] 3) The scheduler uses the allocation matrix A t The allocation information is broadcast to all edge nodes. After receiving the information, the edge nodes unload and perform collaborative computation on all tasks according to the dependencies between tasks, and record the time cost τ from the start of unloading to the completion of the last task. t The reward received by the agent is set to r. t =-τ t And the graph state of this step. Recorded as According to A t The graph state after reallocation is denoted as

[0090] 4) The agent performs the state transition for this step. Save to the experience pool;

[0091] 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence;

[0092] 6) Extract the network parameters of the trained agent as the parameters used for deploying the agent.

[0093] like Figure 3 and Figure 4 As shown, the experimental simulation of the mobile edge computing multi-task offloading and collaborative computing method based on graph neural networks and reinforcement learning was carried out and the results were plotted. Figure 3 The graph shows a performance comparison between the proposed graph neural network-based reinforcement learning method and the traditional PPO reinforcement learning method. The graph shows that the optimized system has a lower system time cost than the traditional reinforcement learning method, which verifies the effectiveness of the proposed method. Figure 4The normalized reward curve of the reinforcement learning agent based on the graph neural network is shown. As can be seen from the figure, the method gradually converges as the training process progresses, and the model is effectively optimized.

[0094] <Example 2>

[0095] This embodiment provides an edge computing offloading system for graph neural network-assisted reinforcement learning, including:

[0096] Module 1: This module is used in mobile edge multi-task offloading and collaboration scenarios to obtain the divisible tasks required for mobile edge node computing services, break them down into multiple subtasks, and model them into a directed acyclic graph based on the dependencies between the subtasks.

[0097] Module 2: This module estimates the wireless channel information between each mobile edge node and other mobile edge nodes, and uploads it to the scheduler. The scheduler then models this information as a weighted graph based on the communication and computational conditions between all edge nodes.

[0098] Module 3: It is used by the scheduler to respectively... and The node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.

[0099] Module 4: It is used by the scheduler to feed the final state feature tensor as the observation state into the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node.

[0100] Module 5: It is used by each mobile edge node to perform calculations of each subtask according to the obtained task-node allocation and the subtask dependency relationship.

[0101] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0102] In the embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above embodiments of the power system transient stability adaptive evaluation method considering PMU data loss. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0106] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for offloading edge computation in graph neural network-assisted reinforcement learning, characterized in that, Includes the following steps: Step S1: Based on the mobile edge multi-task offloading and collaboration scenario, obtain the divisible tasks required for the mobile edge node computing service and split them into multiple sub-tasks. Then, based on the dependencies between the sub-tasks, perform graph modeling as a directed acyclic graph. It includes the following sub-steps: Step S11. The scheduler, based on the computing power and number of mobile edge nodes, as well as the divisibility of the divisible task, splits the divisible task into several subtasks until it becomes indivisible. Step S12. The scheduler records the number of all subtasks obtained from the split that need to be calculated as follows: The set of all tasks is denoted as The scheduler models the dependencies between subtasks using a graph, transforming all dependencies into a directed acyclic graph. ,in A collection of task indexes , A matrix representing the dependencies between tasks; Step S13. The scheduler normalizes or embeds the subtask information to generate feature vectors for subtask graph nodes; Subtask information includes: subtask computational load. Size of subtask calculation results The index of all preceding tasks for this subtask Subsequent task index And the graph node indices to which the subtask and its predecessor tasks are currently assigned. ,in Index for subtasks; Step S2: Estimate the wireless channel information between each mobile edge node and other mobile edge nodes, and upload it to the scheduler. The scheduler models a weighted graph based on the communication and computation conditions between all edge nodes. ; Step S3: The scheduler will respectively... and Node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor. Step S4: The scheduler sends the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node. Step S5: Each mobile edge node performs calculations for each subtask based on the obtained subtask-node allocation and the subtask dependencies.

2. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 1, characterized in that, The mobile edge multi-task offloading collaboration scenario in step 1 includes: Deployed One mobile edge node wireless cellular cell; A base station, at which a scheduler is deployed, broadcasts optimal task allocation and resource allocation strategies to mobile edge nodes.

3. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 1, characterized in that, Step S2 specifically involves: Each mobile edge node estimates the wireless channel between itself and other nodes. After obtaining the estimated channel information, it uploads it to the scheduler. The communication and computation conditions between all edge nodes are then modeled as a weighted graph. ,in Each graph node represents a mobile edge node and its computational conditions. The edges between graph nodes represent the communication conditions between them. The index of the mobile edge node is... The computational conditions between the mobile edge nodes include the processor computing frequency of the mobile edge nodes. The communication conditions are for all other edge nodes to the mobile edge node. Channel state vector .

4. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 3, characterized in that, Step S3 includes the following sub-steps: S31. Will The node features and edge features are modeled as subtask node feature tensors and subtask edge feature tensors, respectively. The node features and edge features are modeled as the moving edge node tensor and the moving edge node edge feature tensor, respectively. S32. Construct a graph convolutional neural network based on communication and computational features. Input the subtask node feature tensor, subtask edge feature tensor, moving edge node tensor, and moving edge node edge feature tensor from step S31 into the graph convolutional neural network for processing and output the final state feature tensor. .

5. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 4, characterized in that, The graph convolutional neural network based on communication and computation features includes several graph node information aggregation layers and several feedforward network layers. The graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer. The task graph node information aggregation layer aggregates graph information for each sub-task graph node based on its characteristics and the characteristics of other task graph nodes. The edge node graph node information aggregation layer aggregates graph information for each mobile edge node graph node based on its computing power and the passage conditions between it and other edge nodes.

6. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 5, characterized in that, The task graph node information aggregation layer aggregates graph information for each subtask graph node based on its features and the features of other task graph nodes, including: If task If it is not a preceding task of this task, then it is related to the task. The weight is 0; if it is a preceding task of this task, then the weight is 0 for all other subtasks. Computation based on communication computing subtasks Information weight : in The function determines whether two tasks are assigned to the same mobile edge node. If the two tasks are on the same server, its value is 1, otherwise it is -1. The edge node graph node information aggregation layer performs graph information aggregation for each mobile edge node based on its computing power and its accessibility with other edge nodes. This includes: for mobile edge nodes... Calculate its relationship with another edge node. Weights between : in For transmission power, For channel background noise power, This represents the channel gain between the two edge nodes.

7. The edge computing offloading method for graph neural network-assisted reinforcement learning according to claim 5, characterized in that, Step S4, the training of the deep reinforcement learning agent based on the PPO algorithm, includes the following sub-steps: 1) Set the number of training cycles Randomly initialize the allocation of subtask nodes; 2) The agent will generate the final state feature tensor of the current time step. The data is fed into a deep neural network, which outputs the current task assignment matrix. ; 3) The scheduler uses this allocation matrix The allocation information is broadcast to all mobile edge nodes. After receiving the information, the mobile edge nodes unload and perform collaborative computation of all tasks according to the dependencies between subtasks, and record the time cost from the start of unloading to the completion of the last task. The reward obtained by the intelligent agent is set to And the graph state of this step. Recorded as According to The graph state after reallocation is denoted as ; 4) The agent performs the state transition for this step. Save to the experience pool; 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence; 6) Extract the network parameters of the trained agent as the parameters used to deploy the agent.

8. An edge computing offloading system for graph neural network-assisted reinforcement learning, characterized in that, include: Module 1: This module is used in mobile edge multi-task offloading and collaboration scenarios to obtain the divisible tasks required for mobile edge node computing services, break them down into multiple subtasks, and model them into a directed acyclic graph based on the dependencies between the subtasks. ; Module 2: This module estimates the wireless channel information between each mobile edge node and other mobile edge nodes, and uploads it to the scheduler. The scheduler then models this information as a weighted graph based on the communication and computational conditions between all edge nodes. ; Module 3: It is used by the scheduler to respectively... and Node features and edge features are modeled as tensors and processed based on graph convolutional neural networks. Based on the communication and computation conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor. Module 4: It is used by the scheduler to feed the final state feature tensor as the observation state into the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node. Module 5: It is used by each mobile edge node to perform calculations of each subtask according to the obtained task-node allocation and the subtask dependency relationship; The edge computing offloading system for graph neural network-assisted reinforcement learning is used to execute the steps in the edge computing offloading method for graph neural network-assisted reinforcement learning according to any one of claims 1-7.

Images