Edge calculation unloading method and system for graph neural network assisted reinforcement learning
Through graph neural network and reinforcement learning methods, the complexity of task and resource allocation in mobile edge computing is solved, and the low-latency multi-task offloading and collaborative computing is realized, which significantly reduces the complexity and meets the needs of mobile communication services. .
Patent Information
- Application Number
- CN202411206943.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-30
AI Technical Summary
In mobile edge computing systems, when there are complex dependencies between multiple tasks, the allocation of tasks and resources and the allocation of communication between nodes is difficult to solve, especially when the number of tasks or edge nodes is large, resulting in exponential complexity. degree of challenge.
The edge computing offload method of graph neural network assisted reinforcement learning is adopted. By splitting the task into subtasks and modeling it into directed acyclic graphs, wireless channel information is estimated, features are extracted using graph convolutional neural networks, and PPO algorithm is used. Deep reinforcement learning agents perform optimal task-node allocation, realizing low-latency multi-task offloading and collaborative computing.
It effectively reduces the complexity of task allocation decisions, realizes low-latency mobile edge computing, significantly improves performance, is suitable for large-scale task offload scenarios, has high scalability, and meets the needs of mobile communication services.
Smart Images

Figure CN119011583A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of wireless communication technology, and specifically relates to a mobile edge multi-task offloading collaborative computing method and system based on graph neural network and reinforcement learning. Background Art
[0002] With the development of the Internet of Things and communication technologies, mobile edge computing has become an effective solution for mobile devices to handle complex computing tasks. Mobile devices can offload difficult computing tasks to nearby edge nodes through wireless channels for computing, thereby achieving low-latency and low-energy computing.
[0003] In future mobile edge computing systems, it is difficult to achieve real-time computing by offloading large-scale tasks to a single edge node. In this context, offloading multiple tasks to multiple edge nodes for collaborative computing has become an effective solution. However, when there are complex dependencies between tasks, this offloading method needs to solve the problem of allocating tasks to nodes (i.e., deciding which task to assign to which node) and the problem of allocating communication computing resources for nodes. Especially when there are a large number of tasks or edge nodes, task and resource allocation faces the challenge of exponential complexity. Therefore, it is necessary to design an efficient multi-task offloading and collaborative computing method to meet the needs of future mobile communication services. Summary of the invention
[0004] Aiming at the offloading and collaborative computing scenarios of multiple tasks in future mobile edge computing networks, in order to solve the above problems, the present invention aims to provide an edge computing offloading method and system for graph neural network assisted reinforcement learning, so as to realize low-latency mobile edge computing services.
[0005] In order to achieve the above purpose, the present invention adopts the following scheme:
[0006] An edge computing offloading method for graph neural network assisted reinforcement learning includes the following steps:
[0007] Step S1: Based on the mobile edge multi-task offloading collaboration scenario, obtain the divisible tasks required for the mobile edge node computing service and split them into multiple subtasks, and model them as a directed acyclic graph based on the dependency relationship between the subtasks.
[0008] Step S2: All mobile edge nodes estimate the wireless channel information between themselves and other mobile edge nodes and upload it to the scheduler. The scheduler models the information into a weighted graph based on the communication conditions and computing conditions between all edge nodes.
[0009] Step S3: The scheduler will and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.
[0010] Step S4: The scheduler sends the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node;
[0011] Step S5: Each mobile edge node performs calculations on each subtask according to the obtained task-node allocation and the subtask dependency relationship.
[0012] Furthermore, the mobile edge multi-task offloading collaboration scenario in step 1 includes:
[0013] K mobile edge node wireless cellular cells are deployed;
[0014] A base station, wherein a regulator is deployed at the base station, and the scheduler broadcasts the optimal task allocation and resource allocation strategy to the mobile edge nodes.
[0015] Furthermore, step S1 includes the following sub-steps:
[0016] Step S11. The scheduler splits the splittable task into several subtasks based on the computing power of the mobile edge nodes, the number of mobile edge nodes, and the divisibility of the splittable task until the task becomes indivisible;
[0017] Step S12: The scheduler records the number of all subtasks that need to be calculated as J, and the set of all tasks as The scheduler models the dependencies between subtasks as a directed acyclic graph. in A collection of task indices ε={0,1} J×J is a matrix that represents the dependencies between tasks;
[0018] Step S13: The scheduler normalizes or embeds the subtask information to generate a subtask graph node feature vector.
[0019] Furthermore, the subtask information in step S13 includes: subtask task calculation load C k , the subtask calculation result size R k , all the previous tasks of this subtask have index Pre k , the successor task index Suc k, and the graph node index Nodes to which the subtask and the previous task are currently assigned k , where k is the subtask index.
[0020] Furthermore, the step S2 is specifically as follows:
[0021] All mobile edge nodes estimate the wireless channels between themselves and other nodes, and upload the estimated channel information to the scheduler. The communication conditions and computing conditions between all edge nodes are modeled as a weighted graph. in Each graph node represents a mobile edge node and its computing conditions, and the edges between graph nodes represent the communication conditions between them. The mobile edge node index is m, and the computing conditions between mobile edge nodes include the processor computing frequency f of the mobile edge node. m The communication condition is the channel state vector h between all other edge nodes and the mobile edge node m m .
[0022] Furthermore, step S3 includes the following sub-steps:
[0023] S31. The node features and edge features of are modeled as subtask node feature tensors and subtask edge feature tensors respectively. The node features and edge features of are modeled as mobile edge node tensor and mobile edge node edge feature tensor respectively;
[0024] S32. Build a graph convolutional neural network based on communication and computing features, and send the subtask node feature tensor, subtask edge feature tensor, mobile edge node tensor, and mobile edge node edge feature tensor in step S31 as input to the graph convolutional neural network for processing and output the final state feature tensor v State .
[0025] Furthermore, the graph convolutional neural network based on communication and computational features includes several graph node information aggregation layers and several feedforward network layers;
[0026] The graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer; the task graph node information aggregation layer performs graph information aggregation for each subtask graph node according to its characteristics and other task graph node characteristics; the edge node graph node information aggregation layer performs graph information aggregation for each mobile edge node graph node according to its computing power and the travel conditions between it and other edge nodes.
[0027] Furthermore, the task graph node information aggregation layer aggregates graph information for each subtask graph node according to its characteristics and other task graph node characteristics, including:
[0028] If task l is not a predecessor task of this task, its weight for task k is 0; if it is a predecessor task of this task, the information weight α of subtask k based on communication calculation is calculated for all other subtasks l l,k :
[0029]
[0030] Where I(k,l) is a function to determine whether two tasks are assigned to the same mobile edge node. If the two tasks are on the same server, its value is 1, otherwise it is -1;
[0031] The edge node graph node information aggregation layer aggregates graph information for each mobile edge node graph node based on its computing power and the traffic conditions between it and other edge nodes, including: for a mobile edge node m, calculate the weight β between it and another edge node n m,n :
[0032]
[0033] Where p is the transmission power, σ 2 is the channel background noise power, h m,n is the channel gain between two edge nodes.
[0034] Furthermore, the deep reinforcement learning agent training based on the PPO algorithm in step S4 includes the following sub-steps:
[0035] 1) Set the number of training cycles N and randomly initialize the subtask node allocation;
[0036] 2) The agent converts the final state feature tensor of the current time step Send it into the deep neural network and output the current task allocation matrix A t ;
[0037] 3) The scheduler calculates the allocation matrix A according to the t The allocation information is broadcast to all mobile edge nodes. After receiving it, the mobile edge nodes perform offloading and collaborative calculation of all tasks according to the dependencies between subtasks, and record the time cost τ from the start of offloading to the completion of the last task t , the reward obtained by the agent is set to r t =-τ t , and the graph state of this step Recorded as According to A t The graph state after reallocation is recorded as
[0038] 4) The agent transfers the state of this step Save to the experience pool;
[0039] 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence;
[0040] 6) Take the trained agent network parameters as the parameters used for deploying the agent.
[0041] On the other hand, the present invention provides an edge computing offloading system for graph neural network assisted reinforcement learning, comprising:
[0042] Module 1: It is used to obtain the divisible tasks required for the mobile edge node computing service based on the mobile edge multi-task offloading collaboration scenario and split them into multiple subtasks, and model the graph as a directed acyclic graph based on the dependency relationship between the subtasks.
[0043] Module 2: It is used to estimate the wireless channel information between all mobile edge nodes and other mobile edge nodes, and upload it to the scheduler. The scheduler is modeled as a weighted graph based on the communication conditions and computing conditions between all edge nodes.
[0044] Module 3: It is used by the scheduler to and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.
[0045] Module 4: It is used by the scheduler to send the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node;
[0046] Module 5: It is used for each mobile edge node to calculate each subtask according to the obtained task-node allocation and the subtask dependency relationship;
[0047] The mobile edge multi-task offloading collaborative computing system based on graph neural network and reinforcement learning is used to execute the steps in the mobile edge multi-task offloading collaborative computing method based on graph neural network and reinforcement learning.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] 1. In mobile edge computing scenarios, it provides an effective, low-latency offloading and computing method for a large number of mobile edge tasks with complex dependencies.
[0050] 2. The proposed method effectively utilizes the tasks in the mobile edge computing scenario, the communication conditions and computing power of the edge nodes to encode the graph nodes. It performs better than the traditional graph neural network in the target scenario and can fully meet the needs of mobile edge computing.
[0051] 3. Compared with traditional convex optimization and combinatorial optimization methods, this method significantly reduces the complexity of task allocation decisions and is easy to deploy at base stations and edge computing devices.
[0052] 4. The proposed optimization method has high scalability and can be extended to other complex scenarios with graph structures in mobile edge computing. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0054] Figure 1 It is a system scenario diagram of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural network and reinforcement learning provided in an embodiment of the present application;
[0055] Figure 2 This is a flowchart of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural network and reinforcement learning provided in an embodiment of the present application.
[0056] Figure 3 This is a system delay simulation result diagram of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural network and reinforcement learning provided in an embodiment of the present application.
[0057] Figure 4 It is a simulation diagram of the normalized reward curve obtained by the intelligent agent during the reinforcement learning training of a mobile edge computing multi-task offloading and collaborative computing method based on graph neural network and reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.
[0060] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0061] <Example 1>
[0062] Figure 1 A schematic diagram of a system scenario for an edge computing offloading method for graph neural network-assisted reinforcement learning provided in an embodiment of the present application. Figure 1 As shown, within a wireless cellular cell, a scheduler is deployed at the base station to broadcast the optimal task allocation and resource allocation strategies to the edge nodes.
[0063] like Figure 2 As shown, in step S1, the scheduler obtains information and preprocesses multiple sub-interdependent tasks required to be calculated by the mobile edge computing service. The scheduler obtains the dependency relationship and task information between all sub-tasks and performs graph modeling.
[0064] In step S2, all mobile edge nodes estimate the wireless channels between themselves and other nodes, and upload the estimated channel information to the scheduler. The communication conditions and computing conditions between all edge nodes are modeled as a weighted graph. in Each graph node represents a mobile edge node and its computing conditions, and the edges between graph nodes represent the communication conditions between them. The mobile edge node index is m, and the computing conditions between mobile edge nodes include the processor computing frequency f of the mobile edge node. mThe communication condition is the channel state vector h between all other edge nodes and the mobile edge node m m .
[0065] Step S3: The scheduler will and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.
[0066] In step S4, the scheduler sends the feature tensor extracted in step S3 as the observation state to the trained deep reinforcement learning agent based on the PPO (Proximal Policy Optimization) algorithm for processing. The agent outputs the optimal task-node allocation. The scheduler then broadcasts the allocation information to each edge node.
[0067] Step S5: Each edge node calculates each task according to the obtained task-node allocation and the task dependency relationship. According to the allocation result, if a node completes a task and the subsequent task of the task is assigned to other nodes, the node forwards the task data through device-to-device (D2D) communication and sends it to the target node for calculation of the subsequent task. This process continues until all tasks are calculated.
[0068] Furthermore, the step S1 comprises the following sub-steps:
[0069] Step S11. The scheduler obtains the information of the divisible task (hereinafter referred to as the original task) required for calculation according to the device requirements. The information includes the task calculation load, the size of the data packet at each stage of the task, the task input data and the task output data. Subsequently, the scheduler splits the task into several subtasks (hereinafter referred to as tasks). Specifically, the scheduler splits the original task into several tasks according to the specific device computing power and the number of available edge nodes in the mobile edge computing network, the parallel computing requirements at each stage of the original task, and the divisibility of the original task until it cannot be split.
[0070] Step S12: The scheduler will record the number of all tasks to be calculated as J, and the set of all tasks as The scheduler then models the dependencies between these tasks as a directed acyclic graph. Where V is the set of task indexes ε={0,1} J×J is a matrix that represents the dependencies between tasks. The element e in the i-th row and j-th column of ε ij=1 indicates that there is a sequential dependency relationship between task i and task j, that is, task j can only start computing after task i is completed. In this case, task i is called the predecessor task of task j, and task j is the successor task of task i; e ij =0 means there is no dependency between the two tasks.
[0071] Step S13: The scheduler performs feature encoding on each task according to the task information. Considering the task index k, the task information includes: task computing load C k (in bits), the size of the task calculation result R k (in bits), the index of all previous tasks of this task Pre k , the successor task index Suc k , and the graph node index Nodes to which the task and the previous task are currently assigned k Subsequently, the scheduler performs normalization or embedding operations on the above task information to generate a graph node feature vector for each task.
[0072] Furthermore, the step S2 is specifically as follows:
[0073] All edge nodes estimate the wireless channels between themselves and other nodes, and upload the estimated channel information to the scheduler. The server models the communication and computing conditions between all edge nodes as a weighted graph. Each graph node represents an edge node and its computing conditions, and the edges between graph nodes represent the communication conditions between them. Considering the edge node index m, the communication and computing characteristics between edge nodes include the server processor computing frequency f m , the channel state h between all other edge nodes and edge node m m , the task index Tasks assigned to edge node m m .
[0074] Furthermore, the step S3 comprises the following sub-steps:
[0075] S31. Through Pytorch and other related tools, and Respective node features and edge features are modeled as tensors.
[0076] S32. Build a graph convolutional neural network based on communication and computing features, and convert the The graph convolutional neural network is used as input for processing. The graph convolutional neural network based on communication and computing features includes several graph node information aggregation layers and several feedforward network layers.
[0077] In order to effectively extract the communication and computing related features of graph structured data, the graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer. The task graph node encoding layer aggregates graph information for each task graph node according to its features and other task graph node features. Specifically, a weight α based on communication computing information is calculated for all other tasks. If task l is not a predecessor task of the task, its weight for task k is 0; if it is a predecessor task of the task, its weight for task k is calculated according to the following formula:
[0078]
[0079] Among them, I(k,l) is a function to determine whether two tasks are assigned to the same server. If the two tasks are on the same server, its value is 1, otherwise it is -1. After finally calculating and normalizing all weights, this layer aggregates the information of its adjacent graph nodes for each task graph node according to the weight to obtain the feature representation of each graph node.
[0080] The edge node graph node coding layer aggregates graph information for each edge node graph node based on its computing power and the traffic conditions between it and other edge nodes. Specifically, for edge node m, the weight β between it and another edge node n is calculated according to the following formula: m,n :
[0081]
[0082] Where p is the transmission power, σ 2 is the channel background noise power, h m,n is the channel gain between two edge nodes. Finally, after calculating and normalizing all weights, this layer aggregates the information of its adjacent edge node graph nodes according to the weight for each edge node to obtain the feature representation of each edge node graph node.
[0083] Then the graph convolutional neural network performs layer-by-layer operations on the extracted features based on graph convolution, where the propagation rules of each layer of the graph convolutional neural network are as follows:
[0084]
[0085] in D is the degree matrix of the input graph, σ is the activation function, H q is the hidden representation of the node in the qth layer, W q is the trainable model parameter matrix, A *=A+I, A is the adjacency matrix of the input graph, and I is the unit matrix of the same size as A. When the graph convolutional neural network propagates layer by layer, each node aggregates its own and its adjacent node features according to the above weights, and sends them to the nonlinear activation function and feedforward layer. After propagating through several layers, the final state feature tensor v is output through the fully connected layer and the softmax layer. State .
[0086] Furthermore, the agent training in step S4 is specifically as follows:
[0087] 1) Set the number of training cycles N and randomly initialize the task node allocation;
[0088] 2) The agent outputs the current time step Send it into the deep neural network and output the current task allocation matrix A t ;
[0089] 3) The scheduler calculates the allocation matrix A according to the t The allocation information is broadcast to all edge nodes. After receiving it, the edge nodes perform unloading and collaborative calculation of all tasks according to the dependencies between tasks, and record the time cost τ from the start of unloading to the completion of the last task t The reward obtained by the agent is set to r t =-τ t , and the graph state of this step Recorded as According to A t The graph state after reallocation is recorded as
[0090] 4) The agent transfers the state of this step Save to the experience pool;
[0091] 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence;
[0092] 6) Take the trained agent network parameters as the parameters used by the deployed agent.
[0093] like Figure 3 and Figure 4 As shown, the mobile edge computing multi-task offloading and collaborative computing method based on graph neural network and reinforcement learning is experimentally simulated and the result graph is drawn. Figure 3 A performance comparison chart of the graph neural network-based reinforcement learning method and the traditional PPO reinforcement learning method is shown. The system time overhead of the optimized method is lower than that of the traditional reinforcement learning method, which verifies the effectiveness of the method. Figure 4The normalized reward curve during the training process of the graph neural network-based reinforcement learning agent is shown. It can be seen from the figure that the method gradually converges with the training process and the model is effectively optimized.
[0094] <Example 2>
[0095] This embodiment provides an edge computing offloading system for graph neural network assisted reinforcement learning, including:
[0096] Module 1: It is used to obtain the divisible tasks required for the mobile edge node computing service based on the mobile edge multi-task offloading collaboration scenario and split them into multiple subtasks, and model the graph as a directed acyclic graph based on the dependency relationship between the subtasks.
[0097] Module 2: It is used to estimate the wireless channel information between all mobile edge nodes and other mobile edge nodes, and upload it to the scheduler. The scheduler is modeled as a weighted graph based on the communication conditions and computing conditions between all edge nodes.
[0098] Module 3: It is used by the scheduler to and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor.
[0099] Module 4: It is used by the scheduler to send the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node;
[0100] Module 5: It is used for each mobile edge node to calculate each subtask according to the obtained task-node allocation and the subtask dependency relationship.
[0101] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0102] In the embodiments provided by the present invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0105] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned embodiments of the power system transient stability adaptive evaluation method considering the loss of PMU data can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.
[0106] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. An edge computing offloading method for graph neural network assisted reinforcement learning, characterized in that: The following steps are involved: Step S1: Based on the mobile edge multi-task offloading collaboration scenario, obtain the divisible tasks required for the mobile edge node computing service and split them into multiple subtasks, and model them as a directed acyclic graph based on the dependency relationship between the subtasks. Step S2: All mobile edge nodes estimate the wireless channel information between themselves and other mobile edge nodes and upload it to the scheduler. The scheduler models the information into a weighted graph based on the communication conditions and computing conditions between all edge nodes. Step S3: The scheduler will and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor. Step S4: The scheduler sends the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node; Step S5: Each mobile edge node performs calculations on each subtask according to the obtained subtask-node allocation and the subtask dependency relationship.
2. According to claim 1, the edge computing offloading method for graph neural network assisted reinforcement learning is characterized in that: The mobile edge multi-task offloading collaboration scenario in step 1 includes: K mobile edge node wireless cellular cells are deployed; A base station, wherein a regulator is deployed at the base station, and the scheduler broadcasts the optimal task allocation and resource allocation strategy to the mobile edge nodes.
3. According to the edge computing offloading method of graph neural network assisted reinforcement learning in claim 1, it is characterized in that: Step S1 includes the following sub-steps: Step S11. The scheduler splits the splittable task into several subtasks based on the computing power of the mobile edge nodes, the number of mobile edge nodes, and the divisibility of the splittable task until the task becomes indivisible; Step S12: The scheduler records the number of all subtasks that need to be calculated as J, and the set of all tasks as The scheduler models the dependencies between subtasks as a directed acyclic graph. Where V is the set of task indexes ε={0,1} J×J is a matrix that represents the dependencies between tasks; Step S13: The scheduler normalizes or embeds the subtask information to generate a subtask graph node feature vector.
4. According to claim 3, the edge computing offloading method for graph neural network assisted reinforcement learning is characterized in that: The subtask information in step S13 includes: subtask task calculation load C k , the subtask calculation result size R k , all the previous tasks of this subtask have index Pre k , the successor task index Suc k , and the graph node index Nodes to which the subtask and the previous task are currently assigned k , where k is the subtask index.
5. According to claim 4, the edge computing offloading method for graph neural network assisted reinforcement learning is characterized in that: The step S2 is specifically as follows: All mobile edge nodes estimate the wireless channels between themselves and other nodes, and upload the estimated channel information to the scheduler. The communication conditions and computing conditions between all edge nodes are modeled as a weighted graph. in Each graph node represents a mobile edge node and its computing conditions, and the edges between graph nodes represent the communication conditions between them. The mobile edge node index is m, and the computing conditions between mobile edge nodes include the processor computing frequency f of the mobile edge node. m The communication condition is the channel state vector h between all other edge nodes and the mobile edge node m m .
6. The edge computing offloading method for graph neural network assisted reinforcement learning according to claim 5 is characterized in that: Step S3 includes the following sub-steps: S31. The node features and edge features of are modeled as subtask node feature tensors and subtask edge feature tensors respectively. The node features and edge features of are modeled as mobile edge node tensor and mobile edge node edge feature tensor respectively; S32. Build a graph convolutional neural network based on communication and computing features, and send the subtask node feature tensor, subtask edge feature tensor, mobile edge node tensor, and mobile edge node edge feature tensor in step S31 as input to the graph convolutional neural network for processing and output the final state feature tensor v state .
7. The edge computing offloading method for graph neural network assisted reinforcement learning according to claim 6 is characterized in that: The graph convolutional neural network based on communication and computing features includes several graph node information aggregation layers and several feedforward network layers; The graph node information aggregation layer is divided into a task graph node information aggregation layer and an edge node graph node information aggregation layer; the task graph node information aggregation layer performs graph information aggregation for each subtask graph node according to its characteristics and other task graph node characteristics; the edge node graph node information aggregation layer performs graph information aggregation for each mobile edge node graph node according to its computing power and the travel conditions between it and other edge nodes.
8. The edge computing offloading method for graph neural network assisted reinforcement learning according to claim 7 is characterized in that: The task graph node information aggregation layer aggregates graph information for each sub-task graph node based on its characteristics and other task graph node characteristics, including: If task l is not a predecessor task of this task, its weight for task k is 0; if it is a predecessor task of this task, the information weight α of subtask k based on communication calculation is calculated for all other subtasks l l,k : Where I(k,l) is a function to determine whether two tasks are assigned to the same mobile edge node. If the two tasks are on the same server, its value is 1, otherwise it is -1; The edge node graph node information aggregation layer aggregates graph information for each mobile edge node graph node based on its computing power and the traffic conditions between it and other edge nodes, including: for a mobile edge node m, calculate the weight β between it and another edge node n m,n : Where p is the transmission power, σ 2 is the channel background noise power, h m,n is the channel gain between two edge nodes.
9. The edge computing offloading method for graph neural network assisted reinforcement learning according to claim 7 is characterized in that: The deep reinforcement learning agent training based on the PPO algorithm in step S4 includes the following sub-steps: 1) Set the number of training cycles N and randomly initialize the subtask node allocation; 2) The agent converts the final state feature tensor of the current time step Send it into the deep neural network and output the current task allocation matrix A t ; 3) The scheduler calculates the allocation matrix A according to the t The allocation information is broadcast to all mobile edge nodes. After receiving it, the mobile edge nodes perform offloading and collaborative calculation of all tasks according to the dependencies between subtasks, and record the time cost τ from the start of offloading to the completion of the last task t , the reward obtained by the agent is set to r t =-τ t , and the graph state of this step Recorded as According to A t The graph state after reallocation is recorded as 4) The agent transfers the state of this step Save to the experience pool; 5) Randomly select several samples from the experience pool and train the agent based on the PPO reinforcement learning algorithm until convergence; 6) Take the trained agent network parameters as the parameters used by the deployed agent.
10. An edge computing offloading system for graph neural network-assisted reinforcement learning, characterized in that: include: Module 1: It is used to obtain the divisible tasks required for the mobile edge node computing service based on the mobile edge multi-task offloading collaboration scenario and split them into multiple subtasks, and model the graph as a directed acyclic graph based on the dependency relationship between the subtasks. Module 2: It is used to estimate the wireless channel information between all mobile edge nodes and other mobile edge nodes, and upload it to the scheduler. The scheduler is modeled as a weighted graph based on the communication conditions and computing conditions between all edge nodes. Module 3: It is used by the scheduler to and The node features and edge features of the network are modeled as tensors and processed based on the graph convolutional neural network. According to the communication conditions and computing conditions between subtasks and mobile edge nodes, feature extraction is performed to output the final state feature tensor. Module 4: It is used by the scheduler to send the final state feature tensor as the observation state to the trained deep reinforcement learning agent based on the PPO algorithm for processing. The agent outputs the optimal subtask-node allocation, and the scheduler broadcasts the allocation information to each mobile edge node; Module 5: It is used for each mobile edge node to calculate each subtask according to the obtained task-node allocation and the subtask dependency relationship; The edge computing offloading system for graph neural network assisted reinforcement learning is used to execute the steps in the edge computing offloading method for graph neural network assisted reinforcement learning described in any one of claims 1-9.
Citation Information
Patent Citations
Multi-user time sequence dependent service calculation unloading method based on graph neural network
CN116450241A
Calculation unloading method and system for task with dependency relationship in edge calculation
CN116755882A
Computing task unloading method based on graph theory and small sample near-end strategy optimization
CN116820621A
Task scheduling method and device for big data job, electronic equipment and storage medium
CN116974729A
Mobility unloading method based on graph neural network
CN117149416A
Cited By
Cooperative data migration scheduling method based on topology awareness and deep reinforcement learning
CN120469982A
