6G network multi-agent real-time task unloading algorithm based on time sequence diagram

Abstract

The invention relates to the field of mobile edge computing, in particular to a time sequence diagram-based 6G network multi-agent real-time task unloading algorithm, which comprises the following steps of: establishing a time sequence diagram-based representation model according to a dynamic connection relationship between an intelligent assistant and an edge node in a moving process; establishing a multi-objective optimization problem task unloading and resource allocation model according to the task unloading selection model and the task execution energy consumption of the unloading node to minimize the energy efficiency cost of task unloading; decoupling the task unloading decision and the resource allocation problem; a network topology structure based on a time sequence diagram and a design drawing reinforcement learning framework are used for realizing an optimal decision of task unloading in a 6G network environment, and an optimal task unloading strategy and an edge resource allocation strategy are solved step by step through multiple rounds of training and updating, so that the efficiency and energy efficiency of multi-agent task unloading are remarkably improved, and the task unloading efficiency is improved. The problem that an existing method is insufficient in optimization effect in a dynamic environment is solved.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing, and in particular to a 6G network multi-agent real-time task offloading algorithm based on a timing diagram. Background Art

[0002] With the rapid development of new-generation information technology, the network is transforming into an integrated platform for communication and computing, providing opportunities for mobile networks to provide multifaceted services beyond data transmission to edge AI agents.

[0003] In the future 6G network, intelligent agents will inevitably become "new citizens" that the future network needs to connect and serve. With the emergence of a large number of intelligent "new citizens", the demand for network intelligence and empowerment has also come along. There is a significant contradiction between the goal of intelligent agents to improve their own capabilities in order to better serve humans and their relatively limited computing power, which will inevitably lead to a strong demand for intelligent agent intelligence and empowerment. In addition, the massive data brought by intelligent agents will bring huge computing pressure to the core of the network cloud. At the same time, the data interaction between the cloud edge will also bring huge load to the capacity-limited front-end link. Therefore, it is necessary to consider the end-network-cloud collaborative construction of a hybrid resource scheduling mechanism to provide distributed AI computing resources at the edge of the network. Edge computing has gradually become a key technology to improve network efficiency and reduce latency, and the application of AI agents in edge computing provides a new solution for real-time offloading and resource optimization of complex tasks. However, the task offloading of AI agents in edge computing faces multiple challenges.

[0004] The main challenge of AI agent task offloading at edge nodes is limited resources. Unlike the traditional deployment of edge servers at the edge to provide computing power support for devices, a large number of idle AI agents have become the main body of computing power support at the edge. Similarly, the computing, storage and energy consumption capabilities of AI agents are usually limited, and the 6G network requires the processing of high-density, low-latency tasks. How to effectively schedule limited resources is one of the key difficulties. Studies have shown that resource allocation problems involve multi-objective optimization, which requires balancing latency, bandwidth, computing power and energy consumption among multiple tasks. In addition, the heterogeneity and coupling between tasks increase the complexity of resource management. Different tasks have different requirements for resources, which makes it difficult for traditional static resource scheduling methods to meet the needs of real-time offloading.

[0005] AI agents rely on accurate prediction of task requirements during task offloading in order to decide which tasks should be performed locally and which should be offloaded to the edge or cloud. However, due to the highly dynamic nature of the network environment, such as the unpredictable behavior of AI agents and the fluctuation of network resources, it becomes extremely difficult to accurately predict task requirements.

[0006] A core goal of edge computing is to reduce latency. Latency is a key factor affecting the quality of service for computing tasks and artificial intelligence tasks. Latency usually consists of three parts: communication delay, queuing delay, and computational delay. Communication delay is affected by the amount of data and communication quality. Queueing delay is affected by the length of the task queue in the computing node. Computational delay is affected by factors such as task workload, model parameters, and the remaining computing power of the node. How to ensure that the task can maintain a stable and low-latency task offloading process during the offloading process without adding additional overhead is an important issue currently faced.

[0007] Energy efficiency and power consumption optimization issues. Although 6G networks provide faster transmission speeds and greater bandwidth for task offloading, this also means higher energy consumption. For AI agents with limited energy, long-term and high-intensity task processing will quickly consume power and affect the endurance of the device. Current research focuses on how to minimize energy consumption while ensuring task offloading efficiency through intelligent scheduling algorithms. However, the balance between energy efficiency optimization and efficient task processing still needs further study. Summary of the invention

[0008] The purpose of the present invention is to provide a 6G network multi-agent real-time task offloading algorithm based on a timing diagram to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A 6G network multi-agent real-time task offloading algorithm based on a timing diagram, the method comprising:

[0011] Step 1: Establish a representation model based on a timing diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during movement;

[0012] Step 2: According to the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established;

[0013] Step 3: Decouple the task offloading decision from the resource allocation problem. Based on the network topology structure of the timing graph, a graph reinforcement learning framework is designed to achieve the optimal decision for task offloading in the 6G network environment. The graph neural network is responsible for extracting the features of nodes and edges from the timing graph, and optimizing the offloading decision through a reinforcement learning algorithm. Through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved.

[0014] Preferably, in step 1, establishing a characterization model based on a timing diagram includes:

[0015] S101, the edge node includes an intelligent terminal and a base station;

[0016] The dynamic connection relationship is: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a user-centric dynamic service network;

[0017] S102. Establishing a representation model G based on a timing diagram based on a dynamic connection relationship u =(v u ,ε u ,T), which is the set of nodes that can be offloaded during task offloading;

[0018] Among them, v u represents a set of offloading nodes, where the offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a 2-hop link from edge node to smart assistant to edge node, which is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Label L u represents the available time period of the edge node for the intelligent assistant; T represents the delay limit of the task to be offloaded.

[0019] Preferably, in step 2, a multi-objective optimization problem task offloading and resource allocation model is established, including:

[0020] S201. It is assumed that a computationally intensive task with a delay limit of D is composed of M subtasks, forming a subtask set M = {1, 2, ..., M}; then the subtask decision variable set based on subtask m is: Satisfying the task offloading selection model

[0021] From the above description, we can see that during the task offloading process, subtask m has three options: the first is to choose to execute locally; the second is to be able to offload to the base station server for calculation; the third is to choose to offload to the idle agent around the intelligent assistant for execution;

[0022] in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; N represents the edge node index set, and S represents the base station index set;

[0023] Among them, when executed locally, When the task is offloaded to the agent, When the Assistant chooses to offload to a base station,

[0024] S202: Based on the task offloading selection model, when a subtask is selected to be processed locally at the intelligent assistant, its completion time is the execution and calculation time; when a subtask is selected to be offloaded to an edge node for service, its completion time includes data transfer time, queue waiting time, execution and calculation time, and calculation result return time; wherein the calculation result of each subtask is eventually returned to the intelligent assistant;

[0025] By analyzing the completion time of offloading and the node energy consumption, the energy efficiency cost of task offloading is minimized by calculating the frequency, and the task offloading and resource allocation model is confirmed.

[0026] Preferably, in S202, by analyzing the completion time of the unloading and the node energy consumption, the energy efficiency cost of the task unloading is minimized by calculating the frequency, and the task unloading and resource allocation model is confirmed, including:

[0027] S1. When the subtask is offloaded and processed locally at the smart assistant, confirm the task completion time separately and node energy consumption

[0028] Get the computing power of the local processing subtask m and calculate the load μ m , and the local task execution time is And the energy consumption of the smart assistant node is

[0029] Where κ represents the effective switch capacitance coefficient determined by the chip architecture of the computing device, and here κ is set to 10 -27 ,Obviously, by adjusting the computation frequency of the device, a trade-off can be made between ,energy consumption and completion time;

[0030] Since in the task offloading process, a task can only be started after receiving all the results of its predecessor tasks and reaching the specified node, the completion time of all predecessor tasks of subtask m is obtained. As the preparation time of subtask m, the task completion time of local processing of subtask m is further obtained.

[0031] S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed respectively. Energy consumption of communication with smart assistant

[0032] The rate at which the intelligent assistant u can transmit subtask m to the intelligent agent n is set as At the same time, obtain the computing power assigned by agent n to subtask m in the task list Based on the transmission rate Calculate the load μ m and computing power Get task execution time Smart Assistant task transfer time and energy consumption of intelligent body

[0033] Further obtain the intelligent assistant transmission energy consumption where p m Indicates the transmit power;

[0034] S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of communication with smart assistant

[0035] The rate at which the intelligent assistant u can transmit task m to the base station s is set to At the same time, obtain the computing power allocated by the base station server for subtask m and get the task execution time Task transfer time and the energy consumption of the base station server Further obtain the intelligent assistant transmission energy consumption

[0036] S4. Analyze the completion time of tasks offloaded to edge agents and base station servers

[0037] Assume that the agent node n receives a certain number of computing tasks in each time slot τ. In the time slot τ, the kth task in the task list of the agent n or the base station s is represented as q k To facilitate the calculation of delay, let task q k The corresponding task sequence in the computationally intensive task initiated by a certain intelligent assistant is subtask m; due to the existence of the task processing queue, task q k There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list:

[0038] Among them, k represents the sequence number of subtask m in the agent task list, and ht0 represents the calculation of the current task q 0 The service time required, Indicates the sum of execution time of all tasks in the queue before this task;

[0039] Based on this, we get the agent n for task q kThe task execution time is: Similarly, we get the value of task q on base station s k The task execution time is:

[0040] Get the preparation time of subtask m to be offloaded to the agent or base station for processing According to task q k The task execution time of agent n and base station s is respectively used to obtain the task completion time offloaded to the edge agent: and completion time of the task offloaded to the base station server

[0041] S5. Get the task completion time respectively and Confirm that the completion delay of subtask m is:

[0042]

[0043] S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model:

[0044]

[0045] in, are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet

[0046] For example, when the initiator and most of the cooperating terminals in the network are not plugged in or have low battery, the main consideration is the energy cost, so we can set a larger To save energy; on the contrary, if the computing task is very sensitive to execution time, you can set a larger To reduce delay.

[0047] Preferably, solving the optimal task offloading strategy and edge resource allocation strategy in step 3 includes:

[0048] S301, based on the task offloading and resource allocation model, decouple the optimization problem of system energy efficiency cost construction into two sub-problems: offloading decision P1 and optimization problem P2 with convex properties;

[0049] P1 involves the decision computation for task offloading in MEC, while P2 involves the resource allocation problem;

[0050] The uninstall decision P1 is:

[0051] Among them, A represents the task offloading selection matrix, T represents the task set that the agent needs to decide, U represents the agent subscript set, M represents the subtask set, and the constraint condition C1 represents that each subtask of the intelligent assistant has three options for offloading: the first is processed locally, the second is processed by the base station server, and the third is executed by the agent, and a task can only be executed by one node;

[0052] The convex optimization problem P2 is:

[0053] Wherein, K represents the number of RB resource blocks allocated by the base station, and P represents the calculation frequency of the calculation terminal;

[0054] Constraints C2 to C6 represent:

[0055] C2: The execution time of subtask m must be within the specified time, and the actual completion time of the intelligent assistant task is less than the calculated delay limit;

[0056] C3: The intelligent assistant subtask m should be performed within the coverage area of ​​the edge node;

[0057] C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users;

[0058] The cellular user indicates that he will choose to offload the task to the intelligent assistant of the base station, and the D2D user indicates that he will choose to offload the task to the intelligent assistant of the intelligent body;

[0059] C5: The number of RB resource blocks obtained by each cellular user is ≥ 0, and the number of RB resource blocks reused by each pair of D2D users is ≥ 0;

[0060] C6: The computing power allocated to the agent ≤ the maximum computing power of the agent;

[0061] S302, using the T-GRL framework to analyze the optimal decision for task offloading;

[0062] In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, the quadruple obtained in each iteration is saved to the memory, and the GNN is trained based on experience replay to approach the optimal strategy through exploration and rapid learning experience;

[0063] At the same time, since the timing graph is used as the GNN input, the difference between the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

[0064] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned 6G network multi-agent real-time task offloading algorithm based on a timing diagram.

[0065] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps in the above-mentioned 6G network multi-agent real-time task offloading algorithm based on a timing diagram are implemented.

[0066] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0067] The present invention accurately captures the dynamic connection relationship between the intelligent assistant and the edge node through a modeling method based on a timing graph, and provides a more flexible and accurate offloading decision. The time-varying edge feature in the timing graph effectively solves the problem of network coverage changes of the intelligent assistant during movement, ensures the accuracy and timeliness of the task offloading decision, and significantly improves the efficiency of multi-agent collaborative task offloading in the 6G network; by adopting the graph reinforcement learning (T-GRL) framework, the nodes and edges in the network are extracted with the graph neural network (GNN), thereby effectively capturing the topological relationship between devices. Through the reinforcement learning algorithm, the task offloading strategy and edge resource allocation are jointly optimized, and the global optimal decision can be achieved in a dynamic network environment. The system can automatically adjust the offloading decision according to the change of the network topology, avoiding the overfitting problem that traditional deep reinforcement learning may encounter when dealing with complex networks; by decoupling the task offloading decision and the resource allocation problem, the coupling effect between the two is effectively avoided in the optimization process. This decoupling method reduces the computational complexity and improves the computational efficiency of the system. At the same time, it also enables the task offloading strategy to optimize energy efficiency and delay without sacrificing resource utilization, further improving the multi-agent collaboration capability in the 6G network. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 This is a timing scenario diagram of the intelligent assistant accessing the base station in the present invention;

[0070] Figure 2 It is a time sequence scene diagram of the intelligent assistant accessing the intelligent agent in the present invention;

[0071] Figure 3 is a timing diagram model of the present invention;

[0072] Figure 4 It is the framework diagram of the T-GRL algorithm of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] See also Figure 1-Figure 4 , the present invention provides a technical solution:

[0075] Embodiment 1:

[0076] A 6G network multi-agent real-time task offloading algorithm based on a timing diagram, the method comprising:

[0077] Step 1: Establish a representation model based on a timing diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during movement;

[0078] Step 2: According to the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established;

[0079] Step 3: Decouple the task offloading decision from the resource allocation problem. Based on the network topology structure of the timing graph, a graph reinforcement learning framework is designed to achieve the optimal decision for task offloading in the 6G network environment. The graph neural network is responsible for extracting the features of nodes and edges from the timing graph, and optimizing the offloading decision through a reinforcement learning algorithm. Through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved.

[0080] Preferably, in step 1, establishing a characterization model based on a timing diagram includes:

[0081] S101, the edge node includes an intelligent terminal and a base station;

[0082] The dynamic connection relationship is: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a user-centric dynamic service network;

[0083] S102. Establishing a representation model G based on a timing diagram based on a dynamic connection relationship u =(v u ,ε u ,T), which is the set of nodes that can be offloaded during task offloading;

[0084] Among them, v u represents a set of offloading nodes, where the offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a 2-hop link from edge node to smart assistant to edge node, which is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Label L u represents the available time period of the edge node for the intelligent assistant; T represents the delay limit of the task to be offloaded;

[0085] Task offloading research scenarios such as Figure 1 As shown,

[0086] The system consists of N intelligent terminals, U user intelligent assistants, and S base stations, which are collectively referred to as edge nodes. During the user's mobile process, the intelligent assistant agent chooses to process local processing or edge offloading for real-time computing tasks with certain business needs. As the user's geographical location changes, the intelligent assistant can offload different subtasks to different edge nodes. During the entire task offloading process, as the user moves, the covered MEC nodes form a user-centric dynamic service network.

[0087] Many dynamic systems can be modeled using time series graphs, such as disease transmission processes, transmission systems, and distributed computing systems. Compared with ordinary graphs, the set of nodes, the set of edges, or both in a time series graph will change over time. Therefore, we use timestamps to model the dynamic access relationship of mobile intelligent assistants;

[0088] Figure 1 The time-series scenario of the intelligent assistant connecting to the intelligent body during movement is demonstrated, focusing on the communication coverage of the base station and the moment when the intelligent assistant establishes a connection with the base station. In the process of establishing a connection between the intelligent assistant and the base station or the intelligent assistant and the intelligent body, the condition for a successful connection is that the distance between the two must be within the communication range of each other. However, since the design of the time-series scenario graph is difficult to directly present the dynamic changes in spatial position, it is assumed in this figure that the intelligent assistant can establish a communication connection when it enters the communication coverage of the base station. The moment when the connection is established in the figure is represented by the time point corresponding to the boundary tangent position of the base station communication coverage;

[0089] Figure 2 It describes the sequential scenarios in which the intelligent assistant accesses the intelligent agent during mobility, characterizing the communication coverage between multiple intelligent agents and the moment when the intelligent assistant establishes connection with them.

[0090] Establish a representation model G based on a timing graph based on dynamic connection relationships u =(v u,ε u ,T), by means of “sampling”, the base stations or agents that the intelligent assistant can access at certain discrete moments are taken as the set of nodes that can be unloaded, and the time set is recorded as T = {T1, T2, ...}; at T i At this moment, the set of edge nodes that can be uninstalled by the intelligent assistant u is The set of edge nodes that can be uninstalled during the entire uninstallation process is The set of agents that can be uninstalled by the intelligent assistant u can be denoted as V u ={v1,v2,...,v N}, the set of base station nodes that can be uninstalled is denoted as BS u ={bs1,bs2,...,bs S The edge node subscript set is recorded as N = {1, 2, ..., N}, the intelligent assistant subscript set is recorded as U = {1, 2, ..., U}, and the base station subscript set is recorded as S = {1, 2, ..., S}.

[0091] The following will be combined Figure 1 , Figure 2 The study scenario shown in the figure further describes the timing diagram. We sampled the entire task offloading process L times. To facilitate the description of the timing diagram, we set L = 4 and the set of nodes that can be offloaded is When the intelligent assistant and the node are both within the communication range of each other, it is determined that the intelligent assistant can be offloaded to the node, where is the set of nodes that can be uninstalled by the smart assistant u at the T1 timestamp. is the set of nodes available at timestamp T2, is the set of nodes available at timestamp T3, is the set of nodes that can be uninstalled at timestamp T4. u Uninstalled node v at timestamp i and T u+1 All the unloadable nodes under the timestamp establish edges, then for T u Base station node v at timestamp i and T u+1 v under timestamp j For a node, the label of its edge It is used to indicate the effective period of intelligent assistant access during the switching process. Is the smart assistant entering node v j At the time of communication coverage, Is the smart assistant leaving the uninstall node v j At the moment of communication range, it needs to be explained that the node v i ,v j Contains base station nodes and intelligent agent nodes.

[0092] Preferably, in step 2, a multi-objective optimization problem task offloading and resource allocation model is established, including:

[0093] S201. It is assumed that a computationally intensive task with a delay limit of D is composed of M subtasks, forming a subtask set M = {1, 2, ..., M}; then the subtask decision variable set based on subtask m is: Satisfying the task offloading selection model

[0094] in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; N represents the edge node index set, and S represents the base station index set;

[0095] Among them, when executed locally, When the task is offloaded to the agent, When the Assistant chooses to offload to a base station,

[0096] Preferably, the computing initiator needs to complete a computing-intensive mobile application. The computing-intensive task with a delay limit of D is composed of M subtasks. The subtasks of the mobile application are interdependent, that is, a subtask may require the results of other subtasks to start. The set of subtask subscripts is represented as M = {1, 2, ..., M}.

[0097] The computationally intensive tasks to be offloaded are represented by a directed acyclic graph (DAG), namely G A =(V A ,ε A ). Where V A is the set of subtasks, the set of edges ε A It is used to express the dependency between subtasks. The relationship is:

[0098]

[0099] where e i,j ∈{0,1}(i,j∈M), and e i,j =1 means that the output data of task i is a necessary condition for executing task j, e i,j = 0 means that task i and task j are independent. Due to the nature of DAG, when i>j, there is e i,j = 0. If subtask v i ∈V A The output result is subtask v j∈V A The input of , we call the subtask v i and subtask v j There is a dependency relationship between i ,v j )∈ε A , and subtask v i It is subtask v j The immediate predecessor subtask, subtask v j It is subtask v i The direct successor subtask of

[0100] From the above description, we can see that during the task offloading process, subtask m has three options: the first is to choose to execute locally; the second is to be able to offload to the base station server for calculation; the third is to choose to offload to the idle agent around the intelligent assistant for execution;

[0101] S202: Based on the task offloading selection model, when a subtask is selected to be processed locally at the intelligent assistant, its completion time is the execution and calculation time; when a subtask is selected to be offloaded to an edge node for service, its completion time includes data transfer time, queue waiting time, execution and calculation time, and calculation result return time; wherein the calculation result of each subtask is eventually returned to the intelligent assistant;

[0102] By analyzing the completion time and node energy consumption of offloading, the energy efficiency cost of task offloading is minimized by calculating the frequency, and the task offloading and resource allocation model is confirmed.

[0103] Preferably, in S202, by analyzing the completion time of the unloading and the node energy consumption, the energy efficiency cost of the task unloading is minimized by calculating the frequency, and the task unloading and resource allocation model is confirmed, including:

[0104] S1. When the subtask is offloaded and processed locally at the smart assistant, confirm the task completion time separately and node energy consumption

[0105] Get the computing power of the local processing subtask m and calculate the load μ m , and the local task execution time is And the energy consumption of the smart assistant node is

[0106] Where κ represents the effective switch capacitance coefficient determined by the chip architecture of the computing device, and here κ is set to 10 -27 ,Obviously, by adjusting the computation frequency of the device, a trade-off can be made between ,energy consumption and completion time;

[0107] Since in the task offloading process, a task can only be started after receiving all the results of its predecessor tasks and reaching the specified node, the completion time of all predecessor tasks of subtask m is obtained. As the preparation time of subtask m, the task completion time of local processing of subtask m is further obtained.

[0108] S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed respectively. Energy consumption of communication with smart assistant

[0109] The rate at which the intelligent assistant u can transmit subtask m to the intelligent agent n is set as At the same time, obtain the computing power assigned by agent n to subtask m in the task list Based on the transmission rate Calculate the load μ m and computing power Get task execution time Smart Assistant task transfer time and energy consumption of intelligent body

[0110] Further obtain the transmission energy consumption of the intelligent assistant where p m Indicates the transmit power;

[0111] S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of communication with smart assistant

[0112] The rate at which the intelligent assistant u can transmit task m to the base station s is set to At the same time, obtain the computing power allocated by the base station server for subtask m and get the task execution time Task transfer time and the energy consumption of the base station server Further obtain the transmission energy consumption of the intelligent assistant

[0113] S4. Analyze the completion time of tasks offloaded to edge agents and base station servers

[0114] Assume that the agent node n receives a certain number of computing tasks in each time slot τ. In the time slot τ, the kth task in the task list of the agent n or the base station s is represented as q k To facilitate the calculation of delay, let task q kThe corresponding task sequence in the computationally intensive task initiated by a certain intelligent assistant is subtask m; due to the existence of the task processing queue, task q k There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list:

[0115] Among them, k represents the sequence number of subtask m in the agent task list, and ht0 represents the calculation of the current task q 0 The service time required, Indicates the sum of execution time of all tasks in the queue before this task;

[0116] Based on this, we get the agent n for task q k The task execution time is: Similarly, we get the value of task q on base station s k The task execution time is:

[0117] Get the preparation time of subtask m to be offloaded to the agent or base station for processing According to task q k The task execution time of agent n and base station s is respectively used to obtain the task completion time offloaded to the edge agent: and completion time of the task offloaded to the base station server

[0118] S5. Get the task completion time respectively and Confirm that the completion delay of subtask m is:

[0119]

[0120] S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model:

[0121]

[0122] in, are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet

[0123] For example, when the initiator and most of the cooperating terminals in the network are not plugged in or have low battery, the main consideration is the energy cost, so we can set a larger To save energy; on the contrary, if the computing task is very sensitive to execution time, you can set a larger To reduce delay.

[0124] Preferably, solving the optimal task offloading strategy and edge resource allocation strategy in step 3 includes:

[0125] S301, based on the task offloading and resource allocation model, decouple the optimization problem of system energy efficiency cost construction into two sub-problems: offloading decision P1 and optimization problem P2 with convex properties;

[0126] P1 involves the decision computation for task offloading in MEC, while P2 involves the resource allocation problem;

[0127] The uninstall decision P1 is:

[0128] Among them, A represents the task offloading selection matrix, T represents the task set that the agent needs to decide, U represents the agent subscript set, M represents the subtask set, and the constraint condition C1 represents that each subtask of the intelligent assistant has three options for offloading: the first is processed locally, the second is processed by the base station server, and the third is executed by the agent, and a task can only be executed by one node;

[0129] The convex optimization problem P2 is:

[0130] Wherein, K represents the number of RB resource blocks allocated by the base station, and P represents the calculation frequency of the calculation terminal;

[0131] Constraints C2 to C6 represent:

[0132] C2: The execution time of subtask m must be within the specified time, and the actual completion time of the intelligent assistant task is less than the calculated delay limit;

[0133] C3: The intelligent assistant subtask m should be performed within the coverage area of ​​the edge node;

[0134] C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users;

[0135] The cellular user indicates that he will choose to offload the task to the intelligent assistant of the base station, and the D2D user indicates that he will choose to offload the task to the intelligent assistant of the intelligent body;

[0136] C5: The number of RB resource blocks obtained by each cellular user is ≥ 0, and the number of RB resource blocks reused by each pair of D2D users is ≥ 0;

[0137] C6: The computing power allocated to the agent ≤ the maximum computing power of the agent;

[0138] S302, using the T-GRL framework to analyze the optimal decision for task offloading;

[0139] In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, the quadruple obtained in each iteration is saved to the memory, and the GNN is trained based on experience replay to approach the optimal strategy through exploration and rapid learning experience;

[0140] At the same time, since the timing graph is used as the GNN input, the difference between the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

[0141] Preferably, when considering problem P1, our goal is to minimize the total energy efficiency of the MEC system. We propose a T-GRL solution to solve the joint optimization problem of RB resource allocation and CPU computing frequency in MEC networks. We need to train the Actor network to quickly infer the best offloading decision action π based on the graph data of the MEC network: (T, K, P)aa * The essence of computation offloading is a MIP problem. In a real dynamic MEC network, both the state space and the action space are very large.

[0142] The connection relationship of the agent in the MEC network is represented by a graph G==(V,E), where V represents the set of entity information nodes around the agent and E represents the connection relationship between network entities. In theory, 2 Ng(U+S) There are feasible solutions a, and the strategy π with the lowest energy efficiency cost of the agent is selected. However, this search method is difficult to implement, and it is almost impossible for reinforcement learning to obtain accurate state representation.

[0143] Therefore, we introduced GNN to enhance the representation ability and continuously approach the optimal solution by updating GNN parameters. Compared with DNN, GNN can learn the relationship features between nodes in the graph, so it is more suitable for graph representation than DNN.

[0144] Chart data is very suitable for representing MEC status information. Here we use Chart G u To represent the entity and structural information of the intelligent assistant u in the MEC system (due to the particularity of GNN, the dimension of the adjacency matrix does not need to be fixed, so in order to reduce the computational complexity, the graph data G u The vertices of the graph represent network entities in MEC, including computing power and wireless communication rate. The edges in the graph represent the communication relationship between network entities, and the labels of the edges are Indicates the effective time period of the intelligent assistant access during the switching process, where is the intelligent assistant u and the intelligent agent vj The moment of establishing a communication connection, Is the assistant leaving the agent v j The communication range of the moment. MEC graph data can be obtained from the timing graph model. Whether a communication connection can be established between network entities depends on the distance between them. The maximum communication distance between two intelligent entities in MEC is expressed as D max In the definition of the timing diagram model in Section 3.1, we can know the location coordinates of the intelligent entity, so we can calculate the communication link through the location coordinates. i With v j The distance between them is calculated as follows:

[0145]

[0146] If D i,j <D max , then the two agents can establish communication.

[0147] Figure 4 The complete structure of the T-GRL framework is shown; T-GRL consists of two parts: offloading decision and decision exploration;

[0148] We combine them through the actor-critic algorithm. Specifically, we construct the GNN reasoning framework as an Actor network, use formula (35) as the Critic part, save the quadruple obtained in each iteration into memory, and train the GNN based on experience replay to approximate the optimal strategy through exploration and rapid learning experience. In the Actor network, the offloading operation depends on the forward propagation of the GNN, which realizes reasoning through message passing and aggregation between nodes in the graph data. The aggregation method learned by GNN depends on the relationship between nodes. The addition and reduction of nodes in the topology and the connection and disconnection of edges directly affect the neighborhood of the vertex. However, this does not affect the aggregation parameters. When faced with a new topology, T-GRL does not need to relearn parameters, but automatically filters information from disconnected nodes according to changes in the graph data. Therefore, T-GRL has strong adaptability in dynamic MEC scenarios. In addition, since we use a time-series graph as GNN input, the network iteration is not similar to the general graph reinforcement learning method, that is, the profit in a certain period of time is used as the system reward and iterative updates are made at a specific time step; for this scenario, we use the difference in the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them as the reward function, and update the network parameters when each new intelligent assistant makes a decision.

[0149] In the T-GRL framework, GNN is responsible for inferring the offloading action based on MEC; to ensure that the T-GRL decision is close to the optimal solution, we save the action with the lowest evaluation into the experience replay and iteratively train the network. As shown in Algorithm 1, GNN generates a′ and evaluates the total energy efficiency cost of the task generated by the intelligent assistant as the cost of the MEC system. Then, multiple exploratory actions are generated based on a′, and different actions {C1, C2, ..., C B}, we select the local optimal solution from the explored B actions, while ensuring that the solution satisfies the constraints in P1, and save it to the experience replay < S u ,A u ,R u ,S u+1 > in;

[0150]

[0151] Minimizing the task response latency of the MEC system requires us to evaluate the predictions and spend some time exploring better decisions. GNN has enough hidden layers to be approximated with continuous mapping. We use the softmax activation function in the output layer and use the final prediction result as a real number between (0,1), which can be considered as the probability distribution of the classification result.

[0152] When GNN predicts a task for an intelligent assistant and outputs a′, we can quantize a′ and expand it into B ternary offloading actions, where B represents the number of tasks that need to be decided. Generally speaking, B is related to the size of the search space of the expansion algorithm. When the search space is fixed, the larger B is, the better the expansion algorithm is, but the computational complexity will increase accordingly. In order to balance performance and complexity, we adopt an equivalent regression algorithm with a proximity factor β. The basic idea is to quantize the prediction vector into a ternary decision based on Euclidean distance, and search non-neighbor space and neighbor space of different densities in the early stage of training to accelerate convergence. The expansion algorithm helps the network converge to the optimal global solution in the later stage of training. Specifically, for a given B and a′, the calculation process of the exploration algorithm is as follows:

[0153]

[0154] Where subscript u represents the intelligent assistant u; i∈|E u |,|E u represents the dimension of the GNN output vector a′; b represents the bth decision to be made; set decision a b,i ={0,1,2} respectively represent that the task is executed locally, offloaded to the intelligent agent, or offloaded to the base station.

[0155] In order to avoid falling into the local optimal solution and better explore the action space, the proximity factor β is introduced as part of generating candidate unloading decisions;

[0156]

[0157] β means the proportion of actions generated by the extended algorithm. β can adjust the value according to the prediction results to prevent the model from falling into the local optimum. Indicates that from y b,i The decision vector in the matrix is ​​made based on minimizing the system energy efficiency cost;

[0158] After making decisions for multiple intelligent processing executions, the combination of state sequences in experience replay will be used as experience to train the GNN. When the total system energy efficiency cost of the GNN decision is greater than the total system energy efficiency cost of the state-action sequence in the experience replay, we will directly update the parameters of the GNN with the latter as the label.

[0159] For the unloading decision, the loss function is as follows:

[0160] Where y′ represents the model prediction value, y * represents the quantified true label, i.e., the unloading decision; I(g) represents the indicator function; Represents the weight of category c. During the training process, the weight is dynamically adjusted according to the current classification performance.

[0161] Embodiment 2:

[0162] The computer-readable storage medium of this embodiment stores a computer program thereon, which, when executed by a processor, implements the steps in a 6G network multi-agent real-time task offloading algorithm based on a timing diagram in Example 1.

[0163] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0164] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0165] Embodiment 3:

[0166] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in a 6G network multi-agent real-time task offloading algorithm based on a timing diagram in Embodiment 1 are implemented.

[0167] In this embodiment, the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, readily available programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0168] Those skilled in the art will appreciate that the disclosed content of the embodiments may be provided as methods, systems, or computer program products. Therefore, the present solution may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, the present solution may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0169] The present solution is described with reference to the method according to the embodiment of the present solution and the flowchart and / or block diagram of the computer program product. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 one or more processes and / or methods Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 one or more processes and / or methods Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 one or more processes and / or methods Figure 1 The steps for the functions specified in one or more boxes.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0173] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A 6G network multi-agent real-time task offloading algorithm based on timing diagram, characterized by: The method comprises: Step 1: Establish a representation model based on a timing diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during movement; Step 2: According to the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established; Step 3: Decouple the task offloading decision from the resource allocation problem. Based on the network topology structure of the timing graph, a graph reinforcement learning framework is designed to achieve the optimal decision for task offloading in the 6G network environment. The graph neural network is responsible for extracting the features of nodes and edges from the timing graph, and optimizing the offloading decision through a reinforcement learning algorithm. Through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved.

2. A 6G network multi-agent real-time task offloading algorithm based on a timing diagram as claimed in claim 1, characterized in that: The step 1 of establishing a characterization model based on a timing diagram includes: S101, the edge node includes an intelligent terminal and a base station; The dynamic connection relationship is: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a user-centric dynamic service network; S102. Establishing a representation model G based on a timing diagram based on a dynamic connection relationship u =(v u ,ε u ,T), which is the set of nodes that can be offloaded during task offloading; Among them, v u represents a set of offloading nodes, where the offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a 2-hop link from edge node to smart assistant to edge node, which is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Label L u represents the available time period of the edge node for the intelligent assistant; T represents the delay limit of the task to be offloaded.

3. A 6G network multi-agent real-time task offloading algorithm based on a timing diagram as claimed in claim 1, characterized in that: In step 2, a multi-objective optimization problem task offloading and resource allocation model is established, including: S201. It is assumed that a computationally intensive task with a delay limit of D is composed of M subtasks, forming a subtask set M = {1, 2, ..., M}; then the subtask decision variable set based on subtask m is: Satisfying the task offloading selection model in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; N represents the edge node index set, and S represents the base station index set; Among them, when executed locally, When the task is offloaded to the agent, When the Assistant chooses to offload to a base station, S202: Based on the task offloading selection model, when a subtask is selected to be processed locally at the intelligent assistant, its completion time is the execution and calculation time; when a subtask is selected to be offloaded to an edge node for service, its completion time includes data transfer time, queue waiting time, execution and calculation time, and calculation result return time; wherein the calculation result of each subtask is eventually returned to the intelligent assistant; By analyzing the completion time and node energy consumption of offloading, the energy efficiency cost of task offloading is minimized by calculating the frequency, and the task offloading and resource allocation model is confirmed.

4. A 6G network multi-agent real-time task offloading algorithm based on a timing diagram as claimed in claim 3, characterized in that: In S202, by analyzing the completion time of the unloading and the node energy consumption, the energy efficiency cost of the task unloading is minimized by calculating the frequency, and the task unloading and resource allocation model is confirmed, including: S1. When the subtask is offloaded and processed locally at the smart assistant, confirm the task completion time separately and node energy consumption Get the computing power of the local processing subtask m and calculate the load μ m , and the local task execution time is And the energy consumption of the smart assistant node is Where κ represents the effective switch capacitance coefficient determined by the chip architecture of the computing device; Since in the task offloading process, a task can only be started after receiving all the results of its predecessor tasks and reaching the specified node, the completion time of all predecessor tasks of subtask m is obtained. As the preparation time of subtask m, the task completion time of local processing of subtask m is further obtained. S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed respectively. Energy consumption of communication with smart assistant The rate at which the intelligent assistant u can transmit subtask m to the intelligent agent n is set as At the same time, obtain the computing power assigned by agent n to subtask m in the task list Based on the transmission rate Calculate the load μ m and computing power Get task execution time Smart Assistant task transfer time and energy consumption of intelligent body Further obtain the intelligent assistant transmission energy consumption where p m Indicates the transmit power; S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of communication with smart assistant The rate at which the intelligent assistant u can transmit task m to the base station s is set to At the same time, obtain the computing power allocated by the base station server for subtask m and get the task execution time Task transfer time and the energy consumption of the base station server Further obtain the intelligent assistant transmission energy consumption S4. Analyze the completion time of tasks offloaded to edge agents and base station servers Assume that the agent node n receives a certain number of computing tasks in each time slot τ. In the time slot τ, the kth task in the task list of the agent n or the base station s is represented as q k To facilitate the calculation of delay, let task q k The corresponding task sequence in the computationally intensive task initiated by a certain intelligent assistant is subtask m; due to the existence of the task processing queue, task q k There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list: Among them, k represents the sequence number of subtask m in the agent task list, and ht0 represents the calculation of the current task q 0 The service time required, Indicates the sum of execution time of all tasks in the queue before this task; Based on this, we get the agent n for task q k The task execution time is: Similarly, we get the value of task q on base station s k The task execution time is: Get the preparation time of subtask m to be offloaded to the agent or base station for processing According to task q k The task execution time of agent n and base station s is respectively used to obtain the task completion time offloaded to the edge agent: and completion time of the task offloaded to the base station server S5. Get the task completion time respectively and Confirm that the completion delay of subtask m is: S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model: in, are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet 5. A 6G network multi-agent real-time task offloading algorithm based on a timing diagram as claimed in claim 1, characterized in that: The optimal task offloading strategy and edge resource allocation strategy are solved in step 3, including: S301, based on the task offloading and resource allocation model, decouple the optimization problem of system energy efficiency cost construction into two sub-problems: offloading decision P1 and optimization problem P2 with convex properties; The uninstall decision P1 is: Among them, A represents the task offloading selection matrix, T represents the task set that the agent needs to decide, U represents the agent subscript set, M represents the subtask set, and the constraint condition C1 represents that each subtask of the intelligent assistant has three options for offloading: the first is processed locally, the second is processed by the base station server, and the third is executed by the agent, and a task can only be executed by one node; The convex optimization problem P2 is: Wherein, K represents the number of RB resource blocks allocated by the base station, and P represents the calculation frequency of the calculation terminal; Constraints C2 to C6 represent: C2: The execution time of subtask m must be within the specified time, and the actual completion time of the intelligent assistant task is less than the calculated delay limit; C3: The intelligent assistant subtask m should be performed within the coverage area of ​​the edge node; C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users; The cellular user indicates that he will choose to offload the task to the intelligent assistant of the base station, and the D2D user indicates that he will choose to offload the task to the intelligent assistant of the intelligent body; C5: The number of RB resource blocks obtained by each cellular user is ≥ 0, and the number of RB resource blocks reused by each pair of D2D users is ≥ 0; C6: The computing power allocated to the agent ≤ the maximum computing power of the agent; S302, using the T-GRL framework to analyze the optimal decision for task offloading; In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, the quadruple obtained in each iteration is saved to the memory, and the GNN is trained based on experience replay to approach the optimal strategy through exploration and rapid learning experience; At the same time, since the timing graph is used as the GNN input, the difference between the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a 6G network multi-agent real-time task offloading algorithm based on a timing diagram as described in any one of claims 1-5 are implemented.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, it implements the steps in a 6G network multi-agent real-time task offloading algorithm based on a timing diagram as described in any one of claims 1-5.

Images