A 6g network multi-agent real-time task offloading algorithm based on a timing diagram

Abstract

The present application relates to the field of mobile edge computing, and particularly to a 6G network multi-agent real-time task offloading algorithm based on a time sequence diagram, comprising: for the dynamic connection relationship between the intelligent assistant and the edge node in the moving process, a representation model based on a time sequence diagram is established; according to the task offloading selection model and the energy efficiency cost of the task execution energy consumption minimization task offloading of the offloading node, a multi-objective optimization problem task offloading and resource allocation model is established; the task offloading decision and the resource allocation problem are decoupled; based on the network topology structure of the time sequence diagram, a graph reinforcement learning framework is designed, which is used to realize the optimal decision of task offloading in the 6G network environment, and through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved, the efficiency and energy efficiency of multi-agent task offloading are significantly improved, and the problem of insufficient optimization effect of the existing method in the dynamic environment is solved.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing, and specifically to a 6G network multi-agent real-time task offloading algorithm based on a timing diagram. Background Art

[0002] With the rapid development of the new generation of information technology, the network is transforming into an integrated platform for communication and computing, providing opportunities for mobile networks to provide multifaceted services beyond data transmission to edge AI agents.

[0003] In the future 6G network, intelligent agents will inevitably become "new citizens" that the network will need to connect and serve. With the emergence of a large number of these "new citizens," the demand for network intelligence and capabilities is increasing. However, there is a significant conflict between the goal of enhancing the capabilities of intelligent agents to better serve humanity and their relatively limited computing power, which inevitably leads to a strong demand for intelligent empowerment. Furthermore, the massive amount of data generated by intelligent agents will place tremendous computing pressure on the core of the network cloud. Furthermore, data exchange between the cloud and the edge will also place a significant load on capacity-constrained front-end links. Therefore, it is necessary to consider the coordinated construction of a hybrid resource scheduling mechanism that integrates end-to-end, network-to-cloud collaboration to provide distributed AI computing resources at the network edge. Edge computing is becoming a key technology for improving network efficiency and reducing latency. The application of AI agents in edge computing offers a new solution for real-time offloading of complex tasks and resource optimization. However, task offloading by AI agents in edge computing faces multiple challenges.

[0004] The main challenge of offloading tasks of AI agents at edge nodes is limited resources. Unlike traditional edge servers deployed at the edge to provide computing power support for devices, a large number of idle AI agents have become the main body of computing power support at the edge. Similarly, the computing, storage and energy consumption capabilities of AI agents are usually limited, and the 6G network requires the processing of high-density, low-latency tasks. How to effectively schedule limited resources is one of the key difficulties. Research shows that resource allocation problems involve multi-objective optimization, which requires balancing latency, bandwidth, computing power and energy consumption among multiple tasks. In addition, the heterogeneity and coupling between tasks increase the complexity of resource management. Different tasks have different resource requirements, which makes it difficult for traditional static resource scheduling methods to meet the needs of real-time offloading.

[0005] During task offloading, AI agents rely on accurate predictions of task demand to determine which tasks should be executed locally and which should be offloaded to the edge or cloud. However, due to the highly dynamic nature of the network environment, such as the unpredictable behavior of AI agents and fluctuations in network resources, accurately predicting task demand becomes extremely difficult.

[0006] A core goal of edge computing is to reduce latency. Latency is a key factor affecting the quality of service for computing and AI tasks. Latency is generally composed of three components: communication latency, queuing latency, and computational latency. Communication latency is affected by data volume and communication quality. Queuing latency is affected by the length of the task queue in the compute node. Computational latency is influenced by factors such as task workload, model parameters, and the node's remaining computing power. Ensuring a stable and low-latency task offloading process without incurring additional overhead is a key challenge currently facing edge computing.

[0007] Energy efficiency and power consumption optimization. While 6G networks offer faster transmission speeds and greater bandwidth for task offloading, this also translates to higher energy consumption. For AI agents with limited energy, prolonged, high-intensity task processing rapidly depletes battery life, impacting device endurance. Current research focuses on using intelligent scheduling algorithms to minimize energy consumption while ensuring efficient task offloading. However, further research is needed to balance energy efficiency optimization with efficient task processing. Summary of the Invention

[0008] The purpose of the present invention is to provide a 6G network multi-agent real-time task offloading algorithm based on a timing diagram to solve the problems raised in the above background technology.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A 6G network multi-agent real-time task offloading algorithm based on a timing diagram, the method includes:

[0011] Step 1: Establish a representation model based on a time sequence diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during its movement;

[0012] Step 2: Based on the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established;

[0013] Step 3: Decouple the task offloading decision from the resource allocation problem. Based on the network topology structure of the timing graph, a graph reinforcement learning framework is designed to achieve the optimal decision for task offloading in the 6G network environment. The graph neural network is responsible for extracting node and edge features from the timing graph and optimizing the offloading decision through a reinforcement learning algorithm. Through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved.

[0014] Preferably, in step 1, establishing a characterization model based on a timing diagram includes:

[0015] S101, the edge node includes an intelligent terminal and a base station;

[0016] The dynamic connection relationship is as follows: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a dynamic service network centered on the user;

[0017] S102. Establish a representation model G based on a timing diagram based on a dynamic connection relationship u =(v u ,ε u ,T), which is the set of nodes that can be offloaded during task offloading;

[0018] Among them, v u Represents a set of offloading nodes, where offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a two-hop link from edge node to smart assistant to edge node, which is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Label L u represents the available period of the edge node for the intelligent assistant; T represents the delay limit of the task to be offloaded.

[0019] Preferably, in step 2, a multi-objective optimization problem task offloading and resource allocation model is established, including:

[0020] S201. Assume that a computationally intensive task with a delay limit of D is composed of M subtasks, forming a subtask set M = {1, 2, ..., M}; then the subtask decision variable set based on subtask m is: Satisfying the task offloading selection model

[0021] As described above, during the task offloading process, subtask m has three options: the first is to choose to execute locally; the second is to offload to the base station server for calculation; the third is to choose to offload to the idle agent around the intelligent assistant for execution;

[0022] in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; N represents the edge node subscript set, and S represents the base station subscript set;

[0023] Among them, when executed locally, When the task is offloaded to the agent, When the Assistant chooses to offload to the base station,

[0024] S202: Based on the task offloading selection model, when a subtask is processed locally at the intelligent assistant, its completion time is the execution and calculation time; when a subtask is offloaded to an edge node for service, its completion time includes data transfer time, queue waiting time, execution and calculation time, and calculation result return time; the calculation result of each subtask is ultimately returned to the intelligent assistant;

[0025] By analyzing the completion time and node energy consumption of offloading, the energy efficiency cost of task offloading is minimized by calculating the frequency, and the task offloading and resource allocation model is confirmed.

[0026] Preferably, in S202, by analyzing the completion time of the offloading and the node energy consumption, calculating the energy efficiency cost of minimizing the frequency of task offloading, and confirming the task offloading and resource allocation model, the following steps are included:

[0027] S1. When the subtask is offloaded and processed locally at the smart assistant, confirm the task completion time separately. and node energy consumption

[0028] Get the computing power of local processing subtask m and calculate the load μ m , and the local task execution time is And the energy consumption of smart assistant node is

[0029] Where κ represents the effective switch capacitance coefficient determined by the chip architecture of the computing device, and here κ is set to 10 -27 ,Obviously, by adjusting the computation frequency of the device, a trade-off can be made between ,energy consumption and completion time;

[0030] Since during the task offloading process, a task can only be started after receiving all the results of its predecessor tasks and reaching the specified node, the completion time of all predecessor tasks of subtask m is obtained. As the preparation time of subtask m, the task completion time of local processing of subtask m is further obtained

[0031] S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed. Energy consumption of smart assistant transmission

[0032] The rate at which the intelligent assistant u transmits subtask m to the intelligent agent n is set as At the same time, obtain the computing power assigned by agent n to subtask m in the task list Based on the transmission rate Calculate load μ m and computing power Get task execution time Smart Assistant task transfer time and intelligent body energy consumption

[0033] Further obtain the intelligent assistant transmission energy consumption where p m Indicates the transmit power;

[0034] S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of smart assistant transmission

[0035] Assume that the rate at which the intelligent assistant u transmits task m to the base station s is denoted as At the same time, obtain the computing power allocated by the base station server for subtask m and obtain the task execution time Task transfer time and energy consumption of base station servers Further obtain the intelligent assistant transmission energy consumption

[0036] S4. Analyze the completion time of tasks offloaded to edge agents and base station servers

[0037] Assume that the agent node n receives a certain number of computing tasks in each time slot τ. In the time slot τ, the kth task in the task list of the agent n or the base station s is represented by q k , in order to facilitate the calculation of delay, let task q k The corresponding task sequence in the computationally intensive task initiated by an intelligent assistant is subtask m; due to the existence of the task processing queue, task q k There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list:

[0038] Among them, k represents the sequence number of subtask m in the agent task list, and ht0 represents the calculation of the current task q 0 The service time required, Indicates the sum of the execution time of all tasks in the queue before this task;

[0039] Based on this, we get the agent n for task q kThe task execution time is: Similarly, we get the task q on base station s k The task execution time is:

[0040] Get the preparation time of subtask m to be offloaded to the agent or base station for processing According to task q k The task execution time of agent n and base station s is respectively used to obtain the task completion time of the task offloaded to the edge agent: and completion time of tasks offloaded to the base station server

[0041] S5. Get task completion time respectively and Confirm that the completion delay of subtask m is:

[0042]

[0043] S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model:

[0044]

[0045] in, are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet

[0046] For example, when the initiator and most of the cooperating terminals in the network are not plugged in or have low battery, the main consideration is the energy consumption cost, so we can set a larger To save energy; on the contrary, if the computing task is very sensitive to the execution time, you can set a larger To reduce delay.

[0047] Preferably, solving the optimal task offloading strategy and edge resource allocation strategy in step 3 includes:

[0048] S301. Based on the task offloading and resource allocation model, the optimization problem of the system energy efficiency cost structure is decoupled into two sub-problems: the offloading decision P1 and the optimization problem P2 with convex properties;

[0049] P1 involves the decision computation for task offloading in MEC, while P2 involves the resource allocation problem;

[0050] The uninstall decision P1 is:

[0051] Where A represents the task offloading selection matrix, T represents the set of tasks that the agent needs to decide, U represents the set of agent subscripts, M represents the set of subtasks, and the constraint C1 indicates that each subtask of the intelligent assistant has three offloading options: the first is processed locally, the second is processed by the base station server, and the third is executed by the agent, and a task can only be executed by one node;

[0052] The optimization problem P2 of the convex property is:

[0053] Where K represents the number of RB resource blocks allocated by the base station, and P represents the calculation frequency of the calculation terminal;

[0054] Constraints C2 to C6 represent:

[0055] C2: The execution time of subtask m must be within the specified time, and the actual completion time of the intelligent assistant task must be less than the calculated delay limit;

[0056] C3: The intelligent assistant subtask m should be performed within the coverage area of ​​the edge node;

[0057] C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users;

[0058] The cellular user indicates that the task will be offloaded to the intelligent assistant of the base station, and the D2D user indicates that the task will be offloaded to the intelligent assistant of the agent;

[0059] C5: The number of RB resource blocks obtained by each cellular user is ≥ 0, and the number of RB resource blocks reused by each pair of D2D users is ≥ 0;

[0060] C6: The computing power allocated to the agent ≤ the maximum computing power of the agent;

[0061] S302, using the T-GRL framework to analyze the optimal decision for task offloading;

[0062] In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, it saves the quadruple obtained in each iteration into memory, trains the GNN based on experience replay, and approaches the optimal strategy through exploration and rapid learning experience;

[0063] At the same time, since the timing graph is used as the GNN input, the difference in the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

[0064] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned 6G network multi-agent real-time task offloading algorithm based on a timing diagram.

[0065] A computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the above-mentioned 6G network multi-agent real-time task offloading algorithm based on a timing diagram.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] The present invention accurately captures the dynamic connection relationship between the intelligent assistant and the edge node through a modeling method based on a timing graph, providing a more flexible and accurate offloading decision. The time-varying edge features in the timing graph effectively solve the problem of network coverage changes during the movement of the intelligent assistant, ensure the accuracy and timeliness of the task offloading decision, and significantly improve the efficiency of multi-agent collaborative task offloading in the 6G network; by adopting the graph reinforcement learning (T-GRL) framework, combined with the graph neural network (GNN), the nodes and edges in the network are extracted to effectively capture the topological relationship between devices. Through the reinforcement learning algorithm, the task offloading strategy and edge resource allocation are jointly optimized to achieve the global optimal decision in a dynamic network environment. The system can automatically adjust the offloading decision according to the changes in the network topology, avoiding the overfitting problem that traditional deep reinforcement learning may encounter when dealing with complex networks; by decoupling the task offloading decision and resource allocation problems, the coupling effect between the two is effectively avoided during the optimization process. This decoupling method reduces the computational complexity and improves the computational efficiency of the system. At the same time, it also enables the task offloading strategy to optimize energy efficiency and delay without sacrificing resource utilization, further improving the multi-agent collaboration capability in the 6G network. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0069] Figure 1 This is a timing diagram of the intelligent assistant accessing the base station in the present invention;

[0070] Figure 2 This is a time sequence scene diagram of the intelligent assistant accessing the intelligent agent in the present invention;

[0071] Figure 3 is a timing diagram model of the present invention;

[0072] Figure 4 It is a framework diagram of the T-GRL algorithm of the present invention. DETAILED DESCRIPTION

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0074] See also Figures 1-4 , the present invention provides a technical solution:

[0075] Example 1:

[0076] A 6G network multi-agent real-time task offloading algorithm based on a timing diagram, the method includes:

[0077] Step 1: Establish a representation model based on a time sequence diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during its movement;

[0078] Step 2: Based on the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established;

[0079] Step 3: Decouple the task offloading decision from the resource allocation problem. Based on the network topology structure of the timing graph, a graph reinforcement learning framework is designed to achieve the optimal decision for task offloading in the 6G network environment. The graph neural network is responsible for extracting node and edge features from the timing graph and optimizing the offloading decision through a reinforcement learning algorithm. Through multiple rounds of training and updating, the optimal task offloading strategy and edge resource allocation strategy are gradually solved.

[0080] Preferably, in step 1, establishing a characterization model based on a timing diagram includes:

[0081] S101, the edge node includes an intelligent terminal and a base station;

[0082] The dynamic connection relationship is as follows: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a dynamic service network centered on the user;

[0083] S102. Establish a representation model G based on a timing diagram based on a dynamic connection relationship u =(v u ,ε u ,T), which is the set of nodes that can be offloaded during task offloading;

[0084] Among them, v u Represents a set of offloading nodes, where offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a two-hop link from edge node to smart assistant to edge node, which is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Label L u represents the available period of the edge node for the intelligent assistant; T represents the delay limit of the task to be offloaded;

[0085] Task offloading research scenarios such as Figure 1 As shown,

[0086] The system consists of N intelligent terminals, U user intelligent assistants, and S base stations, collectively referred to as edge nodes. While users are on the move, intelligent assistant agents choose whether to process real-time computing tasks with specific business needs locally or offload them to the edge. As the user's location changes, the intelligent assistant can offload different subtasks to different edge nodes. Throughout the task offloading process, as the user moves, the covered MEC nodes form a dynamic, user-centric service network.

[0087] Many dynamic systems can be modeled using time series graphs, such as disease transmission processes, transport systems, and distributed computing systems. Unlike regular graphs, the set of nodes, the set of edges, or both in a time series graph can change over time. Therefore, we use timestamps to model the dynamic access relationships of mobile intelligent assistants.

[0088] Figure 1 The diagram shows a time-series scenario of an intelligent assistant connecting to an intelligent agent during movement, focusing on the communication coverage of the base station and the moment when the intelligent assistant establishes a connection with the base station. In the process of establishing a connection between the intelligent assistant and the base station or the intelligent assistant and the intelligent agent, the condition for a successful connection is that the distance between the two must be within the communication range of each other. However, since the design of the time-series scene diagram is difficult to directly present the dynamic changes in spatial position, it is assumed in this diagram that the intelligent assistant can establish a communication connection as soon as it enters the communication coverage of the base station. The moment when the connection is established in the diagram is represented by the time point corresponding to the tangent position of the boundary of the base station's communication coverage;

[0089] Figure 2 It describes the sequential scenarios of intelligent assistants accessing intelligent agents during mobility, characterizing the communication coverage between multiple intelligent agents and the moments when intelligent assistants establish connections with them.

[0090] Establish a representation model G based on a timing diagram based on dynamic connection relationships u =(v u,ε u ,T), through the means of “sampling”, the base stations or agents that the intelligent assistant can access at certain discrete moments are taken as the set of nodes that can be unloaded, and the time set is recorded as T = {T1, T2, ...}; at T i At this moment, the set of edge nodes that can be uninstalled by the intelligent assistant u is The set of edge nodes that can be uninstalled during the entire uninstallation process is The set of agents that can be uninstalled by the intelligent assistant u can be denoted as V u ={v1,v2,...,v N}, the set of base station nodes that can be offloaded is recorded as BS u ={bs1,bs2,...,bs S The edge node subscript set is denoted as N = {1, 2, ..., N}, the intelligent assistant subscript set is denoted as U = {1, 2, ..., U}, and the base station subscript set is denoted as S = {1, 2, ..., S}.

[0091] The following will be combined Figure 1 、 Figure 2 The study scenario shown in the figure further describes the timing diagram. We sampled the entire task offloading process L times. To facilitate the description of the timing diagram, we set L = 4 and the set of nodes that can be offloaded is When the intelligent assistant and the node are both within the communication range of each other, it is determined that the intelligent assistant can be offloaded to the node, where It is the set of nodes that can be uninstalled by the intelligent assistant u at the T1 timestamp. is the set of nodes available at timestamp T2, is the set of nodes available at timestamp T3, is the set of nodes that can be uninstalled at the T4 timestamp. u Uninstall node v at timestamp i and T u+1 All the detachable nodes under the timestamp establish edges, then for T u Base station node v at timestamp i and T u+1 v under timestamp j For a node, the label of its edge Used to indicate the effective period of intelligent assistant access during the switching process, where Is the smart assistant entering node v j At the time of communication coverage, Is the smart assistant leaving the uninstall node v j At the moment of communication range, it needs to be explained that the node v i ,v j Contains base station nodes and intelligent agent nodes.

[0092] Preferably, in step 2, a multi-objective optimization problem task offloading and resource allocation model is established, including:

[0093] S201. Assume that a computationally intensive task with a delay limit of D is composed of M subtasks, forming a subtask set M = {1, 2, ..., M}; then the subtask decision variable set based on subtask m is: Satisfying the task offloading selection model

[0094] in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; N represents the edge node subscript set, and S represents the base station subscript set;

[0095] Among them, when executed locally, When the task is offloaded to the agent, When the Assistant chooses to offload to the base station,

[0096] Preferably, the computation initiator needs to complete a computationally intensive mobile application. This computationally intensive task, with a latency limit of D, consists of M subtasks. The subtasks of a mobile application are interdependent; that is, a subtask may require the results of other subtasks before it can be started. The set of subtask subscripts is represented as M = {1, 2, ..., M}.

[0097] The computationally intensive tasks to be offloaded are represented by a directed acyclic graph (DAG), namely G A =(V A ,ε A ). Where V A is the set of subtasks, the set of edges ε A Used to express the dependency between subtasks. The relationship is:

[0098]

[0099] where e i,j ∈{0,1}(i,j∈M), and e i,j =1 means that the output data of task i is a necessary condition for executing task j, e i,j = 0 means that task i and task j are independent. Due to the nature of DAG, when i>j, there is e i,j = 0. If subtask v i ∈V A The output result is subtask v j∈V A If the input of the subtask v i is the output of the subtask v j , we say that there is a dependency relationship between the subtask v i and the subtask v j (v A , v i )∈ε and the subtask v j is the direct precedent subtask of the subtask v j , and the subtask v i is the direct subsequent subtask of the subtask v m ;

[0100] As can be known from the above description, in the task offloading execution process, the subtask m has three choices, the first is to select local execution, the second is to offload to the base station server for calculation, and the third is to select offloading to the idle intelligent agent around the intelligent assistant to execute;

[0101] S202, based on the task offloading selection model, when the subtask selects local processing at the intelligent assistant, the completion time is the execution calculation time; when the subtask selects offloading to the edge node for service, the completion time includes data transfer time, queuing waiting time, execution calculation time and calculation result return time; wherein the calculation result of each subtask is finally returned to the intelligent assistant;

[0102] By analyzing the completion time and node energy consumption of offloading, the energy efficiency cost of frequency minimization task offloading is calculated, and the task offloading and resource allocation model is confirmed.

[0103] Preferably, in S202, the completion time and node energy consumption of offloading are analyzed, the energy efficiency cost of frequency minimization task offloading is calculated, and the task offloading and resource allocation model is confirmed, including:

[0104] S1, when the subtask offloading selects local processing at the intelligent assistant, the task completion time and the node energy consumption

[0105] The calculation capability and the calculation load μ m of the local processing subtask m are obtained, the local task execution time is and the intelligent assistant node energy consumption is

[0106] Wherein, κ represents the effective switching capacitance coefficient determined by the chip architecture of the computing device, and here κ=10 -27 Obviously, by adjusting the computing frequency of the device, a trade-off between energy consumption and completion time can be made;

[0107] Since during the task offloading process, a task can only be started after receiving all the results of its predecessor tasks and reaching the specified node, the completion time of all predecessor tasks of subtask m is obtained. As the preparation time of subtask m, the task completion time of local processing of subtask m is further obtained

[0108] S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed. Energy consumption of smart assistant transmission

[0109] The rate at which the intelligent assistant u transmits subtask m to the intelligent agent n is set as At the same time, obtain the computing power assigned by agent n to subtask m in the task list Based on the transmission rate Calculate load μ m and computing power Get task execution time Smart Assistant task transfer time and intelligent body energy consumption

[0110] Further obtain the intelligent assistant transmission energy consumption where p m Indicates the transmit power;

[0111] S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of smart assistant transmission

[0112] Assume that the rate at which the intelligent assistant u transmits task m to the base station s is denoted as At the same time, obtain the computing power allocated by the base station server for subtask m and obtain the task execution time Task transfer time and energy consumption of base station servers Further obtain the intelligent assistant transmission energy consumption

[0113] S4. Analyze the completion time of tasks offloaded to edge agents and base station servers

[0114] Assume that the agent node n receives a certain number of computing tasks in each time slot τ. In the time slot τ, the kth task in the task list of the agent n or the base station s is represented by q k , in order to facilitate the calculation of delay, let task q kThe corresponding task sequence in the computationally intensive task initiated by an intelligent assistant is subtask m; due to the existence of the task processing queue, task q k There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list:

[0115] Among them, k represents the sequence number of subtask m in the agent task list, and ht0 represents the calculation of the current task q 0 The service time required, Indicates the sum of the execution time of all tasks in the queue before this task;

[0116] Based on this, we get the agent n for task q k The task execution time is: Similarly, we get the task q on base station s k The task execution time is:

[0117] Get the preparation time of subtask m to be offloaded to the agent or base station for processing According to task q k The task execution time of agent n and base station s is respectively used to obtain the task completion time of the task offloaded to the edge agent: and completion time of tasks offloaded to the base station server

[0118] S5. Get task completion time respectively and Confirm that the completion delay of subtask m is:

[0119]

[0120] S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model:

[0121]

[0122] in, are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet

[0123] For example, when the initiator and most of the cooperating terminals in the network are not plugged in or have low battery, the main consideration is the energy consumption cost, so we can set a larger To save energy; on the contrary, if the computing task is very sensitive to the execution time, you can set a larger To reduce delay.

[0124] Preferably, solving the optimal task offloading strategy and edge resource allocation strategy in step 3 includes:

[0125] S301. Based on the task offloading and resource allocation model, the optimization problem of the system energy efficiency cost structure is decoupled into two sub-problems: the offloading decision P1 and the optimization problem P2 with convex properties;

[0126] P1 involves the decision computation for task offloading in MEC, while P2 involves the resource allocation problem;

[0127] The uninstall decision P1 is:

[0128] Where A represents the task offloading selection matrix, T represents the set of tasks that the agent needs to decide, U represents the set of agent subscripts, M represents the set of subtasks, and the constraint C1 indicates that each subtask of the intelligent assistant has three offloading options: the first is processed locally, the second is processed by the base station server, and the third is executed by the agent, and a task can only be executed by one node;

[0129] The optimization problem P2 of the convex property is:

[0130] Where K represents the number of RB resource blocks allocated by the base station, and P represents the calculation frequency of the calculation terminal;

[0131] Constraints C2 to C6 represent:

[0132] C2: The execution time of subtask m must be within the specified time, and the actual completion time of the intelligent assistant task must be less than the calculated delay limit;

[0133] C3: The intelligent assistant subtask m should be performed within the coverage area of ​​the edge node;

[0134] C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users;

[0135] The cellular user indicates that the task will be offloaded to the intelligent assistant of the base station, and the D2D user indicates that the task will be offloaded to the intelligent assistant of the agent;

[0136] C5: The number of RB resource blocks obtained by each cellular user is ≥ 0, and the number of RB resource blocks reused by each pair of D2D users is ≥ 0;

[0137] C6: The computing power allocated to the agent ≤ the maximum computing power of the agent;

[0138] S302, using the T-GRL framework to analyze the optimal decision for task offloading;

[0139] In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, it saves the quadruple obtained in each iteration into memory, trains the GNN based on experience replay, and approaches the optimal strategy through exploration and rapid learning experience;

[0140] At the same time, since the timing graph is used as the GNN input, the difference in the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

[0141] Preferably, when considering problem P1, our goal is to minimize the total energy efficiency of the MEC system. We propose a T-GRL solution to solve the joint optimization problem of RB resource allocation and CPU computing frequency in MEC networks. We need to train the Actor network to quickly infer the optimal offloading decision action π: (T, K, P)aa based on the graph data of the MEC network. * The essence of computation offloading is a MIP problem. In a real dynamic MEC network, both the state space and the action space are very large.

[0142] The connection relationship of the agent in the MEC network is represented by a graph G==(V,E), where V represents the set of entity information nodes around the agent and E represents the connection relationship between network entities. In theory, we can enumerate 2 Ng(U+S) feasible solutions a, and select the strategy π that minimizes the agent's energy efficiency cost. However, this search method is difficult to implement, and reinforcement learning is almost impossible to obtain an accurate state representation.

[0143] Therefore, we introduced GNN to enhance the representation ability and continuously approach the optimal solution by updating GNN parameters. Compared with DNN, GNN can learn the relationship features between nodes in the graph, so it is more suitable for graph representation than DNN.

[0144] Chart data is very suitable for representing MEC status information. Here we use Figure G u To represent the entity and structural information of the intelligent assistant u in the MEC system (due to the particularity of GNN, the dimension of the adjacency matrix does not need to be fixed, so in order to reduce the computational complexity, the graph data G u With the intelligent assistant u as the center, it only needs to cover the range of two hops). The vertices of the graph represent the network entities in MEC, including computing capabilities and wireless communication rates. The edges in the graph represent the communication relationships between network entities, and the labels of the edges are Indicates the effective period of the intelligent assistant access during the switching process, where It is the intelligent assistant u and the intelligent agent vj The moment a communication connection is established, Is the intelligent assistant leaving the intelligent agent v j The communication range of the moment. MEC graph data can be obtained from the time sequence graph model. Whether a communication connection can be established between network entities depends on the distance between them. The maximum communication distance between two intelligent entities in MEC is expressed as D max Indicates. In the definition of the timing diagram model in Section 3.1, the location coordinates of the intelligent entity can be known, so the communication link can be established by calculating the location coordinates. i With v j The distance between them is calculated as follows:

[0145]

[0146] If D i,j <D max , then communication can be established between the two agents.

[0147] Figure 4 The complete structure of the T-GRL framework is shown; T-GRL consists of two parts: offloading decision and decision exploration;

[0148] We combine them through the actor-critic algorithm. Specifically, we construct the GNN reasoning framework as an Actor network, use formula (35) as the Critic part, save the quadruple obtained in each iteration to memory, and train the GNN based on experience replay, approximating the optimal strategy through exploration and rapid learning experience. In the Actor network, the offloading operation depends on the forward propagation of the GNN, which realizes reasoning through message passing and aggregation between nodes in the graph data. The aggregation method learned by GNN depends on the relationship between nodes. The addition and reduction of nodes in the topology and the connection and disconnection of edges will directly affect the neighborhood of the vertex. However, this will not affect the aggregation parameters. When faced with a new topology structure, T-GRL does not need to relearn parameters, but automatically filters information from disconnected nodes according to the changes in the graph data. Therefore, T-GRL has strong adaptability in dynamic MEC scenarios. In addition, since we use a time-series graph as GNN input, network iteration is different from the general graph reinforcement learning method, that is, the profit in a certain time period is used as the system reward and iterative updates are performed at a specific time step. For this scenario, we use the difference in the average total system energy efficiency cost of the decisions made and executed by the previous and subsequent intelligent assistants as the reward function, and update the network parameters when each new intelligent assistant makes a decision.

[0149] In the T-GRL framework, GNN is responsible for inferring offloading actions based on MEC. To ensure that T-GRL decisions are close to the optimal solution, we save the action with the lowest evaluation to the experience replay and iteratively train the network. As shown in Algorithm 1, GNN generates a′ and evaluates the total energy efficiency cost of the task through the task, and obtains the total energy efficiency cost of the intelligent assistant generation task as the cost of the MEC system. Then, based on a′, multiple exploratory actions are generated and different actions {C1, C2, ..., C B}, we select the local optimal solution from the explored B actions, while ensuring that the solution satisfies the constraints in P1, and save it to the experience replay <S u ,A u ,R u ,S u+1 > in;

[0150]

[0151] Minimizing the task response latency of MEC systems requires us to evaluate predictions and spend some time exploring better decisions. GNNs have enough hidden layers to be approximated with continuous mappings. We use the softmax activation function in the output layer and interpret the final prediction as a real number between (0, 1), which can be considered as the probability distribution of the classification result.

[0152] When GNN predicts a task for an intelligent assistant and outputs a′, we can quantize a′ and expand it into B ternary offloading actions, where B represents the number of tasks that need to be decided. Generally speaking, B is related to the size of the search space of the expansion algorithm. When the search space is fixed, the larger B is, the better the expansion algorithm is, but the computational complexity will increase accordingly. In order to balance performance and complexity, we adopt an equivalent regression algorithm with a proximity factor β. The basic idea is to quantize the prediction vector into a ternary decision based on Euclidean distance, and search non-neighbor space and neighbor space of different densities in the early stage of training to accelerate convergence. The expansion algorithm helps the network converge to the optimal global solution in the later stage of training. Specifically, for a given B and a′, the calculation process of the exploration algorithm is as follows:

[0153]

[0154] Where subscript u represents the intelligent assistant u; i∈|E u |,|E u Represents the dimension of the GNN output vector a′; b represents the bth decision to be made; set decision a b,i ={0,1,2} respectively represent that the task is executed locally, offloaded to the intelligent agent, and offloaded to the base station.

[0155] In order to avoid falling into the local optimal solution and better explore the action space, the proximity factor β is introduced as part of generating candidate unloading decisions;

[0156]

[0157] β is the ratio of actions generated by the extended algorithm. β can be adjusted based on the prediction results to prevent the model from falling into a local optimum. Indicates that from y b,i The decision vector in the matrix is ​​made based on minimizing the system energy efficiency cost;

[0158] After executing decisions on multiple intelligent processes, the combination of state sequences in experience replay will be used as experience to train the GNN. When the total system energy efficiency cost of the GNN decision is greater than the total system energy efficiency cost of the state-action sequence in experience replay, we will directly update the GNN parameters with the latter as the label.

[0159] For the uninstallation decision, the loss function is as follows:

[0160] Where y′ represents the model prediction value, y * represents the quantified true label, i.e., the offloading decision; I(g) represents the indicator function; Represents the weight of category c. During the training process, the weight is dynamically adjusted according to the current classification performance.

[0161] Example 2:

[0162] The computer-readable storage medium of this embodiment stores a computer program thereon, which, when executed by a processor, implements the steps in a 6G network multi-agent real-time task offloading algorithm based on a timing diagram in Example 1.

[0163] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0164] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0165] Example 3:

[0166] The computer device of the embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the 6G network multi-agent real-time task offloading algorithm based on a timing diagram in Embodiment 1 when executing the program.

[0167] In the embodiment, the processor can be a central processing unit, and can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any other conventional processor.

[0168] Those skilled in the art should understand that the embodiments disclosed herein can be provided as a method, a system, or a computer program product. Therefore, the present solution can be in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present solution can be in the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0169] The present solution is described with reference to flowcharts and / or block diagrams of the method and computer program product according to the embodiments of the present solution. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices generate a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or block diagrams.

[0170] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or block diagrams. Figure 1 The device that implements the functions specified in one or more flows and / or block diagrams.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or methods Figure 1 A step that specifies a function in one or more boxes.

[0172] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0173] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A 6G network multi-agent real-time task offloading method based on a timing diagram, characterized by: The method comprises: Step 1: Establish a representation model based on a time sequence diagram for the dynamic connection relationship between the intelligent assistant and the edge nodes during its movement; Step 2: Based on the task offloading selection model and the task execution energy consumption of the offloading node, the energy efficiency cost of task offloading is minimized, and a multi-objective optimization problem task offloading and resource allocation model is established; The step 2 of establishing a multi-objective optimization problem task offloading and resource allocation model includes: S201, set the delay limit to The computationally intensive tasks are performed by subtasks to form a subtask set ; based on subtasks The set of subtask decision variables: , satisfying the task offloading selection model ; in, Indicates that you choose to execute it locally on the smart assistant. Indicates that the program is offloaded to the intelligent agent terminal around the intelligent assistant for execution. Indicates that the calculation is offloaded to the base station server; Represents the edge node subscript set, represents the base station index set; Among them, when executed locally, , , ; When the task is offloaded to the agent, , , ; When the smart assistant chooses to offload to the base station, , , ; S202: Based on the task offloading selection model, when a subtask is processed locally at the intelligent assistant, its completion time is the execution and calculation time; when a subtask is offloaded to an edge node for service, its completion time includes data transfer time, queue waiting time, execution and calculation time, and calculation result return time; the calculation result of each subtask is ultimately returned to the intelligent assistant; By analyzing the completion time of offloading and node energy consumption, the energy efficiency cost of task offloading with the frequency minimum is calculated, and the task offloading and resource allocation model is confirmed. The specific formula is: ； in, 、 are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet ; in, represents the node energy consumption, represents the energy consumption of the intelligent body, Indicates that the intelligent assistant is sending The transmission energy consumption, represents the energy consumption of the base station server, Indicates that the smart assistant sends The transmission energy consumption, Indicates the task completion time of the local processing of the subtask, represents the completion delay of subtask m; Step 3: Decouple task offloading decisions from resource allocation. Based on the network topology of the time-series graph, a graph reinforcement learning framework is designed to achieve optimal decision-making for task offloading in a 6G network environment. The graph neural network extracts node and edge features from the time-series graph and optimizes offloading decisions using a reinforcement learning algorithm. Through multiple rounds of training and updates, the optimal task offloading strategy and edge resource allocation strategy are gradually solved. The step 3 of solving the optimal task offloading strategy and edge resource allocation strategy includes: S301, based on the task offloading and resource allocation model, decouple the optimization problem of system energy efficiency cost structure into two sub-problems: offloading decision and optimization problems with convex properties ; The uninstall decision for: ; in, represents the offloading selection matrix of the task, represents the set of tasks that the agent needs to decide, represents the agent subscript set, Represents a set of subtasks, constraints Each subtask of the intelligent assistant has three options for offloading: the first is to process it locally, the second is to process it on the base station server, and the third is to execute it on the agent. A task can only be executed by one node; The convex nature of the optimization problem for: ; in, Indicates the number of RB resource blocks allocated by the base station, Indicates the calculation frequency of the calculation terminal; Constraints 6 respectively represent: C2: Subtask The execution time must be within the specified time, and the actual completion time of the intelligent assistant task must be less than the calculated delay limit; C3: Intelligent Assistant subtask It should be carried out within the coverage area of ​​the edge node; C4: All cellular users are prohibited from sharing the same RB resource block and each RB resource block can only be reused by one pair of D2D users; The cellular user indicates that the task will be offloaded to the intelligent assistant of the base station, and the D2D user indicates that the task will be offloaded to the intelligent assistant of the agent; C5: The number of RB resource blocks obtained by each cellular user 0, and each pair of D2D users reuses the number of RB resource blocks 0; C6: Computational capacity allocated to the agent The maximum computing power of the agent; S302, using the T-GRL framework to analyze the optimal decision for task offloading; In the T-GRL framework, the GNN reasoning framework is constructed as an Actor network, and the task offloading and resource allocation model As the Critic part, it saves the quadruple obtained in each iteration into memory, trains the GNN based on experience replay, and approaches the optimal strategy through exploration and rapid learning experience; At the same time, since the timing graph is used as the GNN input, the difference in the average total system energy efficiency cost of the previous and subsequent intelligent assistants making decisions and executing them is used as the reward function, and the network parameters are updated when each new intelligent assistant makes a decision.

2. A 6G network multi-agent real-time task offloading method based on a timing diagram according to claim 1, characterized in that: The step 1 of establishing a characterization model based on a timing diagram includes: S101, the edge node includes an intelligent terminal and a base station; The dynamic connection relationship is as follows: when a user has a real-time computing task with certain business needs during the mobile process, the intelligent assistant agent chooses to perform local processing or edge offloading, and offloads the subtask to the edge node. During the entire task offloading process, as the user's geographical location changes, the covered edge nodes form a dynamic service network centered on the user; S102. Establish a representation model based on a timing diagram based on dynamic connection relationships , that is, the set of nodes that can be uninstalled during task offloading; in, Represents a set of offloading nodes, where offloading nodes include smart terminals, base stations, and local smart assistants in edge nodes; Represents a set of edges, where each edge is a two-hop link from edge node to smart assistant to edge node. It is used to represent the switching process of smart assistants between different nodes, i.e., the dynamic access process of smart assistants. Indicates the period of time during which the edge node is available to the intelligent assistant; Indicates the latency limit of the task to be offloaded.

3. A 6G network multi-agent real-time task offloading method based on a timing diagram as claimed in claim 1, characterized in that: In S202, by analyzing the completion time of the offloading and the node energy consumption, the energy efficiency cost of the task offloading is minimized by calculating the frequency, and the task offloading and resource allocation model is confirmed, including: S1. When the subtask is offloaded and processed locally at the smart assistant, confirm the task completion time separately. and node energy consumption : Get local processing subtasks Computing power and computational load , and the local task execution time is And the energy consumption of smart assistant node is ; in, Represents the effective switching capacitance coefficient determined by the chip architecture of the computing device; Since during task offloading, a task cannot start until all results of its predecessor tasks have been received and have reached the specified node, the subtask is obtained. Completion time of all predecessor tasks , as a subtask The preparation time of the subtask is further obtained by the task completion time of the local processing of the subtask. ; S2. When the subtask is selected to be offloaded to the intelligent terminal in the edge node for execution, the energy consumption of the intelligent terminal is confirmed. Energy consumption of smart assistant transmission : Setting up the Assistant Towards the Agent Transfer subtask The achievable rate is denoted as , and obtain the agent Assigned to a subtask in the task list Computing power ; Based on the transmission rate , calculate load and computing power Get task execution time , Intelligent Assistant task transmission time and intelligent body energy consumption ; Further obtain the intelligent assistant transmission energy consumption ;in Indicates the transmit power; S3. When the subtask is offloaded to the base station server for calculation, the energy consumption of the base station server is confirmed. Energy consumption of smart assistant transmission : Setting up the Assistant To the base station Transfer Task The achievable rate is denoted as , and obtain the base station server as a subtask Allocate computing power to get task execution time , task transmission time and energy consumption of base station servers ; Further obtain the intelligent assistant transmission energy consumption ; S4. Analyze the completion time of tasks offloaded to edge agents and base station servers 、 : Setting up agent nodes In each time slot A certain number of computing tasks are received within the time slot. Internal, intelligent or base station The kth task in the task list is represented as , in order to facilitate the calculation of delay, set the task The corresponding task sequence in the computationally intensive task initiated by an intelligent assistant is a subtask ; Due to the existence of the task processing queue, the task There is a waiting delay before being executed. The task execution waiting time can be expressed as the execution time of all tasks before k in the base station task execution list: ; Among them, k represents the subtask The sequence number in the agent's task list, Indicates calculation of the current task The service time required, Indicates the sum of the execution time of all tasks in the queue before this task; Based on this, we get the intelligent agent Targeted tasks The task execution time is: ; Similarly, get the base station Targeted tasks The task execution time is: ; Get subtasks offloaded to agents or base stations for processing Preparation time , then according to the task In the intelligent agent and base stations The task execution time is obtained, and the task completion time unloaded to the edge agent is and completion time of tasks offloaded to the base station server ; S5. Get task completion time respectively 、 and , confirm the subtask The completion delay is: ； S6. Calculate the energy efficiency cost of task offloading with the minimum frequency, and confirm the task offloading and resource allocation model: ； in, 、 are the weights of energy consumption and completion time, respectively. The preference between energy consumption and delay can be dynamically controlled by adjusting the weights to meet .

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a 6G network multi-agent real-time task offloading method based on a timing diagram as described in any one of claims 1-3 are implemented.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, it implements the steps in the 6G network multi-agent real-time task offloading method based on a timing diagram as described in any one of claims 1-3.

Images