A method for offloading multi-user computing in mobile edge computing

Abstract

This invention discloses a multi-user computation offloading method for mobile edge computing, belonging to the field of mobile edge computing system applications. It establishes a task offloading-related model, including a system model, a local computing model, a communication model, and an edge computing model, and optimizes the objective. Based on the established task offloading-related model, an MDP model is built, and a computation offloading decision algorithm based on Actor-Critic multi-agent deep reinforcement learning is used to obtain the corresponding offloading decision for each task. Each user device executes its own offloading decision to perform computation offloading. This invention can solve the computation offloading decision problem for dependent tasks in mobile edge computing and can effectively reduce task execution latency.

Description

Technical Field

[0001] This invention relates to the field of mobile edge computing system applications, specifically a method for offloading multi-user computing in mobile edge computing. Background Technology

[0002] In recent years, with the rapid development of mobile communication technology and the increasing popularity of smart mobile devices, new business applications such as facial recognition, voice recognition, and virtual reality have emerged. These mobile applications are often characterized by high computational demands, latency sensitivity, and high power consumption. However, mobile devices are limited by their physical size, and their computing power and battery capacity are usually relatively limited. This makes it difficult for them to meet the low latency requirements of applications, resulting in significant power consumption and affecting device usage time.

[0003] Mobile edge computing (MEC) provides computing, storage, and network resources at the edge of mobile networks. By offloading computing tasks to edge servers closer to users, it can effectively reduce processing latency and energy consumption. Since MEC server resources are limited, reasonable computing offloading decisions and resource allocation schemes need to be formulated to fully utilize the limited resources in MEC, reduce user latency and energy consumption, and meet the needs of multiple users.

[0004] For applications such as virtual reality and augmented reality, execution can be divided into multiple subtasks. These subtasks often have complex dependencies. Different subtasks can be executed locally on edge servers or devices based on reasonable computation offloading decisions and resource allocation schemes, thereby improving application execution efficiency. Therefore, only by specifying reasonable computation task offloading decisions and rationally allocating computation and communication resources in the MEC system can the performance of the MEC system be improved, the advantages of edge computing be leveraged, and user needs be met. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a multi-user computation offloading method for mobile edge computing, which can solve the computation offloading decision problem of dependent tasks in mobile edge computing and effectively reduce task execution latency.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A mobile edge computing multi-user computing offloading method is proposed. It establishes a task offloading related model, including a system model, a local computing model, a communication model, and an edge computing model, and optimizes the objective. Considering the dependencies between tasks, an MDP model is established based on the established task offloading related model. The corresponding offloading decision for each task is obtained through a computing offloading decision algorithm based on multi-agent deep reinforcement learning of Actor-Critic.

[0008] Each user device executes its own unloading decision and performs calculations for unloading.

[0009] This method is based on system models, computational models, and communication models. By using an Actor-Critic-based offloading decision method, it makes full use of limited computational and communication resources, thereby reducing latency and energy consumption of multi-user tasks and improving user experience.

[0010] In the Actor-Critic multi-agent deep reinforcement learning approach, each agent comprises an Actor network and a Critic network. The Actor is responsible for selecting actions to perform in order to explore different states, while the Critic is responsible for evaluating the actions performed and guiding the Actor's next action. The Actor adjusts the probability distribution of its chosen actions based on the Critic's evaluation. Both the Actor and Critic have two parts: an online network and a target network. The Actor online network simulates the current policy, outputting actions based on the agent's state, interacting with the environment to obtain the current reward, and determining the next state value. The Actor target network's output actions are not actually executed but assist in updating the Critic's parameters. The Critic online network evaluates actions based on the current states of all agents and the actions output by the Actor online network; the Critic target network evaluates the Actor target network's output based on the next states of all agents and the actions output by the Actor target network.

[0011] Preferably, the system model is constructed by assuming that there is an edge server in the MEC system, each user has an application to execute, and the application consists of multiple tasks (subtasks) with dependencies. The tasks can be executed locally or offloaded to the edge server for execution. Considering the periodic time slot system, each user device calculates and specifies the offload decision in each time slot.

[0012] Multiple user equipment are represented as:

[0013]

[0014] The task for each user device is represented as follows:

[0015]

[0016] i is one of the user devices, n i This indicates the number of tasks assigned to the application on user device i.

[0017] The computational unloading decisions for each task of the application are represented as follows:

[0018] a = {a1, a2, ..., a} i ,…,a n}, i∈{1,2,…,n}

[0019]

[0020] Among them, a i,j =0 indicates that the task is computed locally. This indicates that the task unloads data to the MEC server for computation.

[0021] Furthermore, the construction of the local computing model,

[0022] Local computation latency of the task Defined as:

[0023]

[0024] Similarly, the energy consumption generated by the equipment for:

[0025]

[0026] Where, ζ i u represents the energy consumption of user device i per CPU cycle. i,j f represents the computational complexity of the task. i l Provides local computing capabilities for user devices.

[0027] Furthermore, the construction of the communication model,

[0028] If the computation task is offloaded, its uplink data transfer rate is:

[0029]

[0030] Where W represents the system's wireless channel bandwidth, p i G represents the transmission power of user equipment i. i I represents the access channel gain. i This indicates interference during transmission. Indicates Gaussian white noise;

[0031] The transmission delay of the task is expressed as:

[0032]

[0033] Transmission energy consumption during task i's computational unloading for:

[0034]

[0035] in, This represents the transmission power of user equipment i.

[0036] Furthermore, the construction of the edge computing model,

[0037] If the computation task is offloaded, its computation latency on the edge server is:

[0038]

[0039] Among them, f o Assuming the edge server has the computing power to distribute computing power evenly across all unloading tasks, all tasks unloaded to the server will receive the same computing resources.

[0040] Furthermore, the construction optimization objective,

[0041] The latency of the user task is:

[0042]

[0043] For tasks computed locally (a) i,j For (=0), the computation latency is equal to the local computation latency;

[0044] For computational tasks that perform computational unloading (a i,j =1) In terms of the whole uninstallation process, it consists of three steps: uploading task-related data, executing the task on the edge server, and returning the calculation results; since the amount of data transmitted for the calculation results is usually much smaller than the amount of data transmitted for the upload task, the overhead of returning the results is ignored.

[0045] The energy consumption generated by user tasks is:

[0046]

[0047] If the task is computed locally (a i,j =0), the task's energy consumption is equal to the local computation's energy consumption. If the task performs computational unloading (a i,j =0), the task's energy consumption only includes the energy consumption of data transmission during unloading;

[0048] Because an application consists of multiple tasks with dependencies, the total execution latency of the application is equal to the completion time of the last task in the task dependency relationship.

[0049] The completion time of task j on user device i can be calculated recursively as follows:

[0050]

[0051] Where pred(j) represents the set of predecessor tasks of task j, meaning that task j can only be executed after all tasks in this set have been completed; specifically, This indicates there is no precursor mission.

[0052] The set of tasks completed in the t-th time slot is represented as follows: The execution latency of the entire application in the t-th time slot can be expressed as:

[0053]

[0054] When all n i After all tasks are completed, the total execution time of the application is:

[0055]

[0056] To comprehensively consider the latency and energy consumption of task offloading decisions, the optimization objective is set as minimizing the weighted sum of the latency and energy consumption of all applications, expressed as:

[0057]

[0058] Furthermore, the establishment of the MDP model,

[0059] In the MDP model, the state is defined by the offloading decision for each task and the scheduling order of tasks. The state of user equipment i at time slot t is represented as:

[0060]

[0061] In the MDP model, an action is defined as a change in adjacent states, i.e., a change in the unloading decision of a single task. The action at time slot t is represented as:

[0062]

[0063] Where i represents the user device task that needs to change the decision, and a i,j ,b i New decisions and task scheduling order for user task j;

[0064] In the MDP model, the reward function is set as the objective function value of the previous decision minus the objective function value of the new decision, expressed as:

[0065]

[0066] Among them, O(S) t ), O(S t+1 ) represents the objective function value corresponding to the state at time slot t, t+1, which is determined by the unloading decisions of all tasks and the task scheduling order.

[0067] Furthermore, the Actor-Critic-based computational unloading decision algorithm,

[0068] The Actor-Critic multi-agent reinforcement learning algorithm is as follows:

[0069] 1) Randomly initialize the parameters θ of the online network for n intelligent agents (Actors). A With target network parameters Critic online network parameter θ C With network parameters:

[0070]

[0071]

[0072] 2) Initialize the experience replay area Its capacity is defined as Used to store the traversed states, actions, and rewards;

[0073] 3) Initialize the initial state S of the first iteration 1 ;

[0074] 4) Repeat steps 4.1) to 4.6):

[0075] 4.1) For each agent, select a deterministic action from the Actor online network output with a probability of 1-ε:

[0076]

[0077] The remaining ε probability randomly selects any action.

[0078] 4.2) Execute action A t Obtain the new state S t+1 Calculate the reward R t ,

[0079] The data from this step (S) t A t ,R t ,S t+1 Save to experience replay area In the middle, proceed to the next step.

[0080] M batches of interactive data were randomly sampled from the experience replay area, with each batch consisting of (S) k A k ,R k ,S′ k The state value of each agent is input into the target neural network corresponding to that agent to obtain the action. The obtained actions and states are input into the Critic target network of each agent to obtain the evaluation value output by the target network. This represents the evaluation value output by the target network for the i-th agent; the state S... k A k Input the corresponding online Critic network for each agent and obtain its output evaluation value. This represents the evaluation value output by the online network for the i-th agent;

[0081] 4.3) Let the target evaluation value obtained by the k-th experience of the i-th agent be:

[0082]

[0083] Where γ is a parameter set by the user;

[0084] 4.4) The loss function of the Critic target network is:

[0085]

[0086] The objective is to minimize the loss function by updating the parameters of the Critic online network using gradient descent.

[0087] 4.5) Calculate the policy gradient of the Actor online network:

[0088]

[0089] Updating the parameters of the online Actor network based on gradient ascent

[0090] 4.6) Update the Actor and Critic target networks of the agent, where τ is a user-defined parameter:

[0091]

[0092]

[0093] The present invention also claims a mobile edge computing multi-user computing offloading device, comprising: at least one memory and at least one processor;

[0094] The at least one memory is used to store a machine-readable program;

[0095] The at least one processor is configured to invoke the machine-readable program to execute the above-described mobile edge computing multi-user computing offloading method.

[0096] The present invention also claims a computer-readable medium, characterized in that the computer-readable medium stores computer instructions, which, when executed by a processor, cause the processor to perform the above-described mobile edge computing multi-user computing offloading method.

[0097] Compared with the prior art, the mobile edge computing multi-user computing offloading method of the present invention has the following advantages:

[0098] This method takes into account the dependencies between tasks in a multi-user device and comprehensively considers the impact between multiple users. It can make full use of limited computing and communication resources, thereby reducing the latency and energy consumption of multi-user tasks and improving the overall multi-user experience. Attached Figure Description

[0099] Figure 1 This is a schematic diagram of application task dependencies provided in an embodiment of the present invention. Detailed Implementation

[0100] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0101] A multi-user computation offloading method for mobile edge computing is proposed. This method establishes a task offloading-related model, including a system model, a local computing model, a communication model, and an edge computing model, and sets an optimization objective. Considering the dependencies between tasks, an MDP model is established based on the established task offloading-related model. A computation offloading decision algorithm based on multi-agent deep reinforcement learning (Actor-Critic) is used to obtain the corresponding offloading decision for each task. Each user device executes its own offloading decision to perform computation offloading.

[0102] This method is based on system models, computational models, and communication models. By using an Actor-Critic-based offloading decision method, it makes full use of limited computational and communication resources, thereby reducing latency and energy consumption of multi-user tasks and improving user experience.

[0103] The specific implementation is as follows:

[0104] S1: The constructed system model,

[0105] Assume there is an edge server in the MEC system, each user has an application to execute, and the application consists of multiple tasks (subtasks) with dependencies. The tasks can be executed locally or offloaded to the edge server for execution. Considering the periodic time-slot system, each user device calculates and specifies the offload decision in each time slot.

[0106] Multiple user equipment are represented as:

[0107]

[0108] The task for each user device is represented as follows:

[0109]

[0110] i is one of the user devices, n i This indicates the number of tasks assigned to the application on user device i.

[0111] The computational unloading decisions for each task of the application are represented as follows:

[0112] a = {a1, a2, ..., a} i ,…,a n}, i∈{1,2,…,n}

[0113]

[0114] Among them, a i,j =0 indicates that the task is computed locally. This indicates that the task unloads data to the MEC server for computation.

[0115] S2: Building a local computation model:

[0116] Local computation latency of the task Defined as:

[0117]

[0118] Similarly, the energy consumption generated by the equipment for:

[0119]

[0120] Where, ζ i u represents the energy consumption of user device i per CPU cycle. i,j f represents the computational complexity of the task. i l Provides local computing capabilities for user devices.

[0121] S3: Constructing a Communication Model

[0122] If the computation task is offloaded, its uplink data transfer rate is:

[0123]

[0124] Where W represents the system's wireless channel bandwidth, p i G represents the transmission power of user equipment i. i I represents the access channel gain. i This indicates interference during transmission. Indicates Gaussian white noise;

[0125] The transmission delay of the task is expressed as:

[0126]

[0127] Transmission energy consumption during task i's computational unloading for:

[0128]

[0129] in, This represents the transmission power of user equipment i.

[0130] S4: Building an edge computing model:

[0131] If the computation task is offloaded, its computation latency on the edge server is:

[0132]

[0133] Among them, f o Assuming the edge server has the computing power to distribute computing power evenly across all unloading tasks, all tasks unloaded to the server will receive the same computing resources.

[0134] S5: Constructing Optimization Objectives

[0135] The latency of the user task is:

[0136]

[0137] For tasks computed locally (a) i,j For (=0), the computation latency is equal to the local computation latency;

[0138] For computational tasks that perform computational unloading (a i,j =1) In terms of the whole uninstallation process, it consists of three steps: uploading task-related data, executing the task on the edge server, and returning the calculation results. Since the amount of data transmitted for the calculation results is usually much smaller than the amount of data transmitted for the upload task, the overhead of returning the results is ignored.

[0139] The energy consumption generated by user tasks is:

[0140]

[0141] If the task is computed locally (a i,j =0), the task's energy consumption is equal to the local computation's energy consumption. If the task performs computational unloading (a i,j =0), the task's energy consumption only includes the energy consumption of data transmission during unloading;

[0142] Because an application consists of multiple tasks with dependencies, the total execution latency of the application is equal to the completion time of the last task in the task dependency relationship.

[0143] The completion time of task j on user device i can be calculated recursively as follows:

[0144]

[0145] Where pred(j) represents the set of predecessor tasks of task j, meaning that task j can only be executed after all tasks in this set have been completed; specifically, This indicates there is no precursor mission.

[0146] The set of tasks completed in the t-th time slot is represented as follows: The execution latency of the entire application in the t-th time slot can be expressed as:

[0147]

[0148] When all n i After all tasks are completed, the total execution time of the application is:

[0149]

[0150] To comprehensively consider the latency and energy consumption of task offloading decisions, the optimization objective is set as minimizing the weighted sum of the latency and energy consumption of all applications, expressed as:

[0151]

[0152] S6: Establish the MDP model:

[0153] In the MDP model, the state is defined by the offloading decision for each task and the scheduling order of tasks. The state of user equipment i at time slot t is represented as:

[0154]

[0155] In the MDP model, an action is defined as a change in adjacent states, i.e., a change in the unloading decision of a single task. The action at time slot t is represented as:

[0156]

[0157] Where i represents the user device task that needs to change the decision, and a i,j ,b i New decisions and task scheduling order for user task j;

[0158] In the MDP model, the reward function is set as the objective function value of the previous decision minus the objective function value of the new decision, expressed as:

[0159]

[0160] Among them, O(S) t ), O(S t+1 ) represents the objective function value corresponding to the state at time slot t, t+1, which is determined by the unloading decisions of all tasks and the task scheduling order.

[0161] S7: Actor-Critic-based computational unloading decision algorithm

[0162] Because the state space dimension becomes very high and the problem scale becomes very large when there are multiple user devices, traditional reinforcement learning methods can no longer be used to solve this high-dimensional problem. Therefore, artificial neural networks need to be introduced to fit the state value function. A multi-agent deep reinforcement learning method based on Actor-Critic can be used to solve this multi-user computation offloading problem.

[0163] To ensure a more stable and faster learning process, the Actor-Critic multi-agent deep reinforcement learning method incorporates an Actor network and a Critic network for each agent. The Actor selects actions to explore different states, while the Critic evaluates these actions and guides the Actor's next move. The Actor adjusts the probability distribution of its chosen actions based on the Critic's evaluation. Both the Actor and Critic have online and target networks: the Actor online network simulates the current policy, outputting actions based on the agent's state, interacting with the environment to obtain the current reward, and determining the next state value; the Actor target network's output actions are not actually executed but assist in updating the Critic's parameters. The Critic online network evaluates actions based on the current states of all agents and the actions output by the Actor online network; the Critic target network evaluates the Actor target network's output based on the next states of all agents and the actions output by the Actor target network.

[0164] The Actor-Critic multi-agent reinforcement learning algorithm is as follows:

[0165] 1) Randomly initialize the parameters θ of the online network for n intelligent agents (Actors). A With target network parameters Critic online network parameter θ C With network parameters:

[0166]

[0167]

[0168] 2) Initialize the experience replay area Its capacity is defined as Used to store the traversed states, actions, and rewards;

[0169] 3) Initialize the initial state S of the first iteration 1 ;

[0170] 4) Repeat steps 4.1) to 4.6):

[0171] 4.1) For each agent, select a deterministic action from the Actor online network output with a probability of 1-ε:

[0172]

[0173] The remaining ε probability randomly selects any action.

[0174] 4.2) Execute action A t Obtain the new state S t+1 Calculate the reward R t ,

[0175] The data from this step (S) t A t ,R t ,S t+1 Save to experience replay area In the middle, proceed to the next step.

[0176] M batches of interactive data were randomly sampled from the experience replay area, with each batch consisting of (S) k A k ,R k ,S′ k The state value of each agent is input into the target neural network corresponding to that agent to obtain the action. The obtained actions and states are input into the Critic target network of each agent to obtain the evaluation value output by the target network. This represents the evaluation value output by the target network for the i-th agent; the state S... k A k Input the corresponding online Critic network for each agent and obtain its output evaluation value. This represents the evaluation value output by the online network for the i-th agent;

[0177] 4.3) Let the target evaluation value obtained by the k-th experience of the i-th agent be:

[0178]

[0179] Where γ is a parameter set by the user;

[0180] 4.4) The loss function of the Critic target network is:

[0181]

[0182] The objective is to minimize the loss function by updating the parameters of the Critic online network using gradient descent.

[0183] 4.5) Calculate the policy gradient of the Actor online network:

[0184]

[0185] Updating the parameters of the online Actor network based on gradient ascent

[0186] 4.6) Update the Actor and Critic target networks of the agent, where τ is a user-defined parameter:

[0187]

[0188]

[0189] This invention also provides a mobile edge computing multi-user computing offloading device, comprising: at least one memory and at least one processor;

[0190] The at least one memory is used to store a machine-readable program;

[0191] The at least one processor is configured to call the machine-readable program to execute the mobile edge computing multi-user computing offloading method described in the above embodiments.

[0192] This invention also provides a computer-readable medium storing computer instructions. When executed by a processor, the computer instructions cause the processor to perform the mobile edge computing multi-user computing offloading method described in the above embodiments. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the above embodiments can be provided, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0193] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0194] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0195] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0196] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0197] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A method for offloading multi-user computing in mobile edge computing, characterized in that, Establish task offloading related models, including system model, local computing model, communication model, and edge computing model, and optimize the objectives; based on the established task offloading related models, establish an MDP model, and obtain the corresponding offloading decision for the task through a computational offloading decision algorithm based on multi-agent deep reinforcement learning of Actor-Critic; Each user equipment executes its own offloading decision and performs offloading calculations; The constructed system model assumes that there is an edge server in the MEC system, each user has an application to execute, and the application consists of multiple subtasks with dependencies. The tasks can be executed locally or offloaded to the edge server for execution; each user device calculates and specifies the offloading decision in each time slot. Multiple user equipment are represented as: The task for each user device is represented as follows: i is one of the user devices, n i This indicates the number of tasks assigned to the application on user device i. The computational unloading decisions for each task of the application are represented as follows: a={a1,a2,…,a i ,…,a n },i∈{1,2,…,n} Among them, a i,j =0 indicates that the task is computed locally. This indicates that the task unloads data to the MEC server for computation. The construction of a local computing model Local computation latency of the task Defined as: Similarly, the energy consumption generated by the equipment for: Where, ζ i u represents the energy consumption of user device i per CPU cycle. i,j f represents the computational complexity of the task. i l For local computing capabilities of user devices; The construction of the communication model, If the computation task is offloaded, its uplink data transfer rate is: Where W represents the system's wireless channel bandwidth, p i G represents the transmission power of user equipment i. i I represents the access channel gain. i This indicates interference during transmission. Indicates Gaussian white noise; The transmission delay of the task is expressed as: Transmission energy consumption during task i's computational unloading for: in, This represents the transmission power of user equipment i; The construction of the edge computing model If the computation task is offloaded, its computation latency on the edge server is: Among them, f o Assuming the edge server has computing power, it is assumed that the edge server distributes computing power equally to all unloaded tasks, and all tasks unloaded to the server receive the same computing resources. The construction optimization objective The latency of the user task is: For tasks computed locally (a) i,j For (=0), the computation latency is equal to the local computation latency; For computational tasks that perform computational unloading (a i,j =1) In terms of the whole uninstallation process, it consists of three steps: uploading task-related data, executing the task on the edge server, and returning the calculation results; The energy consumption generated by user tasks is: If the task is computed locally (a i,j =0), the task's energy consumption is equal to the local computation's energy consumption. If the task performs computational unloading (a i,j =0), the task's energy consumption only includes the energy consumption of data transmission during unloading; The total execution latency of the application is equal to the completion time of the last task in the task dependency relationship. The completion time of task j on user device i can be calculated recursively as follows: Where pred(j) represents the set of predecessor tasks of task j, meaning that task j can only be executed after all tasks in this set have been completed; specifically, This indicates there is no precursor mission. The set of tasks completed in the t-th time slot is represented as follows: The execution latency of the entire application in the t-th time slot can be expressed as: When all n i After all tasks are completed, the total execution time of the application is: To comprehensively consider the latency and energy consumption of task offloading decisions, the optimization objective is set as minimizing the weighted sum of the latency and energy consumption of all applications, expressed as:

2. The mobile edge computing multi-user computing offloading method according to claim 1, characterized in that, The establishment of the MDP model In the MDP model, the state is defined by the offloading decision for each task and the scheduling order of tasks. The state of user equipment i at time slot t is represented as: In the MDP model, an action is defined as a change in adjacent states, i.e., a change in the unloading decision of a single task. The action at time slot t is represented as: Where i represents the user device task that needs to change the decision, and a i,j ,b i New decisions and task scheduling order for user task j; In the MDP model, the reward function is set as the objective function value of the previous decision minus the objective function value of the new decision, expressed as: Among them, O(S) t ), O(S t+1 ) represents the objective function value corresponding to the state at time slot t, t+1, which is determined by the unloading decisions of all tasks and the task scheduling order.

3. The mobile edge computing multi-user computing offloading method according to claim 2, characterized in that, The Actor-Critic-based computational unloading decision algorithm, The Actor-Critic multi-agent reinforcement learning algorithm is as follows: 1) Randomly initialize the parameters θ of the online network for n intelligent agents (Actors). A With target network parameters Critic online network parameter θ C With network parameters: 2) Initialize the experience replay area Its capacity is defined as Used to store the traversed states, actions, and rewards; 3) Initialize the initial state S of the first iteration 1 ; 4) Repeat steps 4.1) to 4.6): 4.1) For each agent, select a deterministic action from the Actor online network output with a probability of 1-ε: The remaining ε probability randomly selects any action. 4.2) Execute action A t Obtain the new state S t+1 Calculate the reward R t , The data from this step (S) t A t ,R t ,S t+1 Save to experience replay area In the middle, proceed to the next step. M batches of interactive data were randomly sampled from the experience replay area, with each batch consisting of (S) k A k ,R k ,S′ k The state value of each agent is input into the target neural network corresponding to that agent to obtain the action. The obtained actions and states are input into the Critic target network of each agent to obtain the evaluation value output by the target network. This represents the evaluation value output by the target network for the i-th agent; the state S... k A k Input the corresponding online Critic network for each agent and obtain its output evaluation value. This represents the evaluation value output by the online network for the i-th agent; 4.3) Let the target evaluation value obtained by the k-th experience of the i-th agent be: Where γ is a parameter set by the user; 4.4) The loss function of the Critic target network is: The objective is to minimize the loss function by updating the parameters of the Critic online network using gradient descent. 4.5) Calculate the policy gradient of the Actor online network: Updating the parameters of the online Actor network based on gradient ascent 4.6) Update the Actor and Critic target networks of the agent, where τ is a user-defined parameter:

4. A mobile edge computing multi-user computing offloading device, characterized in that... include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to execute the method according to any one of claims 1 to 3.

5. A computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 3.

Images