Multi-user multi-edge server task unloading and computing resource allocation method

Abstract

The invention provides a multi-user multi-edge server task unloading and computing resource allocation method. Members in the system adopted by the invention comprise user equipment, an access point and an edge server. Each user equipment deploys an intelligent agent, the intelligent agent collects information observed by the current user equipment, performs task unloading and access point selection based on the observed information, and formulates an unloading action scheme through an Actor network; after all the intelligent agents formulate unloading action schemes, uniformly carrying out computing resource allocation on all the tasks unloaded to the edge server by a computing resource scheduling center, and adjusting unloading positions of part of the tasks as required; the intelligent agent takes the observed information, the unloading action, the reward and the information observed in the next time slot as a group of experiences and stores the experiences into a global playback cache area; and periodically updating network parameters by adopting a centralized training strategy. According to the invention, the execution urgency of the task can be minimized, and the utilization efficiency of computing resources can be maximized.

Description

Technical Field

[0001] This invention relates to the field of edge computing and resource allocation technology, specifically a method for adapting to dynamic changes in multi-user, multi-edge server task offloading and computing resource allocation. Background Technology

[0002] Existing edge computing resource allocation methods are mostly focused on static or single-user scenarios, typically performing resource scheduling and task execution in a fixed environment. These methods suffer from the following shortcomings:

[0003] 1. Incomplete Task Priority Assessment: Existing technologies primarily rely on basic task execution attributes, such as task size and resource requirements, when assessing task priority, neglecting task deadlines, dependencies between tasks, and the urgency of task execution. This lack of multi-dimensional task prioritization makes task scheduling and resource allocation inefficient.

[0004] 2. Decision delay and inflexibility: Existing methods typically assume that task information is fixed when the application starts, and the decision-making process does not consider real-time updates and flexible adjustments. This results in delays in task unloading and resource allocation, and makes it difficult to cope with real-time changes in network and computing environments.

[0005] 3. Low utilization of computing resources: Existing resource allocation schemes often lack reasonable utilization of task dependencies and often fail to adjust resource allocation according to the urgency of tasks, resulting in waste of computing resources and low task execution efficiency. Summary of the Invention

[0006] This invention introduces a dynamic task offloading and resource allocation method based on multi-agent deep reinforcement learning. Combining this with a greedy algorithm, it proposes a novel solution capable of responding to environmental changes in real time and flexibly adjusting task priorities and computing resource allocation. This invention aims to improve resource utilization in edge computing environments by optimizing task offloading decisions and resource allocation, and to ensure that tasks can be efficiently scheduled according to their execution urgency and dependencies.

[0007] This invention is implemented as follows:

[0008] A method for task offloading and computing resource allocation on a multi-user, multi-edge server includes the following steps:

[0009] (1) Each user equipment deploys an intelligent agent, which collects information observed by the current user equipment;

[0010] The information observed by the user equipment includes information on the ready task set, environmental information, and recent dynamics. The information on the ready task set includes data size and computing load. The environmental information includes local queue information, transmission queue information of each access point around the user equipment, and computing resource idle rate of each edge server. Recent dynamics refer to the offloading action taken by the user equipment in the previous time slot.

[0011] (2) The agent performs task unloading and access point selection based on the observed information, and formulates unloading action plan through the Actor network;

[0012] (3) After all agents have formulated the unloading action plan, the computing resource scheduling center will uniformly allocate computing resources to all tasks unloaded to the edge server, and adjust the unloading position of some tasks as needed.

[0013] (4) The agent stores the observed information, unloading action, reward and the information observed in the next time slot as a set of experience in the global replay buffer; and updates the network parameters periodically using a centralized training strategy.

[0014] Preferably, in step (3), the computing resource scheduling center uses a greedy algorithm to uniformly allocate computing resources to all tasks offloaded to the edge server, specifically:

[0015] S3-1. Based on the priority of the tasks, traverse the tasks from the ready task set of all user devices. For the currently traversed task, determine whether it can be executed locally within a time slot. If it can, change its unloading location to local and update the local queue information. Otherwise, put it into the edge task queue. If the edge task queue is empty after all tasks have been traversed, the resource allocation calculation ends. Otherwise, execute step S3-2.

[0016] S3-2. Calculate the earliest time slot (ESlot) for each task to be completed based on task dependencies. i,j With the latest time slot LSlot i,j It also determines whether the available computing resources of the edge server are sufficient to enable all tasks in the edge task queue to complete in the earliest time slot (ESlot). i,j If the task is completed, then allocate corresponding computing resources to each task, and the computing resource allocation ends; otherwise, proceed to step S3-3.

[0017] S3-3. Traverse the tasks according to their priority. For each traversed task, determine whether the available computing resources of the edge server can enable the task to be scheduled in the latest time slot (LSlot). i,jIf the task is completed within the specified time slot, then allocate computing resources to it and execute steps S3-4; otherwise, determine whether the time slot in which the task was completed locally is earlier than the latest time slot LSlot. i,j If so, change the task's unload location to local and update the local queue information; otherwise, determine that the task and its application are overdue.

[0018] S3-4. Iterate through each task in the ready task set, and take the union of all its direct successor sets as the set of direct successor tasks for that task, denoted as . ; any Its direct predecessor set is denoted as For any set of direct predecessors, calculate the earliest end gap of the set. Gap with the latest end ; , ;

[0019] S3-5. Traverse the direct predecessor sets counted in step S4. If the end slot of all direct predecessor sets is their earliest end slot, then the resource allocation calculation ends; otherwise, the set of all direct predecessor sets that has the greatest reduction in execution urgency by completing execution one slot earlier is called the optimal task set. And determine whether the available computing resources of the edge server are greater than or equal to the optimal task set. If additional computing resources are required to complete the task one time slot ahead of schedule, then allocate the corresponding computing resources directly and proceed to step S3-8; otherwise, proceed to step S3-6.

[0020] S3-6. Among all direct predecessor sets, find the set of tasks whose execution urgency is reduced by completing execution one time slot earlier. Check the set If the execution location of all tasks can be changed to the local location, then change the unloading location to the local location, update the local queue information, and release computing resources. Then execute steps S3-7. If not, find the task set with the second smallest reduced execution urgency that was completed one time slot earlier, and check if the execution location of all tasks in the set can be changed to the local location. Continue this process until all task sets have been checked, then exit the computing resource allocation process.

[0021] S3-7. Among all the task sets that request additional computing resources that finish one time slot early but whose additional computing resources are no greater than the available computing resources of the edge server, find the task set whose execution urgency is reduced the most due to finishing one time slot early, allocate the corresponding computing resources to it, and execute step S8; otherwise, continue to execute step S6.

[0022] S3-8, Assume DP i,q For the task set selected in step S5 or S7, allocate additional computing resources to all tasks whose latest execution end time slot is equal to the latest execution end time slot of the task set, so that these tasks all complete one time slot earlier, thus improving DP. i,q The process is completed one time slot ahead of schedule, and then step S5 is executed.

[0023] Preferably, in step S3-2, the earliest time slot ESlot when each task is completed is calculated. i,j With the latest time slot LSlot i,j The specific calculation formula is as follows:

[0024]

[0025]

[0026]

[0027]

[0028] in, For Task i,j In application y i The highest level in the task set dependency graph, DS i,j (t) represents the task. i,j The set of directly related successor tasks, LT i,j For Task i,j The latest time that can be terminated. for The actual number of tasks in the process, For application y i The task set, LT is the duration of a time slot. i For application y i Maximum tolerable execution delay.

[0029] Preferably, in steps S3-5, S3-6, and S3-7, the reduction in execution urgency by completing the process one time slot earlier specifically means:

[0030] For any set of tasks The additional computing resources required to complete execution one time slot ahead of schedule With reduced execution urgency They are as follows:

[0031]

[0032]

[0033]

[0034]

[0035] in, and They are respectively All tasks in the middle end at any possible time slot When the corresponding task is completed Priorities and required computing resources For the task The computational load, For the task Estimated computational load; TS i,q (t) represents the task. i,q The set of all subsequent tasks; t is the current time slot.

[0036] Preferably, in step (4), a centralized training strategy is used to periodically update the network parameters, and the optimized model of the network is as follows:

[0037]

[0038] Among them, U i For application y i The execution urgency at the end of execution, where T and M represent the total number of time slots allowed by the system and the total number of user devices, respectively, and I, S represents the set of user devices, the set of access points associated with user devices, and the set of edge servers, respectively. i,j For Task i,j The execution location, For application y i The task set, LT i For application y i Maximum tolerable execution delay.

[0039] The technical solution of this invention can effectively solve the problems existing in the prior art, as follows:

[0040] 1. This invention proposes a dynamic task priority evaluation method based on multi-agent deep reinforcement learning by comprehensively considering multi-dimensional information of the task, so as to ensure that the task can be reasonably scheduled according to its actual execution urgency.

[0041] 2. By employing a multi-agent deep reinforcement learning (MADRL) model, this invention can update task offloading strategies in real time and dynamically adjust task execution priorities and computing resource allocation schemes to improve the system's response speed and adaptability.

[0042] 3. This invention proposes a task offloading and resource allocation strategy based on a combination of multi-agent deep reinforcement learning and greedy algorithm, aiming to minimize the execution urgency of tasks and maximize the utilization efficiency of computing resources. Attached Figure Description

[0043] Figure 1 This is a diagram of the architecture for a mobile edge computing scenario.

[0044] Figure 2 This is a diagram illustrating the task unloading process.

[0045] Figure 3 This is a diagram of the DAG (Directed Acyclic Graph) showing the dependencies between tasks.

[0046] Figure 4 This is a framework diagram of the task unloading and computing resource allocation scheme.

[0047] Figure 5 This invention compares the average reward change when the computing resources of the user device change, as described in this invention and existing technologies.

[0048] Figure 6 This invention compares the changes in average computing resource idle rate when the computing resources of user equipment change, as described in this invention and existing technologies.

[0049] Figure 7 This describes the change in average reward when the number of user devices changes, as described in this invention and in the prior art.

[0050] Figure 8 This invention compares the changes in average computing resource idle rate when the number of user devices changes, as described in this invention and existing technologies.

[0051] Figure 9 This invention and existing technologies describe the average reward change when the computing resources of the edge server change.

[0052] Figure 10 This invention and existing technologies describe the changes in average computing resource idle rate when computing resources on edge servers change.

[0053] Figure 11 This describes the average reward variation of the present invention and existing technologies under load scenario I.

[0054] Figure 12 This describes the average reward variation of the present invention and existing technologies under load scenario II.

[0055] Figure 13 This describes the average reward variation of the present invention and existing technologies under load scenario III. Detailed Implementation

[0056] To address the problems in existing technologies, this invention provides an edge computing resource scheduling method based on task priority, task dependencies, and dynamic resource allocation. The specific steps are as follows:

[0057] 1. Application and task information design:

[0058] Analyze application and task information, including but not limited to: execution latency, task dependency construction, task size, and task execution priority.

[0059] 2. Obtain ready task information:

[0060] Obtain detailed information on all ready tasks in the system to facilitate subsequent task unloading and resource allocation.

[0061] 3. Constructing a system model:

[0062] The system model refers to the organizational structure of the system's members. User devices, access points, and edge servers together form a mobile edge computing system with task dependencies, consisting of multiple members and multiple edge servers.

[0063] For information on system members and information transmission, please refer to [link / reference]. Figure 1 See the task unloading process. Figure 2 .

[0064] Figure 1 In the Internet of Things (IoT) device, the pre-generation, formal construction, and result reception of tasks are handled. A User Equipment (UE) capable of communicating with neighboring IoT devices and Access Points (APs) publishes an application. This application consists of tasks generated by multiple cooperating IoT devices. Information regarding the collaboration between the UE and its associated devices is transmitted bidirectionally via a data stream. A group of Access Points (APs) is deployed around the UE. The APs are responsible for relaying task data, including receiving and forwarding task data and distributing the results returned by the edge server after task computation. Communication with the UE is transmitted via a communication data stream. The APs forward task data to the edge server and receive the results from the edge server's task computation via a forward data stream. The edge server sends information about tasks to be computed to the computing resource scheduling center and receives computing resource allocation decisions. Data from this collaboration is transmitted via a computing resource usage and allocation decision stream.

[0065] Figure 2 This is the unloading process for the j-th task of user device i, x i,jThis defines the execution location of the task. For tasks executed at the local user equipment, they are scheduled into the local computing queue and processed according to a first-come, first-served principle. For task sets executed at edge servers, an AP is first selected from the AP cluster associated with the UE, and the task set information is uploaded via the wireless link. Subsequently, this AP also forwards the task set information in its queue to the target edge server according to a first-come, first-served principle. Each edge server sends the received task set information along with its own computing resource usage to the computing resource scheduling center. The computing resource scheduling center allocates appropriate computing resources to the tasks offloaded to the edge server and modifies the task offloading actions accordingly.

[0066] Figure 3 This diagram shows a one-to-one correspondence between the task numbers of all applications and the node numbers in the graph, with directed edges representing task dependencies. Tasks numbered 0 and... Vertices with +1 are all virtual tasks, responsible only for starting and stopping the application. for The actual number of tasks in the process, For application y i A set of tasks.

[0067] Figure 4 China i (t), a i (t) and r i (t) represents the information observed by agent i at time slot t, the discrete action value of task unloading, and the task execution reward, respectively. During the execution phase, within each time slot, the user equipment first observes the system dynamics. The information to be observed is divided into three parts: information on the ready task set, environmental information, and recent dynamics. The ready task set information includes data size and computational load; environmental information consists of local queue information, transmission queue information of each AP, and the idle rate of computational resources of each edge server; and recent dynamics refers to the unloading action taken by the user equipment in the previous time slot. Subsequently, the agent corresponding to the user equipment will use the observed information as input to the local actor network to generate an unloading location and AP selection decision, and then proceed according to... Figure 1 , Figure 2 The described execution process interacts with the MEC (Mobile Edge Computing) environment, computes ready tasks, and calculates the cost of task execution. Each agent stores observation information, actions, rewards, and observation information for the next time slot as a set of experiences in a global replay buffer. Once the required amount of experience is reached, joint training begins, with the Critic network of each agent learning using global information, thereby providing a more comprehensive evaluation for the Actor network.

[0068] 4. Delay calculation:

[0069] Before introducing the objective function, we need to define the urgency level after the application finishes execution. The urgency level is related to the execution latency of the application, so we need to analyze the latency model of the task in the system first.

[0070] The latency for user equipment to execute tasks and the latency for edge server to execute tasks are calculated based on the system model, including:

[0071] The latency for user equipment to execute tasks is calculated using the following formula:

[0072]

[0073]

[0074] Among them UE i For the i-th user equipment, UE at the beginning of time slot t i A queue of tasks waiting to be executed locally. for The computational load, For UE i Rated computing power C is the duration of a time slot. i,j For Task i,j The computational load, For Task i,j Latency in local computing mode.

[0075] The latency of the edge server executing tasks is calculated using the following formula:

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] Among them, User Equipment (UE) i Access points (APs) process tasks in the queue according to a first-come, first-served principle. For the same set of ready tasks... The tasks in the process are executed in order of priority. For UE i The associated AP cluster is numbered as Channel bandwidth, and User Equipment (UE) i to h i (using h) i To indicate the number as AP), h i Channel gain to edge server x, p i and User Equipment (UE) i Access point h i Transmission power during data transmission This is channel noise. For User Equipment (UE) i To the number h i The queue when the AP uploads data. The size of the input data for the ready task set, and User equipment Access point h i Access point h i The rate at which data is transmitted to edge server x. To be assigned to Task i,j computing power , and User Equipment (UE) i Upload information about the ready task set to h i delay, h i The latency of uploading the set of ready tasks to edge server x and the latency of edge server x executing the tasks. For Task i,j Total latency in edge computing mode.

[0084] 5. Application execution urgency definition:

[0085] The urgency of task completion is defined based on the task's execution latency model, task load information, and dependencies to reflect the urgency of task completion. This applies to each task. i,j The definition of execution urgency (priority) in time slot t:

[0086]

[0087] Among them LT i For application y iMaximum tolerable execution delay, tstart i For application y i The time slot in which the release occurs, TS i,j (t) represents the task. i,j The set of all successor tasks, For Task i,z The estimated computational load. The application estimates the computational load and data size of the task at the time of release, and then updates the computational load and data size of the task based on the execution results before actual execution.

[0088] When all subtasks have been completed, it means that the application to which it belongs has finished executing. i Execution priority, task The time of execution completion and the application y i The execution urgency at the end of execution is defined as follows:

[0089]

[0090]

[0091]

[0092] in For application y i The task set, for The actual number of tasks in the process, For virtual tasks At the end of its execution, the virtual task only marks the start and end of the application's execution; it has no actual computational load or scale. The virtual task begins execution when the first task in the application is about to execute. Execution complete; virtual task completed when all tasks have finished executing. Execution complete. The task dependencies are detailed below. Figure 3 Example.

[0093] 6. Objective function construction:

[0094] Construct an optimization objective function to minimize the execution urgency of all applications in the long run, ensuring efficient resource allocation. Based on the execution urgency function, construct the following problem optimization model to minimize the execution urgency of all applications in the long run:

[0095]

[0096] Where T and M represent the total number of time slots allowed by the system and the total number of user equipment, respectively, and I, S and S represent the set of user equipment and UE, respectively.i The associated set of APs and the set of edge servers, for The number of APs in the set, x i,j For Task i,j The execution location has x i,j =x i (t), if Task i,j In UE i Execution at the location, UE i The full computing power will be used to compute the local task. Otherwise, the task will be numbered x. i,j edge servers The computational resources are used for computation. Constraint C1 represents Task i,j It can only be unloaded to one execution location. Constraint C2 indicates that the UE... i The selectable AP range when uploading data. Constraint C3 indicates that the range of computing resources allocated to edge server x for any task is [0, ..., ... Constraint C4 states that the total computing resources acquired by all tasks offloaded to edge server x must not exceed the currently available resources of edge server x. Constraint C5 states that the completion time of any task must not be later than the maximum tolerable execution delay of its respective application, ensuring that the application execution does not exceed its deadline. It is important to note that in most cases, application execution lasts for multiple time slots. The current time slot t represents the task's determined unloading location, AP selection, and computational resource allocation scheme, and its impact will be reflected when calculating the application's execution urgency after several time slots.

[0097] 7. Offloading and allocation model based on multi-agent deep reinforcement learning:

[0098] A task offloading model based on Multi-Agent Deep Reinforcement Learning (MADRL) and a greedy algorithm is proposed. Each user device deploys one agent, which performs task offloading and computational resource selection through the following steps:

[0099] The agent collects information from the current time slot, including information on the set of ready tasks, environmental information, and recent dynamics.

[0100] Based on the collected information, the agent performs task offloading and AP selection, and formulates discrete action plans.

[0101] Once all agents in the system generate discrete actions through the Actor network, the computing resource scheduling center uniformly allocates computing resources to all tasks offloaded to edge servers, and adjusts the offloading locations of some tasks as needed to resolve the computing resource allocation problem. The algorithm used is described below as a greedy algorithm for computing resource allocation.

[0102] The agent stores observation information, actions, rewards, and observation information for the next time slot as a set of experiences in a global replay buffer. A centralized training strategy is used to periodically update the network parameters.

[0103] The following is further information about this model:

[0104] The proposed optimization problem involves both discrete and continuous variables, making it a mixed-integer linear programming problem. It is essentially an NP-hard problem. This invention proposes a scheme combining MADRL and greedy algorithms, PPO-PST. Problem P1 is modeled as a multi-agent Markov decision process (MAMDP), where each user device is considered as a deployed agent, and each agent only considers maximizing its own reward, treating the other members as part of the environment. The task offloading problem is a sequential decision-making problem, generally considered to possess Markov properties, and can be solved using Markov decision processes. In real-world scenarios, the state observed by the agent in the current time slot may change in the next time slot, making it impossible to make decisions for later time slots based on the currently observed state. Furthermore, agents need to exchange information with other members of the network to understand the state of the environment. When multiple agents exist, the Markov decision process can be modeled as a multi-agent partially observable Markov decision process (MPOMDP), with the specific state and observation information, agent actions, and reward functions under specific states and actions as follows.

[0105] (1) State / Observation Space: Based on the agent's need to formulate offloading actions and AP selection, and the range of information available to the user equipment, UE i Observations in any time slot t consist of three parts: information on the ready task set, environmental information, and recent dynamics. The information on the ready task set includes data size and computational load; the environmental information consists of local queue information and UE (User Equipment) data. i With H i The upload queue information for each AP and the idle rate of computing resources for each edge server are included. Recent dynamics refer to the offloading actions taken in the previous time slot. The definition of observation is as follows:

[0106]

[0107] in Given the data size and computational load of the ready task set, (t) represents the computing resource usage of each edge server at the beginning of time slot t. UE i The local task queue at the beginning of time slot t and the upload task queue of all associated APs. Let t be the average computing resource utilization of all edge servers at the beginning of time slot t. For UE i The unloading action in the previous time slot.

[0108] (2) Action space: After acquiring the observation information, agent i formulates the corresponding action a. i (t)=(x i (t), h i (t)), where x i (t) represents the execution position selected by agent i for the entire set of ready tasks, treating all currently ready tasks as a whole to formulate an unloading action, h i (t) represents UE i The AP selected when uploading data.

[0109] (3) Reward Function: The reward of agent i in time slot t is determined based on the execution status of tasks in the ready task set. The reward function is defined as follows:

[0110]

[0111] Where r i (o i (t), a i (t) represents the reward value of agent i after its action in time slot t and its interaction with the environment, abbreviated as r. i (t).

[0112]

[0113] Where ψ is the application y i The penalty coefficient for overdue payments is used to regulate the application y. i The impact of overdue payments on the reward calculation for agent i.

[0114] To enable each agent to make self-interest-maximizing actions based on observation information, each agent employs the Proximal Policy Optimization (PPO) algorithm to learn policies for task set offloading and AP selection. The PPO algorithm is a policy gradient algorithm based on the Actor-Critic mechanism. It consists of a policy network called the Actor and a value network called the Critic. The Actor network represents the agent's policy for selecting actions, while the Critic network evaluates the actions taken by the agent. Compared to traditional policy gradient algorithms, the PPO algorithm makes the training process more stable by limiting the policy update magnitude and reduces the risk of overfitting to some extent, resulting in a better generalization ability for the trained policy. In the PPO algorithm, the loss function of the Actor network is defined as follows:

[0115]

[0116]

[0117]

[0118]

[0119]

[0120] in A policy network used by an agent to generate actions. The ratio coefficient for selecting the probability of specific actions for the agent's old and new policy networks. and These are the parameters for the old and new policy networks. This is the dominance function used to calculate the observation-action pair, employing the generalized dominance estimation (GAE) method, where λ∈[0,1] is used to balance variance and bias. γ is a discount factor to control the weight of future rewards. For timing difference error, Let ι be the state-value function for time slot t, where ι is typically a small number, such as 0.1 or 0.2, representing the magnitude of the constraint policy update, and μ. t (θ i The range is restricted to [1-ι, 1+ι], which ensures the stability of the training phase.

[0121] The Critic network evaluates the agent's observations in the current time slot and facilitates parameter updates in the Actor network. Its goal is to minimize the difference between predicted and observed values, updating the network parameters through mean squared error regression. The definition is as follows:

[0122]

[0123] When there is enough sample information in the global replay buffer, the algorithm will perform centralized training and update the network parameters. During training, all agents share a centralized Critic network to receive global information, including the observations, actions, and environmental states of all agents. Output the predicted value of the current state. Subsequently, the target value is calculated by combining the global reward with discounted cumulative reward or generalized advantage estimation (GAE). GAE effectively smooths the value estimation and reduces variance. The centralized training process of Multi-Agent Proximal Optimization (MAPPO) collects the global rewards from each agent's local observations, action trajectories, and environmental feedback in a decentralized execution mode. A centralized value network, capable of acquiring global information, is used to evaluate state value and calculate action advantage. Then, based on the truncation mechanism of MAPPO, the decentralized policy network of all agents is centrally optimized, while the value network is updated to improve evaluation accuracy. The Actor network maximizes the loss function through gradient ascent, while using a truncation mechanism to limit the policy update magnitude, avoiding training instability due to excessive updates, and updates the Actor parameters of each agent separately. The Critic network updates the Critic's parameters through gradient descent, improving its accuracy in evaluating global state value and providing support for the next round of advantage calculation. After completing one round of policy and value network updates, the experience buffer is cleared or updated, and the next round of trajectory collection and training begins. Through multiple iterations of "trajectory collection - value and advantage calculation - network optimization", all agents learn coordinated and adaptable local strategies for decentralized execution.

[0124] 8. Computational resource allocation algorithm based on greedy thinking:

[0125] In an embodiment of the present invention, a greedy allocation algorithm is designed. Based on task dependencies, urgency, and the current computing resource occupancy of the edge servers, the computing resource scheduling center in the system allocates edge computing resources to tasks offloaded to the edge servers. To prevent unreasonable allocation, the system allows dynamic adjustments to the offloading decisions for some tasks during the scheduling process. The computing resource scheduling center specifically executes the following steps:

[0126] Step S1: Based on the priority of the ready tasks, traverse the ready task sets from all user devices. For the currently traversed task, calculate the local execution time of the task. If it can be executed and completed locally within one time slot, change its unloading location to local and update the local queue. Otherwise, enter the edge task queue. If the edge task queue is empty after all tasks have been traversed, the algorithm ends. Otherwise, proceed to step S2.

[0127] Step S2, record Given the available computing resources for edge server x in time slot t, for tasks in the edge task queue, based on task dependencies, calculate the set of its successor tasks and the level of the task in the DAG graph. Then, calculate the earliest and latest time slots in which each task is completed. Specifically:

[0128]

[0129]

[0130]

[0131]

[0132] in For Task i,j In application y i The highest level of the task set dependency graph it occupies is related to the time slot where it is executed last. i,j (t) represents the task. i,j The set of directly related successor tasks, For Task i,j Theoretically, the earliest time slot to be executed, i.e., the time slot in which the data is transmitted to the edge server, is LT. i,j For Task i,j The latest time that can be terminated. Each refers to a ready task set The latency from user equipment to the access point and the latency from the access point to the edge server. For TS i,j All tasks in (t) are configured to prevent the application from crashing. For tasks that are overdue, at least one time slot should be reserved for execution delay. i,j The earliest slot to be completed is ESlot. i,j The latest time slot to be executed is LSlot. i,j If LSlot i,j <ESlot i,j Then Task i,j The application it belongs to y i Overdue. Otherwise, record it as a Task. i,j The possible set of time slots for execution completion is MSlot i,j ={ESlot i,j ESlot i,j +1,..., LSlot i,j Task i,j In time slot η∈MSlot i,jThe computing resources required for the edge server x to complete the execution are

[0133]

[0134] Check the remaining computing power (i.e., as described above). Can it support the completion of all tasks in the edge task queue within the earliest time slot? If so, allocate corresponding computing resources to each task; otherwise, continue with step S3. This approach makes full use of idle computing resources, and the allocated computing resources will be released as soon as possible and become available again in the next time slot.

[0135] Step S3: For all tasks executed on edge servers, traverse them in descending order of priority. For any task... judge Is it less than The computational resources required at the end of the latest possible timeout. If not, then... Sufficient, therefore The total amount allocated is Computing resources. If Less than Then judge Does the local execution complete its assigned time slot earlier than If earlier, then change. The execution location is local, and the local queue information is updated; otherwise, a judgment is made. and its applications If the deadline is reached, all tasks in the application are discarded. Agent i calculates the corresponding reward value for the overdue task and relaunches a new application. If, after traversing all tasks, there are still tasks executed on edge servers, then proceed to step S4 to optimize the existing allocation scheme.

[0136] Step S4: This step will consider the dependencies between tasks, and the specific steps are as follows.

[0137] Traversal The set of tasks ready at the start of time slot t. For each task in the set, take the union of all its direct successor sets as the result. The direct successor set of tasks, denoted as Any Its direct predecessor set is In addition, all tasks with an out-degree of 0 are included in the direct predecessor set. In the following text, the direct predecessor set will be referred to as the task set. For all tasks belonging to the same task set, the time slots in which they are completed should be aligned. This is because the earlier completion of a task does not mean that its direct successor task can start building in the next time slot. It still needs to wait for other tasks in the task set to be completed later, which will result in a waste of some computing resources.

[0138] For any set of tasks It is necessary to calculate the earliest end gap of the set. Gap with the latest end Considering the constraints of the earliest and latest task completion slots, we define... , The set of possible time slots in which all tasks within this task set are completed. . All tasks in the middle end at any possible time slot When the corresponding task is completed priority With required computing resources Defined as follows

[0139]

[0140]

[0141] The check in step S3 ensures the feasibility of the allocation scheme, and step S5 continues after step S4.

[0142] Step S5: Iterate through the task set counted in step S4. For any task set... The additional computing resources required to complete execution one time slot ahead of schedule With reduced execution urgency Their respective definitions are as follows:

[0143]

[0144]

[0145] If the end slots of all task sets are their respective earliest end slots, then the program exits and the allocation ends. Otherwise, The optimal task set is the set of tasks whose execution urgency is reduced by completing them one time slot earlier than expected. Then, the judgment is... and The size relationship. If If sufficient computing resources are available, the corresponding computing resources will be allocated directly, and step S8 will be executed; otherwise, step S6 will be executed.

[0146] Step S6: Find the set of tasks in the entire task set that minimizes the reduction in execution urgency if completed one time slot earlier. Check the set If the execution location of all tasks can be changed to the local machine, then the unloading location is changed to the local machine, and the local queue information is updated to reduce execution latency. The computing resources released during the unloading location change process are used as a buffer. After supplementing, proceed to step S7. If the reduced set of tasks with the lowest execution urgency is... If all tasks in the set cannot be changed to local execution after being checked, then the next lowest priority task set is searched. The execution locations of all tasks in this set are checked to see if they can be changed to local execution. This process continues until a task set is found where the execution locations of all tasks can be changed to local execution, thus freeing up computing resources. Then, step S7 is executed. If all task sets have been checked and no task set exists that can change the execution locations of all tasks to local execution, then there are no computing resources available to be freed, and the task set cannot be completed one time slot earlier. In this case, the computing resource allocation process is exited, and the current allocation scheme is used as the final allocation scheme.

[0147] Step S7: The additional computing resources requested for all executions completed one time slot earlier shall not exceed [amount missing]. In a set of tasks, find the set of tasks whose execution urgency is reduced the most due to finishing one time slot earlier (DP). i,opt If such a set of tasks can be found, allocate the corresponding computing resources to it and execute step S8; otherwise, continue to execute step S6.

[0148] Step S8, assume DP i,q For the task set selected in step S5 or S7, allocate additional computing resources to all tasks whose latest execution end time slot is equal to the latest execution end time slot of the task set, so that these tasks all complete one time slot earlier, thus improving DP. i,q The process is completed one time slot ahead of schedule, and then step S5 is executed.

[0149] 9. Simulation experiment verification:

[0150] To simulate the diversity of applications, application y i The value ranges of the main parameters are as follows.

[0151] (1) Number of tasks (|J) i |): Application y i The actual number of tasks that need to be calculated is within the range of {5, 7, 9, 11, 13, 15, 17}.

[0152] (2) Upper limit of the outdegree of a vertex max The upper limit of the out-degree of a vertex corresponds to the application y. i The maximum number of direct successor tasks a task can have, where the outdegree of a vertex ranges from {0, 1, ..., Outdegree}. max}, y i The complexity of dependencies between tasks and Outdegree max The values ​​are positively correlated, Outdegree max The range of values ​​for is {1, 2, 3, 4, 5}.

[0153] (3) Task data scale: refers to UE i The data size of the input portion of the published task ranges from [10, 50], and the unit is kB. The larger the data size of the task, the more it affects the transmission latency of task information and also the computational load of the task.

[0154] (4) Computational density of the task: Without loss of generality, the computational load of the task is considered to be related to the data size of the task input part. The value range of computational density is [500, 1000], and the unit is Hz / bit, that is, the number of CPU cycles required to compute one bit of data.

[0155] (5) Tolerable execution latency of the application: refers to the latency of the released application y i The maximum tolerable delay until the execution is completed without overdue is in the range of {7, 8, ..., 15}, in seconds.

[0156] The unloading scene is set to a 1000×1000 m 2 A square area is defined, with all user devices and access points (APs) randomly distributed within it. Three edge servers are evenly deployed within this area. In simulation experiments, the number of user devices, the computing power of the edge servers, and the local computing power of the user devices are adjusted to observe the performance of the proposed scheme. The AP cluster size for each user device is 5, the power consumption for data upload by the user device is 0.1 W, and the channel bandwidth for each AP is 15 MHz. Penalty coefficient... The penalty for an application's delay is set to 10, meaning the penalty for an application's delay is 10 times its priority, to ensure that all applications are executed on time as much as possible. In the proposed scheme, the PPO algorithm deployed by each agent contains an Actor network with the same structure as the Critic network, consisting of two fully connected hidden layers, each with 64 neurons. Each episode has 400 time slots, a pruning parameter of 0.2, a discount factor of 0.99, a learning rate of 0.001 for both networks, and a batch size of 128.

[0157] The following five schemes were compared with the scheme PPO-PST proposed in this invention to evaluate the performance advantages of the scheme PPO-PST proposed in this invention.

[0158] (1) PPO-CO: The PPO algorithm is used to formulate task unloading actions and AP selection actions. The convex optimization toolkit is used to solve the convex optimization problem of computing resource allocation, so as to allocate computing resources to the target task by minimizing the computing latency of all tasks.

[0159] (2) PPO-RA: The PPO algorithm is used to formulate the offloading action for the task, and an AP is randomly selected from the AP cluster associated with the user equipment for communication. This scheme is used to prove the necessity of optimizing the AP selection in PPO-PST.

[0160] (3) DDQN-PST: The DDQN algorithm is used to complete the task unloading and AP selection action formulation, and the algorithm proposed in this invention is used to allocate the computing resources of the edge server. The DDQN algorithm uses two Q networks with the same structure but different parameters to decouple action selection and action value evaluation, and the estimation of Q value is more accurate than that of the DQN algorithm.

[0161] (4) DQN-PST: The DQN algorithm is used to formulate task offloading actions and AP selection actions, and the algorithm proposed in this invention is used to allocate computing resources for edge servers. The DQN algorithm is a classic DRL algorithm that has been widely used in MEC task offloading in recent years.

[0162] (5) Random: Randomly determine the task's unloading action and the AP for communication when uploading data, and be consistent with the computing resource allocation strategy in (1).

[0163] The simulation experiment and results analysis are as follows:

[0164] Figure 5 and Figure 6 This illustrates the impact of varying local computing capabilities of user equipment on the performance of different solutions. The local computing capabilities of all user equipment were set to 0.1 GHz, 0.2 GHz, 0.3 GHz, 0.4 GHz, and 0.5 GHz. Figure 5 As shown, the PPO-PST scheme of the present invention can maintain the highest average reward compared to other schemes when the computing power of the user equipment changes. Figure 6This indicates that PPO-PST consistently maintains the highest idle rate of edge server computing resources compared to other solutions when the local computing power of user devices changes. When user device computing power decreases, user devices rely more on edge computing services to execute tasks. Reasonable task offloading decisions, appropriate AP selection, and the availability of edge computing resources combine to lead to PPO-PST's superior performance in the average reward value metric. However, the difference in average idle rate of computing resources narrows because the limited total computing resources cannot support the rapid completion of tasks, and occupied computing resources are difficult to make available quickly. When user device computing power increases, the dependence of task execution on edge computing services decreases, more tasks are completed locally more quickly, and the consistency of task offloading actions is enhanced. Since tasks are executed in the same pattern locally, although the availability of computing resources is stronger, the difference in average reward value among the solutions still decreases. Therefore, the advantage of PPO-PST is more pronounced in scenarios where user device computing resources are limited.

[0165] Figure 7 and Figure 8 This illustrates the impact of different numbers of user devices in the system on the performance of six task offloading schemes. The performance of each scheme is shown in the scenarios with 5, 10, 15, 20, and 25 user devices. Figure 7 and 8 As shown. With the increase in user devices, such as... Figure 7 As shown, the PPO-PST solution of this invention consistently maintains the highest average reward compared to other solutions. Figure 8 As shown, the PPO-PST solution of this invention has a higher idle rate of edge server computing resources compared to other solutions, and the competition for edge computing resources by user devices also significantly affects the availability of edge service computing resources.

[0166] As the number of user devices increases, the trends in average idle rate and average reward value for each solution are inconsistent. This is because competition for edge computing resources intensifies among user devices, and limited edge computing resources struggle to guarantee task execution efficiency. User devices are more inclined to execute tasks locally, and task offloading strategies and AP selection have a more significant impact on average reward value than computing resource availability. Furthermore, as the number of user devices increases, this competition intensifies, leading to a greater decrease in the average reward value for each solution. Compared to other solutions, the PPO-PST solution prioritizes ensuring all applications meet latency constraints when allocating computing resources during intense edge server computing resource competition, effectively reducing application overdue penalties and thus achieving a more significant performance improvement.

[0167] Figure 9 and Figure 10The study investigated the impact of varying edge server computing power on the performance of six different scenarios. The computing power of the five edge servers in the scenario was set to 10 GHz, 15 GHz, 20 GHz, 25 GHz, and 30 GHz, respectively. Figure 9 This indicates that the PPO-PST scheme consistently offers the highest average reward compared to other schemes. As edge server computing resources increase, the difference in average rewards among the various schemes gradually decreases, indicating that the edge server's impact on application execution rewards diminishes. The five schemes other than Random tend to utilize edge computing services more for task execution. When edge computing resources decrease, the average reward value of PPO-PST significantly increases compared to other schemes. This suggests that when edge computing resources are scarce, appropriate offloading decisions and computing resource allocation strategies have a more significant impact on the urgency of application execution. Figure 10 The results show that, compared to other schemes, the PPO-PST scheme consistently achieves the highest idle rate of edge server computing resources. As the computing resources on edge servers increase, the average idle ratio of computing resources across all schemes increases significantly. This is primarily because ample edge computing resources lead to shorter task execution latency, allowing occupied computing resources to be released and made available more quickly.

[0168] The performance of each scheme under different load scenarios is studied below. This part of the simulation experiment considers the performance of all applications under different computing load scenarios with different average computing resource idle rates, to verify that it is necessary to use the average computing resource idle rate as an evaluation indicator of scheme performance. The computing load level in the scenario is determined by the application configuration information. The scenario information for the three computing load levels is shown in Table 1:

[0169] Table 1 Parameter settings for each task scenario

[0170]

[0171] In each of the three scenarios above, there were 10 user devices and 3 edge servers. The local computing power of the user devices was 0.3 GHz, and the computing power of the edge servers was 20 GHz. All applications published by the same user device had the same parameters. To avoid excessively extreme computational load levels in the simulated scenarios, applications generated by five user devices used the parameters of Scenario II, while applications generated by the other five user devices used the parameters of Scenario I, Scenario II, and Scenario III, respectively. The relevant experimental results are shown below. Figures 11-13 .

[0172] As shown in the figure, the PPO-PST scheme consistently exhibits the highest average reward value across the three different load scenarios. When the average idle rate is 10%, user devices tend to execute tasks locally. The Random scheme, due to its random determination of task offloading locations and communication APs, has a significantly lower average reward value than other schemes. This may be because when offloading to the edge server, the limited edge computing resources cannot meet the task execution demands, leading to application delays. The penalty for delays significantly reduces the average reward value. Furthermore, due to the varying scale of input data for all application tasks and the different transmission latency of tasks in different load scenarios, the AP selection action ultimately has an inconsistent impact on the average reward value under different load scenarios. The computing resource allocation scheme has a more significant impact on the application's execution efficiency. Figure 11 and Figure 12 The performance of the mid-range solution PPO-RA is generally higher than that of PPO-CO. However, in high-computational-load scenarios, such as... Figure 13 As shown, the average reward of the PPO-RA scheme is lower than that of the PPO-CO scheme when the average computing resource idle rate reaches 40%. This is because, on the one hand, the increased input size of the task makes the selection action of the AP more important, and on the other hand, the impact of the computing resource allocation scheme on performance decreases when the availability of computing resources is enhanced.

[0173] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for task offloading and computing resource allocation in a multi-user, multi-edge server, characterized in that, Includes the following steps: (1) Each user equipment deploys an intelligent agent, which collects information observed by the current user equipment; The information observed by the user equipment includes information on the ready task set, environmental information, and recent dynamics. The information on the ready task set includes data size and computing load. The environmental information includes local queue information, transmission queue information of each access point around the user equipment, and computing resource idle rate of each edge server. Recent dynamics refer to the offloading action taken by the user equipment in the previous time slot. (2) The agent performs task unloading and access point selection based on the observed information, and formulates unloading action plan through the Actor network; (3) After all agents have formulated the unloading action plan, the computing resource scheduling center will uniformly allocate computing resources to all tasks unloaded to the edge server, and adjust the unloading position of some tasks as needed. (4) The agent stores the observed information, unloading action, reward and the information observed in the next time slot as a set of experience in the global replay buffer; A centralized training strategy is adopted to update the network parameters regularly.

2. The method for multi-user, multi-edge server task offloading and computing resource allocation according to claim 1, characterized in that, In step (3), the computing resource scheduling center uses a greedy algorithm to allocate computing resources to all tasks offloaded to the edge server. Specifically: S3-1. Based on the priority of the tasks, traverse the tasks from the ready task set of all user devices. For the currently traversed task, determine whether it can be executed locally within a time slot. If it can, change its unloading location to local and update the local queue information. Otherwise, put it into the edge task queue. If the edge task queue is empty after all tasks have been traversed, the resource allocation calculation ends. Otherwise, execute step S3-2. S3-2. Calculate the earliest time slot (ESlot) for each task to be completed based on task dependencies. i,j With the latest time slot LSlot i,j It also determines whether the available computing resources of the edge server are sufficient to enable all tasks in the edge task queue to complete in the earliest time slot (ESlot). i,j Once the task is completed, if possible, allocate corresponding computing resources to each task, and the computing resource allocation is complete. Otherwise, proceed to step S3-3; S3-3. Traverse the tasks according to their priority. For each traversed task, determine whether the available computing resources of the edge server can enable the task to be scheduled in the latest time slot (LSlot). i,j If the task is completed within the specified time slot, then allocate computing resources to it and execute steps S3-4; otherwise, determine whether the time slot in which the task was completed locally is earlier than the latest time slot LSlot. i,j If so, change the task's unload location to local and update the local queue information; otherwise, determine that the task and its application are overdue. S3-4. Iterate through each task in the ready task set, and take the union of all its direct successor sets as the set of direct successor tasks for that task, denoted as . ; any Its direct predecessor set is denoted as For any set of direct predecessors, calculate the earliest end gap of the set. Gap with the latest end ; , ; S3-5. Traverse the direct predecessor sets counted in step S4. If the end slot of all direct predecessor sets is their earliest end slot, then the resource allocation calculation ends; otherwise, the set of all direct predecessor sets that has the greatest reduction in execution urgency by completing execution one slot earlier is called the optimal task set. And determine whether the available computing resources of the edge server are greater than or equal to the optimal task set. If the additional computing resources required to complete the task one time slot ahead of schedule are needed, then allocate the corresponding computing resources directly and proceed to step S3-8; otherwise, proceed to step S3-6. S3-6. Among all direct predecessor sets, find the set of tasks whose execution urgency is reduced by completing execution one time slot earlier. Check the set If the execution location of all tasks can be changed to the local location, then change the unloading location to the local location, update the local queue information, and release computing resources. Then execute steps S3-7. If not, find the task set with the second smallest reduced execution urgency that was completed one time slot earlier, and check if the execution location of all tasks in the set can be changed to the local location. Continue this process until all task sets have been checked, then exit the computing resource allocation process. S3-7. Among all the task sets that request additional computing resources that finish one time slot early but whose additional computing resources are no greater than the available computing resources of the edge server, find the task set whose execution urgency is reduced the most due to finishing one time slot early, allocate the corresponding computing resources to it, and execute step S8; otherwise, continue to execute step S6. S3-8, Assume DP i,q For the task set selected in step S5 or S7, allocate additional computing resources to all tasks whose latest execution end time slot is equal to the latest execution end time slot of the task set, so that these tasks all complete one time slot earlier, thus improving DP. i,q The process is completed one time slot ahead of schedule, and then step S5 is executed.

3. The method for multi-user, multi-edge server task offloading and computing resource allocation according to claim 2, characterized in that, In step S3-2, the earliest time slot ESlot when each task is completed is calculated. i,j With the latest time slot LSlot i,j The specific calculation formula is as follows: ， ， ， ， in, For Task i,j In application y i The highest level in the task set dependency graph, DS i,j (t) represents the task. i,j The set of directly related successor tasks, LT i,j For Task i,j The latest time that can be terminated. for The actual number of tasks in the process. For application y i The task set, LT is the duration of a time slot. i For application y i Maximum tolerable execution delay.

4. The method for multi-user, multi-edge server task offloading and computing resource allocation according to claim 3, characterized in that, In steps S3-5, S3-6, and S3-7, the reduction in execution urgency caused by completing the process one time slot earlier is specifically as follows: For any set of tasks The additional computing resources required to complete execution one time slot ahead of schedule With reduced execution urgency They are as follows: ， ， ， ， in, and They are respectively All tasks in the middle end at any possible time slot When the corresponding task is completed Priorities and required computing resources For the task The computational load, For the task Estimated computational load; TS i,q (t) represents the task. i,q The set of all subsequent tasks; t is the current time slot.

5. The method for multi-user, multi-edge server task offloading and computing resource allocation according to claim 1, characterized in that, In step (4), the network parameters are updated periodically using a centralized training strategy. The optimized network model is as follows: ， Among them, U i For application y i The execution urgency at the end of execution, where T and M represent the total number of time slots allowed by the system and the total number of user devices, respectively, and I, S represents the set of user devices, the set of access points associated with user devices, and the set of edge servers, respectively. i,j For Task i,j The execution location, For application y i The task set, LT i For application y i Maximum tolerable execution delay.

Images