Cloud edge collaborative task scheduling method based on optimized A2C algorithm

Abstract

The invention discloses a cloud-edge collaborative task scheduling method based on an optimized A2C algorithm, and belongs to the technical field of cloud computing and edge computing fusion scheduling. Aiming at the technical problems of low resource utilization rate, high system energy consumption and high delay of the existing cloud edge collaborative task scheduling method, the invention provides a method for constructing a task-host heterogeneous graph, and explicitly modeling a bidirectional constraint and competition relationship between task requirements and host resources; designing a heterogeneous graph pointer network, and respectively extracting the competitive strength of a task to a host and the supply and demand matching quality of the task and the host through a two-stage graph attention mechanism to form a joint environment state feature; a task granularity dominant function is adopted, the dominant value of each task action is calculated in a differentiated mode according to the task scheduling execution result and the global time sequence differential error, and strategy network gradient updating is carried out in combination with the pointer probability. According to the method, accurate perception and fine-grained optimization of the dynamic cloud edge environment are realized, and system energy consumption and task response delay are remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a cloud-edge collaborative task scheduling method based on an optimized A2C algorithm. Background Technology

[0002] Currently, technologies such as 5G, IoT, cloud computing, and edge computing are developing rapidly. Running applications on IoT terminals and processing and analyzing data collected by terminal devices such as sensors in real time through cloud-edge collaboration technology can provide intelligent decision support for end users, further enabling smart IoT.

[0003] In cloud-edge collaborative environments, the types, quantities, and resource requirements of computing tasks, as well as the network bandwidth between data sources and computing nodes, are constantly changing dynamically. At the same time, edge hosts and cloud processing centers are geographically dispersed and have heterogeneous resources and devices, resulting in significant differences in response time and energy consumption when different hosts process different types of computing tasks. How to use intelligent scheduling strategies to dynamically allocate computing tasks generated by IoT terminals to the optimal edge nodes or cloud nodes to achieve efficient computing task scheduling has become a research hotspot in the field of cloud-edge collaboration.

[0004] Making task scheduling decisions in a complex and dynamically changing cloud-edge collaborative environment is an NP-hard problem, and it is impossible to obtain the optimal solution in polynomial time. Therefore, most existing works use mathematical programming methods, heuristic algorithms, and deep reinforcement learning algorithms to establish task scheduling models for solving the problem.

[0005] (1) Cloud-edge collaborative task scheduling method based on mathematical programming

[0006] These methods, based on mathematical principles, can accurately express the objective function and constraints of task scheduling by constructing mathematical models, theoretically yielding a globally optimal solution. For example, in a cloud-edge collaborative environment, the task scheduling and resource allocation problem can be formulated as Mixed-Integer Non-Linear Programming (MINLP). Using piecewise linear approximation and linear relaxation, MINLP is transformed into Mixed-Integer Linear Programming (MILP), and a near-optimal solution is obtained through a gap-adjusted branch-and-bound algorithm, achieving efficient scheduling in large-scale task scenarios. However, due to the dynamic changes in the cloud-edge collaborative environment, such research often requires numerous assumptions and constraints, making model construction complex and idealized, and model solving relatively difficult, thus struggling to cope with the dynamic fluctuations of the scenario.

[0007] (2) Cloud-edge collaborative task scheduling method based on heuristic algorithm These methods utilize predefined, empirical, and intuitive rule sets to make scheduling decisions, or simulate natural and physical processes, iteratively improving to gradually approach a better solution. For example, heuristic rules use the task requirements and host status of the cloud-edge collaborative environment to quickly determine and make task scheduling decisions according to rules; genetic algorithms encode task scheduling decisions as chromosomes, evolving individuals in the population through selection, crossover, and mutation to obtain near-optimal solutions; ant colony algorithms analogize the task scheduling process to ants choosing paths, finding the optimal path through pheromone positive feedback; simulated annealing sets a high initial temperature for task scheduling decisions, effectively exploring multiple candidate solutions in a vast solution space by controlling the cooling rate, eventually approaching the global optimum. However, since the essence of heuristic algorithms is a trade-off between speed and accuracy, the solutions provided by heuristic algorithms in fast decision-making are often locally optimal or near-optimal solutions optimized for a single objective. They typically only consider the current state, and decisions between multiple tasks are relatively isolated, lacking a global perspective and long-term optimization. If we move beyond local optima and consider global optimization, the decision-making time and computational overhead of heuristic algorithms will increase significantly. Therefore, in cloud-edge collaborative task scheduling scenarios, heuristic algorithms still have room for further optimization.

[0008] (3) Cloud-edge collaborative task scheduling method based on deep reinforcement learning algorithm These methods combine the feature extraction capabilities of deep learning with the dynamic decision-making capabilities of reinforcement learning. Through the mapping from state space to action space and the interaction between the agent and the environment, they achieve end-to-end policy optimization, enabling better adaptation to the dynamically changing cloud-edge collaborative environment. For example, the Deep Q-Network (DQN) algorithm uses a neural network to approximate the Q-value function, indirectly mapping the environment state to task scheduling decisions using Q-values, and iteratively optimizing task scheduling decisions through Q-value iteration. The Double DQN (DDQN) algorithm alleviates the overestimation of Q-values ​​by separating the Q-value network and the Q-value evaluation network, further optimizing task scheduling decisions. The Advantage Actor-Critic (A2C) algorithm uses a policy network to directly map the environment state to task scheduling decisions. Furthermore, it uses an advantage function and a value network to measure the quality of task scheduling decisions relative to state values ​​and guides the agent's policy continuous optimization. However, most current research, when characterizing the environment state, typically transforms the physical quantities of resources required for task execution and the physical quantities of resources that the host can provide into feature vectors. These vectors are simply concatenated to form the feature matrix of the environment state, neglecting the competition among multiple tasks for the same host resources. This leads to an incomplete perception of the environment state by the agent, resulting in decisions to schedule multiple tasks to the same host and consequently, low resource utilization. Some studies have proposed using isomorphic graphs to extract task features, essentially capturing the dependencies between tasks, but this also fails to consider the competition among multiple tasks for host resources. Furthermore, although the A2C algorithm improves its evaluation capability for task scheduling decisions by introducing a dominance function and a value network, in multi-task scheduling scenarios, using only the global dominance function is insufficient to accurately evaluate the merits of different decisions. This results in individual failed task scheduling decisions sharing the same dominance function with other decisions, making it difficult for the agent to effectively optimize its strategy.

[0009] In summary, current cloud-edge collaborative task scheduling methods based on deep reinforcement learning algorithms still have the following problems: (1) The feature extraction of the environmental state is insufficient. When representing the environmental state, only simple feature vectors and feature concatenation are used, ignoring the competition of multiple tasks for the same host resources. It is difficult to effectively capture the complex matching relationship between tasks and hosts, resulting in low resource utilization.

[0010] (2) The evaluation of task scheduling decisions is inaccurate. All task scheduling decisions share the global advantage function, making it difficult to accurately evaluate the merits of different decisions, which leads to inaccurate dynamic adjustments of the agent to the strategy. Summary of the Invention

[0011] This invention addresses the problems of low resource utilization, high system energy consumption, and high latency caused by existing cloud-edge collaborative task scheduling methods failing to adequately consider the dynamic fluctuations of the environment and the competition among multiple tasks for the same host resources. It proposes a cloud-edge collaborative task scheduling method based on an optimized A2C (Advantage Actor-Critic) algorithm. This invention constructs a task-host heterogeneous graph to explicitly model the competition among multiple tasks for host resources, and designs a Heterogeneous Graph Pointer Network (HGPN) as the policy network accordingly. It then combines a graph attention mechanism (GAT) to extract the competition and matching relationship features between tasks and hosts. Simultaneously, the advantage function of the A2C algorithm is finely reconstructed, allowing each task scheduling decision to independently enjoy a dynamic advantage value adapted to its execution result, thereby achieving accurate gradient updates to the policy network. This invention can effectively reduce system energy consumption and latency, and improve resource utilization.

[0012] The technical means employed in this invention are as follows: A cloud-edge collaborative task scheduling method based on an optimized A2C algorithm includes the following steps: A set of tasks to be scheduled is received within discrete time intervals. The optimization objective is to minimize the overall energy consumption of the cloud-edge host and the average response time of the tasks completed within the time interval. A cloud-edge collaborative task scheduling model is constructed. A task-host heterogeneous graph is constructed to represent the mapping relationship and resource constraints between task resource requirements and host resource supply. The task-host heterogeneous graph includes a set of task nodes, a set of host nodes, and a set of bidirectional edges with directional semantics. A forward edge indicates that a task can be assigned to a host node, and a reverse edge indicates that a host node can carry a task. Based on the real-time acquired task resource requirement information and host resource supply information, task node features and host node features are generated and embedded into task nodes and host nodes, respectively. A cloud-edge collaborative task scheduling model is solved based on an optimized A2C network. The optimized A2C network includes a state, a task-host heterogeneous graph, a heterogeneous graph pointer network, an environment, a value network, and a loss layer. The state part acquires the real-time environment state at the start of the current scheduling interval; the task-host heterogeneous graph part constructs the task-host heterogeneous graph based on the environment state; the heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and further dynamically generates task scheduling decisions at the decision layer; the environment part provides environmental feedback information and a new environment state after the decision is executed; at the next moment after the task scheduling decision is executed, the environment feedback information and environment state are acquired, and the loss of the policy network and value network is calculated based on the optimized advantage function to update the learnable parameters of the optimized A2C network. When both the value network loss and the policy network loss converge, the optimal task scheduling decision that minimizes system energy consumption and latency is obtained.

[0013] Furthermore, the construction of the task-host heterogeneous graph includes: For any task node and host node, a bidirectional edge is established between them if and only if the following four resource constraints are simultaneously satisfied: The task's computational requirements are less than the available computing resources on the host node; The memory requirements of the task node are less than the available memory resources of the host node; The bandwidth requirement of the task node is less than the available bandwidth resources of the host node; The node's current load rate is below the preset threshold.

[0014] Furthermore, the heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, including: introducing an attention mechanism into the graph convolutional layers of the heterogeneous graph pointer network, extracting environmental state features from the task-host heterogeneous graph input to the heterogeneous graph pointer network, first extracting the competition relationship features between tasks and host resources based on the edges from tasks to hosts combined with node information, and then extracting the matching relationship features between task requirements and host resources based on the edges from hosts to tasks combined with node information and competition relationship features.

[0015] Furthermore, the heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and also includes: concatenating task node features with matching relationship features to obtain the environmental state features of the time interval.

[0016] Furthermore, the heterogeneous graph pointer network dynamically generates task scheduling decisions at the decision layer, including: processing the extracted environmental state features using a multilayer perceptron to generate a probability distribution of task scheduling to each host. use The strategy dynamically generates task scheduling decisions based on the task scheduling probability distribution.

[0017] Furthermore, the optimized advantage function Calculate according to the following segmented logic: when and hour, ; when and hour, ; when hour, ; in For timing difference error, Time interval The optimization target value, This is a subset of task scheduling decisions that failed to execute successfully due to resource competition or insufficient resources. For the range of values ​​within Scaling factor within.

[0018] Furthermore, the loss function of the policy network is:

[0019] in, For the task The pointer records the task. The probability of being scheduled to the target host. To take the logarithm of the probability; The loss function of the value network is:

[0020] in, This refers to timing difference error.

[0021] Furthermore, the updates to the learnable parameters of the optimized A2C network include: The gradients of the value network and policy network are obtained by taking the partial derivatives of all parameters through backpropagation, and then the parameters are updated by gradient descent.

[0022] Compared with the prior art, the present invention has the following advantages: This invention first constructs a task-host heterogeneous graph based on real-time acquired task resource demand information and host resource supply information, explicitly modeling the competitive relationship between tasks and host resources. Then, the A2C algorithm is optimized in two ways: First, a Heterogeneous Graph Pointer Network (HGPN) is designed to optimize the policy network of the A2C algorithm. By introducing an attention mechanism into the graph convolutional layer of HGPN to extract environmental state features, the competitive and matching relationships between multiple tasks and host resources are fully captured. Furthermore, task scheduling decisions are dynamically generated at the decision layer of HGPN using these environmental state features. Second, the advantage function of the A2C algorithm is optimized. The advantage function is independently calculated for each task scheduling decision based on the task execution status, making the dynamic adjustment of the strategy by the A2C agent more accurate. Finally, through continuous interaction between the A2C agent and the cloud-edge environment, the optimal task scheduling decision is learned, simultaneously optimizing system energy consumption and latency. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a cloud-edge collaborative task scheduling method based on an optimized A2C algorithm in an embodiment of the present invention.

[0025] Figure 2 This is a task-host heterogeneous diagram in an embodiment of the present invention.

[0026] Figure 3 This is a diagram of the A2C network structure optimized based on HGPN in an embodiment of the present invention.

[0027] Figure 4 This is a comparison chart of system energy consumption for different methods in the embodiments of the present invention.

[0028] Figure 5 This is a comparison chart of the average scheduling time of different methods in the embodiments of the present invention.

[0029] Figure 6 This is a comparison chart of the average response times of different methods in the embodiments of the present invention.

[0030] Figure 7 This is a comparison chart of service level agreement (SLA) violation rates for different methods in embodiments of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] like Figure 1 As shown, this invention provides a cloud-edge collaborative task scheduling method based on an optimized A2C algorithm, applied to a cloud-edge collaborative computing system. This system includes a scheduler, a cloud-edge host, terminal applications that generate tasks, and a monitor for real-time data monitoring. The method includes the following steps: S1. Receive a set of tasks to be scheduled within discrete time intervals. Optimize the overall energy consumption of the cloud-edge host and the average response time of the tasks completed within a time interval by constructing a cloud-edge collaborative task scheduling model.

[0034] This invention first establishes a cloud-edge collaborative task scheduling model. In a cloud-edge collaborative scenario, the scheduler receives computing tasks generated by IoT terminals and allocates these tasks to cloud-edge hosts for execution according to a scheduling strategy. Since computing tasks in real-world scenarios typically arrive randomly and have different resource requirements, the scheduling process is divided into equal time intervals. It is assumed that the entire scheduling process includes... A time interval, Let the set of time intervals be . ,in For the first The scheduler selects from each time interval. Initially, at the start of each time interval, a scheduling decision is dynamically generated for the set of tasks to be scheduled at that time. Definition For the scheduler at time intervals The start time The set of tasks to be scheduled has been obtained. Time interval The first in One task to be scheduled , It is a set The number of tasks in the middle, Including newly arrived task sets and the set of tasks that failed to be scheduled due to insufficient resources or other reasons. As shown in formula (1).

[0035]

[0036] definition A collection of cloud-edge hosts. For the first One host, , It is a set The number of hosts in the middle, defined For the scheduler in The starting time is the set of tasks to be scheduled. The generated task scheduling decision set can be represented as a task-host pair, as shown in formulas (2) and (3).

[0037]

[0038] in For the task Scheduling decisions, For the task The target host. Task scheduling decisions for all time intervals are generated based on the scheduler's scheduling policy. The quality of the scheduling policy directly affects the overall performance of the cloud-edge collaborative computing system. To find the optimal scheduling policy, we consider minimizing the weighted sum of the overall energy consumption of the cloud-edge host and the average response time of the tasks executed within a time interval, thereby optimizing system energy consumption and latency throughout the scheduling process. To eliminate the difference in dimensions between system energy consumption and latency, we normalize both. Let... Time interval The normalized energy consumption of all cloud-edge hosts is calculated as shown in formula (4).

[0039]

[0040] in For host Regarding time The energy consumption function is calculated based on the host's CPU utilization and power consumption model. Time interval At the end of the day, For host Maximum potential power consumption. It's important to note that different host models have different power consumption models. The power consumption model corresponds to the host's power consumption at 0-100% CPU utilization. The power consumption function here... At any time tThe specific value is obtained by real-time monitoring by the monitor, and the host power consumption for the current time interval is obtained by integrating the values ​​over the time interval.

[0041] set up Time interval The normalized value of the average response time of all tasks completed within the period. The task response time is the time from task creation to completion of execution, including the time for task waiting, scheduling and execution. The calculation process is shown in formulas (5) and (6).

[0042]

[0043] in To be in the time interval The set of tasks to be executed. In order to be in The first execution completed One task, , It is a set The number of tasks in the middle, For the task The moment of completion of execution, For the task The time of arrival For the task Response time Time interval The maximum response time for all tasks to be executed within a given time interval. The optimization objective of the task scheduling model is set as the normalized weighted sum of system energy consumption and latency within each time interval, where the time interval is... optimization objectives The calculation process is shown in formula (7).

[0044]

[0045] in and Pre-defined weights for system energy consumption and latency are applied throughout the scheduling process. and The objective function of the task scheduling model can be expressed as minimizing the normalized weighted sum of system energy consumption and delay throughout the scheduling process, as shown in Equation (8).

[0046]

[0047] S2. Construct a task-host heterogeneous graph that represents the mapping relationship and resource constraints between task resource requirements and host resource supply. The task-host heterogeneous graph includes a set of task nodes, a set of host nodes, and a set of bidirectional edges with directional semantics. A forward edge indicates that a task can be assigned to a host node, and a reverse edge indicates that a host node can carry a task. Based on the real-time acquired task resource requirement information and host resource supply information, generate task node features and host node features, and embed the task node features and host node features into the task nodes and host nodes, respectively.

[0048] This invention explicitly models the competition for host resources among multiple tasks by constructing a task-host heterogeneous graph. First, a time interval is defined. heterogeneous graph structure ,in The set of tasks to be scheduled. For host collection, Let be the set of edges from tasks to hosts, indicating that tasks can be assigned to hosts. This is the set of edges from the host to the task, indicating that the host can run the task.

[0049] Subsequently, to ensure the heterogeneous graph includes real-time acquired task resource demand information and host resource provision information, task nodes and host nodes are constructed and node features are embedded. Definitions Set of tasks to be scheduled Resource demand information, among which For the task Information regarding the requirements for computing resources, memory resources, and bandwidth resources. Definitions. For host collection The resource provides information, among which For host In time interval The available computing, memory, and bandwidth resources at the start time are denoted as follows. To eliminate the dimensional differences between multidimensional resources through normalization, the highest computing, memory, and bandwidth resource limits across all hosts are denoted as follows: , and Next, based on the task's resource requirements and the host's resource availability, normalized node characteristics are calculated and defined. Time interval Task node characteristics, among which For the task The node characteristics are calculated as shown in formula (9).

[0050]

[0051] definition Time interval The host node characteristics, among which For host The node features are calculated as shown in formula (10).

[0052]

[0053] Then, based on the resource constraints between the task and the host and the host's load, a bidirectional edge is established between the task node and the host node. This eliminates scheduling relationships that do not meet resource constraints and prevents host node overload. Any task node with an edge relationship... and host node The constraints of formulas (11), (12), (13), and (14) should be satisfied.

[0054]

[0055] The final constructed task-host heterogeneous graph is as follows: Figure 2 As shown.

[0056] S3. Solve the cloud-edge collaborative task scheduling model based on the optimized A2C network. The optimized A2C network includes a state, a task-host heterogeneous graph, a heterogeneous graph pointer network, an environment, a value network, and a loss layer. The state part acquires the real-time environment state at the start of the current scheduling interval. The task-host heterogeneous graph part constructs the task-host heterogeneous graph based on the environment state. The heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and further dynamically generates task scheduling decisions at the decision layer. The environment part provides environmental feedback information and a new environment state after executing the decision. At the next moment after executing the task scheduling decision, the environment feedback information and environment state are acquired, and the losses of the policy network and value network are calculated based on the optimized advantage function to update the learnable parameters of the optimized A2C network. When the losses of the value network and the policy network converge, the optimal task scheduling decision that minimizes system energy consumption and latency is obtained.

[0057] After constructing the task-host heterogeneous graph, this invention solves the task scheduling model based on the optimized A2C algorithm. The specific solution process is as follows.

[0058] (1) Optimization of A2C network structure based on HGPN The traditional A2C algorithm is a deep reinforcement learning algorithm within the Actor-Critic framework, used to solve models that satisfy Markov decision processes. Therefore, this invention first models the cloud-edge collaborative task scheduling model as a Markov decision process. The Markov decision process uses quintuples. It means that, among them The state space is the environmental state information observed by the agent. Action space refers to the actions that an agent can take. The strategy is the mapping from the state space to the action space. The reward is the feedback given by the environment after the agent performs an action; This is a discount factor used to calculate the importance of controlling future rewards when calculating time-series difference errors. The specific elements of the Markov decision process modeled based on the cloud-edge collaborative task scheduling model are as follows.

[0059] 1) State space: definition This represents the state space containing all environmental state information throughout the entire scheduling process, where , including time intervals Resource requirements of tasks to be scheduled and host resource provision information .

[0060] 2) Action Space: Definition This represents the action space encompassing all scheduling decisions made by the agent during the entire scheduling process, where... For the agent in the time interval The set of task scheduling decisions made.

[0061] 3) Strategy: Definition This indicates that the agent is in a given state. Take action below The probability distribution is in the form of A matrix of size, where = For the first [number] of this matrix Each row vector represents a task to be scheduled. Dispatch to The probability distribution of each host.

[0062] 4) Rewards: Definition This represents the set of feedback provided by the environment after the agent performs an action during the entire scheduling process, where including time intervals optimization objectives That is, the intelligent agent executes task scheduling decisions. The normalized weighted average of system energy consumption and latency within the current time interval, and the average of energy consumption and latency that could not be achieved within the time interval due to resource contention or insufficient resources. Successfully executed , .

[0063] Traditional A2C algorithms, when solving cloud-edge collaborative task scheduling models, map states to actions through a policy network (Actor). After an agent performs an action, it obtains environmental feedback and evaluates the merits of the action taken in the current state using a value network (Critic) and a dominance function. This updates the parameters of the value network and policy network, enabling the agent to take better actions in the current environment. However, traditional A2C algorithms directly concatenate task resource requirements and host resource availability information as environmental state features during decision-making, neglecting the competition for host resources among multiple tasks and making it difficult to measure the matching degree between different task requirements and host resources. Furthermore, multiple task scheduling decisions share a global dominance function, making it difficult to accurately evaluate the merits of different task scheduling decisions. Therefore, this invention addresses the two shortcomings of the traditional A2C algorithm mentioned above by optimizing it: A Heterogeneous Graph Pointer Network (HGPN) is designed to optimize the policy network of the A2C algorithm, enabling the A2C agent to fully capture the competition and matching relationships between tasks and host resources; furthermore, the advantage function is optimized by independently calculating the advantage function for each task scheduling decision, thus more accurately evaluating the decisions made by the A2C agent. The A2C network structure optimized based on HGPN is as follows: Figure 3 As shown.

[0064] The optimized A2C network comprises six parts: state, task-host heterogeneity graph, HGPN, environment, value network, and loss layer. The state part is located within the current scheduling interval. The start time Obtain real-time environmental status The task-host heterogeneous graph section is based on the status. The task-host heterogeneous graph is constructed; the HGPN part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and further dynamically generates task scheduling decisions at the decision layer; the environment part executes the decision at the next time interval. The end time Provide environmental feedback information and environmental conditions The value network and loss layer evaluate the quality of decisions and calculate losses based on the optimized advantage function, and then update all learnable parameters of the value network and the optimized policy network.

[0065] (2) Environmental state feature extraction based on GAT This invention constructs a task-host heterogeneous graph and then uses it as input to an HGPN (Hypergraphical Generalized Networking Network). An attention mechanism (Graph Attention, GAT) is introduced into the graph convolutional layer of the HGPN to extract environmental state features, capturing the competition and matching relationships between multiple tasks and host resources. First, the competition relationship features between tasks and host resources are extracted based on the edges from tasks to hosts combined with node information. This is done using host nodes... For example, based on the characteristics of task nodes and host node characteristics Compute host node For neighboring task nodes Attention score The calculation process is shown in formula (15).

[0066]

[0067] Let the hidden layer dimension be . , and The purpose is to create a learnable weight matrix for task nodes. and host node The features are projected onto a latent space of the same dimension through a linear transformation, and || represents feature concatenation. It is a learnable parameter vector. This is the activation function. For the host node. Attention scores are calculated for all neighboring task nodes. Then, attention weights are calculated based on the host node's attention scores for all neighboring task nodes, with the host node as the weight. For neighboring task nodes Attention weights For example, the calculation process is shown in formula (16).

[0068]

[0069] in For host node The set of indices of all neighboring task nodes. Next, based on the attention weights of the host node towards all neighboring task nodes, the features of the corresponding task nodes are aggregated to obtain the task-to-host relationship. Characteristics of resource competition The calculation process is shown in formula (17).

[0070]

[0071] in It is a non-linear activation function. The competition relationship characteristics are obtained after calculation over all host nodes. .

[0072] Subsequently, to assess the compatibility between the task and the host, matching features between task requirements and host resources were extracted based on the host-to-task edge, node information, and competition relationship characteristics. This was done using task nodes... For example, combining host node characteristics Task node characteristics Characteristics of competitive relationships Compute task nodes For neighboring host nodes Attention score The calculation process is shown in formula (18).

[0073]

[0074] in It is a learnable parameter vector. For task nodes Attention scores are calculated for all neighboring host nodes. Then, attention weights are calculated based on the task node's attention scores for all neighboring host nodes, with the task node... For neighboring host nodes Attention weights For example, the calculation process is shown in formula (19).

[0075]

[0076] in For task nodes The set of indices of all neighboring host nodes. Next, based on the attention weights of the task node towards all neighboring host nodes, the features of the corresponding host nodes are aggregated to obtain the host and task... Matching relationship features The calculation process is shown in formula (20).

[0077]

[0078] The matching relationship features are obtained after calculating all task nodes. Finally, the task node features are... Matching relationship features splicing to obtain time interval Environmental characteristics The calculation process is shown in formula (21).

[0079]

[0080] (3) Task scheduling decision generation based on MLP After extracting environmental state features, this invention uses a multilayer perceptron (MLP) in the decision layer of HGPN to generate the probability distribution of task scheduling to each host. The calculation process is shown in formula (22).

[0081]

[0082] in It is a multilayer perceptron, belonging to the feedforward neural network category. These are the parameters of the network. The normalization function is obtained after normalization. The sum of probabilities for any row vector is 1.

[0083] Subsequently, to prevent the algorithm from getting trapped in local optima too early and to increase the exploratory nature of the agent's decision-making, the following approach was adopted. The strategy dynamically generates task scheduling decisions based on the task scheduling probability distribution. This strategy is based on The probability is used to randomly select a host as the target host for the task. The probability is based on the probability distribution The host with the highest probability is selected as the target host of the task, and the calculation process is shown in formula (23).

[0084]

[0085] in To select the host with the highest probability, To randomly select a host, pointers are used to record the probability of each task being scheduled to the target host, defined as follows: Time interval The set of pointers to tasks to be scheduled, where Tasks to be scheduled The pointer, whose value is a probability distribution. The probability of being located in the target host's location.

[0086] (4) Environmental feedback information acquisition and loss calculation This invention obtains environmental feedback information at the next moment after the intelligent agent executes a task scheduling decision. and environmental conditions The losses of the policy network and value network are calculated to update the learnable parameters of the algorithm. First, the value network is used to evaluate the current environmental state. and the next environmental state The environmental state value is shown in formulas (24) and (25).

[0087]

[0088] in To integrate the information on task resource requirements and host resource provision, the environment state is obtained by concatenating the task node features and host node features after performing global average pooling on each. It is a multilayer perceptron, belonging to the feedforward neural network category. These are the parameters of the network. For activation function, It is the value network's response to the environmental state The value estimation is similar to the calculation of environmental state. Value estimation .

[0089] The timing difference error was then calculated. , is used to measure the difference between the assessed value and the actual value of the value network, and the calculation process is shown in formula (26).

[0090]

[0091] in As a discount factor, Environmental feedback The optimization objective value is then calculated. Next, the value network loss is calculated. As shown in formula (27).

[0092]

[0093] in The mean square error of the time-series difference is minimized. This can make the environmental state value assessed by the value network more accurate.

[0094] Unlike traditional A2C algorithms that directly use time-series difference error as the advantage function Reflected in the current environmental state Take action The probabilities of the selected action are determined by their relative strengths: a positive advantage increases the probability of the selected action, while a negative advantage decreases the probability of the selected action. This invention relates to the action... In fact, it includes For task scheduling decisions, timing difference errors only reflect the overall quality of the decisions. Therefore, this invention optimizes the advantage function of the A2C algorithm and independently calculates the advantage function for each decision based on the timing difference error and the execution status of the task scheduling decisions. The calculation process is shown in formula (28).

[0095]

[0096] Since the environmental state value estimate reflects system energy consumption and latency, and the optimization objective is to minimize system energy consumption and latency, it is necessary to negatively evaluate the timing difference error as an advantage for successfully executed task scheduling decisions. Furthermore, for task scheduling decisions that fail to execute, the positive advantage should be reduced when the overall advantage is positive, and the negative advantage should be amplified when the overall advantage is negative. This is a scaling factor used to control the advantage gap between decisions.

[0097] Finally, the network loss of the strategy is calculated based on the advantage function and pointers. As shown in formula (29).

[0098]

[0099] in For the task The pointer records the probability that the task will be scheduled to its target host. To take the logarithm of the probability, The sum of losses from task scheduling decisions is minimized here. This means increasing the probability of each decision when the overall advantage is positive, and decreasing the probability of each decision when the overall advantage is negative. The degree of increase and decrease depends on the advantage function of each decision.

[0100] (5) Optimize A2C parameter updates definition The set of parameters that needs to be updated to optimize the A2C algorithm includes the parameters of the value network. Parameters of GAT in the policy network , , , and parameters Next, based on the losses of the value network and the policy network, the partial derivatives with respect to all parameters are obtained through backpropagation to obtain their respective gradients. For example, the gradient calculation process is shown in formula (30).

[0101]

[0102] The parameters are then updated using gradient descent. For example, the parameter update process is shown in formula (31).

[0103]

[0104] in For parameters By adjusting the learning rate and updating all learnable parameters, the agent completes one training iteration. The new parameters will then generate task scheduling decisions with lower losses, reducing system energy consumption and latency.

[0105] (6) Determining the optimal task scheduling decision If the value network loss and the policy network loss do not converge, it means that the task scheduling decision at this time is not the optimal decision, and the training process needs to be repeated to update the algorithm parameters so that the loss of the next decision is smaller; if the value network loss and the policy network loss converge, the agent learns the optimal task scheduling policy and obtains the optimal task scheduling decision that minimizes system energy consumption and latency.

[0106] The following specific application examples will further illustrate the solution and effects of the present invention.

[0107] This invention evaluates the proposed method using the Bitbrains dataset, a publicly available dataset, in a simulated cloud-edge environment. This dataset consists of workload traces running on 1750 virtual machines in a Bitbrains distributed data center, including CPU utilization, requested RAM size, disk bandwidth characteristics, etc., in millions of instructions per second (MIPS) for each time period. Since these workload traces correspond to applications running on real infrastructure, the Bitbrains dataset is used to simulate task generation at each scheduling interval.

[0108] This invention models cloud-edge hosts by referencing publicly available performance metrics from Microsoft's Azure B-series virtual machines, with specific parameters shown in Table 1.

[0109]

[0110] This invention was tested on the COSCO cloud-edge simulation platform. This platform, as a framework, allows for customized workloads and virtual machine configurations and can acquire system performance metrics at each time interval. The evaluation metrics of this invention are Total Energy, Average Schedule Time, Average Response Time, and Service Level Agreement Violations Percentage (SLA Violations Percentage). Total Energy represents the total energy consumption of all hosts from the start of the simulation to the current moment; Average Schedule Time is the average scheduling time for all scheduled tasks to be scheduled to the target host up to the current moment; Average Response Time is the average response time for all completed tasks from the start of scheduling to completion up to the current moment; and SLA Violations Percentage is the percentage of completed tasks that violated the SLA out of the total number of completed tasks up to the current moment.

[0111] This invention selects seven cloud-edge collaborative task scheduling methods as baseline models for comparative experiments, including Random scheduling, heuristic rule-based cloud-edge collaborative task scheduling methods LRMMT, MADMMT, and MADMC, metaheuristic algorithm-based cloud-edge collaborative task scheduling methods POND and GA, and deep reinforcement learning algorithm-based cloud-edge collaborative task scheduling method DDQN. In addition, ablation experiments are set up to verify the effectiveness of HGPN as an A2C algorithm policy network, as detailed below.

[0112] Random: A completely random task scheduling method that randomly selects tasks to be assigned and randomly assigns them to any host.

[0113] LRMMT: Prioritizes scheduling tasks with low memory requirements through Minimum Memory Transfer (MMT) and predicts host overload through Local Regression (LR), assigning tasks to hosts that are not overloaded.

[0114] MADMMT: Prioritizes scheduling tasks with low memory requirements using MMT, and selects the target host based on Median Absolute Deviation (MAD) to prevent host overload.

[0115] MADMC: Prioritizes scheduling tasks that have the greatest impact on other tasks through the Maximum Correlation Policy (MC), and selects target hosts through MAD to prevent host overload.

[0116] POND: It explores the performance history of a specific task assigned to different hosts through the Upper-Confidence Bound (UCB) algorithm and greedy neighborhood search, and then calculates the weight of the host to the task, and selects the host with the largest weight for task scheduling.

[0117] GA: Genetic Algorithm (GA) optimizes task scheduling decisions. The current environmental state is used as the initial chromosome. Different task scheduling decisions are explored through selection, crossover and mutation operations. The fitness of each scheme is evaluated using a pre-trained neural network model. The optimal task scheduling decision is obtained after multiple generations of evolution.

[0118] DDQN: It directly uses the feature matrix concatenated from the task requirements, host resources, and allocation matrix as the environment state. It uses a Double Deep Q Network (DDQN) to optimize task scheduling decisions. It uses the current Q network to evaluate the Q value of the task-host pair in real time, makes task scheduling decisions based on the Q value of the current Q network, and accelerates agent learning through periodic experience replay and periodically synchronizes the target Q network to stabilize the training process.

[0119] A2C: The feature matrix, which is a concatenation of task requirements, host resources, and allocation matrix, is used as the environment state. A multilayer perceptron (MLP) is used as the policy network to optimize task scheduling decisions.

[0120] PNA2C: The pointer network PN is used as the policy network of the A2C algorithm. In the encoder part of the pointer network, the task resource requirement features and host available resource features are extracted by the Long Short-Term Memory (LSTM) network and the attention mechanism, respectively. The task scheduling decision is generated in the decoder part of the pointer network and optimized by training A2C.

[0121] GPNA2C: The task nodes and host nodes are embedded into their respective isomorphic graphs. The pointer network PN is used as the policy network of the A2C algorithm. In the encoder part of the pointer network, the features of the task nodes and host nodes are extracted by the graph neural network (GNN) and the attention mechanism, respectively. The task scheduling decision is generated in the decoder part of the pointer network. The optimization is achieved by training A2C.

[0122] The above ten methods were compared with the HGPNA2C method proposed in this invention. Each method was run 5 times on the simulation platform, with each run lasting 3000 seconds. The average value of the 5 experiments was taken. The experimental results are as follows: Figure 4 , Figure 5 , Figure 6 , Figure 7 As shown.

[0123] As shown in the results, although the Random algorithm did not violate SLA overall, it performed poorly among all methods due to the lack of an optimization strategy. Among the heuristic rule-based scheduling methods LRMMT, MADMMT, and MADMC, LRMMT had the highest system energy consumption and the lowest average response time, MADMC had the lowest system energy consumption and the highest average response time, and MADMMT had the shortest average scheduling time. However, even with a very short average scheduling time, the average response time was still higher than LRMMT and an SLA violation occurred, reflecting that heuristic rule-based scheduling methods usually only focus on single-objective optimization, and task decisions are local optima. Among the metaheuristic algorithm-based scheduling methods POND and GA, POND performed better than GA and was similar to heuristic rule-based scheduling methods and PNA2C. This may be because POND uses the upper bound confidence algorithm UCB from the multi-armed slot machine model, combining the advantages of dynamic decision-making in reinforcement learning algorithms in terms of optimization mechanism. However, like heuristic rule-based scheduling methods, it focuses more on single-objective optimization. GA performed particularly poorly among the four methods. The poor performance across performance metrics indicates that GA not only incurs high computational overhead and requires longer decision-making time, but also struggles to learn in complex and dynamic cloud-edge collaborative scenarios, failing to generate effective task scheduling strategies. This reflects the convergence problem of heuristic algorithms based on global optimization in dynamic scenarios. Among deep reinforcement learning-based scheduling methods, DDQN and A2C performed poorly overall, indicating that insufficient feature extraction of the cloud-edge collaborative environment directly affects the agent's interaction and learning process with the environment, resulting in incomplete perception of the environment state and an inability to generate effective task scheduling strategies. PNA2C and GPNA2C outperformed DDQN and A2C, showing better system energy consumption metrics than heuristic rule-based scheduling methods, but slightly worse system average response time metrics. This is because although PNA2C and GPNA2C enabled the agent to learn better strategies by further extracting environmental state features of the cloud-edge collaborative computing system, they focused more on optimizing system energy consumption metrics during training, specifically the weighting of system energy consumption when calculating immediate rewards from environmental feedback. Weights greater than system latency This reflects that in scenarios where a specific goal is more focused, deep reinforcement learning algorithms achieve better performance than heuristic algorithms for that goal, and have greater potential in solving complex dynamic optimization problems such as cloud-edge collaborative task scheduling. Further balancing... and This could enable the agent to learn better task scheduling strategies.

[0124] Among all methods, the HGPNA2C method proposed in this invention performs best in three performance evaluation metrics: system energy consumption, average response time, and service level agreement violation rate. It performs second best in average scheduling time, and its overall performance is superior to other methods. There are two reasons for this: 1) This method designs a policy network that optimizes the A2C algorithm using HGPN. This leverages the advantage of deep reinforcement learning algorithms in effectively handling dynamic environmental fluctuations. Compared to PNA2C and GPNA2C, which only focus on the relationship between homogeneous data, HGPNA2C can capture the competition between multiple tasks for the same host resources and the matching relationship between task resource requirements and available host resources. This allows the agent to more comprehensively perceive the cloud-edge collaborative environment and make better task scheduling decisions. 2) This method optimizes the advantage function of A2C, calculating the advantage function independently for each task scheduling decision. The agent can more accurately adjust the probability of scheduling different tasks to the target host based on the advantage function, and learn the optimal task scheduling strategy more effectively through interaction in the dynamically changing cloud-edge environment.

[0125] In summary, the cloud-edge collaborative task scheduling method based on the optimized A2C algorithm proposed in this invention can fully extract environmental state features and improve the evaluation capability of task scheduling decisions, realizing dynamic scheduling of tasks on cloud-edge hosts and effectively reducing system energy consumption and latency. Experiments simulated in a cloud-edge environment using real workload datasets show that the proposed method outperforms other baseline methods in terms of comprehensive performance evaluation indicators, including system energy consumption, average scheduling time, average response time, and service level agreement violation rate.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud-edge collaborative task scheduling method based on an optimized A2C algorithm, characterized in that, Includes the following steps: A set of tasks to be scheduled is received within discrete time intervals. The optimization objective is to minimize the overall energy consumption of the cloud-edge host and the average response time of the tasks completed within the time interval. A cloud-edge collaborative task scheduling model is constructed. A task-host heterogeneous graph is constructed to represent the mapping relationship and resource constraints between task resource requirements and host resource supply. The task-host heterogeneous graph includes a set of task nodes, a set of host nodes, and a set of bidirectional edges with directional semantics. A forward edge indicates that a task can be assigned to a host node, and a reverse edge indicates that a host node can carry a task. Based on the real-time acquired task resource requirement information and host resource supply information, task node features and host node features are generated and embedded into task nodes and host nodes, respectively. A cloud-edge collaborative task scheduling model is solved based on an optimized A2C network. The optimized A2C network includes a state, a task-host heterogeneous graph, a heterogeneous graph pointer network, an environment, a value network, and a loss layer. The state part acquires the real-time environment state at the start of the current scheduling interval; the task-host heterogeneous graph part constructs the task-host heterogeneous graph based on the environment state; the heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and further dynamically generates task scheduling decisions at the decision layer; the environment part provides environmental feedback information and a new environment state after the decision is executed; at the next moment after the task scheduling decision is executed, the environment feedback information and environment state are acquired, and the loss of the policy network and value network is calculated based on the optimized advantage function to update the learnable parameters of the optimized A2C network. When both the value network loss and the policy network loss converge, the optimal task scheduling decision that minimizes system energy consumption and latency is obtained.

2. The cloud-edge collaborative task scheduling method based on an optimized A2C algorithm according to claim 1, characterized in that, The construction of the task-host heterogeneous graph includes: For any task node and host node, a bidirectional edge is established between them if and only if the following four resource constraints are simultaneously satisfied: The task's computational requirements are less than the available computing resources on the host node; The memory requirements of the task node are less than the available memory resources of the host node; The bandwidth requirement of the task node is less than the available bandwidth resources of the host node; The node's current load rate is below the preset threshold.

3. The cloud-edge collaborative task scheduling method based on an optimized A2C algorithm according to claim 1, characterized in that, The heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers. This includes: introducing an attention mechanism into the graph convolutional layers of the heterogeneous graph pointer network to extract environmental state features from the task-host heterogeneous graph input to the heterogeneous graph pointer network; firstly, extracting the competition relationship features between tasks and host resources based on the edges from tasks to hosts combined with node information; and secondly, extracting the matching relationship features between task requirements and host resources based on the edges from hosts to tasks combined with node information and competition relationship features.

4. The cloud-edge collaborative task scheduling method based on the optimized A2C algorithm according to claim 3, characterized in that, The heterogeneous graph pointer network part captures the competition and matching relationships between tasks and host resources through graph convolutional layers, and also includes: concatenating task node features with matching relationship features to obtain the environmental state features of the time interval.

5. A cloud-edge collaborative task scheduling method based on an optimized A2C algorithm according to claim 4, characterized in that, The heterogeneous graph pointer network dynamically generates task scheduling decisions at the decision layer, including: using a multilayer perceptron to process the extracted environmental state features and generate a probability distribution of task scheduling to each host. use The strategy dynamically generates task scheduling decisions based on the task scheduling probability distribution.

6. The cloud-edge collaborative task scheduling method based on the optimized A2C algorithm according to claim 5, characterized in that, Optimized advantage function Calculate according to the following segmented logic: when and hour, ; when and hour, ; when hour, ; in For timing difference error, Time interval The optimization target value, This is a subset of task scheduling decisions that failed to execute successfully due to resource competition or insufficient resources. For the range of values ​​within Scaling factor within.

7. A cloud-edge collaborative task scheduling method based on an optimized A2C algorithm according to claim 6, characterized in that, The loss function of the policy network is: in, For the task The pointer records the task. The probability of being scheduled to the target host. To take the logarithm of the probability; The loss function of the value network is: in, This refers to timing difference error.

8. A cloud-edge collaborative task scheduling method based on an optimized A2C algorithm according to claim 7, characterized in that, The optimized A2C network learnable parameter updates include: The gradients of the value network and policy network are obtained by taking the partial derivatives of all parameters through backpropagation, and then the parameters are updated by gradient descent.

Images