Multi-cloud environment task-oriented graph enhancement two-level architecture A3C scheduling method and system

Abstract

The invention relates to the technical field of cloud computing, in particular to a multi-cloud environment task-oriented graph-enhanced two-level architecture A3C scheduling method and a multi-cloud environment task-oriented graph-enhanced two-level architecture A3C scheduling system. The method comprises the steps of obtaining cloud user task data; constructing a heterogeneous scheduling graph based on the obtained cloud user task data; gNN embedding calculation is carried out based on the heterogeneous scheduling graph; designing a hierarchical scheduling architecture based on GNN embedding calculation to respectively process mapping tasks with different strengths; performing reward calculation according to a processing result of the mapping task; and outputting an optimal strategy. According to the scheme, further refinement is carried out in an energy consumption and charging model, the model not only considers sub-task communication energy consumption, but also supports layered charging strategies of on-demand, reservation and bidding instances, and it is ensured that a scheduling decision is closer to an energy consumption and cost structure of a real cloud platform.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to a graph-enhanced two-level architecture A3C scheduling method and system for tasks in a multi-cloud environment. Background Art

[0002] In a multi-cloud environment, task scheduling is extremely complex due to factors such as resource heterogeneity, network latency, and diverse billing models. Common scheduling methods include combining meta-heuristic algorithms with machine learning or deep learning techniques, such as the cat swarm optimization (CSO) algorithm combined with a deep neural network, the ant colony optimization (ACO) algorithm combined with Q-Learning, the particle swarm optimization (PSO) algorithm combined with a neural network algorithm, and various hybrid modes of ant colony and deep reinforcement learning, genetic algorithm and deep reinforcement learning techniques. These methods use heuristic search to quickly obtain better solutions and use learning models to predict and optimize historical scheduling results. In addition, the HEFT (Heterogeneous Earliest Finish Time) algorithm and its variants have also been widely studied, such as combining HEFT with Q-Learning, decision trees, or gravitational search algorithms in order to automatically adjust its priority strategy; there are also pure deep reinforcement learning methods, such as DQN, A3C, PPO, etc., which provide end-to-end adaptive scheduling strategies for multi-cloud scheduling problems by constructing parallel threads or continuous action spaces.

[0003] However, the above methods also have significant deficiencies. Although the meta-heuristic hybrid learning method performs well in offline tasks, its heuristic parameter tuning is cumbersome, the model convergence speed is slow, and its adaptability to sudden online loads is insufficient. HEFT and its variants are difficult to absorb historical scheduling experience due to relying on fixed heuristic rules, and are prone to performance fluctuations in a highly dynamic environment. The pure deep reinforcement learning method faces the problem of state-action space explosion, and a slight increase in the number of tasks, virtual machines, and physical machines will lead to a sharp increase in the network scale and a sharp rise in training and inference costs. Summary of the Invention

[0004] To solve the problems mentioned above, the present invention provides a graph-enhanced two-level architecture A3C scheduling method and system for tasks in a multi-cloud environment.

[0005] In a first aspect, a graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment provided by the present invention adopts the following technical solutions: A graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment includes: Obtain cloud user task data; Construct a heterogeneous scheduling graph based on the obtained cloud user task data; Perform GNN embedding calculation based on the heterogeneous scheduling graph; Design a hierarchical scheduling architecture based on GNN embedding calculation to process mapping tasks with different intensities; Calculate the reward according to the processing results of the mapping tasks; Output the optimal policy.

[0006] Furthermore, the heterogeneous scheduling graph constructed based on the obtained cloud user task data includes tasks provided by cloud users , where each task is divided into several subtasks , and the length, data block size, deadline, etc. of each subtask are described by a feature vector . All available virtual machines are denoted as , and the processing capacity, memory, network bandwidth, etc. of each virtual machine are described by a feature vector . Physical nodes are denoted as . The subtasks, virtual machines, and physical machines are used to form a three-layer heterogeneous graph node set as , and the edge set includes candidate edges for subtask-virtual machine mapping, VM-PM attribution edges, and data dependency edges between subtasks under the same task.

[0007] Furthermore, the GNN embedding calculation based on the heterogeneous scheduling graph includes calculating the graph structure information and through several layers of graph neural network GNN to obtain a low-dimensional state representation, which is expressed as: , where: represents the hidden state of node at the th layer. When is 0, is defined; represents the set of all neighbor nodes directly connected to node . The types of edges are distinguished by different weight matrices or attention mechanisms; is the learnable weight matrix at the th layer, which is used to linearly transform the hidden state from neighbor nodes ; is the bias vector at the th layer; is the element-wise non-linear activation function, which is used to enhance the expression ability of the model.

[0008] Furthermore, the design of the hierarchical scheduling architecture based on GNN embedding calculation to process mapping tasks with different intensities includes implementing the category mapping of subtasks to virtual machines in the first-layer scheduling. Among them, the input state is the GNN embedding of all subtask nodes and the common embedding of three virtual machine categories , and for each subtask calculate the category selection distribution, take the maximum probability from the distribution to obtain the category decision , expressed as: wherein, represents the probability distribution of the first-layer agent selecting the VM category in the current subtask state ; is the d-dimensional hidden representation obtained by the GNN for the subtask node ; is the predefined category embedding vector representing three VM categories is the vector obtained by sequentially concatenating the subtask representation and the three category representations, serving as the input feature of the first-layer Actor is the learnable weight matrix of the first-layer Actor, linearly mapping the input feature to the category space

[0009] Furthermore, the hierarchical scheduling architecture designed based on GNN embedding calculates and processes mapping tasks with different intensities, and also includes using the second layer to implement the mapping from subtasks to specific VMs. Among them, for each subtask node , according to the output of the first layer , the candidate VM set is restricted to the VM subset under the category , the Actor network L2 receives the subtask embedding and all VM embeddings of this subset , and outputs the specific VM selection distribution, expressed as: wherein, represents the probability that the second-layer agent selects the th specific VM under the state input of the current subtask and the candidate VM subset of its affiliated VM category; is the set of GNN embeddings of all VMs corresponding to the category already selected by the subtask in the first layer ; represents the vector obtained by sequentially concatenating the subtask and the embeddings of K candidate VMs under this category, serving as the input of the second-layer Actor is the learnable weight matrix of the second-layer Actor, mapping the input feature to the K-dimensional virtual machine category space

[0010] Furthermore, the reward calculation based on the processing result of the mapping task includes setting the reward as a weighted sum , where: , and are the relative weights of the three physical quantities of the maximum completion time, total energy consumption, and resource cost respectively. , and are the relative weights of the three physical quantities, which are usually dynamically adjusted according to the task SLA strictness and satisfy the sum of the three being 1; the output of the designed Critic network is a three-dimensional value function , and the three-dimensional value function predicts the future cumulative return starting from the current state respectively, and there is the following relationship expressed as: , where: is the system state at the t-th scheduling time slot, which is composed of GNN embedding and other features; is the discount factor, which is used to control the attenuation degree of future returns; , , represent the current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling is completed at time slot t respectively.

[0011] Further, the reward calculation according to the processing result of the mapping task further includes, for each virtual machine The total time required to execute all the assigned subtasks includes the computing time and the communication time. Among them, the execution time calculation method is , where: indicates whether the subtask is mapped to the virtual machine ; is the computing amount of the subtask ; is the processing rate of the virtual machine ; The communication time calculation method is , where indicates whether the subtasks l and k of the same task are mapped to different virtual machines. If they are mapped to the same virtual machine, there is no need for communication. represents the amount of data to be exchanged between subtasks, represents the network bandwidth between the two.

[0012] Further, the reward calculation according to the processing result of the mapping task further includes calculating the total energy consumption based on the execution energy consumption and the communication energy consumption. Among them, the execution energy consumption calculation method is , where is the power consumption of the virtual machine at full load; The communication energy consumption calculation method is , where is the average power consumption for network transmission; then the total energy consumption is .

[0013] Further, the reward calculation based on the processing result of the mapping task further includes, based on the fact that different virtual machines in a multi-cloud environment support multiple billing models, making , and and are the per-unit-time billing amounts of virtual machine in three modes respectively, , and correspond to the weights of different modes, then the resource cost The calculation method is .

[0014] In a second aspect, a graph-enhanced two-level architecture A3C scheduling system for tasks in a multi-cloud environment includes: A data acquisition module, configured to acquire cloud user task data; A heterogeneous scheduling module, configured to construct a heterogeneous scheduling graph based on the acquired cloud user task data; An embedding calculation module, configured to perform GNN embedding calculation based on the heterogeneous scheduling graph; A mapping module, configured to design a hierarchical scheduling architecture based on the GNN embedding calculation to process mapping tasks with different intensities respectively; A reward module, configured to calculate a reward according to the processing result of the mapping task; A policy module, configured to output an optimal policy.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions are suitable for being loaded and executed by a processor of a terminal device for the graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment as described above.

[0016] In a fourth aspect, the present invention provides a terminal device, including a processor and a computer-readable storage medium, the processor is used to implement each instruction; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor for the graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment as described above.

[0017] In summary, the present invention has the following beneficial technical effects: The present invention uses a graph neural network (GNN) to perform dimensionality reduction and abstraction on the relationship between physical machines, virtual machines, and tasks, embedding high-dimensional states into a low-dimensional feature space. Secondly, a hierarchical decision-making mechanism is further designed. The upper-layer agent is responsible for selecting the type of virtual machine (such as high-performance, general-purpose, low-power) to which the subtask should be scheduled, and the lower-layer agent finely selects a specific virtual machine within this type, thus greatly compressing the state-action space. Thirdly, for the multi-objective optimization problem, a multi-head Critic structure is designed to evaluate latency, energy consumption, and resource costs respectively, and the weights are dynamically adjusted according to the urgency of the task SLA to achieve an adaptive balance of immediate rewards. Finally, the present solution is further refined in the energy consumption and billing model, which not only considers the energy consumption of subtask communication but also supports a hierarchical billing strategy for on-demand, reserved, and bid instances, ensuring that the scheduling decision is closer to the energy consumption and cost structure of a real cloud platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 FIG. is a schematic diagram of a graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The present invention will be further described in detail below with reference to the accompanying drawings.

[0020] Embodiment 1 Referring to Figure 1 , a graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment in this embodiment includes: Obtaining cloud user task data; Constructing a heterogeneous scheduling graph based on the obtained cloud user task data; Performing GNN embedding calculation based on the heterogeneous scheduling graph; Designing a hierarchical scheduling architecture based on the GNN embedding calculation to process mapping tasks with different intensities; Calculating rewards according to the processing results of the mapping tasks; Outputting the optimal policy.

[0021] Specifically: 1. Constructing a heterogeneous scheduling graph, Assume that the task provided by the cloud user is , where each task is divided into several subtasks , and the length, data block size, deadline, etc. of each subtask are described by the feature vector . All available virtual machines are denoted as , and the processing capacity, memory, network bandwidth, etc. of each virtual machine are described by the feature vector . Physical nodes are denoted as , and their characteristics such as power consumption model and geographical location are determined by Characterization. The collection of data centers across regions is , which is used to calculate the cross-center communication cost and latency. For each node in the graph , the initial feature is obtained by splicing the task length, data block size, and VM processing rate or physical machine energy consumption, etc.

[0022] The heterogeneous graph node set consisting of subtasks, virtual machines, and physical machines is , and the edge set includes candidate edges for subtask-VM mapping, VM-PM belonging edges, and data dependency edges between subtasks under the same task.

[0023] 2. GNN embedding calculation, The graph structure information and are used to calculate the low-dimensional state representation through several layers of graph neural network GNN: , where: represents the hidden state of node at the th layer. When is 0, is defined; represents the set of all neighbor nodes directly connected to node . The type of edge (such as subtask-VM, VM-PM, or data dependency) can be distinguished by different weight matrices or attention mechanisms; is the learnable weight matrix at the th layer, which is used to perform a linear transformation on the hidden state from neighbor nodes ; is the bias vector at the th layer; is the element-wise non-linear activation function, which is used to enhance the expressive ability of the model.

[0024] 3. Hierarchical scheduling architecture, This algorithm adopts a hierarchical scheduling architecture, which is responsible for mapping tasks with different intensities respectively, so as to effectively compress the state-action space and improve the scheduling efficiency.

[0025] (1) The first layer of scheduling, The first layer realizes the category mapping from subtasks to virtual machines. The input state is the GNN embedding of all subtask nodes , and the common embedding of three virtual machine categories (high-performance (numbered 1), general (numbered 2), low-power (numbered 3)). The Actor network 1 receives the above embeddings and for each subtask Calculate the category selection distribution. Take the maximum probability from the distribution to obtain the category decision .

[0026] Among them, represents the probability distribution of the first-layer agent selecting the VM category under the current subtask state. is the d-dimensional hidden representation obtained by the GNN for the subtask node , which synthesizes its own attributes and neighborhood information. is the predefined category embedding vector representing three VM categories, used to provide category semantic information to the network. is the vector obtained by sequentially concatenating the subtask representation and the three category representations, serving as the input feature of the first-layer Actor. is the learnable weight matrix of the first-layer Actor, linearly mapping the input feature to the category space. softmax normalizes the vector of length 3 and outputs a legal probability distribution such that .

[0027] (2) The second-layer scheduling, The second layer realizes the mapping from subtasks to specific VMs. For each subtask node , according to the output of the first layer, the candidate VM set is restricted to the subset of VMs under the category . The Actor network L2 receives the subtask embedding and all VM embeddings of this subset, and outputs the specific VM selection distribution: Among them, represents the probability that the second-layer agent selects the th specific VM under the state input of the current subtask and the candidate VM subset under its affiliated VM category; is the set of GNN embeddings of all VMs corresponding to the category already selected by the subtask in the first layer. represents the vector obtained by sequentially concatenating the subtask and the embeddings of K candidate VMs under this category, serving as the input of the second-layer Actor. is the learnable weight matrix of the second-layer Actor, mapping the input feature to the K-dimensional virtual machine category space. softmax normalizes the vector of length K and outputs the probability distribution over K candidate VMs, ensuring that the sum of the probabilities of selecting a specific VM is 1.

[0028] 4. Reward calculation Set the reward as a weighted sum , where , and are the relative weights of the three physical quantities of the maximum completion time, total energy consumption, and resource cost respectively , and are the relative weights of the three physical quantities, which are usually dynamically adjusted according to the task SLA strictness and satisfy the sum of the three equal to 1. Design the output of the Critic network as a three-dimensional value function The three-dimensional value function predicts the future cumulative return starting from the current state respectively, and there is the following relationship. Where is the system state at the t-th scheduling time slot, which is composed of GNN embedding and other features is the discount factor, which is used to control the attenuation degree of future returns , , represent the current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling is completed at time slot t respectively

[0029] , (1) Maximum completion time , For each virtual machine The total time required for it to execute all the subtasks assigned to it includes the computing time and the communication time. The execution time calculation method is , where indicates whether the subtask is mapped to the virtual machine ; is the computing volume of the subtask ; is the processing rate of the virtual machine . The communication time calculation method is , where indicates whether the subtasks l and k of the same task are mapped to different virtual machines. If they are mapped to the same virtual machine, there is no need for communication represents the amount of data that needs to be exchanged between subtasks represents the network bandwidth between the two. Then the completion time of each VM is . Further, the MSP of the system is defined as the maximum completion time among all VMs .

[0030] (2) Total energy consumption , The total energy consumption consists of two parts: execution energy consumption and communication energy consumption. The calculation method of execution energy consumption is , where is the power consumption of the virtual machine at full load. The calculation method of communication energy consumption is , where is the average power consumption of network transmission. Then the total energy consumption is .

[0031] (3) Total resource cost , In a multi-cloud environment, different virtual machines support multiple billing models (on-demand, reserved, and spot). Let , and be the billing amounts per unit time of the virtual machine in the three models respectively, and , and correspond to the weights of different models. Then the resource cost is calculated as: .

[0032] 5. Output the optimal strategy, (1) Calculate the advantage, After calculating the reward, for each worker, combine the immediate reward obtained at the time slot , as well as the value function estimates of the current environmental state and the next environmental state by the Critic, to calculate the advantage. Among them, is the action executed by the Actor network data in the state , that is, the selection of the sub-task category and the specific VM in the hierarchical scheduling. is the discount factor, which is used to attenuate the influence of future rewards on the current decision. The closer the value is to 1, the more attention is paid to long-term rewards.

[0033] (2) Local gradient accumulation, The gradient of the Actor. Among them, are the learnable parameters of the Actor network, including all the weights and biases in the two-layer scheduling network. is the probability of the Actor network taking the action given the state . is the derivative of the parameter The gradient, the influence of the very bright parameter change on the action probability.

[0034] The gradient of the Critic network. Among them, are the learnable parameters of the Critic network. is the value function estimation of the Critic network for the current state value. is the target value constructed based on the immediate reward at time t and the estimated value of the next state. is the square of the temporal difference error, used as the loss function of the Critic.

[0035] (3) Hybrid synchronous - asynchronous parameter update, The gradients of the Actor output layer and the Critic output layer are centralized to the parameter server or the main thread, and a synchronous update of gradient descent / ascent is uniformly performed once. For the parameters of the GNN layer and the hidden layer, each Worker can asynchronously apply small-step asynchronous updates locally and periodically perform "Gossip"-style partial synchronization of the parameters with the global model.

[0036] (4) New parameter broadcast, After the global parameter update is completed, the latest Actor and Critic network parameters are distributed back to each Worker, so that the next round of decision-making is based on the latest model.

[0037] (5) Convergence judgment, Check whether the change amplitude of the policy or the value function, the sliding average of the rewards reaches the preset threshold, or whether the number of training rounds reaches the upper limit. If it has not converged yet, return to "Construct heterogeneous scheduling graph → GNN embedding calculation" to start a new round of iteration; otherwise, proceed to the next step.

[0038] (6) Output policy, When the convergence condition is met, fix the Actor network parameters . This network represents the optimal task - VM hierarchical scheduling policy. During deployment, directly input the subtask and resource status, and the optimal VM category and specific VM allocation decision can be output.

[0039] Example 2 This example provides a graph-enhanced two-level architecture A3C scheduling system for tasks in a multi-cloud environment, including: A data acquisition module, configured to acquire cloud user task data; A heterogeneous scheduling module, configured to construct a heterogeneous scheduling graph based on the acquired cloud user task data; An embedding computing module, configured to perform GNN embedding computation based on a heterogeneous scheduling graph; A mapping module, configured to design a hierarchical scheduling architecture based on the GNN embedding computation to process mapping tasks with different intensities respectively; A reward module, configured to perform reward computation according to the processing results of the mapping tasks; A policy module, configured to output an optimal policy.

[0040] A computer-readable storage medium, in which multiple instructions are stored, and the instructions are adapted to be loaded and executed by a processor of a terminal device for the graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment as described above.

[0041] A terminal device, including a processor and a computer-readable storage medium, where the processor is used to implement each instruction; the computer-readable storage medium is used to store multiple instructions, and the instructions are adapted to be loaded and executed by the processor for the graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment as described above.

[0042] The above are all preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment, characterized in that, including: Obtain cloud user task data; Construct a heterogeneous scheduling graph based on the obtained cloud user task data; Perform GNN embedding calculation based on the heterogeneous scheduling graph; Design a hierarchical scheduling architecture based on the GNN embedding calculation to process mapping tasks with different intensities respectively; Calculate rewards according to the processing results of the mapping tasks; Output the optimal policy.

2. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 1, characterized in that, Constructing a heterogeneous scheduling graph based on the obtained cloud user task data, including tasks provided for cloud users , where each task is divided into several subtasks , the length, data block size, and deadline of each subtask are described by a feature vector . All available virtual machines are denoted as , and the processing power, memory, and network bandwidth of each virtual machine are described by a feature vector . Physical nodes are denoted as . The subtasks, virtual machines, and physical machines form a three-layer heterogeneous graph node set as . The edge set includes candidate edges for subtask-virtual machine mapping, VM-PM affiliation edges, and data dependency edges between subtasks under the same task.

3. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 2, characterized in that, The GNN embedding calculation based on the heterogeneous scheduling graph includes the graph structure information and which are calculated through several layers of graph neural networks (GNNs) to obtain a low-dimensional state representation, denoted as: ， Wherein: represents a node The hidden state of the layer. When is 0, it is defined as represents the set of all neighbor nodes directly connected to the node . The types of edges are distinguished by different weight matrices or attention mechanisms; is the learnable weight matrix of the layer, which is used to perform a linear transformation on the hidden state from neighbor nodes ; is the bias vector of the layer; is the element-wise non-linear activation function, which is used to enhance the expressive power of the model.

4. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 3, characterized in that, The hierarchical scheduling architecture designed based on GNN embedding processes mapping tasks with different intensities respectively, including implementing the category mapping of subtasks to virtual machines in the first-level scheduling, where the input state is the GNN embedding of all subtask nodes , and the common embedding of three virtual machine categories , and for each subtask calculate the category selection distribution, take the maximum probability from the distribution to obtain the category decision , expressed as: Among them, represents the probability distribution of the first-layer agent selecting the VM category in the current subtask state; The probability distribution; is the hidden representation of dimension d obtained by the subtask node through GNN, is the category embedding vector pre-defined to represent three VM categories, is the vector obtained by sequentially concatenating the subtask representation and the three category representations, serving as the input feature of the first-layer Actor, is the learnable weight matrix of the first-layer Actor, linearly mapping the input feature to the category space.

5. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 4, characterized in that, The hierarchical scheduling architecture designed based on GNN embedding processes mapping tasks with different intensities, and also includes using the second layer to implement the mapping of subtasks to specific VMs. Among them, for each subtask node , according to the output of the first layer , the candidate VM set is restricted to the VM subset under the category . The Actor network L2 receives the subtask embedding and all VM embeddings in this subset , and outputs the specific VM selection distribution, expressed as: Among them, represents the probability that the second-layer agent selects the th specific VM under the state input of the current subtask and its candidate VM subset under the corresponding VM category; is the GNN embedding set of all VMs corresponding to the category selected in the first layer for the subtask ; represents the vector obtained by sequentially concatenating the embedding of the subtask and K candidate VMs under this category, which is used as the input of the second-layer Actor; is the learnable weight matrix of the second-layer Actor, which maps the input features to the K-dimensional VM category space.​​ 6. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 5, characterized in that, The reward calculation based on the processing result of the mapping task includes setting the reward as a weighted sum , where: , and are the relative weights of the three physical quantities of the makespan, total energy consumption, and resource cost respectively, , and are the relative weights of the three physical quantities, usually dynamically adjusted according to the task SLA strictness, and the sum of the three is 1; the output of the designed Critic network is a three-dimensional value function , the three-dimensional value function predicts the future cumulative return starting from the current state respectively, and there is the following relationship expressed as: ， Wherein: is the system state of the t-th scheduling time slot, which is composed of GNN embeddings and other features; is the discount factor, which is used to control the attenuation degree of future rewards; , , respectively represent the current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling in time slot t.

7. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 6, characterized in that, The reward calculation based on the processing result of the mapping task further includes, for each virtual machine The total time required to execute all the subtasks assigned to a virtual machine includes the calculation time and the communication time. Among them, the execution time calculation method is , where: Indicates the subtask Whether it is mapped to a virtual machine ; Is the computational workload of the subtask ; Is the processing rate of the virtual machine . The communication time calculation method is , where Indicates whether the subtasks l and k of the same task Are mapped to different virtual machines. If they are mapped to the same virtual machine, there is no need for communication Represents the amount of data that needs to be exchanged between subtasks Represents the network bandwidth between the two 8. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 7, characterized in that, The reward calculation based on the processing result of the mapping task further includes calculating the total energy consumption based on two parts: execution energy consumption and communication energy consumption. Among them, the calculation method of execution energy consumption is , where is the power consumption of the virtual machine at full load; the calculation method of communication energy consumption is , where is the average power consumption of network transmission; then the total energy consumption is .

9. The graph-enhanced two-level architecture A3C scheduling method for tasks in a multi-cloud environment according to claim 8, characterized in that, The reward calculation based on the processing result of the mapping task further includes, due to multiple charging modes supported by different virtual machines in a multi-cloud environment, making , and be the charging amounts per unit time of the virtual machine under three modes respectively, , and be the weights corresponding to different modes, then the resource cost is calculated as .

10. A graph-enhanced two-level architecture A3C scheduling system for tasks in a multi-cloud environment, characterized in that, including: A data acquisition module, configured to obtain cloud user task data; A heterogeneous scheduling module, configured to construct a heterogeneous scheduling graph based on the obtained cloud user task data; An embedding calculation module, configured to perform GNN embedding calculation based on the heterogeneous scheduling graph; A mapping module, configured to design a hierarchical scheduling architecture based on the GNN embedding calculation to process mapping tasks with different intensities respectively; A reward module, configured to calculate rewards according to the processing results of the mapping tasks; A policy module, configured to output the optimal policy.

Images