A graph-enhanced two-level architecture A3C scheduling method and system for multi-cloud environment tasks

Abstract

The present invention relates to the field of cloud computing technology, and in particular to a graph-enhanced two-level architecture A3C scheduling method and system for multi-cloud environment tasks. The method comprises: obtaining cloud user task data; constructing a heterogeneous scheduling graph based on the obtained cloud user task data; performing GNN embedding calculation based on the heterogeneous scheduling graph; designing a hierarchical scheduling architecture based on the GNN embedding calculation to separately process mapping tasks of different intensities; performing reward calculation based on the processing results of the mapping tasks; and outputting the optimal strategy. This solution is further refined in the energy consumption and billing model. The model not only considers the communication energy consumption of subtasks, but also supports hierarchical billing strategies for on-demand, reserved, and bidding instances, ensuring that scheduling decisions are closer to the energy consumption and cost structure of the actual cloud platform.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a graph-enhanced two-level architecture A3C scheduling method and system for multi-cloud environment tasks. Background Art

[0002] In multi-cloud environments, task scheduling is extremely complex due to factors such as resource heterogeneity, network latency, and diverse billing models. Common scheduling approaches include combining metaheuristic algorithms with machine learning or deep learning techniques, such as the Cat Swarm Optimization (CSO) with deep neural networks, the Ant Colony Optimization (ACO) with Q-Learning, the Particle Swarm Optimization (PSO) with neural networks, and hybrids of ant colonies with deep reinforcement learning, genetic algorithms with deep reinforcement learning. These approaches leverage heuristic search to quickly obtain optimal solutions and use learning models to predict and optimize historical scheduling results. Furthermore, the HEFT (Heterogeneous Earliest Finish Time) algorithm and its variants have also been widely studied, such as combining HEFT with Q-Learning, decision trees, or gravitational search algorithms to automatically adjust their priority strategies. Pure deep reinforcement learning methods, such as DQN, A3C, and PPO, provide end-to-end adaptive scheduling strategies for multi-cloud scheduling by constructing parallel threads or continuous action spaces.

[0003] However, these approaches also have significant shortcomings. While meta-heuristic hybrid learning methods perform well on offline tasks, their heuristic parameter tuning is cumbersome, model convergence is slow, and they lack adaptability to sudden online loads. HEFT and its variants, due to their reliance on fixed heuristic rules, struggle to incorporate historical scheduling experience and are prone to performance fluctuations in highly dynamic environments. Pure deep reinforcement learning methods face the problem of state-action space explosion. Even a slight increase in the number of tasks, virtual machines, and physical machines leads to a dramatic increase in network size, significantly increasing training and inference overhead. Summary of the Invention

[0004] In order to solve the above-mentioned problems, the present invention provides a graph-enhanced two-level architecture A3C scheduling method and system for multi-cloud environment tasks.

[0005] In the first aspect, the present invention provides a graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks, which adopts the following technical solutions:

[0006] A graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks includes:

[0007] Get cloud user task data;

[0008] Build a heterogeneous scheduling graph based on the acquired cloud user task data;

[0009] GNN embedding calculation based on heterogeneous scheduling graph;

[0010] Design a hierarchical scheduling architecture based on GNN embedded computing to handle mapping tasks of different strengths;

[0011] Calculate rewards based on the results of the mapping task;

[0012] Output the optimal strategy.

[0013] Furthermore, the heterogeneous scheduling graph is constructed based on the acquired cloud user task data, including tasks provided by cloud users. , each task Divided into several subtasks , the length, data block size, deadline, etc. of each subtask are represented by the feature vector Description, all available virtual machines are recorded as , the processing power, memory and network bandwidth of each virtual machine are represented by the feature vector Description, physical nodes are recorded as , the subtasks, virtual machines and physical machines are organized into a three-layer heterogeneous graph node set as , edge set It includes subtask-virtual machine mapping candidate edges, VM-PM attribution edges, and data dependency edges between subtasks under the same task.

[0014] Furthermore, the GNN embedding calculation based on the heterogeneous scheduling graph includes the graph structure information and The low-dimensional state representation is obtained by calculating several layers of graph neural network GNN, which is expressed as:

[0015] ,

[0016] in: Representation node No. The hidden state of the layer, when When it is 0, define ; Represents all nodes A set of directly connected neighbor nodes, where the edge types are distinguished by different weight matrices or attention mechanisms; For the The learnable weight matrix of the layer is used to learn the weights of the neighboring nodes The hidden state Perform linear transformation; For the The bias vector of the layer; It is an element-by-element nonlinear activation function used to enhance the expressive power of the model.

[0017] Furthermore, the hierarchical scheduling architecture designed based on GNN embedding calculation processes mapping tasks of different strengths respectively, including implementing the category mapping of subtasks to virtual machines in the first-level scheduling, where the input state is the GNN embedding of all subtask nodes. , and common embeddings for the three virtual machine categories , and for each subtask Calculate the category selection distribution, take the maximum probability from the distribution, and get the category decision , expressed as:

[0018]

[0019] in, Indicates that the first-layer agent selects the VM category in the current subtask state The probability distribution of For subtask nodes The d-dimensional hidden representation obtained by GNN, are predefined category embedding vectors representing three VM categories, The vector obtained by sequentially concatenating the subtask representation and the three category representations is used as the input feature of the first layer Actor. is the learnable weight matrix of the first layer Actor, which linearly maps the input features to the category space.

[0020] Furthermore, the hierarchical scheduling architecture designed based on GNN embedded computing processes mapping tasks of different strengths respectively, and also includes using the second layer to realize the mapping of subtasks to specific VMs, wherein for each subtask node , according to the output of the first layer , narrowing the candidate VM set to categories VM subset under , Actor network L2 receives subtask embedding All VMs with this subset are embedded , and output the specific VM selection distribution, expressed as:

[0021]

[0022] in, Indicates that the second-layer agent is in the current subtask and the candidate VM subsets under their VM categories Under Status Input, select The probability of a specific VM; For subtasks The set of GNN embeddings of all VMs corresponding to the selected categories in the first layer; Represents the vector obtained by sequentially concatenating the embedding of the subtask and the K candidate VMs under this category, which serves as the input of the second layer Actor; is the learnable weight matrix of the second-layer Actor, which maps the input features to the K-dimensional virtual machine category space.

[0023] Furthermore, the reward calculation is performed based on the processing result of the mapping task, including setting the reward as a weighted sum ,in: 、 and are the relative weights of the three physical quantities of maximum completion time, total energy consumption and resource cost, 、 and is the relative weight of the three physical quantities, which is usually adjusted dynamically according to the strictness of the task SLA, and the sum of the three is 1; the output of the critic network is designed to be a three-dimensional value function , the three-dimensional value function predicts the future cumulative returns starting from the current state, and the following relationship exists:

[0024] ,

[0025] in: is the system state at the t-th scheduling time slot, which is composed of GNN embedding and other features; is the discount factor used to control the degree of attenuation of future returns; 、 、 They represent the current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling is completed in time slot t.

[0026] Furthermore, the reward calculation based on the processing result of the mapping task also includes for each virtual machine The total time required to complete all assigned subtasks includes computation time and communication time, where the execution time is calculated as ,in: Instruction subtask Whether to map to a virtual machine ; For subtasks The amount of calculation; For virtual machines The processing rate of communication time is calculated as ,in Instructing the same task Whether subtasks l and k are mapped to different virtual machines, if they are mapped to the same virtual machine, no communication is required. Indicates the amount of data that needs to be exchanged between subtasks, Indicates the network bandwidth between the two.

[0027] Furthermore, the reward calculation based on the processing result of the mapping task also includes calculating the total energy consumption based on the execution energy consumption and the communication energy consumption, wherein the execution energy consumption calculation method is ,in For virtual machines Power consumption at full load; communication energy consumption is calculated as follows: ,in is the average power consumption of network transmission; the total energy consumption is .

[0028] Furthermore, the reward calculation based on the processing result of the mapping task also includes supporting multiple billing modes based on different virtual machines in a multi-cloud environment. 、 and and virtual machines respectively The unit time billing amount in three modes, 、 and Corresponding to the weights of different modes, the resource cost The calculation method is .

[0029] Secondly, a graph-enhanced two-level architecture A3C scheduling system for multi-cloud environment tasks includes:

[0030] The data acquisition module is configured to acquire cloud user task data;

[0031] The heterogeneous scheduling module is configured to construct a heterogeneous scheduling graph based on the acquired cloud user task data;

[0032] The embedding calculation module is configured to perform GNN embedding calculation based on the heterogeneous scheduling graph;

[0033] The mapping module is configured to design a hierarchical scheduling architecture based on GNN embedding calculation to handle mapping tasks of different strengths;

[0034] The reward module is configured to calculate a reward based on the processing result of the mapping task;

[0035] The policy module is configured to output the optimal policy.

[0036] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks.

[0037] In a fourth aspect, the present invention provides a terminal device comprising a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor. A graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks.

[0038] In summary, the present invention has the following beneficial technical effects:

[0039] This invention uses graph neural networks (GNNs) to reduce the dimensionality of the relationships between physical machines, virtual machines, and tasks, embedding high-dimensional states into a low-dimensional feature space. Secondly, a hierarchical decision-making mechanism is designed. The upper-level agent is responsible for selecting the VM category (e.g., high-performance, general-purpose, or low-power) to schedule subtasks, while the lower-level agents carefully select specific VMs within this category, significantly compressing the state-action space. Thirdly, for multi-objective optimization problems, a multi-critic architecture is designed to separately evaluate latency, energy consumption, and resource costs, dynamically adjusting weights based on the urgency of the task's SLA to achieve an adaptive balance of immediate rewards. Finally, this solution further refines the energy consumption and billing model. This model not only considers subtask communication energy consumption but also supports tiered billing strategies for on-demand, reserved, and competitive bidding instances, ensuring that scheduling decisions more closely align with the energy consumption and cost structure of real-world cloud platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of a graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks according to Example 1 of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described in detail below with reference to the accompanying drawings.

[0042] Example 1

[0043] Reference Figure 1 In this embodiment, a graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks includes:

[0044] Get cloud user task data;

[0045] Build a heterogeneous scheduling graph based on the acquired cloud user task data;

[0046] GNN embedding calculation based on heterogeneous scheduling graph;

[0047] Design a hierarchical scheduling architecture based on GNN embedded computing to handle mapping tasks of different strengths;

[0048] Calculate rewards based on the results of the mapping task;

[0049] Output the optimal strategy.

[0050] Specifically:

[0051] 1. Build a heterogeneous scheduling graph,

[0052] Assume that the task provided by the cloud user is , each task Divided into several subtasks , the length, data block size, deadline, etc. of each subtask are represented by the feature vector Description. All available virtual machines are recorded as , the processing power, memory and network bandwidth of each virtual machine are represented by the feature vector Description. Physical nodes are recorded as , whose power consumption model, geographical location and other characteristics are determined by Representation. The cross-region data center set is , used to calculate the cross-center communication cost and delay. For each node in the graph , initial features It is obtained by combining task length, data block size, VM processing rate or physical machine energy consumption.

[0053] Subtasks, virtual machines and physical machines are organized into a three-layer heterogeneous graph node set as , edge set It includes subtask-virtual machine mapping candidate edges, VM-PM attribution edges, and data dependency edges between subtasks under the same task.

[0054] 2. GNN embedding calculation,

[0055] Graph structure information and The low-dimensional state representation is obtained by calculating several layers of graph neural network GNN:

[0056] ,

[0057] in: Representation node No. The hidden state of the layer, when When it is 0, define ; Represents all nodes The set of directly connected neighbor nodes, where the edge types (e.g., subtask-VM, VM-PM, or data dependency) can be distinguished by different weight matrices or attention mechanisms; For the The learnable weight matrix of the layer is used to learn the weights of the neighboring nodes The hidden state Perform linear transformation; For the The bias vector of the layer; It is an element-by-element nonlinear activation function used to enhance the expressive power of the model.

[0058] 3. Hierarchical scheduling architecture,

[0059] This algorithm adopts a hierarchical scheduling architecture, which is responsible for mapping tasks of different strengths to effectively compress the state-action space and improve scheduling efficiency.

[0060] (1) First-level scheduling,

[0061] The first layer implements the category mapping from subtask to virtual machine. The input state is the GNN embedding of all subtask nodes. , and three virtual machine categories (high performance (number 1), general (number 2), low power (number 3)) public embedding Actor network 1 receives the above embedding and performs a Calculate the category selection distribution. Take the maximum probability from the distribution to get the category decision .

[0062]

[0063] in, Indicates that the first-layer agent selects the VM category in the current subtask state The probability distribution of . For subtask nodes The d-dimensional hidden representation obtained by GNN combines its own attributes and neighborhood information. It is a pre-defined category embedding vector representing three VM categories, which is used to provide category semantic information to the network. The vector obtained by sequentially concatenating the subtask representation and the three category representations is used as the input feature of the first layer Actor. is the learnable weight matrix of the first layer Actor, which linearly maps the input features to the category space. Softmax normalizes the vector of length 3 and outputs a legal probability distribution such that .

[0064] (2) Second-level scheduling,

[0065] The second layer implements the mapping of subtasks to specific VMs. For each subtask node , according to the output of the first layer , narrowing the candidate VM set to categories VM subset under Actor network L2 receives subtask embedding All VMs with this subset are embedded , and output the specific VM selection distribution:

[0066]

[0067] in, Indicates that the second-layer agent is in the current subtask and the candidate VM subsets under their VM categories Under Status Input, select The probability of a specific VM; For subtasks The set of GNN embeddings for all VMs corresponding to the selected category in the first layer. It represents the vector obtained by sequentially concatenating the embedding of the subtask and the K candidate VMs under this category, which serves as the input of the second-layer Actor. is the learnable weight matrix of the second-layer actor, mapping the input features to a K-dimensional VM category space. Softmax normalizes the K-length vector and outputs a probability distribution over the K candidate VMs, ensuring that the sum of the probabilities of selecting a specific VM is 1.

[0068] 4. Reward calculation,

[0069] Set the reward as a weighted sum ,in: 、 and are the relative weights of the three physical quantities of maximum completion time, total energy consumption and resource cost, 、 and The relative weights of the three physical quantities are usually adjusted dynamically according to the strictness of the task SLA, and the sum of the three is 1. The output of the Critic network is designed to be a three-dimensional value function The three-dimensional value function predicts the future cumulative returns starting from the current state, and the following relationship exists. is the system state at the t-th scheduling time slot, which is composed of GNN embedding and other features; is the discount factor used to control the degree of attenuation of future returns; 、 、 They represent the current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling is completed in time slot t.

[0070] ,

[0071] (1) Maximum completion time ,

[0072] For each virtual machine The total time required to complete all subtasks assigned to it includes computing time and communication time. The execution time is calculated as ,in: Instruction subtask Whether to map to a virtual machine ; For subtasks The amount of calculation; For virtual machines The communication time is calculated as ,in Instructing the same task Whether subtasks l and k are mapped to different virtual machines, if they are mapped to the same virtual machine, no communication is required. Indicates the amount of data that needs to be exchanged between subtasks, represents the network bandwidth between the two. Then the completion time of each VM is Furthermore, the MSP of the system is defined as the maximum completion time among all VMs .

[0073] (2) Total energy consumption ,

[0074] The total energy consumption includes execution energy consumption and communication energy consumption. The execution energy consumption is calculated as follows: ,in For virtual machines The power consumption at full load. The communication energy consumption is calculated as ,in is the average power consumption of network transmission. The total energy consumption is .

[0075] (3) Total resource cost ,

[0076] Different virtual machines in a multi-cloud environment support multiple billing modes (on-demand, reserved, and bidding). 、 and and virtual machines respectively The unit time billing amount in three modes, 、 and Corresponding to the weights of different modes, the resource cost The calculation method is:

[0077] .

[0078] 5. Output the optimal strategy,

[0079] (1) Computational advantages,

[0080] After the reward calculation is completed, each worker is combined with the time slot The instant rewards obtained , and Critic's assessment of the current state of the environment , the next environmental state The value function estimate of 、 , computing advantage. Among them, In state The actions that are executed by the Actor network data are downloaded, that is, the category of subtasks and the selection of specific VMs in hierarchical scheduling. It is a discount factor used to attenuate the impact of future returns on current decisions. The closer the value is to 1, the more emphasis is placed on long-term returns.

[0081]

[0082] (2) Local gradient accumulation,

[0083] The gradient of the Actor. are the learnable parameters of the Actor network, including all weights and biases in the two-layer scheduling network. For the Actor network, under Parameters, set the action Given state probability. For the parameters The gradient of , clearly shows the effect of parameter changes on action probability.

[0084]

[0085] The gradient of the Critic network. is the learnable parameter of the Critic network. The current state of the Critic network The value function estimate of . The target value constructed based on the immediate reward at time t and the next state estimate. It is the square of the temporal difference error and is used as the loss function of Critic.

[0086]

[0087] (3) Hybrid synchronous-asynchronous parameter update,

[0088] The gradients of the actor and critic output layers are centralized to the parameter server or main thread, and a single gradient descent / ascent update is performed simultaneously. Each worker can asynchronously apply small-step updates to the GNN layer and hidden layer parameters locally, and periodically perform a "gossip"-style partial synchronization of these parameters with the global model.

[0089] (4) New parameter broadcast,

[0090] After completing the global parameter update, the latest Actor and Critic network parameters are distributed back to each Worker so that the next round of decisions are based on the latest model.

[0091] (5) Convergence judgment,

[0092] Check whether the change in the policy or value function, the sliding average of the reward, or the maximum number of training rounds has reached the preset threshold. If convergence has not yet occurred, return to "Build Heterogeneous Scheduling Graph → GNN Embedding Computation" to start a new round of iteration; otherwise, proceed to the next step.

[0093] (6) Output strategy,

[0094] When the convergence conditions are met, fix the Actor network parameters ,This network represents the optimal task-VM hierarchical scheduling strategy.,During deployment, simply input subtasks and resource status, and,then output the optimal VM category and specific VM allocation,decision.

[0095] Example 2

[0096] This embodiment provides a graph-enhanced two-level architecture A3C scheduling system for multi-cloud environment tasks, including:

[0097] The data acquisition module is configured to acquire cloud user task data;

[0098] The heterogeneous scheduling module is configured to construct a heterogeneous scheduling graph based on the acquired cloud user task data;

[0099] The embedding calculation module is configured to perform GNN embedding calculation based on the heterogeneous scheduling graph;

[0100] The mapping module is configured to design a hierarchical scheduling architecture based on GNN embedding calculation to handle mapping tasks of different strengths;

[0101] The reward module is configured to calculate a reward based on the processing result of the mapping task;

[0102] The policy module is configured to output the optimal policy.

[0103] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks.

[0104] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded and executed by the processor. A graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks is described.

[0105] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks, characterized by: include: Get cloud user task data; Build a heterogeneous scheduling graph based on the acquired cloud user task data; GNN embedding calculation based on heterogeneous scheduling graph; Design a hierarchical scheduling architecture based on GNN embedded computing to handle mapping tasks of different strengths; Calculate rewards based on the results of the mapping task; Output the optimal strategy; The heterogeneous scheduling graph is constructed based on the acquired cloud user task data, including tasks provided by cloud users. , each task Divided into several subtasks , the length, data block size, and deadline of each subtask are represented by the feature vector Description, all available virtual machines are recorded as , the processing power, memory and network bandwidth of each virtual machine are represented by the feature vector Description, physical nodes are recorded as , the subtasks, virtual machines and physical machines are organized into a three-layer heterogeneous graph node set as , edge set Includes subtask-virtual machine mapping candidate edges, VM-PM attribution edges, and data dependency edges between subtasks under the same task; The GNN embedding calculation based on the heterogeneous scheduling graph includes the graph structure information and The low-dimensional state representation is obtained by calculating several layers of graph neural network GNN, which is expressed as: ， in: Representation node No. The hidden state of the layer, when When it is 0, define ; Represents all nodes A set of directly connected neighbor nodes, where the edge types are distinguished by different weight matrices or attention mechanisms; For the The learnable weight matrix of the layer is used to learn the weights of the neighboring nodes The hidden state Perform linear transformation; For the The bias vector of the layer; It is an element-by-element nonlinear activation function used to enhance the expressive power of the model; The hierarchical scheduling architecture designed based on GNN embedding calculation processes mapping tasks of different strengths respectively, including implementing the category mapping of subtasks to virtual machines in the first-level scheduling, where the input state is the GNN embedding of all subtask nodes. , and common embeddings for the three virtual machine categories , and for each subtask Calculate the category selection distribution, take the maximum probability from the distribution, and get the category decision , expressed as: ， in, Indicates that the first-layer agent selects the VM category in the current subtask state The probability distribution of For subtask nodes Obtained through GNN d dimensional hidden representation, are predefined category embedding vectors representing three VM categories, The vector obtained by sequentially concatenating the subtask representation and the three category representations is used as the input feature of the first layer Actor. is the learnable weight matrix of the first layer Actor, which linearly maps the input features to the category space; The hierarchical scheduling architecture designed based on GNN embedded computing processes mapping tasks of different strengths respectively, and also includes using the second layer to realize the mapping of subtasks to specific VMs, wherein for each subtask node , according to the output of the first layer , narrowing the candidate VM set to categories VM subset under , Actor network L2 receives subtask embedding All VMs with this subset are embedded , and output the specific VM selection distribution, expressed as: ， in, Indicates that the second-layer agent is in the current subtask and the candidate VM subsets under their VM categories Under Status Input, select The probability of a specific VM; For subtasks The set of GNN embeddings of all VMs corresponding to the selected categories in the first layer; Indicates that the subtask is associated with the category K The vector obtained by sequentially concatenating the embeddings of the candidate VMs is used as the input of the second layer Actor; is the learnable weight matrix of the second layer Actor, mapping the input features to K Dimensional virtual machine category space; The reward calculation is performed based on the processing results of the mapping task, including setting the reward as a weighted sum ,in: 、 and are the relative weights of the three physical quantities of maximum completion time, total energy consumption and resource cost, 、 and is the relative weight of the three physical quantities, which is usually adjusted dynamically according to the strictness of the task SLA, and the sum of the three is 1; the output of the critic network is designed to be a three-dimensional value function , the three-dimensional value function predicts the future cumulative returns starting from the current state, and the following relationship exists: , in: For the t The system state for each scheduling time slot is composed of GNN embedding and other features; is the discount factor used to control the degree of attenuation of future returns; 、 、 Respectively represent the time slot t The current maximum completion time, total energy consumption, and total resource cost measured after the top-level scheduling is completed; The reward calculation is performed based on the processing result of the mapping task, and also includes for each virtual machine The total time required to complete all assigned subtasks includes computation time and communication time. The execution time is calculated as follows: ， in: Instruction subtask Whether to map to a virtual machine ; For subtasks The amount of calculation; For virtual machines The processing rate of communication time is calculated as follows: ， in Instructing the same task Subtasks l and k Whether they are mapped to different virtual machines, if they are mapped to the same virtual machine, no communication is required. Indicates the amount of data that needs to be exchanged between subtasks, Indicates the network bandwidth between the two; The reward calculation based on the processing result of the mapping task also includes calculating the total energy consumption based on the execution energy consumption and the communication energy consumption. The execution energy consumption is calculated as follows: ， in For virtual machines Power consumption at full load; communication energy consumption is calculated as follows: ,in is the average power consumption of network transmission; the total energy consumption is ; The reward calculation is performed based on the processing results of the mapping task, and also includes supporting multiple billing modes based on different virtual machines in a multi-cloud environment. 、 and and virtual machines respectively The unit time billing amount in three modes, 、 and Corresponding to the weights of different modes, the resource cost The calculation method is: 。 2. A graph-enhanced two-level architecture A3C scheduling system for multi-cloud environment tasks, executing the graph-enhanced two-level architecture A3C scheduling method for multi-cloud environment tasks according to claim 1, characterized in that: include: The data acquisition module is configured to acquire cloud user task data; The heterogeneous scheduling module is configured to construct a heterogeneous scheduling graph based on the acquired cloud user task data; The embedding calculation module is configured to perform GNN embedding calculation based on the heterogeneous scheduling graph; The mapping module is configured to design a hierarchical scheduling architecture based on GNN embedding calculation to handle mapping tasks of different strengths; The reward module is configured to calculate a reward based on the processing result of the mapping task; The policy module is configured to output the optimal policy.

Images