Multi-data center green electricity optimization scheduling method and device based on deep reinforcement learning

Abstract

The application provides a multi-data center green electricity optimization scheduling method and device based on deep reinforcement learning, and belongs to the technical field of intelligent scheduling. The method provided by the application comprises the following steps: modeling resources of multi-data center nodes, determining resource parameters of the nodes and output characteristics of renewable energy sources varying with time; modeling a computing task with a hierarchical structure, and decomposing the computing task into hierarchical task units for migration across data centers; based on the resource modeling result and the computing task modeling result, constructing a multi-data center simulation environment, and generating an input state vector of an agent; designing an action space of the agent; designing a reward function; based on a proximal policy optimization algorithm, carrying out multi-round training in the simulation environment, combining the action space and the reward function, taking the input state vector as input, and iteratively updating a scheduling strategy until the scheduling strategy meets a standard in a comprehensive performance index; and applying the scheduling strategy meeting the standard to a real multi-data center environment to optimize a scheduling task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent scheduling, in particular to a multi-data center green electricity optimization scheduling method and device based on deep reinforcement learning. BACKGROUND

[0002] With the rapid development of cloud computing, big data and artificial intelligence, large-scale data centers have become the key infrastructure to support computationally intensive tasks such as model inference and image recognition. Cross-regional multi-data center architecture is widely adopted because it can achieve algorithm redundancy and business disaster recovery. However, this architecture has huge overall power consumption and is highly dependent on traditional power grids, resulting in high operating costs and high carbon emission pressure. At the same time, to improve green energy utilization, the industry has access to renewable energy such as photovoltaic, wind power and hydropower. However, due to natural conditions and scheduling constraints, renewable energy is volatile, random and regionally different. This results in an uneven distribution of green electricity in the time and space dimensions of multi-data centers. When there is a surplus of green electricity, it is not fully utilized, and when there is a shortage, it is still running at high load. Therefore, optimizing the green electricity of multi-data centers has become a key requirement to reduce dependence on traditional power grids, improve green electricity utilization, and alleviate carbon emission pressure.

[0003] Currently, the industry's scheduling methods mainly fall into two categories: one is based on static rules, load balancing or simple heuristics. Traditional solutions use CPU utilization, queue length, geographic location and other indicators to allocate tasks and select nodes. The other is related research that introduces deep reinforcement learning, focusing on virtual machine allocation or single-data center energy consumption control. However, existing methods have obvious shortcomings: traditional solutions treat data centers as a unified power load, lack detailed modeling of green electricity output curves, and rely on human experience to set rules, lacking adaptive learning ability. They cannot make precise scheduling based on "where is the green electricity abundant and when is the green electricity abundant." Reinforcement learning-related research is mostly limited to single-data centers or simplified scenarios, with optimization goals biased towards a single indicator. They do not fully consider the structural characteristics of computing tasks, which are executed forward by layer and can be migrated in segments. The scheduling granularity is coarse, making it difficult to achieve multi-dimensional collaborative optimization of green electricity utilization, task latency and network congestion. The core problem of these issues is that the multi-data center environment is dynamically coupled and complex, making it difficult for traditional methods to design scheduling rules that take into account multiple objectives. The deep integration of green electricity output, task structure and scheduling decisions presents a technical threshold, making existing technologies unable to meet the comprehensive optimization needs of multi-data center scenarios.

[0004] Therefore, there is an urgent need for a method to provide intelligent scheduling strategies to improve green electricity utilization, reduce operating costs and ensure service quality. SUMMARY

[0005] Therefore, the application provides a multi-data center green electricity optimization scheduling method and device based on deep reinforcement learning to provide an intelligent scheduling strategy and improve green electricity utilization, reduce operation cost and guarantee service quality.

[0006] Specifically, the application is implemented through the following technical solutions.

[0007] The first aspect of the application provides a multi-data center green electricity optimization scheduling method based on deep reinforcement learning, which comprises the following steps:

[0008] Resource modeling is performed on the nodes of the multi-data center to determine the resource parameters of each node and the output characteristics of the renewable energy accessed by each node over time.

[0009] The computing task with a hierarchical structure is modeled, and the computing task is decomposed into hierarchical task units that are migrated across data centers.

[0010] Based on the resource modeling results and the computing task modeling results, a multi-data center simulation environment is constructed, and an input state vector of an agent is generated.

[0011] The action space of the agent is designed, and the action is defined as the joint decision of the selection of the target data center node and the number of task layers continuously executed in the target data center node.

[0012] A reward function is designed, which takes the output characteristics of the renewable energy, the running state of the simulation environment and the feedback after the action is executed as the evaluation basis to evaluate the scheduling decisions of the agent from multiple dimensions.

[0013] Based on the proximal policy optimization algorithm, multiple rounds of training are carried out in the simulation environment with the input state vector as the input, combined with the action space and the reward function, the scheduling strategy is iteratively updated until the scheduling strategy meets the standard in terms of comprehensive performance indicators.

[0014] The scheduling strategy that meets the standard is applied to the real multi-data center environment to optimize the scheduling task.

[0015] The second aspect of the application provides a multi-data center green electricity optimization scheduling device based on deep reinforcement learning, which comprises a modeling module, a construction module, a design module, a training module and an optimization module.

[0016] The modeling module is used to perform resource modeling on the nodes of the multi-data center to determine the resource parameters of each node and the output characteristics of the renewable energy accessed by each node over time.

[0017] The modeling module is also used to model the computing task with a hierarchical structure, and decompose the computing task into hierarchical task units that are migrated across data centers.

[0018] the construction module is configured to construct a multi-data center simulation environment based on the resource modeling result and the computing task modeling result, and generate an input state vector of the agent;

[0019] the design module is configured to design an action space of the agent, and an action is defined as a joint decision of a target data center node and a number of layers of tasks to be continuously executed at the target data center node;

[0020] the design module is further configured to design a reward function, and the reward function takes the output characteristics of the renewable energy, the running state of the simulation environment and the feedback after the action is executed as evaluation bases to evaluate the scheduling decision of the agent from multiple dimensions;

[0021] the training module is configured to perform multiple rounds of training in the simulation environment based on a proximal policy optimization algorithm, take the input state vector as an input, combine the action space and the reward function, iteratively update the scheduling policy until the scheduling policy meets a standard in a comprehensive performance index;

[0022] the optimization module is configured to apply the scheduling policy that meets the standard to a real multi-data center environment to optimize the scheduling task.

[0023] The method and device for green electricity optimization scheduling of multiple data centers based on deep reinforcement learning provided by the application, through sequentially setting the whole process steps of multiple data center node resource modeling, hierarchical structure calculation task modeling, simulation environment construction and state vector generation, action space design, reward function design, strategy training based on PPO algorithm and real environment application, a closed-loop deep reinforcement learning scheduling scheme is constructed. Firstly, by constructing a heterogeneous resource model of multiple data centers and a model of renewable energy output changing with time, the green electricity output information is explicitly introduced into the task scheduling decision process, which can arrange more computing tasks to be executed in the nodes and time periods with sufficient green electricity under the premise of guaranteeing business demand, significantly improve the utilization rate of renewable energy, reduce the dependence on traditional power supply, and reduce carbon emissions and electricity cost; at the same time, by dividing the neural network inference task into fine-grained hierarchical task units and supporting the segmented migration of the task during execution, the task execution path can be flexibly adjusted between multiple data centers, the execution location can be dynamically selected according to the real-time computing power and green electricity state of each node, and more precise and flexible scheduling control is realized, which is superior to the traditional scheme which only supports whole task migration or single node execution; by designing a reward function which comprehensively considers multiple dimensional factors and using PPO deep reinforcement learning algorithm for strategy training, the scheduling strategy of the application can automatically weigh between multiple objectives and obtain better latency-energy balance performance, which has better adaptability and global optimization ability than the scheduling strategy based on fixed rules or heuristic method; finally, by training and evaluating the scheduling strategy in the simulation environment and then migrating to the real environment, parameter adjustment and strategy improvement can be carried out without affecting the actual business, which has good engineering realizability and popularization and application value. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The flowchart of the method for green electricity optimization scheduling of multiple data centers based on deep reinforcement learning provided by the embodiment one of the application;

[0025] Figure 2 The structural schematic diagram of the device for green electricity optimization scheduling of multiple data centers based on deep reinforcement learning provided by the embodiment two of the application. DETAILED DESCRIPTION

[0026] The exemplary embodiments will be described in detail hereinbelow with reference to the drawings. When the following description refers to the drawings, identical numbers on different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0027] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this application, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0028] It is to be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to differentiate one piece of information from another. For example, a first information can also be termed a second information, similarly, a second information can also be termed a first information, without departing from the scope of the present application. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining".

[0029] The specific embodiments are given below to detail the technical solutions of the present application.

[0030] Figure 1 A flowchart of the method for multi-data center green electricity optimization scheduling based on deep reinforcement learning provided by Embodiment One of the present application is shown in FIG. 1. Please refer to Figure 1 The method provided by the present embodiment can include:

[0031] S101, model the resources of the multi-data center nodes, determine the resource parameters of each node and the output characteristics of the renewable energy accessed by each node over time.

[0032] Specifically, the multi-data center nodes refer to each independent data center unit that is geographically distributed in different physical locations and constitutes a cross-regional multi-data center architecture. These nodes are the core carriers that carry out computing task execution, power consumption and renewable energy access. Each node is an independent scheduling object, and its hardware configuration, energy supply type and running state can all be different, so it needs to be modeled and monitored separately.

[0033] Further, the resource parameters are core quantitative indicators that represent the hardware capabilities, network connection capabilities and energy configuration basis of the multi-data center nodes, and specifically include node computing power related parameters, power configuration related parameters and network connection related parameters. Among them, the node computing power related parameters include equivalent computing power (measuring the hardware performance of the node in processing computing tasks); the power configuration related parameters include the upper limit of the rated power (the maximum power consumption threshold allowed by the node when running), the installed capacity proportion of renewable energy (the proportion of green power capacity accessed by the node in the total power supply capacity); the network connection related parameters include the upper limit of the link bandwidth between the node and other data center nodes (representing the maximum transmission capacity of data migration between nodes).

[0034] The output characteristic refers to the power supply change rule of renewable energy such as photovoltaic and wind power accessed by the node in the time dimension, and is mainly embodied in the volatility, randomness and regional difference of green power supply. The characteristic is quantitatively represented by a time-output function (normalized green power output curve). The output characteristic of photovoltaic energy is affected by day and night alternation and weather changes, and presents a single-peak fluctuation rule that the output gradually rises to a peak in the daytime and then falls, and the output at night approaches zero. The output characteristic of wind power energy is a periodic fluctuation depending on wind speed, and the power output is related to the wind speed. The overall output fluctuates irregularly with the change of meteorological conditions.

[0035] In a specific implementation, resource basic parameters of each data center node are collected; the resource basic parameters include an upper limit of rated power of the node, a renewable energy installed capacity proportion, an equivalent computing capacity, and an upper limit of link bandwidth between different nodes; renewable energy types accessed by each data center node are identified; the renewable energy types include photovoltaic energy and wind power energy; for different types of renewable energy, corresponding time-output functions are respectively constructed; the time-output function represents the change rule of green power output of the node at different time periods in a day, and the time-output function is constructed according to the inherent characteristic difference of the corresponding renewable energy; the green power output of each node at different time is calculated in combination with the resource basic parameters and the time-output function.

[0036] Optionally, for different types of renewable energy, corresponding time-output functions are respectively constructed, including: when the renewable energy type is photovoltaic energy, a first time-output function is constructed by combining the characteristics affected by day and night and weather, and using a function with a single-peak distribution in the daytime; the first time-output function represents the change rule that the output of photovoltaic energy gradually rises to a peak in the daytime and then falls; when the renewable energy type is wind power energy, a second time-output function is constructed by combining the cubic correlation between power output and wind speed based on the periodic fluctuation characteristic of wind speed; the second time-output function represents the output fluctuation of wind power energy with the change of wind speed.

[0037] Specifically, for each node in the multi-data center architecture, four types of core resource parameters are collected: the node's rated power limit, the proportion of renewable energy installed capacity, equivalent computing power, and the link bandwidth limit between the node and other nodes. The specific type of renewable energy accessed by each data center node is confirmed, limited to photovoltaic (PV) energy and wind power. For each type of renewable energy, a corresponding time-output function is constructed. If PV energy is accessed, considering its susceptibility to day / night cycles and weather conditions, a first time-output function is constructed using a function exhibiting a unimodal distribution during the daytime to characterize the pattern of PV energy output rising to a peak and then falling back during the day. If wind power is accessed, a second time-output function is constructed based on the periodic fluctuation characteristics of wind speed and the cubic correlation between power output and wind speed to characterize the output fluctuation of wind power as wind speed changes. Furthermore, the product of the rated power limit and the proportion of renewable energy installed capacity from the collected node resource parameters is multiplied by the constructed corresponding type of time-output function, and the final result is used to determine the actual green power output of each data center node at different times.

[0038] For example, in one embodiment, the first time-output function can be expressed as:

[0039] ;

[0040] in, For a moment Photovoltaic power output; Photovoltaic energy conversion efficiency; Area required per unit capacity; For a moment Solar irradiance; The ambient temperature is (°C). To adapt to different installed capacities, the first time-output function can be normalized as follows:

[0041] ;

[0042] Based on this, the normalized photovoltaic power output coefficient is obtained. .

[0043] The second time-output function can be represented as a piecewise power curve:

[0044] ;

[0045] in, For a moment Wind power output; Wind speed; To cut in wind speed; Rated wind speed; To cut off the wind speed; The rated power; the second time-output function can be normalized as:

[0046] ;

[0047] To obtain the normalized output coefficient of wind power .

[0048] Further, the actual green power output of each data center node at different times can be represented as:

[0049] ;

[0050] Wherein, is the actual green power output of the data center node at time; is the rated power upper limit of the node; is the proportion of renewable energy installed capacity of the node; is the corresponding time-output function.

[0051] S102, model the computing task with hierarchical structure, and decompose the computing task into hierarchical task units that can be migrated across data centers.

[0052] Specifically, the hierarchical structure refers to the inherent structural characteristics of the computing task itself, which is executed in order in layers. The overall execution process of the computing task can be decomposed into multiple hierarchical steps associated in sequence. The intermediate data output after the execution of the previous hierarchical task is the input premise for the execution of the next hierarchical task. Each hierarchical task is sequentially advanced in a fixed order to jointly complete the overall computing goal.

[0053] Further, the hierarchical task unit is the smallest task execution fragment that can be independently executed and migrated across data centers after the computing task with hierarchical structure is split. Each hierarchical task unit contains two core parameters: one is the floating-point operation amount corresponding to the unit, which determines the computing resource required for task execution; the other is the intermediate activation data amount generated in the execution process of the unit, which determines the network transmission cost during cross-node migration. A single hierarchical task unit can be independently executed on a data center node, or multiple consecutive hierarchical task units can be combined into an execution fragment, which can be flexibly migrated between different data center nodes and completed according to the scheduling decision.

[0054] In a specific implementation, a target computing task with a hierarchical structure is screened, the target computing task being a computing task with a forward execution feature by layer; feature extraction is performed on the target computing task by an offline analysis tool to obtain floating point operation amounts and intermediate activation data amounts of each layer of the target computing task; based on the extracted floating point operation amounts and intermediate activation data amounts of each layer, a forward propagation process of the target computing task is converted into an ordered hierarchical task sequence; based on the computation amount and intermediate data amount of a hierarchical task unit, a migration adaptation rule is set, and a migration condition of a single hierarchical task unit or an execution fragment formed by combination of a plurality of continuous hierarchical task units is determined.

[0055] Optionally, each hierarchical task unit in the hierarchical task sequence corresponds to one layer or a plurality of continuous layers of the target computing task, and each hierarchical task unit includes a computation amount and an intermediate data amount, the computation amount corresponding to the floating point operation amount of each layer, and the intermediate data amount corresponding to the intermediate activation data amount of each layer.

[0056] Specifically, a computing task with a forward execution feature by layer is screened as a target computing task, and it is clear that the target computing task has a hierarchical structure that can be hierarchically decomposed. Feature extraction is performed on the target computing task by an offline analysis tool to obtain floating point operation amounts and intermediate activation data amounts corresponding to each layer of the target computing task. Further, based on the extracted floating point operation amounts and intermediate activation data amounts of each layer, a forward propagation process of the target computing task is converted into an ordered hierarchical task sequence; each hierarchical task unit corresponds to one layer or a plurality of continuous layers of the target computing task, and each hierarchical task unit includes a computation amount and an intermediate data amount, the computation amount corresponding to the floating point operation amount of each layer, and the intermediate data amount corresponding to the intermediate activation data amount of each layer. Based on the computation amount and intermediate data amount of the hierarchical task unit, a migration adaptation rule is set, and a migration condition of a single hierarchical task unit or an execution fragment formed by combination of a plurality of continuous hierarchical task units is determined, so that the hierarchical task unit supports cross-data-center migration.

[0057] S103, based on the resource modeling result and the computing task modeling result, a multi-data-center simulation environment is constructed, and an input state vector of the agent is generated.

[0058] Specifically, the multi-data-center simulation environment is a digital simulation platform built based on the resource modeling result and the task modeling result, and is used to simulate the running state and task execution process of a real multi-data-center. The multi-data-center simulation environment provides a controllable and repeatable interactive scene for training of a deep reinforcement learning agent. The input state vector is the core basis for decision-making of the agent, and is a quantitative integration result of multi-dimensional state information in the simulation environment.

[0059] Further, the agent refers to a decision-making subject constructed based on a deep reinforcement learning algorithm, and is a core execution unit of the scheduling method in the embodiment. It takes the state vector generated by the simulation environment as input, autonomously selects the scheduling decision (target node selection + continuous execution layer determination) according to the preset action space, and continuously optimizes the scheduling strategy through the feedback of the reward function, and finally realizes the multi-objective scheduling optimization of "green electricity utilization maximization, task time delay and network congestion minimization".

[0060] In a specific implementation, based on the resource modeling, the resource parameters of each data center node, the renewable energy time-output function, and the core parameters of the hierarchical task unit obtained by task modeling are used to build a multi-data center simulation environment. At any decision step of the simulation environment, the green power output of each node at the current time is calculated according to the current simulation time, combined with the time-output function and node resource parameters of the resource modeling. The load level and remaining computing power of each node are updated in combination with the task execution result of the previous decision step, and the network link bandwidth utilization is updated according to the data flow generated by task migration and the link bandwidth parameters between nodes. The number of hierarchical task units completed by the current computing task and the location of the data center node where the hierarchical task unit to be executed are recorded. The green power output, load level, remaining computing power, network link bandwidth utilization, and task progress information of each node are normalized according to the preset standard, and the processed data are spliced and integrated according to the preset format to form the input state vector of the agent.

[0061] Specifically, all data output in the resource modeling stage are called, including the rated power upper limit of each data center node, the renewable energy installation proportion, the equivalent computing capacity, the link bandwidth upper limit between nodes, and the renewable energy time-output function adapted to each node; at the same time, the core parameters of the hierarchical task unit output in the computing task modeling stage are called, including the calculation amount, intermediate data amount and migration adaptation rule of each hierarchical task unit. Based on the above two types of data, a multi-data center simulation environment is built. Further, at any decision step of the simulation environment running, the current simulation time is first obtained; then for each data center node, the renewable energy time-output function corresponding to the node is called, and the value of the function at the current simulation time is queried; then combined with the rated power upper limit and renewable energy installation proportion of the node, the actual green power output value of the node at the current time is obtained through data operation; finally, the green power output calculation of all data center nodes is completed, and the quantitative data set of the current green power supply capacity of each node is formed.

[0062] The scheduling action output by the agent in the last decision step and the task execution result are combined, the calculation amount of the hierarchical task unit completed by each node in the last decision step is counted, the current load level and the remaining computing power of each node are calculated and updated based on the equivalent computing power of the node; the intermediate data amount of the hierarchical task unit migrated across nodes in the last decision step is counted, the occupation of the link in the data transmission process is calculated combined with the upper limit of the bandwidth of the inter-node link, and then the current bandwidth utilization of each link is updated; the two types of updated data are summarized to form a complete running state data set of the simulation environment. Based on the task execution record of the last decision step, the number of hierarchical task units completed by the current computing task is counted, and the overall task completion progress is calculated; at the same time, the current location of each to-be-executed hierarchical task unit in the data center node is located to form a task progress tracking table, ensuring that the subsequent scheduling decision can accurately point to the distribution state of the to-be-executed task.

[0063] Finally, first, three types of basic data are collected, which are the current green power output data of each node, the updated running state data of the simulation environment (node load level, remaining computing power, link bandwidth utilization), and task execution progress information (completed task layer number, to-be-executed task node position); then all the above data are normalized, and data of different dimensions and different value ranges are uniformly mapped to the [0, 1] interval; finally, according to the preset vector dimension order, all the normalized data are sequentially spliced to form a one-dimensional vector containing environment state, resource state and task state. The vector is the input state vector of the agent.

[0064] S104, design the action space of the agent, the action is defined as the joint decision of the selection of the target data center node and the number of task layers continuously executed in the target data center node.

[0065] Specifically, the action space refers to the set of all selectable scheduling decisions of the agent in the multi-data center green electricity optimization scheduling scenario, which clearly defines the boundary and dimension of the agent's decision-making, and is a prerequisite constraint for the agent to perform scheduling operations. The action space takes "target node selection + continuous execution layer number determination" as the core two-dimensional decision dimension, covers the selectable range of all data center nodes under the multi-data center architecture and the number range of hierarchical task units that can be continuously executed on each node, and the agent can only select a specific action from the set for execution in each decision-making round.

[0066] Further, the action refers to a single specific scheduling decision selected by the agent from the action space, which is the joint decision result of the selection of the target data center node and the number of consecutive execution layers of the task on the target node. The action contains two inseparable key elements. The first element is the target data center node, i.e., the specific data center node selected by the agent for executing the subsequent hierarchical task unit. The second element is the number of consecutive execution layers of the task, i.e., the number of hierarchical task units determined by the agent to be consecutively executed on the selected target node.

[0067] In a specific implementation, the first decision dimension of determining the action is the selection of the target data center node, which filters the nodes that meet the current resource and green electricity state adaptation requirements from all available multi-data center nodes as candidate target nodes. The second decision dimension of determining the action is the determination of the number of consecutive execution layers of the task, which sets a reasonable range of the number of consecutive execution layers according to the degree of computational power surplus, the level of green electricity supply, and the task migration cost of the candidate target node. The execution logic of the action is defined. If the selected target data center node is consistent with the current task execution node and the number of consecutive execution layers is set to 1, no cross-node task migration is triggered, and only the next hierarchical task unit is executed on the current node. If the selected target data center node is inconsistent with the current task execution node and the number of consecutive execution layers is greater than 0, the migration of the intermediate data of the current task is performed between the source node and the target node based on the intermediate data obtained by task modeling and the link bandwidth parameters obtained by resource modeling, and then the subsequent hierarchical task units of the specified number of layers are consecutively executed on the target node. The constraint condition of the action is set. The constraint condition includes that the number of consecutive execution layers of the task does not exceed the total number of hierarchical task units that are not completed, and the target data center node meets the minimum requirements of computational power and green electricity supply for executing the task segment.

[0068] Specifically, first, the running state data of the multi-data center simulation environment at the current time is retrieved, including the core indicators such as the green power output, the remaining computational power, and the load level of each node. Then, the node adaptation filtering rule is set to filter out the nodes that meet the minimum demand for task execution, have sufficient remaining computational power to carry the task segment, and have a load level in the non-overload interval. Finally, all nodes that meet the filtering rule are listed as candidate target nodes to form the selectable set of the first decision dimension of the action. For each candidate target node selected in the first step, the degree of computational power surplus, the real-time green electricity supply level data of the node are retrieved, and the transmission cost of migrating the task segment to the node is calculated by combining the intermediate data of the hierarchical task unit obtained by task modeling and the link bandwidth parameters obtained by resource modeling. Based on the above multi-dimensional data such as computational power, green electricity, and migration cost, a reasonable range of the number of consecutive execution layers is set for each candidate target node, the upper limit of which does not exceed the total number of hierarchical task units that are not completed, and the lower limit meets the efficiency requirements of task execution to form the selectable interval of the second decision dimension of the action.

[0069] Furthermore, two types of action execution scenarios are defined, and logic is formulated for each. If the target data center node selected by the agent is the same as the current task execution node, and the number of consecutive execution layers is set to 1, the cross-node task migration process is not triggered. Instead, the remaining computing power of the current node is directly used to continue executing the next level of task units. After execution, the load level and remaining computing power data of the node are updated. If the target data center node selected by the agent is different from the current task execution node, and the number of consecutive execution layers is greater than 0, the data transmission time is calculated based on the amount of intermediate data of the task unit to be migrated obtained from task modeling and the link bandwidth parameters between the source node and the target node obtained from resource modeling, and the migration process is started. After all the intermediate data has been migrated to the target node, the computing power resources of the target node are used to continuously execute the subsequent level of task units of the specified number of layers. After execution, the running status data of the source node and the target node are updated respectively.

[0070] Finally, two hard constraints are set for the action. The number of consecutively executed task layers must not exceed the total number of currently incomplete task units at each level to avoid invalid decisions where the number of task layers exceeds the actual scope to be executed. The selected target data center node must meet the minimum requirements for computing power and green electricity supply needed to execute the task segment. The minimum computing power requirement must match the total computing volume of the task segment, and the minimum green electricity supply requirement must ensure that the task does not rely on traditional power grid supplementation during execution to avoid problems such as task failure or insufficient green electricity utilization due to insufficient node resources.

[0071] S105. Design the reward function.

[0072] The reward function evaluates the scheduling decisions of the agent from multiple dimensions based on the output characteristics of renewable energy, the operating status of the simulation environment, and the feedback after the action is executed.

[0073] Specifically, the reward function is a core evaluation model in the deep reinforcement learning framework used to quantify the quality of a single scheduling decision by an agent, and it is a key basis for guiding the agent to iteratively optimize the scheduling strategy. In this embodiment, the reward function takes the output characteristics of renewable energy at each node of the multi-data center, the real-time operating status of the simulation environment, and the environmental feedback data after the action is executed as three core inputs. It constructs a comprehensive evaluation system from multiple dimensions such as green electricity utilization level, task execution latency, and network congestion level, and clarifies the effect of the agent's current scheduling decision by outputting specific reward values ​​(positive rewards or negative penalties).

[0074] In a specific implementation, dimensions of the reward function are determined; the dimensions include green electricity utilization level, task delay, and network congestion degree; a positive reward rule for the green electricity utilization level dimension is set, based on renewable energy output characteristics obtained through resource modeling, distribution of the load after action execution is determined, and when the load is preferentially distributed to nodes and time periods with green electricity output greater than a preset value, corresponding positive rewards are given according to the green electricity utilization improvement range; a punishment rule for the task delay dimension is set, based on the running state of the simulation environment and action execution feedback, migration delay and calculation delay corresponding to the action are estimated, and if the migration delay or the calculation delay exceeds a preset threshold, rewards are deducted according to the delay exceeding degree; a punishment rule for the network congestion dimension is set, based on network link bandwidth utilization updated by the simulation environment after action execution, if the bandwidth occupancy rate of a key link approaches a congestion threshold, rewards are deducted according to the congestion risk level; weight coefficients are configured for each dimension, and the reward function is determined through weighted summation.

[0075] Specifically, three core evaluation dimensions of the reward function are determined, which are green electricity utilization level, task delay, and network congestion degree, and a multi-dimensional comprehensive evaluation framework is constructed. Based on renewable energy output characteristics obtained through resource modeling, distribution of the load at each data center node and time period after action execution is counted; when the load is preferentially distributed to nodes and time periods with high green electricity output, corresponding positive reward values are given according to the actual improvement range of green electricity utilization. Based on the real-time running state of the simulation environment and feedback data after action execution, task migration delay and calculation delay corresponding to the action are estimated; if either of the migration delay or the calculation delay is large, rewards are deducted in proportion according to the specific degree of delay exceeding. Network link bandwidth utilization data updated by the simulation environment after action execution is extracted, and the bandwidth occupancy state of key links is monitored; if the bandwidth occupancy rate of a key link approaches a congestion state, rewards are deducted in different degrees according to the gradient of the congestion risk level.

[0076] Further, the positive reward of the green electricity utilization level dimension is denoted as The punishment of the task delay dimension is denoted as The punishment of the network congestion dimension is denoted as And , , are normalized to the interval [0, 1] respectively; the reward function is represented as:

[0077] ;

[0078] Wherein, is the reward function; , and satisfies ; the weight coefficient is preset according to the scheduling target preference: when the target is more inclined to improve the green electricity utilization level, the ; when the target is more inclined to guarantee the service delay, the ; when the target is more inclined to reduce the network congestion risk, the ; for example, in the green electricity priority scenario, it is advisable that = 0.5, = 0.3, = 0.2, in the delay priority scenario, it is advisable that = 0.3, = 0.5, = 0.2.

[0079] S106, based on the proximal policy optimization algorithm, in the simulation environment, taking the input state vector as input, combining the action space and the reward function to carry out multiple rounds of training, and iteratively updating the scheduling policy until the scheduling policy meets the comprehensive performance index.

[0080] Specifically, in this embodiment, the proximal policy optimization algorithm (Proximal Policy Optimization, PPO) is a policy gradient optimization algorithm for training deep reinforcement learning agents. The core function is to iteratively adjust the scheduling policy of the agent to continuously optimize the decision-making ability of the agent under multi-target constraints (green electricity utilization, task delay, network congestion).

[0081] In specific implementation, the parameters of the policy network and the value network of deep reinforcement learning are initialized, the total number of training rounds, the maximum number of decision steps per round, and the comprehensive performance threshold are set; in each training round, based on the input state vector generated by the simulation environment in real time, the agent selects an appropriate action from the action space and executes it, while recording the current state, the selected action, and the corresponding environmental feedback information; after the action is executed, the immediate reward of this decision is calculated through the reward function, and the next state vector is obtained combined with the simulation environment state update result to form a complete interaction sample. Multiple interaction samples are continuously collected and stored; based on the collected interaction samples, the long-term value of different state-action pairs is evaluated using the advantage estimation method to determine the advantages and disadvantages of each decision; based on the evaluation results, the parameters of the policy network and the value network are iteratively updated using the proximal policy optimization algorithm. During the updating process, the parameter updating amplitude is limited and an exploratory constraint is introduced; after completing the preset number of training rounds, the performance of the current scheduling policy is evaluated through multiple typical scenarios in the simulation environment. If the evaluation results show that all indicators meet the preset threshold and the performance has no fluctuations for multiple consecutive evaluations, it is determined that the comprehensive performance of the scheduling policy meets the standard, and the training is stopped.

[0082] Specifically, first, the basic network structure of the deep reinforcement learning model is configured, and the policy network for outputting scheduling decisions and the value network for evaluating state values are respectively built, and random initial values are assigned to the weights, biases and other parameters of the two types of networks; Then, according to the actual needs of multi-data center task scheduling, the total number of training rounds, the maximum number of decision steps allowed per round, and the comprehensive performance threshold are set, which includes the minimum threshold of green power utilization, the maximum limit of task average delay, and the maximum tolerance of network congestion rate.

[0083] Further, a round of training is started, and at each decision step in the round, the real-time generated input state vector is retrieved from the multi-data center simulation environment and input into the policy network of the agent. The policy network outputs an action selection probability distribution based on the current parameters, and the agent selects an appropriate scheduling action (target node + continuous execution layer) from the preset action space according to the distribution and issues it to the simulation environment for execution. The key information of the decision step is recorded synchronously, including the current input state vector, the selected specific action, and the state change feedback data of the simulation environment after the action is executed, forming a preliminary decision record.

[0084] After the action is executed in the simulation environment, the reward function is retrieved, and the green power utilization level, task delay, and network congestion level data after the action is executed are input to calculate the immediate reward value of this decision. Then, the next state vector updated after the action is executed is extracted from the simulation environment. Finally, the "current state vector-selected action-immediate reward value-next state vector" four pieces of information are associated and integrated to form a complete interaction sample, which is stored in the data buffer. The above operation is repeated continuously until the training round is completed, and a sufficient number of interaction samples are collected.

[0085] After the data buffer accumulates a sufficient number of interaction samples, the advantage estimation method is used to process the samples in batches. For the state-action pair in each sample, the reward value of the subsequent decision step and the state value output by the value network are combined to calculate the advantage value of the state-action pair, which quantitatively evaluates the value contribution of the decision in the long-term training process and clearly defines the advantages and disadvantages of different scheduling decisions, providing a quantitative basis for subsequent network parameter updates.

[0086] Based on the evaluation results obtained by advantage estimation, the parameter updating process of the proximal policy optimization algorithm is started; for the policy network, the policy loss function is calculated, and a clipping mechanism is introduced in the loss function optimization process to strictly limit the difference between the new policy and the old policy, avoiding training shock caused by excessive parameter update amplitude; for the value network, the value loss function is calculated, and the network parameters are updated by gradient descent method to improve the accuracy of the value network in evaluating the state value; at the same time, an exploratory constraint is introduced in the parameter updating process to retain the exploration ability of the agent to the new scheduling strategy and prevent the strategy from converging to a local optimal solution too early.

[0087] After completing the preset number of training rounds, the parameter updating process is suspended and the policy performance evaluation stage is entered; multiple typical business scenarios are constructed in the multi-data center simulation environment, including green power output peak scenario, green power output valley scenario, task load surge scenario, etc., the scheduling strategy obtained by the current training is deployed to the agent, and full-process testing is carried out in each typical scenario; the core indicators such as green power utilization rate, task average delay, network congestion rate of the strategy in each scenario are counted and calculated; if all indicators meet the preset comprehensive performance threshold and the performance data of continuous multiple rounds of evaluation has no obvious fluctuation, it is determined that the comprehensive performance of the scheduling strategy meets the standard, and the training is terminated; if it does not meet the standard, the data buffer is emptied and the training round execution step is returned to continue the next round of training until the strategy meets the standard.

[0088] S107, apply the scheduling strategy that meets the standard to the real multi-data center environment to optimize the scheduling task.

[0089] Specifically, a scheduling strategy model meeting the comprehensive performance target is derived from the trained deep reinforcement learning framework, including the final parameters of the policy network, the action selection logic, and the decision threshold configuration file. The derived scheduling strategy model is integrated into the core scheduling system of the real multi-data center, and the interface adaptation and joint debugging of the policy and the data center node monitoring module, the task distribution module, and the network transmission module are completed. Real-time data acquisition components are deployed to interface with the monitoring systems of each data center node to collect real-time green power output, residual computing power, load level, link bandwidth utilization, and other state data of the nodes, as well as hierarchical structure information of the computing tasks to be executed. According to the unified normalization standard and data splicing format in the simulation environment, the collected real environment state data and task information are processed to generate real-time input state vectors consistent with the format in the training phase. The real-time input state vectors are input into the deployed scheduling strategy model, and the model outputs specific scheduling actions, i.e., the target data center node and the number of consecutive execution tasks; the scheduling system issues task execution instructions to the target node according to the decision, and if it involves cross-node task migration, it synchronously starts the intermediate data transmission process. The execution of the task in the real node is continuously monitored, including the task completion progress, node resource occupation changes, green power consumption ratio, and other data, and the running state is fed back to the scheduling strategy model; if abnormal situations such as node overload and link congestion occur, the preset emergency adjustment mechanism is triggered to dynamically optimize the scheduling decision. Based on the running data and feedback information of the real environment, new collected interaction samples are input into the training framework for incremental training and parameter fine-tuning of the scheduling strategy model, continuously improving the adaptability and optimization effect of the strategy in the real scene.

[0090] The method provided in the embodiment constructs a closed-loop deep reinforcement learning scheduling scheme through sequentially setting the whole process steps of multi-data center node resource modeling, hierarchical structure computing task modeling, simulation environment construction and state vector generation, action space design, reward function design, strategy training based on the PPO algorithm, and real environment application. Firstly, by constructing a multi-data center heterogeneous resource model and a model of renewable energy output varying with time, the green power output information is explicitly introduced into the task scheduling decision process, which can arrange more computing tasks to be executed at nodes and time periods with sufficient green power under the premise of guaranteeing business requirements, significantly improve the renewable energy utilization rate, reduce the dependence on traditional power grid power supply, and reduce carbon emissions and power consumption; at the same time, by dividing the neural network inference task into fine-grained hierarchical task units and supporting the segmented migration of the task in the execution process, the execution path of the task can be flexibly adjusted between the multi-data centers, the execution location is dynamically selected according to the real-time computing power of each node and the green power state, more fine and flexible scheduling control is realized, which is superior to the traditional scheme supporting only overall task migration or single-node execution; by designing a reward function considering multiple dimension factors and using the PPO deep reinforcement learning algorithm for strategy training, the scheduling strategy of the application can automatically weigh between multiple objectives, obtain a better latency-energy consumption balance performance, and has better adaptability and global optimization capability than the scheduling strategy based on fixed rules or heuristic methods; finally, by completing the training and evaluation of the scheduling strategy in the simulation environment and then migrating to the real environment, parameter adjustment and strategy improvement can be carried out without affecting the actual business, which has good engineering realizability and popularization and application value.

[0091] Corresponding to the foregoing embodiment of the method for green power optimized scheduling of multi-data centers based on deep reinforcement learning, the application also provides an embodiment of a device for green power optimized scheduling of multi-data centers based on deep reinforcement learning.

[0092] Figure 2 The structure schematic diagram of the device for green power optimized scheduling of multi-data centers based on deep reinforcement learning provided in Embodiment Two of the application is shown in Figure 2 The device provided in the embodiment includes a modeling module 210, a construction module 220, a design module 230, a training module 240, and an optimization module 250.

[0093] The modeling module 210 is configured to model the resources of the multi-data center nodes, determine the resource parameters of each node and the output characteristics of the renewable energy accessed by each node varying with time.

[0094] The modeling module 210 is further configured to model the computing task with a hierarchical structure, and decompose the computing task into hierarchical task units that can be migrated across data centers.

[0095] The construction module 220 is configured to construct a multi-data center simulation environment based on the resource modeling result and the computing task modeling result, and generate an input state vector of the agent.

[0096] The design module 230 is configured to design an action space of the agent, and the action is defined as a joint decision of a target data center node and a task layer number continuously executed at the target data center node.

[0097] The design module 230 is further configured to design a reward function, and the reward function takes the output characteristics of the renewable energy, the running state of the simulation environment and the feedback after the action is executed as evaluation basis to evaluate the scheduling decision of the agent from multiple dimensions.

[0098] The training module 240 is configured to perform multiple rounds of training in the simulation environment based on a proximal policy optimization algorithm, take the input state vector as input, combine the action space and the reward function, iteratively update the scheduling policy until the scheduling policy meets the standard in the comprehensive performance index.

[0099] The optimization module 250 is configured to apply the scheduling policy meeting the standard to a real multi-data center environment to optimize the scheduling task.

[0100] The device of the embodiment can be used to perform the method. Figure 1 The steps of the method embodiment are similar in implementation principle and process, and will not be repeated here.

[0101] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0102] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The device embodiment described above is only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement without creative labor.

[0103] The above is only the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for multi-data center green electricity optimization scheduling based on deep reinforcement learning, characterized in that, The method comprises: resource modeling of multiple data center nodes, determining resource parameters of each node and output characteristics of renewable energy accessed by each node over time; modeling a computing task with a hierarchical structure, decomposing the computing task into hierarchical task units that are migrated across data centers; based on the resource modeling results and the computing task modeling results, building a multi-data center simulation environment, and generating an input state vector of an agent; designing an action space of the agent, where an action is defined as a joint decision of target data center node selection and task layer number continuously executed in the target data center node; wherein a first decision dimension of the action is target data center node selection, and a node that meets the current resource and green electricity state adaptation requirements is selected as a candidate target node from all available multiple data center nodes; a second decision dimension of the action is the determination of the number of continuously executed task layers, and a reasonable range of the number of continuously executed layers is set according to the degree of computational power surplus, the green electricity supply level and the task migration cost of the candidate target node; the execution logic of the action is defined; and the constraint conditions of the action are set; designing a reward function; the reward function takes the output characteristics of renewable energy, the running state of the simulation environment and the feedback after the action is executed as the evaluation basis to evaluate the scheduling decisions of the agent from multiple dimensions; wherein the evaluation dimensions of the reward function are determined; the evaluation dimensions include green electricity utilization level, task latency and network congestion level; the positive reward rules of the green electricity utilization level dimension are set; the punishment rules of the task latency dimension are set; the punishment rules of the network congestion dimension are set; weight coefficients are configured for each evaluation dimension, and the reward function is determined by weighted summation; based on the proximal policy optimization algorithm, in the simulation environment, taking the input state vector as the input, combining the action space and the reward function to carry out multiple rounds of training, iteratively updating the scheduling strategy until the scheduling strategy meets the standard in the comprehensive performance index; applying the scheduling strategy that meets the standard to a real multi-data center environment to optimize the scheduling task; the modeling of a computing task with a hierarchical structure, decomposing the computing task into hierarchical task units that are migrated across data centers, comprises: selecting a target computing task with a hierarchical structure, the target computing task being a computing task with a forward execution feature by layers; extracting features of the target computing task through an offline analysis tool to obtain floating point operation amounts and intermediate activation data amounts of each layer of the target computing task; based on the extracted floating point operation amounts and intermediate activation data amounts of each layer, converting the forward propagation process of the target computing task into an ordered hierarchical task sequence; based on the computation amount and intermediate data amount of the hierarchical task unit, setting a migration adaptation rule to determine the migration conditions of a single hierarchical task unit or an execution segment formed by a combination of a plurality of continuous hierarchical task units.

2. The method of claim 1, wherein, the resource modeling of multiple data center nodes, determining resource parameters of each node and output characteristics of renewable energy accessed by each node over time, comprises: Collect resource basic parameters of each data center node; the resource basic parameters include the upper limit of the rated power of the node, the installed capacity proportion of renewable energy, the equivalent computing capacity, and the upper limit of the link bandwidth between different nodes; Identify the types of renewable energy accessed by each data center node; the types of renewable energy include photovoltaic energy and wind power energy; For different types of renewable energy, a corresponding time-output function is constructed, which represents the change rule of green power output of the node at different time periods in a day, and the time-output function is constructed according to the inherent characteristics of the corresponding renewable energy; Combine the resource basic parameters and the time-output function to calculate the green power output of each node at different times.

3. The method of claim 2, wherein, For different types of renewable energy, a corresponding time-output function is constructed, including: When the type of renewable energy is photovoltaic energy, a first time-output function is constructed by combining the characteristics affected by day and night and weather, which is a unimodal distribution function in the daytime; the first time-output function represents the change rule that the output of photovoltaic energy gradually rises to a peak and then falls; When the type of renewable energy is wind power energy, a second time-output function is constructed based on the periodic fluctuation characteristics of wind speed, combining the cubic relationship between power output and wind speed; the second time-output function represents the output fluctuation of wind power energy with wind speed.

4. The method of claim 1, wherein, Each hierarchical task unit in the hierarchical task sequence corresponds to one or multiple consecutive layers of the target computing task, and each hierarchical task unit includes a calculation amount and an intermediate data amount, the calculation amount corresponds to the floating point operation amount of each layer, and the intermediate data amount corresponds to the intermediate activation data amount of each layer.

5. The method of claim 1, wherein, Based on the resource modeling results and the computing task modeling results, a multi-data center simulation environment is constructed to generate an input state vector of the agent, including: Relying on the resource parameters of each data center node obtained by resource modeling, the renewable energy time-output function, and the core parameters of the hierarchical task unit obtained by task modeling, a multi-data center simulation environment is built; At any decision step in the simulation environment, the green power output of each node at the current time is calculated according to the current simulation time, combined with the time-output function and node resource parameters of resource modeling; Combine the task execution results of the last decision step to update the load level and remaining computing power of each node, and update the network link bandwidth utilization according to the data flow generated by task migration and the link bandwidth parameters between nodes; Record the number of hierarchical task units completed by the current computing task and the location of the data center node where the hierarchical task units to be executed are located; The green power output, load level, remaining computing power, network link bandwidth utilization, and task progress information of each node are normalized according to the preset standard, and the processed data are spliced and integrated according to the preset format to form the input state vector of the agent.

6. The method of claim 1, wherein, Design the action space of the agent, and the action is defined as the joint decision of the selection of the target data center node and the number of layers of tasks continuously executed in the target data center node, including: The first decision dimension of the determining action is target data center node selection, and a node meeting the current resource and green electricity state adaptation requirement is selected as a candidate target node from all available multi-data center nodes; The second decision dimension of the determining action is continuous execution task layer number determination, and a reasonable continuous execution layer number range is set according to the algorithm power surplus degree, green electricity supply level and task migration cost of the candidate target node; The execution logic of the action is defined, if the selected target data center node is consistent with the current task execution node and the continuous execution layer number is set to 1, no cross-node task migration is triggered, and only the next level task unit is executed in the current node; If the selected target data center node is inconsistent with the current task execution node and the continuous execution layer number is greater than 0, the migration of the intermediate data of the current task is performed between the source node and the target node based on the intermediate data obtained by task modeling and the link bandwidth parameters between nodes obtained by resource modeling, and the subsequent level task unit of the specified layer number is continuously executed on the target node; The constraint condition of the action is set; the constraint condition includes that the continuous execution task layer number does not exceed the total number of unfinished level task units, and the target data center node meets the minimum requirement of algorithm power and green electricity supply for executing the task segment.

7. The method of claim 1, wherein, The reward function is designed, including: The evaluation dimension of the reward function is determined; the evaluation dimension includes the green electricity utilization level, the task time delay and the network congestion degree; The positive reward rule of the green electricity utilization level dimension is set, the load allocation after the action is executed is judged based on the renewable energy output characteristics obtained by resource modeling, when the load is preferentially allocated to the node and period with green electricity output greater than a preset value, the corresponding positive reward is given according to the green electricity utilization improvement amplitude; The punishment rule of the task time delay dimension is set, the migration time delay and the calculation time delay corresponding to the action are estimated based on the running state of the simulation environment and the action execution feedback, if the migration time delay or the calculation time delay exceeds the preset threshold, the reward is deducted according to the time delay exceeding degree; The punishment rule of the network congestion dimension is set, if the bandwidth occupancy rate of the key link is close to the congestion threshold, the reward is deducted according to the congestion risk level according to the network link bandwidth utilization rate updated after the action is executed; The weight coefficients are configured for each evaluation dimension, and the reward function is determined by weighted summation.

8. The method of claim 1, wherein, Based on the proximal policy optimization algorithm, in the simulation environment, the input state vector is input, the action space and the reward function are combined, and multiple rounds of training are carried out, the scheduling policy is iteratively updated, and the scheduling policy is up to standard in the comprehensive performance index, including: The parameters of the policy network and the value network of deep reinforcement learning are initialized, the total number of training rounds, the maximum number of decision steps per round and the comprehensive performance threshold are set; In each training round, based on the input state vector generated by the simulation environment in real time, the agent selects the adaptive action from the action space and executes it, and records the current state, the selected action and the corresponding environment feedback information; After the action is executed, an instant reward of the current decision is calculated through the reward function, a next state vector is obtained in combination with a simulation environment state update result, a complete interaction sample is formed, and multiple rounds of interaction samples are continuously collected and stored; Based on the collected interaction samples, the long-term value of different state-action pairs is evaluated using the advantage estimation method to determine the advantages and disadvantages of each decision; In combination with the proximal policy optimization algorithm, the parameters of the policy network and the value network are iteratively updated based on the evaluation results, and the parameter update amplitude is limited and an exploratory constraint is introduced during the update process; After each preset round of training is completed, the performance of the current scheduling strategy is evaluated through multiple typical scenarios in the simulation environment, and if the evaluation results show that each index meets the preset threshold and the performance is stable for multiple rounds of evaluation, it is determined that the comprehensive performance of the scheduling strategy meets the standard, and the training is stopped.

9. A deep reinforcement learning based multi-data center green electricity optimization scheduling apparatus, characterized in that, The device comprises a modeling module, a construction module, a design module, a training module, and an optimization module; The modeling module is configured to model resources of multiple data center nodes, determine resource parameters of each node, and determine output characteristics of renewable energy accessed by each node over time; The modeling module is further configured to model a computing task having a hierarchical structure, and decompose the computing task into hierarchical task units that are migrated across data centers; The construction module is configured to construct a multi-data center simulation environment based on the resource modeling results and the computing task modeling results, and generate an input state vector of an agent; The design module is configured to design an action space of the agent, wherein an action is defined as a joint decision of a target data center node selection and a number of task layers that are continuously executed in the target data center node; a first decision dimension of the action is determined as the target data center node selection, and a node that meets current resource and green power state adaptation requirements is selected as a candidate target node from all available multiple data center nodes; a second decision dimension of the action is determined as the number of continuously executed task layers, and a reasonable range of the number of continuously executed task layers is set according to an algorithmic surplus degree of the candidate target node, a green power supply level, and a task migration cost; an execution logic of the action is defined; and a constraint condition of the action is set; The design module is further configured to design a reward function; the reward function takes the output characteristics of the renewable energy, the running state of the simulation environment, and feedback after the action is executed as evaluation basis, and evaluates the scheduling decision of the agent from multiple dimensions; an evaluation dimension of the reward function is determined; the evaluation dimension comprises a green power utilization level, a task latency, and a network congestion degree; a positive reward rule of the green power utilization level dimension is set; a penalty rule of the task latency dimension is set; a penalty rule of the network congestion degree dimension is set; a weight coefficient is configured for each evaluation dimension, and the reward function is determined by weighted summation; The training module is configured to perform multiple rounds of training in the simulation environment based on the proximal policy optimization algorithm, take the input state vector as input, combine the action space and the reward function, iteratively update the scheduling strategy, and stop the training until the scheduling strategy meets the standard in terms of comprehensive performance indicators. The optimization module is configured to apply the optimized scheduling strategy to a real multi-data center environment to optimize the scheduling task. The target computing task with a hierarchical structure is screened, and the target computing task is a computing task with a forward execution characteristic by layer. Feature extraction is performed on the target computing task by an offline analysis tool to obtain floating point operation amounts and intermediate activation data amounts of each layer of the target computing task. Based on the extracted floating point operation amounts and intermediate activation data amounts of each layer, the forward propagation process of the target computing task is converted into an ordered hierarchical task sequence. Based on the computation amount and intermediate data amount of the hierarchical task unit, a migration adaptation rule is set, and a migration condition of an execution fragment formed by a single hierarchical task unit or a combination of a plurality of continuous hierarchical task units is determined.

Images