Multi-energy green data center task scheduling method and device, medium and product

Abstract

The invention discloses a multi-energy green data center task scheduling method and device, a medium and a product, and relates to the field of task scheduling, and the method comprises the steps: converting a multi-energy green data center task scheduling process into a Markov process; constructing a multi-agent reinforcement learning model according to the Markov process; acquiring the generating capacity of a renewable energy generator, demand information of an energy green data center and electricity price information; based on an XGBoost prediction method, determining a renewable energy power generation amount, an energy green data center demand quantity sequence and a current state of each energy green data center; according to the current state, adopting a multi-agent reinforcement learning model to determine an action; the root action obtains a real reward from the environment; and updating the multi-agent reinforcement learning model according to the expected reward and the real reward. According to the invention, lower SLO violation rate, total energy currency cost and total carbon emission are realized.

Description

Technical Field

[0001] The present application relates to the field of task scheduling, and in particular to a method, device, medium and product for scheduling tasks in a multi-energy green data center. Background Art

[0002] Geographically distributed data centers provide global cloud services, which requires a large amount of energy (megawatts of electricity) to run data centers, resulting in cloud service provider companies such as Google, Microsoft, and Amazon spending tens of millions of dollars on electricity bills each year. Estimates in 2018 show that global data center electricity consumption accounts for about 1% of the world's total electricity consumption. A large number of data centers in the world are powered by brown energy such as fossil fuels, coal, and oil, which produce a large amount of carbon emissions. In fact, by 2020, the total carbon emissions from data centers will reach 14.3 million tons, accounting for 18% of global carbon emissions. Carbon emissions increase global temperatures, which not only changes water supply and weather patterns and the growing season of food crops, but also causes sea level rise to threaten coastal communities. In order to mitigate the adverse environmental impacts and energy crises caused by the rapid increase in brown energy consumption, many companies (such as Google, Microsoft, and Amazon) have begun to use renewable energy to power their data centers as much as possible. For example, Apple built a 40MW solar array for its data center in North Carolina. However, due to the different amounts of renewable energy generated by renewable energy generators, data centers cannot always use renewable energy generators to meet their energy needs. Even when using brown energy supply, a common cause of data center failure is power outages. Since the CPU of the machine cannot be fully utilized in the case of insufficient energy supply, and it takes a while for the machine to switch to brown energy supply in the case of renewable energy shortage, the job on the machine cannot be executed at full speed, thereby increasing the job completion time. Therefore, the shortage of renewable energy supply may lead to violation of the service level objective (SLO) of the job, such as the percentage of customer requests or jobs that can be served or completed before the job deadline. Therefore, it is very important for data centers to determine the amount of renewable energy to obtain to avoid renewable energy shortage and over-acquisition. Assume that in a large area, there are many data centers belonging to different cloud providers and many renewable energy generators; each data center obtains renewable energy from one or more energy generators, and the generators distribute the generated renewable energy to their requesters. In this scenario, each data center needs to determine how much renewable energy to request from each generator in order to obtain enough energy to support its job SLO while minimizing cost and carbon emissions. Specifically, the renewable energy supply and demand matching problem can be described as:

[0003] Given a set of data centers belonging to different cloud providers and many renewable energy generators (each generating one energy source) distributed across a region, how can we match the renewable energy generators to different data centers to meet their energy needs over the long term (e.g., one month) to minimize the application's SLO violation rate (outages due to insufficient renewable energy), total carbon emissions, and monetary cost per data center?

[0004] The above problem has not been investigated in previous studies. Previous studies on the renewable energy supply-demand matching problem mainly focus on multiple data centers belonging to the same cloud provider. Therefore, the information of the data centers can be obtained through a centralized agent, and there is no demand competition between data centers, so it cannot be applied to the problem. In addition, the above methods can only achieve short-term (e.g., 1 hour) generator-data center matching plans, which leads to frequent changes in the plan and thus incurs additional overhead.

[0005] In order to solve this problem, CN112949195A proposed a multi-objective task scheduling method for power cloud data centers to unify and coordinate the efficiency and energy consumption of task scheduling in power cloud data centers. However, this method does not consider the risks that task heterogeneity and arrival uncertainty may bring to task scheduling. On the other hand, CN117076100A proposed a hybrid data center task scheduling mathematical model based on queuing theory, which analyzes the delay and energy consumption of each node to more effectively distribute the load, thereby reducing energy consumption and delay. However, this method only optimizes the energy consumption and delay of the server, and does not consider the impact of new energy output on the data center during task scheduling.

[0006] Therefore, in order to solve the above problems and achieve long-term planning to achieve lower SLO violation rate, total energy monetary cost and total carbon emissions, it is urgent to provide a new data center task scheduling method. Summary of the invention

[0007] The purpose of this application is to provide a multi-energy green data center task scheduling method, equipment, medium and product, which can achieve lower SLO violation rate, total energy monetary cost and total carbon emissions.

[0008] To achieve the above objectives, this application provides the following solutions:

[0009] In a first aspect, the present application provides a multi-energy green data center task scheduling method, the multi-energy green data center task scheduling method comprising:

[0010] Transform the multi-energy green data center task scheduling process into a Markov process;

[0011] A multi-agent reinforcement learning model is constructed according to the Markov process; the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and obtain the expected return of each state-action pair after processing by the corresponding neural network; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator;

[0012] Obtain information on the power generation of renewable energy generators and the demand for energy from green data centers, as well as electricity price information;

[0013] According to the information of renewable energy generator power generation and energy green data center demand, the sequence of renewable energy power generation and energy green data center demand is determined based on the XGBoost prediction method;

[0014] Determine the current status of each energy green data center based on the renewable energy power generation and energy green data center demand sequence and electricity price information;

[0015] According to the current status of each energy green data center, a multi-agent reinforcement learning model is used to determine the action of each energy green data center;

[0016] Obtain true rewards from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model based on the expected rewards and true rewards.

[0017] Optionally, the step of converting the multi-energy green data center task scheduling process into a Markov process specifically includes:

[0018] Using the formula Determine the state space S i ;

[0019] Using the formula Determine the action space A of the i-th agent i ;

[0020] Determine the reward function based on monetary costs, carbon emissions, and SLO violations caused by energy shortages;

[0021] in, is the kth renewable energy generator at t z The predicted power generation at the time, is the unit price of the generator set at different time periods, D i is the predicted time series energy demand vector of the i-th energy green data center; From the energy generator G kGet the energy value.

[0022] Optionally, the reward function is determined based on the monetary cost, carbon emissions, and SLO violations caused by energy shortages, specifically including:

[0023] Using the formula Determine the reward function R for Z time period i ;

[0024] Among them, α1, α2, and α3 are weight coefficients of different features. is the reward function z The number of SLO violations at the time, For the generator set G k In t z Carbon emissions per kilowatt-hour at any given moment, For the i-th energy green data center at t z At the moment from the energy generator G k Buy is the monetary cost of the energy paid, c is the monetary cost of switching on the renewable energy generator, b is the monetary cost of switching on the renewable energy generator, tz t z is a binary variable indicating whether there is a generator switch at the moment, and K is the total number of renewable energy generators.

[0025] Optionally, the loss function of the power generation prediction process of the renewable energy generator at different time periods specifically includes:

[0026] Using the formula Determine the loss function L(θ) of the XGBoost prediction method;

[0027] Among them, e Gk,tz is the actual power generation, g Gk,tz To predict the power generation, Ω(f x ) is the complexity regularization term used to avoid overfitting, and n is the total number of energy green data centers.

[0028] Optionally, updating the multi-agent reinforcement learning model according to the expected reward and the actual reward specifically includes:

[0029] According to the expected reward and the actual reward, the MiniMax Q-learning algorithm is used to update the multi-agent reinforcement learning model.

[0030] In a second aspect, the present application provides a multi-energy green data center task scheduling device, the multi-energy green data center task scheduling device comprising:

[0031] The process conversion module is used to convert the multi-energy green data center task scheduling process into a Markov process;

[0032] A model building module is used to build a multi-agent reinforcement learning model according to a Markov process; the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and obtain the expected return of each state-action pair after processing by the corresponding neural network; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator;

[0033] A data acquisition module is used to obtain information on the power generation of renewable energy generators, demand for energy green data centers, and electricity price information;

[0034] A demand determination module is used to determine the sequence of renewable energy power generation and energy green data center demand based on the XGBoost prediction method according to the information of renewable energy generator power generation and energy green data center demand;

[0035] A state determination module, used to determine the current state of each energy green data center according to the renewable energy power generation and energy green data center demand sequence and electricity price information;

[0036] An action determination module is used to determine the action of each energy green data center based on the current state of each energy green data center using a multi-agent reinforcement learning model;

[0037] A model updating module is used to obtain the real reward from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model according to the expected reward and the real reward.

[0038] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-energy green data center task scheduling method.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-energy green data center task scheduling method.

[0040] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-energy green data center task scheduling method.

[0041] According to the specific embodiments provided in this application, this application has the following technical effects:

[0042] The present application provides a multi-energy green data center task scheduling method, equipment, medium and product, which converts the multi-energy green data center task scheduling process into a Markov process, and then constructs a multi-agent reinforcement learning model (MARL); uses the XGBoost prediction method to predict the renewable energy generation of each renewable energy generator and the energy demand in each time period of the next month; determines how much renewable energy each generator needs through the multi-agent reinforcement learning model; the present application implements an optimization model for matching energy generators with energy green data centers based on the multi-agent reinforcement learning model, and can generate a matching plan energy generator and energy green data center to meet the energy of each energy green data center while using renewable energy as much as possible, and minimize SLO violations caused by energy shortages, carbon emissions and capital costs; and thus the present application can achieve a lower SLO violation rate, total energy monetary cost and total carbon emissions. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 This is a flowchart of a multi-energy green data center task scheduling method in one embodiment of the present application;

[0045] Figure 2 Schematic diagram of the multi-agent reinforcement learning model structure. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0048] In an exemplary embodiment, Figure 1 As shown, a multi-energy green data center task scheduling method is provided, and the method includes the following S101 to S107. Among them:

[0049] S101, transform the multi-energy green data center task scheduling process into a Markov process;

[0050] Specifically, this application uses Markov game to describe the energy green data center generator matching problem, which can take into account the competition between intelligent agents and enable each intelligent agent to make the best decision. Definition. N={n1,n2,…,n N} represents the set of agents in the Markov game. i represents the state space of the ith agent, S:=S 1 ×,…,×S N represents the state space observed by all agents. s∈S(s={s 1 ,…,s i ,…,s N}) represents the joint observation state, s i A represents the observed state of the i-th agent. i Represents the action space A of the ith agent: =A 1 ×…×A N . a∈A(a={a 1 ,…,a i ,…,a N}) indicates a joint action, a i represents the action taken by the ith agent. P:S×A→Δ(S) represents the transition probability from state s∈S to action a∈A. R i :S*A→R is the reward function, which determines the agent n i The immediate reward obtained when transitioning from s to s' via a. i Indicated by R i The reward calculated by the function. γ(0<γ<1) is a discount factor that controls the degree of influence of future rewards on the optimal decision; a smaller γ means that the weight of recent rewards is higher, and a larger γ means that the weight of future rewards is higher. In the Markov game, the goal of each agent is to maximize its expected sum of discounted rewards.

[0051] Based on the above principles, S101 of this application specifically includes:

[0052] S11, using formula Determine the state space S i ;

[0053] Among them, the state space S of the i-th agent i , including the information of all renewable energy generators and energy green data centers of the agent. ZThe predicted power generation at the time is expressed as: In the formula, is the renewable energy power generation of the kth renewable energy generator.

[0054] u Gk The unit price vector represents the unit price of a generator set at different time periods: In the formula, Indicates the generator G k At time t z The energy unit price is different at different times and is pre-acquired for all energy green data centers;

[0055] The predicted time series energy demand vector D of the i-th energy green data center i for:

[0056]

[0057] S12, using the formula Determine the action space A of the i-th agent i ;

[0058] The RL agent in the energy green data center makes decisions for the energy green data center. Each action in the action space represents a set of energy generators selected at each moment in the next time period (e.g., one month). How much energy is requested from each energy generator. 0 requests for a generator means that the generator is not selected. For the i-th energy green data center, the action space of the i-th agent is A i ;

[0059] in, in, In the time period t z The energy generator G k How much energy is obtained;

[0060] When an action is performed by the Energy Green Data Center agent to decide how much energy should be requested from each energy generator, the state is changed deterministically. Therefore, the probability between each state is always 1.

[0061] S13, determining the reward function based on the SLO violations caused by monetary costs, carbon emissions, and energy shortages;

[0062] Using the formula Determine the reward function R for Z time period i ;

[0063] Among them, α1, α2, and α3 are weight coefficients of different characteristics, which are determined by the owner of the energy green data center. is the reward function z The number of SLO violations at the time. For the generator set G k In t z Carbon emissions per kilowatt-hour at any time, using Determine t z Moment Energy Generator G k Total Energy Total carbon emissions, For the i-th energy green data center at t z At the moment from the energy generator G k Buy is the monetary cost of the energy paid, c is the monetary cost of switching on the renewable energy generator, t z is a binary variable indicating whether there is a generator switch at the moment, and K is the total number of renewable energy generators.

[0064] In order to better avoid the violation of SLO of energy green data center jobs and the reduction of service quality of energy green data center, a method of job delay with guaranteed deadline, DGJP, is provided. It defines the time that must continue to run to meet its deadline requirements and estimates the energy consumption of the remaining operation of the job based on the amount of computing resources allocated to the job. When a renewable energy power outage occurs, DGJP first sorts the running jobs in descending order of the urgency coefficient and sums up the energy requirements of the jobs from top to bottom until the sum is not less than the energy shortage of the time period. Then, the selected running job will be suspended and placed in the suspended job queue.

[0065] S102, construct a multi-agent reinforcement learning model based on the Markov process; Figure 2 As shown, the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and after processing through the corresponding neural network, obtain the expected return of each state-action pair; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator;

[0066] ,S103, obtain the power generation information of renewable energy generators, the demand information of energy green data centers, and the electricity price information;

[0067] S104, determining the renewable energy power generation and energy green data center demand sequence based on the XGBoost prediction method according to the renewable energy generator power generation and energy green data center demand information;

[0068] Take the kth generator as an example:

[0069] Generator at t Z The actual power generation sequence per hour before the moment is combined with other features (such as geographic location, weather information, etc.) as input features for training the XGBoost model. In the prediction stage, the model uses the input features to infer the power generation at future moments. The output is the predicted future power generation sequence, i.e., t Z The future hourly predicted power generation sequence after time.

[0070]

[0071] The XGBoost model builds multiple decision trees, each of which learns based on the input features, with the goal of gradually reducing the prediction error. The XGBoost model optimizes the tree structure by minimizing the following loss function:

[0072] Using the formula Determine the sum of squared errors with the XGBoost model predictions;

[0073] in, is the actual power generation, is the predicted power generation, Ω(f x ) is a regularization term for the complexity of the XGBoost model, which is used to avoid overfitting.

[0074] During the training process, the XGBoost model builds each tree by stepping through iterations. Assume that there is historical data at the current time point. The XGBoost model will be modified based on the prediction value of the previous iteration. The initial prediction value is a constant In each subsequent iteration, each new tree f j (x) will adjust the current prediction. The new prediction value is updated through iteration as:

[0075] Among them, f j (x) represents the correction value of the jth tree. After multiple iterations, the XGBoost model will output the predicted value of future power generation.

[0076] The energy green data center demand prediction model and method are the same as the above-mentioned new energy generator power generation prediction process.

[0077] S105, determining the current state of each energy green data center according to the renewable energy power generation and energy green data center demand sequence and electricity price information;

[0078] S106, determining the action of each energy green data center using a multi-agent reinforcement learning model according to the current state of each energy green data center;

[0079] S106 is as follows:

[0080] Take the i-th energy green data center as an example. The energy green data center has an intelligent agent to make decisions. The intelligent agent obtains the current state S i , the state S i Input into the neural network, the neural network will output the expected reward Q(S) for each action selected in the current state i ,A i ). The agent will choose the action with the largest expected reward, i.e.

[0081] S107, obtain real rewards from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model according to the expected rewards and the real rewards.

[0082] According to the expected reward and the actual reward, the MiniMax Q-learning algorithm is used to update the multi-agent reinforcement learning model.

[0083] In the MiniMax Q-learning algorithm, a single agent considers the behavior of its competitors and then chooses a behavior based on the current state to maximize its benefits in the worst case scenario (assuming that all competitors will take actions that reduce their benefits);

[0084] MiniMax Q-learning algorithm is a Q-learning algorithm whose purpose is to find the optimal strategy:

[0085] π i :s i →a i ;

[0086] The MiniMax Q-learning algorithm redefines the Q-value function as the expected reward for taking an action when the competitor chooses an action from a state and continues to act optimally in this state. i The MiniMax return value calculation formula is:

[0087] Among them A i ' represents the behavior of all competitors of the i-th agent. The state is S i Action A i Action A on competitor i The expected reward function of ' is:

[0088]

[0089] Among them, k is the iteration step size for the i-th agent to update its Q value, and α is the learning rate to ensure the convergence of the updated reward.

[0090] The following is a comparison between different existing methods and the method provided by the present application to further illustrate the beneficial effects of the present application:

[0091] (1) The GS method was developed when the energy supply was higher with reference to the “green scheduling” of an energy green data center. In GS, the energy green data center uses FFT to predict the power generation at each moment in the next month and sends its demand to the generator with the highest predicted power generation (the sum of the predicted value sequence). The job scheduler calculates the energy demand of each job based on the computing resources allocated to each job. The generator distributes energy among multiple requesters and notifies them. If the energy green data center cannot receive the required amount of electricity from the generator, it will request the remaining electricity from the next generator with the highest predicted power generation. This process is repeated until the total demand of the energy green data center is met.

[0092] (2) REM (Renewable Energy Management) was developed by referring to a job scheduler-based approach of an energy green data center, which aims to minimize the monetary cost of energy. REM is similar to the above method, except that the energy green data center uses a method to make predictions and first selects the generator with the lowest average unit price within a month for request to minimize the monetary cost.

[0093] (3) The intention approach draws on renewable energy-aware reinforcement learning. This approach is the same as GS, except that when the energy green data center does not have sufficient renewable energy, it uses RL to determine which jobs in each hour need to be postponed to the next moment (i.e., the next hour). The input to RL includes the deadline and energy requirement of each job, and the output is the energy green data center that runs each job.

[0094] (4) SRL (Single Reinforcement Learning) draws on the method of using RL for single energy green data center renewable energy management by a single energy green data center to determine which energy green data center to use. In this method, each energy green data center uses LSTM ML technology to predict the power generation of each generator, and uses RL to determine which energy generator to request power from from one energy green data center to minimize monetary costs and carbon emissions under lower SLO violations. However, due to the competition for energy demand among all energy green data centers, each energy green data center may not be able to achieve its goal easily, and they also need several times to reach sufficient energy, resulting in higher energy prices.

[0095] The effectiveness of each component in the multi-agent reinforcement learning model can be seen from the performance differences of different comparison methods. First, GS uses FFT for power generation and demand forecasting, while REM uses the same forecasting method (XGBoost) as the multi-agent reinforcement learning model, so the performance difference between GS and REM shows the performance improvement obtained by the forecasting method in the multi-agent reinforcement learning model. Specifically, REM improves GS by 1%, 10%, and 9% in SLO satisfaction, monetary cost, and carbon emissions, respectively. Second, SRL adopts a single-agent RL method that does not consider agent competition, while the multi-agent reinforcement learning model adopts a multi-agent RL method that considers agent competition, so the performance difference between SRL and the multi-agent reinforcement learning model W / oD shows the performance improvement obtained by the multi-agent RL. Specifically, the multi-agent reinforcement learning model W / oD improves SRL by 20%, 13%, and 10% in SLO satisfaction, monetary cost, and carbon emissions, respectively. The performance improvement obtained by the delay of the job with guaranteed deadlines is represented by the difference between the multi-agent reinforcement learning model W / OD and the multi-agent reinforcement learning model. The multi-agent reinforcement learning model improves marw / oD by 3%, 5%, and 4% in terms of SLO satisfaction rate, monetary cost, and carbon emissions, respectively.

[0096] The predicted and actual renewable energy output and prediction accuracy of solar and wind energy for a randomly selected solar generator, a randomly selected solar generator, and a randomly selected wind generator using XGBoost for three consecutive days. It is seen that the renewable energy generation follows a one-day cycle pattern and most of the predicted values ​​are very close to the actual values. The measurement accuracy of solar energy is always above 90% and the measurement accuracy of wind energy is always above 70%. The results also show that the accuracy of solar energy prediction is higher than that of wind energy prediction. From the actual solar energy values, it is observed that solar energy has a stronger seasonality, so its prediction accuracy is higher than that of wind energy.

[0097] Energy consumption of a single randomly selected energy green data center and all 90 energy green data centers from the generated energy green data centers from March 1, 2015 to May 31, 2015. Within a certain period of time, energy consumption presents a periodic pattern (in units of 7 days), which makes energy demand prediction possible.

[0098] According to the relationship between SLO satisfaction and the number of days in the time series (the first six months). MARL is represented by MARLw / oD, excluding DGJP. The results are: MARL>MARLw / oD>SRL>REA>REMGS. MARL and MARLw / oD achieve SLO satisfaction rates of more than 97% and 96%, respectively. The proportions of SRL, REA, REM and GS are about 76%, 75%, 72% and 72%, respectively. MARL improves the comparison method by 35%. Both REM and GS use a forecasting method based on FFT patterns to predict the power generation of renewable energy. Compared with XGBoost, the lower prediction accuracy may mislead energy allocation, so that REM and GS cannot always use renewable energy to meet energy demand, resulting in lower SLO satisfaction. REA uses the same forecasting model as GS and uses RL for job scheduling, which can achieve better job scheduling decisions because RL can react to poor job scheduling decisions. SRL uses LSTM for workload and energy generation in long-term forecasting, and uses RL in an energy green data center to determine how much energy should be requested from which energy generator in a month. Achieving lower SLO violation is one of its goals. Meanwhile, its prediction accuracy is higher than FFT, resulting in higher SLO satisfaction than REA. However, the RL agents on different energy green data centers do not consider the competition between energy green data centers, resulting in some energy green data centers not being able to obtain sufficient energy. MARL and MARLw / oD implement a MARL agent on each energy green data center and use XGBoost to predict the long-term energy demand and energy consumption. XGBoost has a higher prediction accuracy than LSTM. Therefore, MARL and MARLw / oD can generate better energy green data center-generator matching plans based on more accurate predictions, thereby reducing the possibility that renewable energy cannot meet the needs of one energy green data center. At the same time, using the MARL algorithm, in MARL and MARLw / oD, the RL agent considers the competition between agents and makes decisions on the energy green data center to generate matching plans to always meet the energy needs of the energy green data center, regardless of the actions of other competitors. In addition, MARL also has a DGJP algorithm that can postpone the running time of some jobs from the time when renewable energy is insufficient to the time when renewable energy is sufficient, while ensuring the deadline of the job. Therefore, MARL and MARL / oD can always achieve the highest SLO performance ratio than other methods. When energy supply is insufficient, MARLw / oD will not postpone the running of some jobs, so that some jobs may not be completed before the deadline due to energy shortage. Therefore, the SLO satisfaction of MARLw / oD is lower than that of MARL.

[0099] According to the relationship between the total cost of each method and the number of energy-efficient green data centers. MARL < MARL w / o D < SRL < REM < REA < GS. Among the default 90 energy-efficient green data centers, the monetary costs of MARL and MARL w / o D are 1.25 billion US dollars and 1.29 billion US dollars respectively. The monetary costs of SRL, REA, REM, and GS are 1.32 billion; 1.41 billion, 1.54 billion, and 1.58 billion respectively. MARL reduces the monetary cost of the comparison methods by 19% and earns 330 million US dollars within 90 days. GS and REA do not consider the monetary cost, so the monetary costs generated by these two methods are higher than those of other methods. At the same time, the lower SLO satisfaction due to the lower prediction accuracy of power generation means that these two methods use more brown energy to meet the energy demand, and due to the higher unit price of brown energy, the monetary cost increases. Therefore, GS and REA generate the highest monetary costs. Since REA has an RL algorithm, it can delay jobs when there is not enough renewable energy in the energy-efficient green data center, thus reducing brown energy consumption. Therefore, the monetary cost of REA is lower than that of GS. It is also seen that the monetary cost generated by REA is lower than that of GS and REM because the goal of REM is to reduce the total monetary cost while using as much renewable energy as possible. In addition, its SLO satisfaction is lower than that of GS, which means it uses less brown energy than GS.

[0100] Minimizing the monetary cost is one of the goals of SRL. SRL can formulate a better energy-efficient green data center - generator matching plan to provide an adequate supply of renewable energy, which helps to reduce the use of brown energy due to the shortage of renewable energy. This helps to reduce the monetary cost. Therefore, the monetary cost generated by SRL is lower than that of REM, REA, and GS. MARL and MARL w / o D formulate a better energy-efficient green data center - generator matching plan than SRL, reducing the number of shortages of renewable energy.

[0101] According to the relationship between the carbon emissions of each method and the number of energy-efficient green data centers. MARL = MARL w / o D < SRL < REA < REM < GS. The carbon emissions of MARL and MARL w / o D are 1.21 tons and 1.23 tons respectively. The carbon emissions of GS, REM, REA, and SRL are 1.8 tons, 1.6 tons, 1.31 tons, and 1.27 tons respectively. MARL and MARL w / o D can reduce carbon emissions by up to 33% and 0.59 tons within 90 days. GS has the largest carbon emissions because GS uses FFT energy prediction with low accuracy. The less accurate the energy prediction, the lower the efficiency of the energy-efficient green data center - generator matching plan, and the more brown energy is required, resulting in the highest carbon emissions. Since renewable energy is cheaper than brown energy, even with the same prediction method, REM aims to reduce the monetary cost while attempting to use more renewable energy, which leads to using more renewable energy than GS, thus reducing carbon emissions. REA has an RL to decide which tasks are postponed to the next time period, so it uses less brown energy, thereby reducing carbon emissions.

[0102] According to the average time overhead for generating a demand - supply matching plan for an energy-efficient green data center for each method. The time overhead only measures the decision-making time and does not consider the training time of reinforcement learning-based methods. MARL w / o D < MARL < SRL < GS REM ≈ REA. The time overheads of GS, REM, REA, SRL, MARL w / o D, and MARL are 102 ms, 95 ms, 94 ms, 53 ms, 43 ms, and 48 ms respectively. GS, REM, and REA need more time to find an energy generator that can meet their energy demands because they do not consider long-term planning and energy competition. MARL and MARL w / o D use multi-agent RL. Considering energy competition, these two methods use fewer energy requests when generating an energy-efficient green data center generator matching plan, thus reducing the decision-making time overhead. The overhead of MARL is a bit larger than that of MARL w / o D because MARL needs to execute DGJP, which brings additional time overhead. Therefore, MARL and MARL w / o D can achieve better time overhead performance.

[0103] According to the average SLO satisfaction for different numbers of energy-efficient green data centers within two years. The obtained results are in the order of MARL > MARL W / OD > SRL > REA > GS REM. It is worth noting that as the number of energy-efficient green data centers increases, MARL can always achieve an SLO satisfaction of over 98%, indicating its high scalability.

[0104] Based on the same inventive concept, the embodiment of the present application also provides a multi-energy green data center task scheduling device for implementing the multi-energy green data center task scheduling method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more multi-energy green data center task scheduling device embodiments provided below can refer to the limitations of the multi-energy green data center task scheduling method above, and will not be repeated here.

[0105] In an exemplary embodiment, a multi-energy green data center task scheduling device is provided, comprising:

[0106] The process conversion module is used to convert the multi-energy green data center task scheduling process into a Markov process;

[0107] A model building module is used to build a multi-agent reinforcement learning model according to a Markov process; the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and obtain the expected return of each state-action pair after processing by the corresponding neural network; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator;

[0108] A data acquisition module is used to obtain information on the power generation of renewable energy generators, demand for energy green data centers, and electricity price information;

[0109] A demand determination module is used to determine the sequence of renewable energy power generation and energy green data center demand based on the XGBoost prediction method according to the information of renewable energy generator power generation and energy green data center demand;

[0110] A state determination module, used to determine the current state of each energy green data center according to the renewable energy power generation and energy green data center demand sequence and electricity price information;

[0111] An action determination module is used to determine the action of each energy green data center based on the current state of each energy green data center using a multi-agent reinforcement learning model;

[0112] A model updating module is used to obtain the real reward from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model according to the expected reward and the real reward.

[0113] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a multi-energy green data center task scheduling method is implemented.

[0114] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0115] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0116] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0117] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0118] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0119] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A multi-energy green data center task scheduling method, characterized in that: The multi-energy green data center task scheduling method includes: Transform the multi-energy green data center task scheduling process into a Markov process; A multi-agent reinforcement learning model is constructed according to the Markov process; the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and obtain the expected return of each state-action pair after processing by the corresponding neural network; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator; Obtain information on the power generation of renewable energy generators and the demand for energy from green data centers, as well as electricity price information; According to the information of renewable energy generator power generation and energy green data center demand, the sequence of renewable energy power generation and energy green data center demand is determined based on the XGBoost prediction method; Determine the current status of each energy green data center based on the renewable energy power generation and energy green data center demand sequence and electricity price information; According to the current status of each energy green data center, a multi-agent reinforcement learning model is used to determine the action of each energy green data center; Obtain true rewards from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model based on the expected rewards and true rewards.

2. The multi-energy green data center task scheduling method according to claim 1 is characterized in that: The multi-energy green data center task scheduling process is converted into a Markov process, specifically including: Using the formula Determine the state space S i ; Using the formula Determine the action space A of the i-th agent i ; Determine the reward function based on monetary costs, carbon emissions, and SLO violations caused by energy shortages; in, is the kth renewable energy generator at t z The predicted power generation at the time, is the unit price of the generator set at different time periods, D i is the predicted time series energy demand vector of the i-th energy green data center; From the energy generator G k Get the energy value.

3. The multi-energy green data center task scheduling method according to claim 2 is characterized in that: The reward function is determined based on the SLO violation caused by monetary cost, carbon emissions and energy shortage, and specifically includes: Using the formula Determine the reward function R for Z time period i ; Among them, α1, α2, and α3 are weight coefficients of different features. is the reward function z The number of SLO violations at the time, For the generator set G k In t z Carbon emissions per kilowatt-hour at any given moment, For the i-th energy green data center at t z At the moment from the energy generator G k Buy is the monetary cost of the energy paid, c is the monetary cost of switching on the renewable energy generator, t z is a binary variable indicating whether there is a generator switch at the moment, and K is the total number of renewable energy generators.

4. The multi-energy green data center task scheduling method according to claim 3 is characterized in that: The loss function of the renewable energy generator power generation prediction process at different time periods includes: Using the formula Determine the loss function L(θ) of the XGBoost prediction method; in, is the actual power generation, To predict the power generation, Ω(f x ) is the complexity regularization term used to avoid overfitting, and n is the total number of energy green data centers.

5. The multi-energy green data center task scheduling method according to claim 4 is characterized in that: The multi-agent reinforcement learning model is updated according to the expected reward and the actual reward, specifically including: According to the expected reward and the actual reward, the MiniMax Q-learning algorithm is used to update the multi-agent reinforcement learning model.

6. A multi-energy green data center task scheduling device, characterized in that: The multi-energy green data center task scheduling device includes: The process conversion module is used to convert the multi-energy green data center task scheduling process into a Markov process; A model building module is used to build a multi-agent reinforcement learning model according to a Markov process; the multi-agent reinforcement learning model includes multiple agents; each agent includes a neural network composed of multiple energy green data centers; the agent is used to take the predicted sequence of generator power generation and energy green data center demand as input, and obtain the expected return of each state-action pair after processing by the corresponding neural network; the agent selects an action according to the expected return; and determines the expected reward and the next new state; the action is the value of energy requested by the energy green data center from each energy generator; A data acquisition module is used to obtain information on the power generation of renewable energy generators, demand for energy green data centers, and electricity price information; A demand determination module is used to determine the sequence of renewable energy power generation and energy green data center demand based on the XGBoost prediction method according to the information of renewable energy generator power generation and energy green data center demand; A state determination module, used to determine the current state of each energy green data center according to the renewable energy power generation and energy green data center demand sequence and electricity price information; An action determination module is used to determine the action of each energy green data center based on the current state of each energy green data center using a multi-agent reinforcement learning model; A model updating module is used to obtain the real reward from the environment according to the actions of each energy green data center; and update the multi-agent reinforcement learning model according to the expected reward and the real reward.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-energy green data center task scheduling method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-energy green data center task scheduling method described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-energy green data center task scheduling method described in any one of claims 1 to 5 is implemented.

Images