Adaptive power capping method and system considering uncertainty of cloud data center

Abstract

The invention relates to an adaptive power capping method and system considering uncertainty of a cloud data center. The dynamic power capping process of the cloud data center is modeled as a Markov decision process, the task execution condition, the energy consumption condition and the electricity price fluctuation are used as uncertain parameters, a power capping decision process model of the cloud data center is obtained, and the dynamic power capping decision process of the cloud data center is obtained through a reinforcement learning algorithm of two-stage uncertainty perception. Training the power capping decision process model, and outputting a self-adaptive power capping decision; the robustness and adaptability of the power capping strategy to complex scenes such as electricity price fluctuation, task random arrival and regulation and control decision change are improved, the efficient power capping strategy is learned under fewer interaction samples, the power capping decision is dynamically adjusted to respond to the electricity price change, the energy consumption cost is reduced, and meanwhile the service quality of the calculation task is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an adaptive power capping method and system considering the uncertainty of a cloud data center. Background Art

[0002] The rapid development of the digital economy and artificial intelligence has made cloud data center energy consumption increasingly prominent. Power capping, a commonly used technical measure, effectively reduces energy usage during peak electricity price periods and optimizes cloud data center operating energy consumption by setting a power limit for cloud data centers.

[0003] However, current power capping methods mostly focus on ensuring the effectiveness and performance of computing tasks, without considering how to track and respond to electricity prices through power capping technology. Existing methods typically make adjustments based on preset rules and static thresholds, or set power caps for the entire day based on day-ahead forecasts. Because these methods either adjust power caps based on preset rules or make decisions based on day-ahead forecasts, they lack the ability to dynamically couple with the real-time operating environment (such as electricity price fluctuations and random task arrivals). Faced with the highly uncertain and dynamically changing computing environments of cloud data centers, they face poor dynamic adaptability and an inability to adapt to changing information in the environment. This limits their ability to optimize energy consumption in real-world industrial scenarios and can lead to serious problems such as task execution failure. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the existing technology, the present invention aims to provide an adaptive power capping method and system that takes into account the uncertainty of cloud data centers, responds to electricity price fluctuations, reduces energy consumption costs, and ensures the normal execution of computing tasks.

[0005] In order to solve the above problems, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides an adaptive power capping method that considers uncertainty in a cloud data center, comprising:

[0007] The dynamic power capping process of cloud data centers is modeled as a Markov decision process. Task execution, energy consumption and electricity price fluctuations are taken as uncertain parameters to obtain the power capping decision process model of cloud data centers.

[0008] The power capping decision process model is trained through a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision.

[0009] As an implementable embodiment, the power capping decision process model includes states, actions, reward functions, transition functions, and policy functions.

[0010] The status Includes: actual energy consumption information of the cloud data center at time t , power capping value , Task execution violation rate , the total amount of resources required for the task , electricity price information .

[0011] The action Defined as the power cap set for a cloud data center, a discrete value between 0 and 100%.

[0012] The reward function Including energy consumption cost, task execution violation penalty and power cap adjustment cost, the energy consumption cost for , the task execution violation penalty for ,in is the penalty value, the power cap adjustment cost for ,in is the upper limit of the rated power of the cloud data center. To adjust costs.

[0013] The transfer function is established by setting up an environment simulation network Indicates that the status Take action After transfer to state , θ is the network parameter of the environment simulation network.

[0014] The policy function Indicates that the status Take action probability.

[0015] As an implementation method, the optimal strategy of the power capping decision process model is Defined as:

[0016] .

[0017] in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, E is the expected operation, are the network parameters of the decision value network.

[0018] As an implementable method, the training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes:

[0019] (1) Cloud data center based on decision value network Output selection power capping action .

[0020] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory , and the trajectory Stored in the trajectory buffer middle.

[0021] (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, until the loss function drops to the preset threshold , Environmental Simulation Network Training completed.

[0022] (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer In the figure, the superscript v stands for virtual.

[0023] (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training completed.

[0024] As an implementation method, the decision value network The loss function for:

[0025]

[0026] .

[0027] in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input to the time t. and t+1 , and For counting subscripts, is the differential error, is the indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise, and γ is the attenuation factor.

[0028] The environment simulation network The loss function for:

[0029]

[0030] .

[0031] in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, It is the state value that the t+1 sample simulated by the environment simulation network can transfer to the most. represents all possible t+1 samples simulated by the environment simulation network, is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, It is the estimation function of the decision value network for the value of the next state.

[0032] On the other hand, the present invention provides an adaptive power capping system that considers uncertainty in a cloud data center, comprising: a power capping decision process model construction module and a power capping decision process model training output module;

[0033] The power capping decision process model construction module is used to model the dynamic power capping process of the cloud data center as a Markov decision process, taking the task execution situation, energy consumption situation and electricity price fluctuation as uncertain parameters to obtain the power capping decision process model of the cloud data center.

[0034] The power capping decision process model training output module is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision.

[0035] As an implementable embodiment, the power capping decision process model includes states, actions, reward functions, transition functions, and policy functions.

[0036] The status Includes: actual energy consumption information of the cloud data center at time t , power capping value , Task execution violation rate , the total amount of resources required for the task , electricity price information .

[0037] The action Defined as the power cap set for a cloud data center, a discrete value between 0 and 100%.

[0038] The reward function Including energy consumption cost, task execution violation penalty and power cap adjustment cost, the energy consumption cost for , the task execution violation penalty for ,in is the penalty value, the power cap adjustment cost for ,in is the upper limit of the rated power of the cloud data center. To adjust costs.

[0039] The transfer function is established by setting up an environment simulation network Indicates that the status Take action After transfer to state , θ is the network parameter of the environment simulation network.

[0040] The policy function Indicates that the status Take action probability.

[0041] As an implementation method, the optimal strategy of the power capping decision process model is Defined as:

[0042] .

[0043] in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, E is the expected operation, are the network parameters of the decision value network.

[0044] As an implementable method, the training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes:

[0045] (1) Cloud data center based on decision value network Output selection power capping action .

[0046] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory , and the trajectory Stored in the trajectory buffer middle.

[0047] (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, until the loss function drops to the preset threshold , Environmental Simulation Network Training completed.

[0048] (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer In the figure, the superscript v stands for virtual.

[0049] (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training completed.

[0050] As an implementation method, the decision value network The loss function for:

[0051]

[0052] .

[0053] in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input to the time t. and t+1 , and For counting subscripts, is the differential error, is the indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise, and γ is the attenuation factor.

[0054] The environment simulation network The loss function for:

[0055]

[0056] .

[0057] in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, It is the state value that the t+1 sample simulated by the environment simulation network can transfer to the most. represents all possible t+1 samples simulated by the environment simulation network, is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, It is the estimation function of the decision value network for the value of the next state.

[0058] The beneficial effects of the present invention are: the adaptive power capping method and system considering the uncertainty of cloud data centers proposed in the present invention, and the design of a model-based reinforcement learning optimization algorithm for uncertainty perception of cloud data centers, significantly improve the robustness and adaptability of the power capping strategy to complex scenarios such as electricity price fluctuations, random arrival of tasks and changes in control decisions, and realize dynamic adjustment of power capping decisions in response to changes in electricity prices, reducing energy consumption costs while ensuring the service quality of computing tasks; at the same time, the sample efficiency of the present invention is significantly improved, and it is possible to learn efficient power capping strategies with fewer interactive samples, providing cloud data centers with an energy consumption management solution that takes into account both energy consumption cost optimization and service quality assurance. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of an adaptive power capping method considering the uncertainty of a cloud data center according to the present invention.

[0060] Figure 2 This is a schematic diagram of an adaptive power capping system that considers the uncertainty of a cloud data center according to the present invention.

[0061] Figure 3 The figure is a comparison chart of CPU resource usage under the method of the present invention and the comparative method.

[0062] Figure 4 The probability distribution diagram of the data center operating cost under the method of the present invention and the comparative method. DETAILED DESCRIPTION

[0063] The present invention will be further described in detail below with reference to specific embodiments.

[0064] It should be noted that these embodiments are only used to illustrate the present invention, rather than to limit the present invention. Simple improvements to the method based on the concept of the present invention fall within the scope of protection claimed by the present invention.

[0065] Example 1

[0066] See also Figure 1 , is an adaptive power capping method considering the uncertainty of cloud data centers, including:

[0067] S100. Model the dynamic power capping process of the cloud data center as a Markov decision process, taking task execution conditions, energy consumption conditions, and electricity price fluctuations as uncertain parameters, and obtain a power capping decision process model of the cloud data center.

[0068] The power capping decision process model of the cloud data center established by the present invention is an MDP, which includes five parts: {state, action, reward function, transfer function, and policy function}.

[0069] (1) Status Includes: actual energy consumption information of the cloud data center at time t , power capping value , Task execution violation rate , the total amount of resources required for the task and electricity price information Among them, the task execution situation, energy consumption situation and electricity price fluctuation are status Parameters included in .

[0070] (2) Action It is defined as the power cap set for the cloud data center, which is a discrete value from 0 to 100%, that is, [0, 1%, 2%, ..., 100%].

[0071] (3) Reward Function Energy consumption cost, task execution violation penalty and power cap adjustment cost are taken into account. Defined as actual energy consumption and electricity prices The negative product of .

[0072] If the task is not completed before the specified deadline as required by the user, it will incur a task execution violation penalty. , defined as the task execution violation rate multiplied by the penalty value ,Right now .

[0073] The power cap adjustment cost is due to the fact that large-scale power cap adjustment may affect task execution and cause hardware wear. In order to minimize the impact of the adjustment, the power cap adjustment cost is Defined as the power capping decision value Data center power rating cap The distance between the two times the adjustment cost ,Right now .

[0074] (4) Transfer function is achieved by establishing an environment simulation network Indicates that the status Take action After transfer to state The probability of θ is the network parameter of the environment simulation network. Represents the predicted value of the next state output by the environment simulation network.

[0075] (5) Strategy function Indicates that the status Take action probability.

[0076] The goal of this patent for the cloud data center power capping problem is to find the optimal strategy , to achieve the best trade-off between energy consumption cost and task execution.

[0077] Therefore, the optimal strategy of the power capping decision process model is Defined as:

[0078] .

[0079] in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, the meaning of the optimal strategy is to select the state The action with the largest mean value is the one with the largest mean value. E is the expected operation. The mean value of the value distribution is calculated according to the expected calculation formula. is the network parameter of the decision value network, Represents the selection of the expected value function Action to reach maximum As the action to be executed at time step t.

[0080] S200 : Training the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision.

[0081] This paper proposes a two-stage uncertainty-aware reinforcement learning algorithm to train the power capping decision process model, as follows:

[0082] Build two neural networks and environment simulation network and decision value network , representing the environment simulation phase and the decision-making phase, respectively. The environment simulation network is used to learn the dynamics of the cloud data center's operating environment, while the decision-making value network is used to learn the pros and cons of the cloud data center's power capping decisions. Through the continuous interaction of these two phases, the cloud data center learns about environmental uncertainties, makes decisions, and continuously updates these two networks until convergence is achieved, resulting in the final power capping strategy, which includes:

[0083] (1) Cloud data center based on decision value network Output selection power capping action .

[0084] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory ,in is the next state of the real cloud operating environment, and the trajectory Stored in the trajectory buffer middle.

[0085] (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, that is, to let the environment simulate the predicted value of the network output and the true value Closer until the loss function drops to a preset threshold , Environmental Simulation Network The training is completed, and the environment simulation network is considered It can effectively simulate the transfer dynamics of the cloud operating environment.

[0086] (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer In the example, the superscript v stands for virtual, i.e. For virtual state, For virtual actions, is the virtual reward function, For the virtual next state, etc.

[0087] (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training is complete. Implementing adaptive power capping strategy learning by utilizing the environment to simulate the network The generated virtual trajectories can accelerate the learning process of the power capping strategy and make the strategy more adaptable to the uncertainty and dynamic changes of the environment.

[0088] The neural network loss functions of the two stages are constructed as follows:

[0089] Decision Value Network The loss function for:

[0090]

[0091] .

[0092] in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input into the decision value network at time t. and the decision value network at time t+1 , and For counting subscripts, is the differential error, is an indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise. γ is the attenuation factor, which represents the attenuation of the estimate at time t+1 from the current time t.

[0093] By using the aforementioned quantile-based loss function, the uncertainty in cloud data center operations is expressed as a probability distribution and incorporated into the loss function of the decision-making process. This allows the power capping strategy to take into account the dynamic changes and uncertainties in the environment.

[0094] Loss function of the environment simulation network for:

[0095]

[0096] .

[0097] in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, and is the state value that the t+1 sample simulated by the environment simulation network can transfer to the maximum possible value. The next step for the real cloud operating environment, Represents all possible t+1 samples simulated by the environment simulation network. The integral of represents all possible t+1 samples simulated by the environment simulation network The mean of is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, is the estimation function of the decision value network for the next state value, Take the brackets When represents the state value estimation of the environment simulation network, Take the brackets When represents the estimation of the state value of the actual sample.

[0098] The loss function of the environment simulation network embeds the decision value network's estimation of policy value and uncertainty knowledge into the learning process. This not only captures the inherent uncertainty in cloud data center environments but also incorporates policy-related information. Tightly integrating the environment simulation process with the decision-making process helps cloud data centers better exploit the dynamic information feedback within the cloud environment, thereby improving the adaptability of power capping policies to dynamically changing environments.

[0099] The power capping decision process model trained by the present invention outputs an adaptive power capping decision.

[0100] See also Figure 2, is an adaptive power capping system that considers the uncertainty of cloud data centers, including a power capping decision process model construction module 100 and a power capping decision process model training output module 200.

[0101] The power capping decision process model building module 100 is used to model the dynamic power capping process of the cloud data center as a Markov decision process, taking the task execution situation, energy consumption situation and electricity price fluctuation as uncertain parameters to obtain the power capping decision process model of the cloud data center.

[0102] The power capping decision process model training and output module 200 is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision.

[0103] The power capping decision process model includes states, actions, reward functions, transition functions and policy functions.

[0104] state Includes: actual energy consumption information of the cloud data center at time t , power capping value , Task execution violation rate , the total amount of resources required for the task , electricity price information .

[0105] action Defined as the power cap set for a cloud data center, a discrete value between 0 and 100%.

[0106] Reward Function Including energy consumption cost, task execution violation penalty and power cap adjustment cost, energy consumption cost for , Penalties for violations in task execution for ,in is the penalty value, the power cap adjustment cost for ,in is the upper limit of the rated power of the cloud data center. To adjust costs.

[0107] Transfer function is achieved by establishing an environment simulation network Indicates that the status Take action After transfer to state The probability of θ is the network parameter of the environment simulation network.

[0108] Policy Function Indicates that the status Take action probability.

[0109] Optimal Strategy for Power Capping Decision Process Model Defined as:

[0110] .

[0111] in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, E is the expected operation, is the network parameter of the decision value network, Represents the selection of the expected value function Action to reach maximum As the action to be executed at time step t.

[0112] The power capping decision-making process model is trained using a two-stage uncertainty-aware reinforcement learning algorithm, including:

[0113] (1) Cloud data center based on decision value network Output selection power capping action .

[0114] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory , and the trajectory Stored in the trajectory buffer middle.

[0115] (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, until the loss function drops to the preset threshold , Environmental Simulation Network Training completed.

[0116] (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer middle.

[0117] (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training completed.

[0118] Decision Value Network The loss function for:

[0119]

[0120] .

[0121] in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input to the time t. and t+1 , and For counting subscripts, is the differential error, is the indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise, and γ is the attenuation factor.

[0122] Environmental Simulation Network The loss function for:

[0123]

[0124] .

[0125] in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, It is the state value that the t+1 sample simulated by the environment simulation network can transfer to the most. represents all possible t+1 samples simulated by the environment simulation network, is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, It is the estimation function of the decision value network for the value of the next state.

[0126] Example 2

[0127] This paper compares existing power capping technologies. Under the same electricity price and task arrival conditions, and the same environmental changes and uncertainty settings, the power capping method proposed in this paper has significant advantages. Comparison method 1 is a power capping setting method disclosed in the paper Radovanović A, Koningstein R, Schneider I, et al. Carbon-aware computing for datacenters[J]. IEEE Transactions on Power Systems, 2022, 38(2): 1270-1280. Comparison method 2 is a power capping value determination method, device, and storage medium disclosed in patent CN116204380A.

[0128] Depend on Figure 3 As can be seen, the rule-based approach (Comparative Method 2) lacks flexibility in responding to electricity price fluctuations and task arrivals, often failing to effectively adapt to these changes. Its overall power capping curve is relatively flat, demonstrating insufficient responsiveness. The predict-then-optimize approach (Comparative Method 1) better tracks electricity price trends, but it lacks consideration for uncertainty and dynamically changing environments, often resulting in excessively low power caps (as indicated by the shaded area). These overly restrictive constraints can hinder task execution, increase latency, and increase the risk of exceeding SLAs. In contrast, the proposed method can adapt to price fluctuations while accounting for uncertainty in task arrival and execution, thus avoiding overly restrictive constraints and ensuring smooth task completion.

[0129] See also Figure 4 , Figure 4 The distribution of total cost and task execution violation rates for three methods in an uncertain cloud data center environment is presented. The rule-based method (Comparison Method 2) relies solely on predefined rules and has limited adaptability to dynamic fluctuations in tasks and electricity prices, resulting in higher energy consumption and violations. Compared with the rule-based method, the prediction-then-optimization method (Comparison Method 1) offers some improvement, but its effectiveness is highly dependent on the accuracy of the prediction. Furthermore, because this method performs a one-time optimization one day in advance, it ignores differences in task scheduling strategies and real-time dynamic changes, limiting its adaptability and flexibility, leading to increased energy consumption and task violation rates. In contrast, the proposed method can dynamically learn and adapt to environmental uncertainties, achieving adaptive power capping adjustments. This adaptability reduces costs and violation rates.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes can be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An adaptive power capping method considering uncertainty in cloud data centers, characterized in that: include: The dynamic power capping process of cloud data centers is modeled as a Markov decision process. Task execution, energy consumption, and electricity price fluctuations are taken as uncertain parameters to obtain a power capping decision process model for cloud data centers. The power capping decision process model is trained through a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision.

2. The adaptive power capping method considering uncertainty of a cloud data center according to claim 1, characterized in that: The power capping decision process model includes states, actions, reward functions, transfer functions and policy functions; The status Includes: actual energy consumption information of the cloud data center at time t , power capping decision value , Task execution violation rate , the total amount of resources required for the task and electricity price information ; The action Defined as the power cap set for a cloud data center, a discrete value between 0 and 100%; The reward function Including energy consumption cost, task execution violation penalty and power cap adjustment cost, the energy consumption cost for , the task execution violation penalty for ,in is the penalty value, the power cap adjustment cost for ,in is the upper limit of the rated power of the cloud data center. To adjust costs; The transfer function is established by setting up an environment simulation network Indicates that the status Take action After transfer to state , θ is the network parameter of the environment simulation network; The policy function Indicates that the status Take action probability.

3. The adaptive power capping method considering uncertainty of cloud data center according to claim 2, characterized in that: The optimal strategy of the power capping decision process model Defined as: ； in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, E is the expected operation, are the network parameters of the decision value network.

4. The adaptive power capping method considering uncertainty of a cloud data center according to claim 3, characterized in that: The training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes: (1) Cloud data center based on decision value network Output selection power capping action ; (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory , and the trajectory Stored in the trajectory buffer middle; (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, until the loss function drops to the preset threshold , Environmental Simulation Network Training completed; (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer In the figure, the superscript v stands for virtual; (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training completed.

5. The adaptive power capping method considering uncertainty of cloud data center according to claim 4, characterized in that: The decision value network The loss function for: ； in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input to the time t. and t+1 , and For counting subscripts, is the differential error, is the indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise, and γ is the attenuation factor; The environment simulation network The loss function for: ； in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, It is the state value that the t+1 sample simulated by the environment simulation network can transfer to the most. represents all possible t+1 samples simulated by the environment simulation network, is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, It is the estimation function of the decision value network for the value of the next state.

6. An adaptive power capping system considering uncertainty in cloud data centers, characterized in that: It includes a power capping decision process model building module and a power capping decision process model training output module; The power capping decision process model building module is used to model the dynamic power capping process of the cloud data center as a Markov decision process, taking the task execution status, energy consumption status and electricity price fluctuation as uncertain parameters to obtain the power capping decision process model of the cloud data center; The power capping decision process model training output module is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision.

7. The adaptive power capping system considering uncertainty of cloud data center according to claim 6, characterized in that: The power capping decision process model includes states, actions, reward functions, transfer functions and policy functions; The status Includes: actual energy consumption information of the cloud data center at time t , power capping value , Task execution violation rate , the total amount of resources required for the task and electricity price information ; The action Defined as the power cap set for a cloud data center, a discrete value between 0 and 100%; The reward function Including energy consumption cost, task execution violation penalty and power cap adjustment cost, the energy consumption cost for , the task execution violation penalty for ,in is the penalty value, the power cap adjustment cost for ,in is the upper limit of the rated power of the cloud data center. To adjust costs; The transfer function is established by setting up an environment simulation network Indicates that the status Take action After transfer to state , θ is the network parameter of the environment simulation network; The policy function Indicates that the status Take action probability.

8. The adaptive power capping system considering cloud data center uncertainty according to claim 7, characterized in that: The optimal strategy of the power capping decision process model Defined as: ； in, is the decision value network, representing the state Take action The probability distribution of the value, the probability distribution through quantiles To express, E is the expected operation, are the network parameters of the decision value network.

9. The adaptive power capping system considering cloud data center uncertainty according to claim 8, characterized in that: The training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes: (1) Cloud data center based on decision value network Output selection power capping action ; (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save it as a trajectory , and the trajectory Stored in the trajectory buffer middle; (3) Using the trajectory buffer The trajectory data in the training environment simulates the network , in order to learn the dynamic response of the real cloud operating environment to the power capping action, until the loss function drops to the preset threshold , Environmental Simulation Network Training completed; (4) Use training completed and Generate virtual trajectory , and the virtual trajectory It is stored in the virtual track buffer In the figure, the superscript v stands for virtual; (5) Buffer the trajectory and virtual trajectory buffer Take out a part of the real and virtual trajectories to jointly train the decision value network , return to (1) until the loss function of the decision value network converges, the decision value network Training completed.

10. The adaptive power capping system considering cloud data center uncertainty according to claim 9, characterized in that: The decision value network The loss function for: ； in, is the Huber loss function, is the threshold of the Huber loss function, N is the number of quantile samples, and They are two groups of quantiles sampled from the uniform distribution U[0,1] from 0 to 1, and are input to the time t. and t+1 , and For counting subscripts, is the differential error, is the indicator function, which takes the value 1 when the inequality in the subscript holds, and takes the value 0 otherwise, and γ is the attenuation factor; The environment simulation network The loss function for: ； in, Represents the set of sampled trajectories , , t is the trajectory sample number, N is the total number of trajectory samples, It is the state value that the t+1 sample simulated by the environment simulation network can transfer to the most. represents all possible t+1 samples simulated by the environment simulation network, is the expectation operator, Represents the arithmetic mean of all sampled trajectories. Representatives All possible actions are taken as the arithmetic average, It is the estimation function of the decision value network for the value of the next state.

Images