An Adaptive Power Capping Method and System Considering Uncertainties in Cloud Data Centers

Abstract

This invention relates to an adaptive power capping method and system that considers the uncertainties of cloud data centers. This method models the dynamic power capping process of cloud data centers as a Markov decision process, using task execution status, energy consumption, and electricity price fluctuations as uncertain parameters to obtain a power capping decision process model. The model is then trained using a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision. This improves the robustness and adaptability of the power capping strategy to complex scenarios such as electricity price fluctuations, random task arrival, and changes in control decisions. It learns an efficient power capping strategy with fewer interaction samples, enabling dynamic adjustment of power capping decisions to respond to electricity price changes, reducing energy costs while ensuring the quality of service for computing tasks.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and more specifically to an adaptive power capping method and system that takes into account the uncertainties of cloud data centers. Background Technology

[0002] The rapid development of the digital economy and artificial intelligence has made the energy consumption of cloud data centers increasingly prominent. Power capping, as a commonly used technical means, can effectively reduce energy consumption during peak electricity price periods and optimize the energy consumption of cloud data center operations by setting an upper limit on the power usage of cloud data centers.

[0003] However, current power capping methods mostly focus on ensuring the execution effect and computing performance of computing tasks, without considering how to follow and respond to electricity prices through power capping technology. Furthermore, existing methods typically make judgments and adjustments based on preset rules and static thresholds, or set a power cap value for the entire day based on the previous day's forecast. Because these methods either adjust the power cap value based on preset rules or make decisions based on the previous day's forecast, they lack dynamic coupling with the real-time operating environment (such as electricity price fluctuations and random task arrival). When facing the highly uncertain and dynamically changing computing environment of cloud data centers, they suffer from poor dynamic adaptability and cannot adaptively adjust to changes in environmental information, limiting their ability to optimize energy consumption in real-world industrial scenarios and potentially leading to serious problems such as task execution failures. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, this invention aims to provide an adaptive power capping method and system that takes into account the uncertainties of cloud data centers, responding to electricity price fluctuations, reducing energy costs while ensuring the normal execution of computing tasks.

[0005] To solve the above problems, the present invention adopts the following technical solution:

[0006] On one hand, the present invention provides an adaptive power capping method that considers the uncertainties of cloud data centers, comprising:

[0007] The dynamic power capping process of cloud data centers is modeled as a Markov decision process, with task execution status, energy consumption, and electricity price fluctuations as uncertain parameters, to obtain a power capping decision process model for cloud data centers.

[0008] The power capping decision process model is trained using a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision.

[0009] As one possible implementation, the power capping decision process model includes states, actions, reward functions, transition functions, and policy functions.

[0010] The state Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping value Task execution violation rate Total resource requirements for the task Electricity price information .

[0011] The action Defined as a power cap set for cloud data centers, it is a discrete value ranging from 0 to 100%.

[0012] The reward function This includes energy consumption costs, penalties for task execution violations, and power limit adjustment costs. The energy consumption costs... for The task execution violation penalty for ,in The power cap adjustment cost is the penalty value. for ,in This represents the maximum rated power for cloud data centers. To adjust costs.

[0013] The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state θ represents the network parameters of the environment simulation network.

[0014] The strategy function Indicates the state Take action below The probability of.

[0015] As one possible implementation method, the optimal strategy of the power capping decision process model Defined as:

[0016] .

[0017] in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. Let E represent the expected value operation. These are the network parameters of the decision-making value network.

[0018] As one possible implementation, training the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes:

[0019] (1) Cloud data centers are based on decision value networks Output selection power capping action .

[0020] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. and the trajectory Stored in the trajectory buffer middle.

[0021] (3) Use the trajectory buffer Trajectory data training environment simulation network The goal is to learn the dynamic response of a real cloud operating environment to power capping actions until the loss function drops to a preset threshold. Environmental simulation network Training complete.

[0022] (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. In this context, the superscript 'v' represents virtual.

[0023] (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete.

[0024] As one possible implementation method, the decision value network loss function for:

[0025]

[0026] .

[0027] in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and Let U and y be two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1, respectively, and input to time t. and time t+1 , and For counting subscripts, For difference error, The function is an indicator function, which takes the value 1 if the inequality in the subscript is true, and 0 otherwise. γ is the decay factor.

[0028] The environmental simulation network loss function for:

[0029]

[0030] .

[0031] in, Represents the set of sampled trajectories , t is the trajectory sample number, and N is the total number of trajectory samples. It is the state value that the sample at time t+1 can most likely transition to, as simulated by the environmental simulation network. This represents all possible t+1 samples simulated by the environment simulation network. It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state.

[0032] On the other hand, the present invention provides an adaptive power capping system that considers the uncertainty of cloud data centers, including: a power capping decision process model construction module and a power capping decision process model training output module;

[0033] The power capping decision process model building module is used to model the dynamic power capping process of cloud data centers as a Markov decision process, taking the task execution status, energy consumption status and electricity price fluctuations as uncertain parameters to obtain the power capping decision process model of cloud data centers.

[0034] The power capping decision process model training output module is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision.

[0035] As one possible implementation, the power capping decision process model includes states, actions, reward functions, transition functions, and policy functions.

[0036] The state Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping value Task execution violation rate Total resource requirements for the task Electricity price information .

[0037] The action Defined as a power cap set for cloud data centers, it is a discrete value ranging from 0 to 100%.

[0038] The reward function This includes energy consumption costs, penalties for task execution violations, and power limit adjustment costs. The energy consumption costs... for The task execution violation penalty for ,in The power cap adjustment cost is the penalty value. for ,in This represents the maximum rated power for cloud data centers. To adjust costs.

[0039] The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state θ represents the network parameters of the environment simulation network.

[0040] The strategy function Indicates the state Take action below The probability of.

[0041] As one possible implementation method, the optimal strategy of the power capping decision process model Defined as:

[0042] .

[0043] in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. Let E represent the expected value operation. These are the network parameters of the decision-making value network.

[0044] As one possible implementation, training the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes:

[0045] (1) Cloud data centers are based on decision value networks Output selection power capping action .

[0046] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. and the trajectory Stored in the trajectory buffer middle.

[0047] (3) Use the trajectory buffer Trajectory data training environment simulation network The goal is to learn the dynamic response of a real cloud operating environment to power capping actions until the loss function drops to a preset threshold. Environmental simulation network Training complete.

[0048] (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. In this context, the superscript 'v' represents virtual.

[0049] (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete.

[0050] As one possible implementation method, the decision value network loss function for:

[0051]

[0052] .

[0053] in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and Let U and y be two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1, respectively, and input to time t. and time t+1 , and For counting subscripts, For difference error, The function is an indicator function, which takes the value 1 if the inequality in the subscript is true, and 0 otherwise. γ is the decay factor.

[0054] The environmental simulation network loss function for:

[0055]

[0056] .

[0057] in, Represents the set of sampled trajectories , t is the trajectory sample number, and N is the total number of trajectory samples. It is the state value that the sample at time t+1 can most likely transition to, as simulated by the environmental simulation network. This represents all possible t+1 samples simulated by the environment simulation network. It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state.

[0058] The beneficial effects of this invention are as follows: The adaptive power capping method and system considering the uncertainties of cloud data centers proposed in this invention design a model-based reinforcement learning optimization algorithm for cloud data center uncertainty awareness, which significantly improves the robustness and adaptability of the power capping strategy to complex scenarios such as electricity price fluctuations, random task arrival, and changes in control decisions. It realizes dynamic adjustment of power capping decisions to respond to changes in electricity prices, reducing energy consumption costs while ensuring the service quality of computing tasks. At the same time, the sample efficiency of this invention is significantly improved, realizing the learning of efficient power capping strategies with fewer interaction samples, providing cloud data centers with an energy management solution that balances energy consumption cost optimization and service quality assurance. Attached Figure Description

[0059] Figure 1 This is a flowchart of an adaptive power capping method for cloud data centers that takes into account uncertainties, according to the present invention.

[0060] Figure 2 This is a schematic diagram of an adaptive power capping system that takes into account the uncertainties of cloud data centers according to the present invention.

[0061] Figure 3 This is a comparison chart of CPU resource usage under the method of the present invention and the comparative method.

[0062] Figure 4 This is a probability distribution diagram of data center operating costs under the method of the present invention and the comparative method. Detailed Implementation

[0063] The present invention will be further described in detail below with reference to specific embodiments.

[0064] It should be noted that these embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Simple improvements to the method under the premise of the present invention are all within the scope of protection claimed by the present invention.

[0065] Example 1

[0066] See Figure 1 An adaptive power capping method considering the uncertainties of cloud data centers includes:

[0067] S100. The dynamic power capping process of the cloud data center is modeled as a Markov decision process. The task execution status, energy consumption status and electricity price fluctuations are used as uncertain parameters to obtain the power capping decision process model of the cloud data center.

[0068] The cloud data center power capping decision process model established in this invention is an MDP, which includes five parts: {state, action, reward function, transition function, and policy function}.

[0069] (1) State Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping value Task execution violation rate Total resource requirements for the task Electricity price information The status quo refers to the task execution status, energy consumption, and electricity price fluctuations. The parameters included.

[0070] (2) Actions Defined as the power cap set for a cloud data center, it is a discrete value from 0 to 100%, i.e., [0, 1%, 2%, ..., 100%].

[0071] (3) Reward function Energy costs, penalties for task execution violations, and power limit adjustment costs were considered. Energy costs... Defined as actual energy consumption With electricity price The negative product, i.e. .

[0072] If the task is not completed by the specified deadline as required by the user, a task execution violation penalty will be imposed. Defined as the task execution violation rate multiplied by the penalty value. ,Right now .

[0073] The cost of adjusting the power limit is due to the fact that large-scale adjustments to the power limit may affect task execution and cause hardware wear. To minimize the impact of such adjustments, the cost of adjusting the power limit is [not specified]. Defined as power capping decision value With the upper limit of the rated power of the data center Distance multiplied by adjustment cost ,Right now .

[0074] (4) The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state The probability is given by θ, where θ is the network parameter of the environment simulation network. This represents the predicted value of the next state output by the environment simulation network.

[0075] (5) Policy function Indicates the state Take action below The probability of.

[0076] The goal of this patent in addressing the power capping problem in cloud data centers is to find the optimal strategy. This is to achieve the best balance between energy consumption costs and task execution.

[0077] Therefore, the optimal strategy of the power capping decision process model is... Defined as:

[0078] .

[0079] in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. In other words, the optimal strategy means choosing a state. The action that maximizes the mean value is taken. E represents the expected value operation, and the mean of the value distribution is calculated according to the expected value calculation formula. For the network parameters of the decision value network, Representative selection enables the expected value function Actions that reach the maximum value This is the action to be executed at time step t.

[0080] S200. The power capping decision process model is trained using a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision.

[0081] This invention proposes a two-stage uncertainty-aware reinforcement learning algorithm to train a power capping decision process model, as detailed below:

[0082] Construct two neural networks: an environment simulation network. and decision-making value network These represent the environment simulation phase and the decision-making phase, respectively. The environment simulation network learns the dynamic transitions of the cloud data center's operating environment, while the decision value network learns the merits of power capping decisions within the cloud data center. Through continuous interaction between these two phases, the cloud data center learns about uncertainties in the environment, makes decisions, and continuously updates these two networks until convergence, yielding the final power capping strategy, including:

[0083] (1) Cloud data centers are based on decision value networks Output selection power capping action .

[0084] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. ,in The next state of the real cloud operating environment, and the trajectory Stored in the trajectory buffer middle.

[0085] (3) Use the trajectory buffer Trajectory data training environment simulation network To learn the dynamic response of a real cloud operating environment to power capping actions, that is, to let the environment simulate the predicted values ​​output by the network. Compared with the true value Get closer, until the loss function drops to a preset threshold. Environmental simulation network Training is complete; at this point, the environment simulation network is considered to be... It can effectively simulate the migration dynamics of the cloud operating environment.

[0086] (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. In this context, the superscript 'v' represents virtual, i.e. For virtual state, For virtual actions, For virtual reward function, For example, the virtual next state.

[0087] (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete. The adaptive power capping strategy was learned by utilizing an environmental simulation network. The generated virtual trajectory can accelerate the learning process of the power capping policy and make the policy more adaptable to environmental uncertainties and dynamic changes.

[0088] The loss functions for the two stages of the neural network are constructed as follows:

[0089] Decision Value Network loss function for:

[0090]

[0091] .

[0092] in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and The two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1 are respectively input into the decision value network at time t. The decision value network at time t+1 , and For counting subscripts, For difference error, The indicator function takes the value 1 when the inequality in the subscript is true, and 0 otherwise. γ is the decay factor, which represents the estimated decay of time t+1 from the current time t.

[0093] By using the aforementioned quantile-based loss function, the uncertainties in the operation of cloud data centers can be expressed using probability distributions and incorporated into the loss function of the decision-making process. This allows the power capping strategy to take into account dynamic changes and uncertainties in the environment.

[0094] Loss function of environmental simulation network for:

[0095]

[0096] .

[0097] in, Represents the set of sampled trajectories , t is the trajectory sample number, N is the total number of trajectory samples, and N is the maximum possible state value that the t+1 sample can transition to, as simulated by the environment simulation network. The next step for a real cloud operating environment, Represents all possible t+1 samples simulated by the environment simulation network, for The integral represents all possible values ​​of sample t+1 simulated by the environment simulation network. The mean, It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state. Take from parentheses Time represents the state value estimate simulated by the environmental simulation network. Take from parentheses Time represents the value estimate of the actual sampled state.

[0098] The loss function of the aforementioned environmental simulation network embeds the estimation of policy value and uncertainty knowledge from the decision value network into the learning process of the environmental simulation network. This not only captures the inherent uncertainties in the cloud data center environment but also includes policy-related information. By closely integrating the environmental simulation process with the decision-making process, cloud data centers can better explore dynamic information feedback in the cloud environment, thereby improving the adaptability of the power cap policy to dynamically changing environments.

[0099] The power capping decision process model trained by this invention outputs an adaptive power capping decision.

[0100] See Figure 2An adaptive power capping system that takes into account the uncertainties of cloud data centers includes a power capping decision process model building module 100 and a power capping decision process model training output module 200.

[0101] The power capping decision process model building module 100 is used to model the dynamic power capping process of cloud data centers as a Markov decision process, taking the task execution status, energy consumption status and electricity price fluctuations as uncertain parameters to obtain the power capping decision process model of cloud data centers.

[0102] The power capping decision process model training output module 200 is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision.

[0103] The power capping decision process model includes states, actions, reward functions, transition functions, and policy functions.

[0104] state Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping value Task execution violation rate Total resource requirements for the task Electricity price information .

[0105] action Defined as a power cap set for cloud data centers, it is a discrete value ranging from 0 to 100%.

[0106] reward function This includes energy consumption costs, penalties for violations during task execution, and costs associated with adjusting power limits. for Penalties for violations in task execution for ,in The penalty value represents the cost of adjusting the power cap. for ,in This represents the maximum rated power for cloud data centers. To adjust costs.

[0107] The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state The probability is given by θ, where θ is the network parameter of the environment simulation network.

[0108] Policy function Indicates the state Take action below The probability of.

[0109] Optimal strategy of the power capping decision process model Defined as:

[0110] .

[0111] in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. Let E represent the expected value operation. For the network parameters of the decision value network, Representative selection enables the expected value function Actions that reach the maximum value This is the action to be executed at time step t.

[0112] The training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes:

[0113] (1) Cloud data centers are based on decision value networks Output selection power capping action .

[0114] (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. and the trajectory Stored in the trajectory buffer middle.

[0115] (3) Use the trajectory buffer Trajectory data training environment simulation network The goal is to learn the dynamic response of a real cloud operating environment to power capping actions until the loss function drops to a preset threshold. Environmental simulation network Training complete.

[0116] (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. middle.

[0117] (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete.

[0118] Decision Value Network loss function for:

[0119]

[0120] .

[0121] in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and Let U and y be two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1, respectively, and input to time t. and time t+1 , and For counting subscripts, For difference error, The function is an indicator function, which takes the value 1 if the inequality in the subscript is true, and 0 otherwise. γ is the decay factor.

[0122] Environmental simulation network loss function for:

[0123]

[0124] .

[0125] in, Represents the set of sampled trajectories , t is the trajectory sample number, and N is the total number of trajectory samples. It is the state value that the sample at time t+1 can most likely transition to, as simulated by the environmental simulation network. This represents all possible t+1 samples simulated by the environment simulation network. It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state.

[0126] Example 2

[0127] This invention compares with existing power capping technologies. Under the same electricity price, task arrival, environmental changes, and uncertainty settings, the power capping method proposed in this invention has significant advantages. Comparison method 1 is a power capping setting method disclosed in the paper Radovanović A, Koningstein R, Schneider I, et al. Carbon-aware computing for datacenters[J]. IEEE Transactions on Power Systems, 2022, 38(2): 1270-1280. Comparison method 2 is a power capping value determination method, device, and storage medium disclosed in patent CN116204380A.

[0128] Depend on Figure 3 It can be seen that the rule-based method (comparison method 2) lacks flexibility in dealing with electricity price fluctuations and task arrival, and often fails to adapt effectively to changes. Its overall power capping curve is relatively flat, indicating insufficient responsiveness. The method of predicting first and then optimizing (comparison method 1) can better track electricity price trends, but it lacks consideration for uncertainties and dynamically changing environments, often resulting in excessively low power caps (as shown in the shaded area). These overly strict restrictions hinder task execution, leading to increased latency and raising the risk of exceeding SLA requirements. In contrast, the method proposed in this invention can adapt to price changes while taking into account the uncertainties in task arrival and execution, thus avoiding overly strict restrictions and ensuring successful task completion.

[0129] See Figure 4 , Figure 4 This paper presents the distribution of total cost and task execution violation rate for three methods in an uncertain cloud data center environment. The rule-based method (compared to method 2) relies solely on predefined rules, exhibiting limited adaptability to dynamic fluctuations in tasks and electricity prices, resulting in higher energy consumption and violations. Compared to the rule-based method, the prediction-then-optimization method (compared to method 1) offers some improvement, but its effectiveness is highly dependent on the accuracy of the prediction. Furthermore, because this method performs a one-time optimization a day in advance, it ignores differences in task scheduling strategies and real-time dynamic changes, limiting its adaptability and flexibility, thus leading to increased energy consumption and task violation rate. In contrast, the method proposed in this invention can dynamically learn and adapt to environmental uncertainties, achieving adaptive power capping adjustments. This adaptability reduces costs and violation rates.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An adaptive power capping method considering the uncertainties of cloud data centers, characterized in that, include: The dynamic power capping process of cloud data centers is modeled as a Markov decision process, with the task execution status, energy consumption status and electricity price fluctuations as uncertain parameters, to obtain the power capping decision process model of cloud data centers. The power capping decision process model is trained using a two-stage uncertainty-aware reinforcement learning algorithm to output an adaptive power capping decision. The power capping decision-making process model includes states, actions, reward functions, transition functions, and policy functions; The state Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping decision value Task execution violation rate Total resource requirements for the task Electricity price information ; The action Defined as a power cap set for cloud data centers, which is a discrete value ranging from 0 to 100%; The reward function This includes energy consumption costs, penalties for task execution violations, and power limit adjustment costs. The energy consumption costs... for The task execution violation penalty for ,in The power cap adjustment cost is the penalty value. for ,in This represents the maximum rated power for cloud data centers. To adjust costs; The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state θ represents the network parameters of the environmental simulation network; The strategy function Indicates the state Take action below The probability of; The optimal strategy of the power capping decision process model Defined as: ； in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. Let E represent the expected value operation. These are the network parameters of the decision-making value network.

2. The adaptive power capping method considering the uncertainties of cloud data centers according to claim 1, characterized in that, The training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes: (1) Cloud data centers are based on decision value networks Output selection power capping action ; (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. and the trajectory Stored in the trajectory buffer middle; (3) Use the trajectory buffer Trajectory data training environment simulation network The goal is to learn the dynamic response of a real cloud operating environment to power capping actions until the loss function drops to a preset threshold. Environmental simulation network Training complete; (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. In this context, the superscript 'v' represents virtual; (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete.

3. The adaptive power capping method considering the uncertainties of cloud data centers according to claim 2, characterized in that, The decision value network loss function for: ； in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and Let U and y be two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1, respectively, and input to time t. and time t+1 , and For counting subscripts, For difference error, The function is an indicator function, which takes the value 1 when the inequality in the subscript is true, and 0 otherwise. γ is the decay factor. The environmental simulation network loss function for: ； in, Represents the set of sampled trajectories , t is the trajectory sample number, and N is the total number of trajectory samples. It is the state value that the sample at time t+1 can most likely transition to, as simulated by the environmental simulation network. This represents all possible t+1 samples simulated by the environment simulation network. It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state.

4. An adaptive power capping system considering the uncertainties of cloud data centers, characterized in that, It includes a power capping decision process model construction module and a power capping decision process model training output module; The power capping decision process model building module is used to model the dynamic power capping process of cloud data centers as a Markov decision process, taking the task execution status, energy consumption status and electricity price fluctuations as uncertain parameters to obtain the power capping decision process model of cloud data centers. The power capping decision process model training output module is used to train the power capping decision process model through a two-stage uncertainty-aware reinforcement learning algorithm and output an adaptive power capping decision. The power capping decision-making process model includes states, actions, reward functions, transition functions, and policy functions; The state Includes: Current actual energy consumption information of the T-Cloud Data Center Power capping value Task execution violation rate Total resource requirements for the task and electricity price information ; The action Defined as a power cap set for cloud data centers, which is a discrete value ranging from 0 to 100%; The reward function This includes energy consumption costs, penalties for task execution violations, and power limit adjustment costs. The energy consumption costs... for The task execution violation penalty for ,in The power cap adjustment cost is the penalty value. for ,in This represents the maximum rated power for cloud data centers. To adjust costs; The transfer function is established by creating an environmental simulation network. Indicates the state Take action below After transitioning to state θ represents the network parameters of the environmental simulation network; The strategy function Indicates the state Take action below The probability of; The optimal strategy of the power capping decision process model Defined as: ； in, For the decision-making value network, it represents the state. Take action below The probability distribution of the value, the probability distribution is expressed by quantiles. Let E represent the expected value operation. These are the network parameters of the decision-making value network.

5. The adaptive power capping system considering the uncertainties of cloud data centers according to claim 4, characterized in that, The training of the power capping decision process model using a two-stage uncertainty-aware reinforcement learning algorithm includes: (1) Cloud data centers are based on decision value networks Output selection power capping action ; (2) Interact with the real cloud operating environment to obtain real energy consumption feedback information and task execution information, and save them as a trajectory. and the trajectory Stored in the trajectory buffer middle; (3) Use the trajectory buffer Trajectory data training environment simulation network The goal is to learn the dynamic response of a real cloud operating environment to power capping actions until the loss function drops to a preset threshold. Environmental simulation network Training complete; (4) Using training completed and Generate virtual trajectory and virtual trajectory It is stored in the virtual trajectory cache area. In this context, the superscript 'v' represents virtual; (5) Set the trajectory buffer and virtual trajectory cache area A portion of both real and virtual trajectories is extracted from each to jointly train the decision value network. Return (1) until the loss function of the decision value network converges, the decision value network Training complete.

6. The adaptive power capping system considering the uncertainties of cloud data centers according to claim 5, characterized in that, The decision value network loss function for: ； in, Here is the Huber loss function. Here, N is the threshold for the Huber loss function, and N is the number of quantile samples. and Let U and y be two sets of quantiles sampled from a uniform distribution U[0,1] from 0 to 1, respectively, and input to time t. and time t+1 , and For counting subscripts, For difference error, The function is an indicator function, which takes the value 1 when the inequality in the subscript is true, and 0 otherwise. γ is the decay factor. The environmental simulation network loss function for: ； in, Represents the set of sampled trajectories , t is the trajectory sample number, and N is the total number of trajectory samples. It is the state value that the sample at time t+1 can most likely transition to, as simulated by the environmental simulation network. This represents all possible t+1 samples simulated by the environment simulation network. It is the expectation operator. This represents the arithmetic mean of all sampled trajectories. Representative to The arithmetic average of all possible actions is calculated. It is the estimation function of the decision value network for the value of the next state.

Images