Task scheduling method and system considering data center uncertainty risk

Abstract

The present application relates to a kind of computing task scheduling method and system considering the uncertainty risk of data center;The present application is based on Markov decision process to the task scheduling process of data center is modeled, obtains task scheduling model, and the information and electricity price of task are as uncertain parameter, using policy network to represent the decision process of task scheduling, based on conditional risk value measurement risk of running cost being too high, and conditional risk value is embedded as constraint under reinforcement learning framework, optimize data center running cost under the condition of guaranteeing risk controllable, solve task scheduling strategy by fusing the improved policy gradient algorithm of Monte Carlo sampling, train the task scheduling model, and output the task scheduling strategy of data center by the task scheduling model trained, the present application can carry out task scheduling according to time-varying electricity price, reduce data center running cost, and can perceive the risk caused by uncertainty, avoid potential loss.

Description

Technical Field

[0001] This invention relates to the field of data center task scheduling technology, and specifically to a computing task scheduling method and system that takes into account the uncertainty risks of data centers. Background Technology

[0002] With the rapid development of digital technologies such as cloud computing, big data, and artificial intelligence, data centers are expanding in scale, leading to a significant increase in electricity consumption and energy costs. It's important to note that data centers need to handle a large number of latency-tolerant computing loads. These loads do not require immediate response, providing flexibility in energy usage over time. Based on this flexibility, data centers can optimize their energy consumption over time through task scheduling methods. Specifically, while ensuring Quality of Service (QoS) requirements, these latency-tolerant loads can be shifted to periods with lower electricity prices, avoiding excessive power consumption during peak hours. This allows for dynamic adjustment of power demand, effectively reducing the operating costs of data centers.

[0003] CN117667639B, based on Markov decision processes and queuing theory, constructs a scheduling model capable of adapting to the random arrival of computational tasks, thus improving resource utilization. However, this method does not consider the impact of electricity price uncertainty on data center energy costs during task scheduling. CN118964032A uses task information and electricity price as uncertainty parameters, employing a quantile regression reinforcement learning algorithm to select scheduling actions and reduce energy costs. However, this method only optimizes energy costs under uncertain conditions, without considering the decision-making risks brought about by uncertainty.

[0004] Current main technical approaches aim to optimize operating costs by providing task scheduling strategies for data centers. However, in real-world applications, these strategies have significant limitations, primarily for two reasons: 1. The operating environment of data centers in real-world production environments is inherently uncertain. Firstly, user behavior cannot be accurately predicted, leading to randomness in task arrival rates and resource demands. Secondly, the large-scale integration of renewable energy sources often results in fluctuating electricity prices. Agents need to learn and make decisions in this dynamic environment, significantly increasing the complexity of the task scheduling problem. 2. These scheduling strategies lack effective management of decision-making risks arising from uncertainty. Specifically, tasks may be assigned during periods of high electricity prices, or inappropriate resource allocation strategies may prevent timely task completion, exposing data centers to excessive operating costs. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, this invention aims to provide a computing task scheduling method and system that takes into account the uncertainty risks of data centers, thereby enabling data centers to reduce costs and avoid risks caused by uncertainty when task arrival is uncertain, thus preventing potential losses.

[0006] To solve the above problems, the present invention adopts the following technical solution:

[0007] On the one hand, the present invention provides a computing task scheduling method that takes into account the uncertainty risk of data centers, including:

[0008] The task scheduling process of the data center is modeled based on the Markov decision process, resulting in a task scheduling model. In the task scheduling model, task information and electricity price are used as uncertain parameters.

[0009] The task scheduling model uses a policy network to represent the decision-making process of task scheduling. It measures the risk of excessively high operating costs based on conditional risk value and embeds conditional risk value as a constraint into a reinforcement learning framework to optimize data center operating costs while ensuring that risks are controllable.

[0010] The improved policy gradient algorithm that incorporates Monte Carlo sampling is used to solve the task scheduling policy. The improved policy gradient algorithm that incorporates Monte Carlo sampling uses the Monte Carlo sampling method to obtain the trajectory under different task scheduling policies and estimate the gradient required for the policy gradient algorithm to solve the policy.

[0011] The task scheduling model is trained, and the trained task scheduling model outputs the task scheduling strategy for the data center.

[0012] As one possible implementation, the task scheduling model includes a state space, an action space, a reward function, and a value function.

[0013] The state space S includes the execution order of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization rate and memory resource utilization rate of each server, current time, and current electricity price.

[0014] The action space A includes task execution actions and empty actions; the task execution action a i :

[0015]

[0016] Indicates the task Assigned to server se m Execute above.

[0017] The empty action φ indicates that the optimal action at this time is not to assign any more tasks.

[0018] The reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time.

[0019] The value function J θ (ξ) is:

[0020] J θ (ξ)=r1+...+γ T-1 r T .

[0021] Where γ∈[0,1], it represents the trade-off between future costs and current costs. The larger γ is, the more the agent values ​​future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T a T r T ), where θ is the parameter of the policy network and T is the length of the trajectory sequence.

[0022] As one possible implementation, the task scheduling model uses a policy network to represent the task scheduling decision-making process, measures the risk of excessively high operating costs based on conditional value of risk (VoV), and embeds VoV as a constraint into a reinforcement learning framework to optimize data center operating costs while ensuring controllable risk. This includes:

[0023] The decision-making process of task scheduling is represented by a policy network. The input of the policy network is the data included in the state space, and the output is the probability of executing each action. The task scheduling model selects the action to be executed according to the probability.

[0024] The goal of the optimization problem is to optimize the network parameters θ of the strategy under controllable risk, so as to minimize the data center operating cost, i.e.:

[0025] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0026] Where β is the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost (CVaR) of a data center under extreme conditions. α (J θ (ξ)):

[0027]

[0028] Among them, (J θ (ξ)-v) + = nax(J θ (ξ)-v, 0), representing the part greater than 0; v is the VaR parameter, which is the risk threshold for calculating CVaR, E is the expectation calculation, and α is the confidence level.

[0029] Based on the Lagrangian relaxation method, transform the optimization problem into an unconstrained optimization problem:

[0030]

[0031] Among them, λ is the Lagrange multiplier, and L(v, θ, λ) is the loss function of the task scheduling model.

[0032] As an implementable manner, solve the task scheduling strategy through the improved policy gradient algorithm that integrates Monte Carlo sampling. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling strategies and estimate the gradients required when solving the policy gradient algorithm, including:

[0033] During each iteration, perform N times of Monte Carlo sampling. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0034]

[0035] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ. j is the actual sampling number and k is the actual training number. λ k is the Lagrange multiplier for the k-th training, and v k is the VaR parameter for the k-th training.

[0036] As an implementable manner, the training of the task scheduling model includes:

[0037] S1. Input the confidence level α, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0038] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter V, and the Monte Carlo sampling scale N.

[0039] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0040] S4. If j < N, go to S5; otherwise, go to S6.

[0041] S5. Form a trajectory according to the policy, set j = j + 1, and return to S4.

[0042] S6. Estimate the gradient

[0043] S7. Update the model parameters of each data center:

[0044]

[0045] where θ k is the policy network parameter for the k-th training.

[0046] S8. Set k = k + 1, and return to S3.

[0047] On the other hand, the present invention provides a computing task scheduling system considering the uncertainty risk of data centers, including a task scheduling model construction module, an optimization module, a gradient estimation module, and a training scheduling module.

[0048] The task scheduling model construction module is used to model the task scheduling process of data centers based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of tasks and electricity prices are used as uncertain parameters.

[0049] The optimization module is used to represent the decision-making process of task scheduling by a policy network, measure the risk of excessive operating costs based on conditional value-at-risk, and embed the conditional value-at-risk as a constraint into the reinforcement learning framework to optimize the operating costs of data centers while ensuring risk controllability.

[0050] The gradient estimation module is used to solve the task scheduling policy by an improved policy gradient algorithm that combines Monte Carlo sampling. The improved policy gradient algorithm that combines Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimate the gradients required when solving the policy gradient algorithm.

[0051] The training scheduling module is used to train the task scheduling model and output the task scheduling policy of the data center through the trained task scheduling model.

[0052] As an implementable manner, the task scheduling model includes a state space, an action space, a reward function, and a value function.

[0053] The state space S includes the execution sequence precedence between tasks, CPU resource requirements, memory resource requirements, and execution time, the CPU resource utilization rate, memory resource utilization rate, current time, and current electricity price of each server.

[0054] The action space A includes task execution actions and empty actions; the task execution action a i :

[0055]

[0056] Task execution action a i Indicates the task Assigned to server se m Execute above.

[0057] The empty action φ indicates that the optimal action at this time is not to assign any more tasks.

[0058] The reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time.

[0059] The value function J θ (ξ) is:

[0060] J θ (ξ)=r1+...+γ T-1 r T .

[0061] Where γ∈[0,1], it represents the trade-off between future costs and current costs. The larger γ is, the more the agent values ​​future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T a T r T ), where θ is the parameter of the policy network and T is the length of the trajectory sequence.

[0062] As one possible implementation, the decision-making process for task scheduling is represented by a policy network. It measures the risk of excessively high operating costs based on conditional risk value and embeds this conditional risk value as a constraint into a reinforcement learning framework. This optimizes data center operating costs while ensuring controllable risk, including:

[0063] The decision-making process of task scheduling is represented by a policy network. The input of the policy network is the data included in the state space, and the output is the probability of executing each action. The agent selects the action to be executed according to the probability.

[0064] The goal of the optimization problem is to optimize the network parameters θ of the strategy under controllable risk, so as to minimize the data center operating cost, i.e.:

[0065] min θ J θ (ξ)subject to CVaRα (J θ (ξ))≤β.

[0066] Where β is the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost (CVaR) of a data center under extreme conditions. α (J θ (ξ)):

[0067]

[0068] Among them, (J) θ (ξ)-v) + =max(J θ (ξ)-v,0), representing the part greater than 0; v is the VaR parameter, used to calculate the risk threshold of CVaR, E is the expected value, and α is the confidence level.

[0069] Based on the Lagrange relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0070]

[0071] Where λ is a Lagrange multiplier, and L(v,θ,λ) is the loss function of the task scheduling model.

[0072] As one possible implementation, the improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required for solving the policy gradient algorithm, including:

[0073] In each iteration, N Monte Carlo samplings are performed. The agent simulates the data center task scheduling process according to the current policy network, obtains the running trajectories of N data centers, and performs an unbiased estimation of the gradient of L(v, θ, λ) using the following formula, and updates the parameters:

[0074]

[0075] in, Indicates the generated trajectory ξ j，k The probability, These represent the derivatives of the loss function with respect to parameters v, θ, and λ, respectively; j is the actual number of samples; k is the actual number of training iterations; and λ is the number of training iterations. k For the Lagrange multiplier trained in the kth iteration, v k Let be the VaR parameters for the k-th training iteration.

[0076] As one possible implementation, the improved policy gradient algorithm that incorporates Monte Carlo sampling to solve the task scheduling policy includes:

[0077] S1. Input the confidence level a, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0078] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter v, and the Monte Carlo sampling scale N.

[0079] S3. If k < K, go to S4; otherwise, the algorithm ends.

[0080] S4. If j < N, go to S5; otherwise, go to S6.

[0081] S5. Form a trajectory according to the policy, let j = j + 1, and return to S4.

[0082] S6. Estimate the gradient

[0083] S7. Update the model parameters of each data center:

[0084]

[0085] where θ k is the policy network parameter for the k-th training.

[0086] S8. Let k = k + 1, and return to S3.

[0087] The beneficial effects of the present invention are as follows: The method of the present invention can perform task scheduling according to the time-varying electricity price, reduce the operating cost of the data center, and can perceive the risks brought by uncertainty to avoid potential losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 is a flowchart of a calculation task scheduling method considering the uncertainty risk of a data center according to the present invention.

[0089] Figure 2 is a schematic structural diagram of the policy network according to the present invention.

[0090] Figure 3 is a schematic diagram of a calculation task scheduling system considering the uncertainty risk of a data center according to the present invention.

[0091] Figure 4 is a diagram of the CPU resource usage under the method of the present invention

[0092] Figure 5 is a curve graph of the probability distribution of the operating cost of the data center under the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0093] The present invention will be further described in detail below with reference to specific embodiments.

[0094] It should be noted that these embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Simple improvements to the method under the premise of the present invention are all within the scope of protection claimed by the present invention.

[0095] Example 1

[0096] See Figure 1 This is a computing task scheduling method that takes into account the uncertainty risks of data centers, including:

[0097] S100. The task scheduling process of the data center is modeled based on the Markov decision process to obtain the task scheduling model; in the task scheduling model, the task information and electricity price are used as uncertain parameters.

[0098] The information for a task includes its characteristics and the number of tasks arriving per unit of time.

[0099] The task scheduling model includes a state space S, an action space A, and a reward function r. i And value function J θ (ξ).

[0100] State space S: states s i Includes: (1) Task characteristics: the execution order of tasks, CPU resource requirements, memory resource requirements and execution time; (2) Data center resource usage: CPU resource utilization and memory resource utilization of each server; (3) Environmental characteristics: current time and current electricity price.

[0101] Gaussian noise was superimposed on both the electricity price data and the task arrival data to reflect the uncertainty in the environment.

[0102] Action space A includes two types of actions: task execution actions and empty actions; task execution action a i for:

[0103]

[0104] Task execution action a i Indicates the task Assigned to server se m Execute above.

[0105] An empty action φ indicates that the optimal action at this point is to no longer assign any tasks.

[0106] reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time.

[0107] Based on the above definition, the value function J of the trajectory ξ can be derived. θ (ξ) is:

[0108] J θ (ξ)=r1+...+γ T-1 r T .

[0109] Where γ∈[0,1], it represents the trade-off between future costs and current costs. The larger γ is, the more the agent values ​​future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T a T r T ), where θ is the parameter of the policy network and T is the length of the trajectory sequence.

[0110] The S200 task scheduling model uses a policy network to represent the decision-making process of task scheduling. It measures the risk of excessively high operating costs based on conditional risk value and embeds conditional risk value as a constraint into the reinforcement learning framework to optimize data center operating costs while ensuring that risks are controllable.

[0111] First, a policy network is used to represent the decision-making process of task scheduling. The structure of the policy network is shown in [link to policy network diagram]. Figure 2 The input to the policy network is the data included in the state space, and the output is the probability of executing each action. The task scheduling model selects the action to be executed according to the probability.

[0112] Secondly, the goal of the optimization problem is to optimize the strategy network parameters θ under the premise of controllable risk, so as to minimize the data center operating cost, that is:

[0113] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0114] Where β is the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost (CVaR) of a data center under extreme conditions. α (J θ (ξ)):

[0115]

[0116] Among them, (J) θ (ξ)-v) + =max(J θ(ξ)-v, 0), representing the part greater than 0; v is the VaR parameter, used as the risk threshold for calculating CVaR, representing an approximate estimate of VaR (Value at Risk), E is for expectation calculation, and α is the confidence level.

[0117] Finally, based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0118]

[0119] Among them, λ is the Lagrange multiplier, used to measure the weights of cost and risk, and L(v, θ, λ) is the loss function of the task scheduling model.

[0120] S300. Solve the task scheduling policy through an improved policy gradient algorithm that integrates Monte Carlo sampling. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm.

[0121] In each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0122]

[0123] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ. j is the actual sampling number, k is the actual training number, λ k is the Lagrange multiplier for the k-th training, and v k is the VaR parameter for the k-th training.

[0124] S400. Train the task scheduling model, and output the task scheduling policy of the data center through the trained task scheduling model, including:

[0125] S1. Input the confidence level α, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0126] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter V, and the Monte Carlo sampling scale N.

[0127] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0128] S4. If i < N, go to S5; otherwise, go to S6.

[0129] S5. Form a trajectory according to the policy, let j = j + 1, and return to S4.

[0130] S6. Estimate the gradient

[0131] S7. Update the model parameters of each data center:

[0132]

[0133] where, θ k is the policy network parameter for the k-th training.

[0134] S8. Let k = k + 1, and return to S3.

[0135] See Figure 3 , which is a computing task scheduling system considering the uncertainty risk of data centers, including a task scheduling model construction module 100, an optimization module 200, a gradient estimation module 300, and a training scheduling module 400.

[0136] The task scheduling model construction module 100 is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of the task and the electricity price are used as uncertain parameters.

[0137] The task scheduling model includes a state space, an action space, a reward function, and a value function.

[0138] The state space S includes the execution order, CPU resource requirements, memory resource requirements, and execution time between tasks, the CPU resource utilization rate, memory resource utilization rate, current time, and current electricity price of each server.

[0139] The action space A includes a task execution action and a null action; the task execution action a i :

[0140]

[0141] represents allocating the task to the server se m for execution.

[0142] The null action φ indicates that the optimal action at this time is not to allocate any tasks.

[0143] The reward function r iIt is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time.

[0144] The value function J θ (ξ) is:

[0145] J θ (ξ)=r1+...+γ T-1 r T .

[0146] Where γ∈[0,1], it represents the trade-off between future costs and current costs. The larger γ is, the more the agent values ​​future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T a T r T ), where θ is the parameter of the policy network and T is the length of the trajectory sequence.

[0147] The optimization module 200 is used to represent the decision-making process of task scheduling using a policy network, measures the risk of excessive operating costs based on conditional risk value, and embeds conditional risk value as a constraint into the reinforcement learning framework to optimize the operating costs of the data center while ensuring that the risks are controllable.

[0148] The decision-making process of task scheduling is represented by a policy network. The input of the policy network is the data included in the state space, and the output is the probability of executing each action. The task scheduling model selects the action to be executed according to the probability.

[0149] The goal of the optimization problem is to optimize the network parameters θ of the strategy under controllable risk, so as to minimize the data center operating cost, i.e.:

[0150] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0151] Where β is the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost (CVaR) of a data center under extreme conditions. α (J θ (ξ)):

[0152]

[0153] Among them, (J) θ (ξ)-v) + =max(Jθ (ξ)-v, 0), representing the part greater than 0; v is the VaR parameter, which is the risk threshold for calculating CVaR, E is the expectation calculation, and α is the confidence level.

[0154] Based on the Lagrangian relaxation method, transform the optimization problem into an unconstrained optimization problem:

[0155]

[0156] Among them, λ is the Lagrangian multiplier, and L(v, θ, λ) is the loss function of the task scheduling model.

[0157] The gradient estimation module 300 is used to solve the task scheduling policy through an improved policy gradient algorithm that fuses Monte Carlo sampling. The improved policy gradient algorithm that fuses Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm.

[0158] In each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0159]

[0160] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ. j is the actual sampling number, k is the actual training number, λ k is the Lagrangian multiplier for the kth training, and v k is the VaR parameter for the kth training.

[0161] The training scheduling module 400 is used to train the task scheduling model, and outputs the task scheduling policy of the data center through the trained task scheduling model.

[0162] Training the task scheduling model includes:

[0163] S1. Input the confidence level α, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0164] S2. Initialize the policy network parameters θ, the Lagrangian multiplier λ, the VaR parameter v, and the Monte Carlo sampling scale N.

[0165] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0166] S4. If j < N, go to S5; otherwise go to S6.

[0167] S5. Form a trajectory according to the policy, set j = j + 1, and return to S4.

[0168] S6. Estimate the gradient

[0169] S7. Update the model parameters of each data center:

[0170]

[0171] where θ k is the policy network parameter for the k-th training.

[0172] S8. Set k = k + 1 and return to S3.

[0173] Embodiment 2

[0174] The present invention compares with the existing algorithms that do not consider risks. Under the same electricity prices, task arrivals, and the same uncertainty settings, the method of the present invention has obvious advantages in the optimization effect.

[0175] Referring to Figure 4 , it can be seen that the method proposed by the present invention can perform task scheduling according to the time-varying electricity price and dynamically adjust the energy consumption curve. A large number of tasks are postponed to the periods with lower electricity prices, reducing the operating cost of the data center.

[0176] Referring to Figure 5 , the traditional optimization algorithm only optimizes the operating cost J θ (ξ) of the data center and does not use CVaR to measure the operating cost risk as a risk constraint. Compared with the optimization algorithms that do not consider risks, the method proposed by the present invention can perceive the risks brought by uncertainties. Although the relatively conservative scheduling strategy gives up some opportunities to obtain lower operating costs, the operating cost distribution under the method of the present invention is more concentrated, avoiding potential high losses.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit. Although the present invention has been described by referring to the preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A computing task scheduling method that takes into account the uncertainty risk of data centers, characterized in that, including: Modeling the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; In the task scheduling model, the information of tasks and electricity prices are regarded as uncertain parameters; The task scheduling model uses a policy network to represent the decision-making process of task scheduling, measures the risk of excessive operating costs based on conditional value-at-risk, and embeds the conditional value-at-risk as a constraint into the reinforcement learning framework to optimize the operating cost of the data center while ensuring risk controllability; Solving the task scheduling policy through an improved policy gradient algorithm that combines Monte Carlo sampling. The improved policy gradient algorithm that combines Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm; Training the task scheduling model, and outputting the task scheduling policy of the data center through the trained task scheduling model; The task scheduling model includes a state space. Action space Reward function and value function ; The state space This includes the execution order of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization and memory resource utilization of each server, current time, and current electricity price; The action space This includes task execution actions and empty actions; the task execution actions : ； Task execution actions Indicates the task Assigned to server Execute above; The empty motion This indicates that the optimal action at this point is to no longer assign any tasks; The reward function It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time. The value function for: ； in This represents a trade-off between future costs and current costs. The larger the value, the more the intelligent agent values ​​future interests. For trajectory, , Here are the policy network parameters, and T is the length of the trajectory sequence.

2. The computing task scheduling method considering data center uncertainty risk according to claim 1, characterized in that, The task scheduling model uses a policy network to represent the decision-making process of task scheduling, measures the risk of excessive operating costs based on conditional value-at-risk, and embeds the conditional value-at-risk as a constraint into the reinforcement learning framework to optimize the operating cost of the data center while ensuring risk controllability, including: Using a policy network to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of each action execution. The task scheduling model selects the action to be executed according to the probability; The goal of the optimization problem is to optimize the network parameters of the strategy while keeping the risks under control. This minimizes the operating costs of the data center, i.e.: ； in, To achieve the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost of a data center under extreme conditions. : ； in, , represents the part greater than 0; α is the VaR parameter, used to calculate the risk threshold of CVaR, where E is the expected value and α is the confidence level. Based on the Lagrangian relaxation method, transforming the optimization problem into an unconstrained optimization problem: ； in, It is a Lagrange multiplier. This is the loss function for the task scheduling model.

3. The computational task scheduling method considering data center uncertainty risk according to claim 2, characterized in that, The improved policy gradient algorithm that combines Monte Carlo sampling is used to solve the task scheduling policy. The improved policy gradient algorithm that combines Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm, including: In each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process based on the current policy network, obtaining the running trajectories of N data centers. The following formula is used to... The gradient is unbiasedly estimated, and the parameters are updated: ； in, Indicates the generated trajectory The probability, , , These represent the loss function in terms of parameters. The derivative in the direction, where j is the actual number of samples and k is the actual number of training iterations. For the Lagrange multiplier of the k-th training iteration, Let be the VaR parameters for the k-th training iteration.

4. The computational task scheduling method considering data center uncertainty risk according to claim 3, characterized in that, The training of the task scheduling model includes: S1, Input confidence level Maximum acceptable operating cost Number of training sessions and update step size , , ; S2. Initialize policy network parameters Lagrange multipliers VaR parameters And Monte Carlo sampling size N; S3. If k < K, go to S4; otherwise, the algorithm ends; S4. If j < N, go to S5; otherwise, go to S6; S5. Form a trajectory according to the strategy, and let Return to S4; S6. Estimating the gradient , , ; S7. Updating the model parameters of each data center: ； in, These are the parameters of the policy network during the k-th training iteration; S8, Order Return to S3.

5. A computing task scheduling system that takes into account the uncertainty risk of data centers, characterized in that, including a task scheduling model construction module, an optimization module, a gradient estimation module, and a training scheduling module; The task scheduling model construction module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of tasks and electricity prices are regarded as uncertain parameters; The optimization module is used to use a policy network to represent the decision-making process of task scheduling, measure the risk of excessive operating costs based on conditional value-at-risk, and embed the conditional value-at-risk as a constraint into the reinforcement learning framework to optimize the operating cost of the data center while ensuring risk controllability; The gradient estimation module is used to solve the task scheduling policy through an improved policy gradient algorithm that combines Monte Carlo sampling. The improved policy gradient algorithm that combines Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm; The training scheduling module is used to train the task scheduling model, and output the task scheduling policy of the data center through the trained task scheduling model; The task scheduling model includes a state space. Action space Reward function and value function ; The state space This includes the execution order of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization and memory resource utilization of each server, current time, and current electricity price; The action space This includes task execution actions and empty actions; the task execution actions : ； Task execution actions Indicates the task Assigned to server Execute above; The empty motion This indicates that the optimal action at this point is to no longer assign any tasks; The reward function It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user's task submission time. The value function for: ； in This represents a trade-off between future costs and current costs. The larger the value, the more the intelligent agent values ​​future interests. For trajectory, , Here are the policy network parameters, and T is the length of the trajectory sequence.

6. The computing task scheduling system considering data center uncertainty risk according to claim 5, characterized in that, The decision-making process of task scheduling is represented by a policy network, the risk of excessive operating costs is measured based on conditional value at risk, and the conditional value at risk is embedded as a constraint into the reinforcement learning framework to optimize the operating costs of the data center while ensuring controllable risks, including: The decision-making process of task scheduling is represented by a policy network. The input of the policy network is the data included in the state space, and the output is the probability of each action execution. The agent selects the action to be executed according to the probability. The goal of the optimization problem is to optimize the network parameters of the strategy while keeping the risks under control. This minimizes the operating costs of the data center, i.e.: ； in, To achieve the maximum acceptable operating cost, CVaR α Defined as the tail expectation of the distribution, it reflects the expected operating cost of a data center under extreme conditions. : ； in, , represents the part greater than 0; α is the VaR parameter, used to calculate the risk threshold of CVaR, where E is the expected value and α is the confidence level. Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem: ； in, It is a Lagrange multiplier. This is the loss function for the task scheduling model.

7. The computing task scheduling system considering data center uncertainty risk according to claim 6, characterized in that, The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimate the gradients required when solving the policy gradient algorithm, including: In each iteration, N Monte Carlo samplings are performed. The agent simulates the data center task scheduling process according to the current policy network, obtains the operating trajectories of N data centers, and uses the following formula to... The gradient is unbiasedly estimated, and the parameters are updated: ； in, Indicates the generated trajectory The probability, , , These represent the loss function in terms of parameters. The derivative in the direction, where j is the actual number of samples and k is the actual number of training iterations. For the Lagrange multiplier of the k-th training iteration, Let be the VaR parameters for the k-th training iteration.

8. The computing task scheduling system considering data center uncertainty risk according to claim 7, characterized in that, Solving the task scheduling policy through the improved policy gradient algorithm that integrates Monte Carlo sampling, including: S1, Input confidence level Maximum acceptable operating cost Number of training sessions and update step size , , ; S2. Initialize policy network parameters Lagrange multipliers VaR parameters And Monte Carlo sampling size N; S3. If k < K, go to S4; otherwise, the algorithm ends. S4. If j < N, go to S5; otherwise, go to S6. S5. Form a trajectory according to the strategy, and let Return to S4; S6. Estimating the gradient , , ; S7. Update the model parameters of each data center: ； in, These are the parameters of the policy network during the k-th training iteration; S8, Order Return to S3.

Images