Computing task scheduling method and system considering uncertainty risk of data center

Abstract

The invention relates to a calculation task scheduling method and system considering the uncertainty risk of a data center. The task scheduling process of the data center is modeled based on the Markov decision process to obtain the task scheduling model, the task information and the electricity price serve as uncertain parameters, the decision process of task scheduling is represented by adopting the strategy network, and the risk of too high operation cost is measured based on the conditional value-at-risk. And embedding the conditional value-at-risk as a constraint into a reinforcement learning framework, optimizing the operation cost of the data center under the condition of ensuring the controllable risk, solving a task scheduling strategy through an improved strategy gradient algorithm fused with Monte Carlo sampling, training the task scheduling model, and obtaining a task scheduling result. Outputting a task scheduling strategy of the data center through the trained task scheduling model; task scheduling can be carried out according to the time-varying electricity price, the operation cost of the data center is reduced, risks caused by uncertainty can be sensed, and potential losses are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of data center task scheduling, and in particular to a computing task scheduling method and system taking into account uncertainty risks of data centers. Background Art

[0002] With the rapid development of digital technologies such as cloud computing, big data, and artificial intelligence, the scale of data centers continues to expand, and power consumption and energy costs have increased significantly. Data centers are required to process a large number of delay-tolerant computing loads. These computing loads do not require instant response, providing time-dependent energy flexibility. Based on this flexibility, data centers can optimize time-dependent energy consumption through task scheduling methods, thereby achieving effective energy management. Specifically, while ensuring quality of service (QoS) requirements, these delay-tolerant loads can be shifted to time periods with lower electricity prices for processing, avoiding high electricity consumption during peak periods. This enables dynamic regulation of power demand and effectively reduces data center operating costs.

[0003] CN117667639B, based on Markov decision processes and queuing theory, constructs a scheduling model that adapts to the random arrival of computing tasks, improving resource utilization. However, this method does not consider the impact of electricity price uncertainty on data center energy costs during task scheduling. CN118964032A uses task information and electricity prices as uncertainty parameters and selects scheduling actions using a quantile regression reinforcement learning algorithm to reduce energy costs. However, this method only optimizes energy costs under uncertain conditions and does not consider the decision-making risks brought about by uncertainty.

[0004] The current main technical approach aims to optimize operating costs and provides task scheduling strategies for data centers. In real-world applications, this scheduling strategy has significant limitations, primarily for two reasons: 1. The operational environment of data centers in real-world production environments is characterized by uncertainty. First, because user behavior cannot be accurately predicted, the arrival rate of tasks and the amount of resources required are random; at the same time, due to the large-scale integration of renewable energy, electricity prices are often volatile. Intelligent agents need to learn and make decisions in a dynamic environment, significantly increasing the complexity of the task scheduling problem. 2. This scheduling strategy lacks effective management of decision-making risks caused by uncertainty. Specifically, tasks may be assigned to times when electricity prices are too high, or tasks may not be completed on time due to incorrect resource allocation strategies, exposing data centers to the risk of excessive operating costs. Summary of the Invention

[0005] In response to the above-mentioned deficiencies in the existing technology, the present invention aims to provide a computing task scheduling method and system that takes into account the uncertainty risks of data centers, so as to reduce costs of data centers when task arrival is uncertain and avoid risks brought by uncertainty, thereby avoiding potential losses.

[0006] In order to solve the above problems, the present invention adopts the following technical solutions:

[0007] In one aspect, the present invention provides a computing task scheduling method that takes into account data center uncertainty risks, comprising:

[0008] The task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity price are used as uncertain parameters.

[0009] The task scheduling model adopts a policy network to represent the decision-making process of task scheduling, measures the risk of excessively high operating costs based on the conditional value at risk, and embeds the conditional value at risk as a constraint into the reinforcement learning framework to optimize the operating costs of the data center while ensuring that the risks are controllable.

[0010] The task scheduling policy is solved by integrating an improved policy gradient algorithm with Monte Carlo sampling. The improved policy gradient algorithm with Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradient required for solving the policy gradient algorithm.

[0011] The task scheduling model is trained, and the task scheduling strategy of the data center is output through the trained task scheduling model.

[0012] As an implementable method, the task scheduling model includes a state space, an action space, a reward function and a value function.

[0013] The state space S includes the execution sequence of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization rate of each server, memory resource utilization rate, current time, and current electricity price.

[0014] The action space A includes task execution actions and empty actions; the task execution action a i :

[0015]

[0016] Indicates that the task Assign to server m Execute on.

[0017] The empty action φ indicates that the optimal action at this time is not to assign any tasks.

[0018] The reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time.

[0019] The value function J θ (ξ) is:

[0020] J θ (ξ)=r1+...+γ T-1 r T .

[0021] Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ is, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T , a T , r T ), θ is the parameter of the policy network, and T is the length of the trajectory sequence.

[0022] As an implementable method, the task scheduling model uses a policy network to represent the decision-making process of task scheduling, measures the risk of excessive operating costs based on conditional value at risk, and embeds the conditional value at risk as a constraint into a reinforcement learning framework to optimize data center operating costs while ensuring that risks are controllable. This includes:

[0023] A policy network is used to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of each action being executed. The task scheduling model selects the action to be executed according to the probability.

[0024] The goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the operating costs of the data center, that is:

[0025] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0026] Among them, β is the maximum acceptable operating cost, CVaR α Defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions α (J θ (ξ)):

[0027]

[0028] Among them, (J θ (ξ)-v) + = max(J θ (ξ)-v, 0), representing the part greater than 0; v is the VaR parameter, which is the risk threshold for calculating CVaR, E is the expectation calculation, and α is the confidence level.

[0029] Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0030]

[0031] Among them, λ is the Lagrange multiplier, and L(v, θ, λ) is the loss function of the task scheduling model.

[0032] As an implementable manner, the task scheduling strategy is solved by the improved policy gradient algorithm that integrates Monte Carlo sampling. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain the trajectories under different task scheduling strategies and estimate the gradients required when solving the policy gradient algorithm, including:

[0033] In each iteration process, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0034]

[0035] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ. j is the actual sampling number, k is the actual training number, λ k is the Lagrange multiplier of the k-th training, and v k is the VaR parameter of the k-th training.

[0036] As an implementable manner, the training of the task scheduling model includes:

[0037] S1. Input the confidence level α, the maximum acceptable operating cost β, the training number K, and the update steps η1, η2, η3.

[0038] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter V, and the Monte Carlo sampling scale N.

[0039] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0040] S4. If j < N, go to S5; otherwise go to S6.

[0041] S5. Form a trajectory according to the policy, set j = j + 1, and return to S4.

[0042] S6. Estimate the gradient

[0043] S7. Update the model parameters of each data center:

[0044]

[0045] where θ k is the policy network parameter for the k-th training.

[0046] S8. Set k = k + 1 and return to S3.

[0047] On the other hand, the present invention provides a computing task scheduling system considering the uncertainty risk of data centers, including a task scheduling model construction module, an optimization module, a gradient estimation module, and a training scheduling module.

[0048] The task scheduling model construction module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of the task and the electricity price are used as uncertain parameters.

[0049] The optimization module is used to represent the decision-making process of task scheduling by a policy network, measure the risk of excessive operating costs based on conditional value at risk, and embed the conditional value at risk as a constraint into the reinforcement learning framework to optimize the operating cost of the data center while ensuring controllable risk.

[0050] The gradient estimation module is used to solve the task scheduling policy through an improved policy gradient algorithm that combines Monte Carlo sampling. The improved policy gradient algorithm that combines Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm.

[0051] The training scheduling module is used to train the task scheduling model and output the task scheduling policy of the data center through the trained task scheduling model.

[0052] As an implementable manner, the task scheduling model includes a state space, an action space, a reward function, and a value function.

[0053] The state space S includes the execution sequence, CPU resource requirements, memory resource requirements, and execution time between tasks, the CPU resource utilization rate, memory resource utilization rate, current time, and current electricity price of each server.

[0054] The action space A includes task execution actions and empty actions; the task execution action a i :

[0055]

[0056] Task execution action a i Indicates that the task Assign to server m Execute on.

[0057] The empty action φ indicates that the optimal action at this time is not to assign any tasks.

[0058] The reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time.

[0059] The value function J θ (ξ) is:

[0060] J θ (ξ)=r1+...+γ T-1 r T .

[0061] Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ is, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,……,s T , a T , r T ), θ is the parameter of the policy network, and T is the length of the trajectory sequence.

[0062] As an implementable method, the policy network is used to represent the decision-making process of task scheduling. The risk of excessive operating costs is measured based on the conditional value at risk. The conditional value at risk is embedded as a constraint in the reinforcement learning framework to optimize the operating costs of the data center while ensuring that the risks are controllable. The method includes:

[0063] A policy network is used to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of executing each action. The agent selects the action to be executed according to the probability.

[0064] The goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the operating costs of the data center, that is:

[0065] min θ J θ (ξ)subject to CVaRα (J θ (ξ))≤β.

[0066] Among them, β is the maximum acceptable operating cost, CVaR α Defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions α (J θ (ξ)):

[0067]

[0068] Among them, (J θ (ξ)-v) + =max(J θ (ξ)-v, 0) represents the part greater than 0; v is the VaR parameter, which is used to calculate the risk threshold of CVaR, E is the expected calculation, and α is the confidence level.

[0069] Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0070]

[0071] Among them, λ is the Lagrange multiplier, and L(v,θ,λ) is the loss function of the task scheduling model.

[0072] As an implementable method, the improved policy gradient algorithm integrating Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling strategies and estimates the gradient required for solving the policy gradient algorithm, including:

[0073] In each iteration, N Monte Carlo samplings are performed. The agent simulates the data center task scheduling process based on the current policy network, obtains the operation trajectories of N data centers, and makes an unbiased estimate of the gradient of L(v, θ, λ) using the following formula and updates the parameters:

[0074]

[0075] in, Represents the generated trajectory ξ j，k The probability of Represent the derivatives of the loss function in the direction of parameters v, θ, and λ respectively, j is the actual number of sampling times, k is the actual number of training times, and λ k is the Lagrange multiplier of the kth training, v k is the VaR parameter of the kth training.

[0076] As an implementable method, the improved policy gradient algorithm integrated with Monte Carlo sampling is used to solve the task scheduling policy, including:

[0077] S1. Input the confidence level a, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0078] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter v, and the Monte Carlo sampling scale N.

[0079] S3. If k < K, go to S4; otherwise, end the algorithm.

[0080] S4. If j < N, go to S5; otherwise, go to S6.

[0081] S5. Form a trajectory according to the policy, set j = j + 1, and return to S4.

[0082] S6. Estimate the gradient

[0083] S7. Update the model parameters of each data center:

[0084]

[0085] where θ k is the policy network parameter for the k-th training.

[0086] S8. Set k = k + 1 and return to S3. <了

[0087] The beneficial effects of the present invention are as follows: The method of the present invention can perform task scheduling according to the time-varying electricity price, reduce the operating cost of the data center, and can perceive the risks brought by uncertainties to avoid potential losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 is a flowchart of a calculation task scheduling method considering the uncertainty risk of a data center according to the present invention.

[0089] Figure 2 is a schematic structural diagram of the policy network according to the present invention.

[0090] Figure 3 is a schematic diagram of a calculation task scheduling system considering the uncertainty risk of a data center according to the present invention.

[0091] Figure 4 is a diagram of the CPU resource usage under the method of the present invention.

[0092] Figure 5 is a curve graph of the probability distribution of the operating cost of the data center under the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0093] The following further elaborates the present invention in conjunction with specific embodiments.

[0094] It should be noted that these embodiments are only used to illustrate the present invention, rather than to limit the present invention. Simple improvements to the method based on the concept of the present invention fall within the scope of protection claimed by the present invention.

[0095] Example 1

[0096] See also Figure 1 , a computing task scheduling method that takes into account the uncertainty risk of data centers, including:

[0097] S100. Modeling the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity price are used as uncertain parameters.

[0098] The task information includes the characteristics of the task and the number of tasks arriving per unit time.

[0099] Among them, the task scheduling model includes state space S, action space A, reward function r i and value function J θ (ξ).

[0100] State space S: state s i These include: (1) Task characteristics: the execution order of tasks, CPU resource requirements, memory resource requirements, and execution time; (2) Data center resource usage: the CPU resource utilization and memory resource utilization of each server; and (3) Environmental characteristics: current time and current electricity price.

[0101] A Gaussian noise is superimposed on the electricity price data and task arrival data to reflect the uncertainty in the environment.

[0102] The action space A includes two types of actions: task execution action and empty action; task execution action a i for:

[0103]

[0104] Task execution action a i Indicates that the task Assign to server m Execute on.

[0105] Empty action φ means that the optimal action at this time is not to assign any tasks.

[0106] Reward function r i It is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption brought by the task execution by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time.

[0107] According to the above definition, the value function J of the trajectory ξ can be derived θ (ξ) is:

[0108] J θ (ξ)=r1+...+γ T-1 r T .

[0109] Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ is, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,……,s T , a T , r T ), θ is the parameter of the policy network, and T is the length of the trajectory sequence.

[0110] S200, the task scheduling model uses a policy network to represent the decision-making process of task scheduling, measures the risk of excessively high operating costs based on the conditional risk value, and embeds the conditional risk value as a constraint into the reinforcement learning framework to optimize the data center operating costs while ensuring that the risks are controllable.

[0111] First, a policy network is used to represent the decision-making process of task scheduling. The structure of the policy network is shown in Figure 2 The input of the policy network is the data included in the state space, and the output is the probability of executing each action. The task scheduling model selects the action to be executed according to the probability.

[0112] Secondly, the goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the data center operation cost, that is:

[0113] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0114] Among them, β is the maximum acceptable operating cost, CVaR α Defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions α (J θ (ξ)):

[0115]

[0116] Among them, (J θ (ξ)-v) + =max(J θ(ξ)-v, 0), representing the part greater than 0; v is the VaR parameter, which is the risk threshold for calculating CVaR, representing an approximate estimate of VaR (Value at Risk), E is the expectation calculation, and α is the confidence level.

[0117] Finally, based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0118]

[0119] Among them, λ is the Lagrange multiplier, which is used to measure the weight of cost and risk, and L(v, θ, λ) is the loss function of the task scheduling model.

[0120] S300. Solve the task scheduling policy through an improved policy gradient algorithm that integrates Monte Carlo sampling. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimate the gradients required when solving the policy gradient algorithm.

[0121] In each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0122]

[0123] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ, j is the actual sampling number, k is the actual training number, λ k is the Lagrange multiplier for the k-th training, and v k is the VaR parameter for the k-th training.

[0124] S400. Train the task scheduling model, and output the task scheduling policy of the data center through the trained task scheduling model, including:

[0125] S1. Input the confidence level α, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0126] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter V, and the Monte Carlo sampling scale N.

[0127] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0128] S4. If i < N, go to S5; otherwise, go to S6.

[0129] S5. Form a trajectory according to the policy, let j = j + 1, and return to S4.

[0130] S6. Estimate the gradient

[0131] S7. Update the model parameters of each data center:

[0132]

[0133] where θ k is the policy network parameter for the k-th training.

[0134] S8. Let k = k + 1, and return to S3.

[0135] See Figure 3 , a computing task scheduling system considering the uncertainty risk of data centers, including a task scheduling model construction module 100, an optimization module 200, a gradient estimation module 300, and a training scheduling module 400.

[0136] The task scheduling model construction module 100 is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of the task and the electricity price are used as uncertain parameters.

[0137] The task scheduling model includes a state space, an action space, a reward function, and a value function.

[0138] The state space S includes the execution sequence, CPU resource requirements, memory resource requirements, and execution time between tasks, the CPU resource utilization rate, memory resource utilization rate, current time, and current electricity price of each server.

[0139] The action space A includes a task execution action and a null action; the task execution action a i :

[0140]

[0141] represents allocating the task to the server se m for execution.

[0142] The null action φ represents that the optimal action at this time is not to allocate any tasks.

[0143] The reward function r iIt is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time.

[0144] The value function J θ (ξ) is:

[0145] J θ (ξ)=r1+...+γ T-1 r T .

[0146] Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ is, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,......,s T , a T , r T ), θ is the parameter of the policy network, and T is the length of the trajectory sequence.

[0147] The optimization module 200 is used to use a policy network to represent the decision-making process of task scheduling, measure the risk of excessively high operating costs based on the conditional risk value, and embed the conditional risk value as a constraint into the reinforcement learning framework to optimize the data center operating costs while ensuring that the risks are controllable.

[0148] A policy network is used to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of each action being executed. The task scheduling model selects the action to be executed according to the probability.

[0149] The goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the operating costs of the data center, that is:

[0150] min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β.

[0151] Among them, β is the maximum acceptable operating cost, CVaR α Defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions α (J θ (ξ)):

[0152]

[0153] Among them, (J θ (ξ)-v) + =max(Jθ (ξ)-v, 0), represents the part greater than 0; v is the VaR parameter, which is used as the risk threshold for calculating CVaR, E is the expectation calculation, and α is the confidence level.

[0154] Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem:

[0155]

[0156] Among them, λ is the Lagrange multiplier, and L(v, θ, λ) is the loss function of the task scheduling model.

[0157] The gradient estimation module 300 is used to solve the task scheduling policy through an improved policy gradient algorithm that integrates Monte Carlo sampling. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradients required when solving the policy gradient algorithm.

[0158] In each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process according to the current policy network, obtains N data center operation trajectories, and unbiasedly estimates the gradient of L(v, θ, λ) through the following formula and updates the parameters:

[0159]

[0160] Among them, represents the probability of generating the trajectory ξ j，k of, respectively represent the derivatives of the loss function in the directions of the parameters v, θ, λ. j is the actual sampling number, k is the actual training number, and λ k is the Lagrange multiplier for the k-th training, and v k is the VaR parameter for the k-th training.

[0161] The training scheduling module 400 is used to train the task scheduling model, and outputs the task scheduling policy of the data center through the trained task scheduling model.

[0162] Training the task scheduling model includes:

[0163] S1. Input the confidence level α, the maximum acceptable operating cost β, the number of training times K, and the update steps η1, η2, η3.

[0164] S2. Initialize the policy network parameters θ, the Lagrange multiplier λ, the VaR parameter v, and the Monte Carlo sampling scale N.

[0165] S3. If k < K, enter S4; otherwise, the algorithm ends.

[0166] S4. If j < N, go to S5; otherwise go to S6.

[0167] S5. Form a trajectory according to the policy, set j = j + 1, and return to S4.

[0168] S6. Estimate the gradient

[0169] S7. Update the model parameters of each data center:

[0170]

[0171] where θ k is the policy network parameter for the k-th training.

[0172] S8. Set k = k + 1 and return to S3.

[0173] Embodiment 2

[0174] The present invention compares with the existing algorithms that do not consider risks. Under the same electricity prices, task arrivals, and the same uncertainty settings, the method of the present invention has obvious advantages in the optimization effect.

[0175] Referring to Figure 4 , it can be seen that the method proposed by the present invention can perform task scheduling according to the time-varying electricity price and dynamically adjust the energy consumption curve. A large number of tasks are postponed to the periods with lower electricity prices, reducing the operating cost of the data center.

[0176] Referring to Figure 5 , the traditional optimization algorithm only optimizes the operating cost J θ (ξ) of the data center and does not use CVaR to measure the risk of the operating cost as a risk constraint. Compared with the optimization algorithms that do not consider risks, the method proposed by the present invention can perceive the risks brought by uncertainties. Although the relatively conservative scheduling strategy gives up some opportunities to obtain lower operating costs, the operating cost distribution under the method of the present invention is more concentrated, avoiding potential high losses.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described by referring to the preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A computing task scheduling method taking into account the uncertainty risk of a data center, characterized in that: include: The task scheduling process of the data center is modeled based on the Markov decision process to obtain the task scheduling model; In the task scheduling model, task information and electricity prices are used as uncertain parameters; The task scheduling model uses a policy network to represent the decision-making process of task scheduling, measures the risk of excessive operating costs based on conditional value at risk, and embeds the conditional value at risk as a constraint into the reinforcement learning framework to optimize data center operating costs while ensuring that risks are controllable. Solving the task scheduling policy by integrating an improved policy gradient algorithm with Monte Carlo sampling. The improved policy gradient algorithm with Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradient required for solving the policy gradient algorithm; The task scheduling model is trained, and the task scheduling strategy of the data center is output through the trained task scheduling model.

2. The computing task scheduling method taking into account the uncertainty risk of data centers according to claim 1 is characterized in that: The task scheduling model includes a state space, an action space, a reward function and a value function; The state space S includes the execution order of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization rate of each server, memory resource utilization rate, current time, and current electricity price; The action space A includes task execution actions and empty actions; the task execution action a i : Task execution action a i Indicates that the task Assign to server m Execute on; The empty action φ indicates that the optimal action at this time is not to assign any tasks; The reward function r i is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time; The value function J θ (ξ) is: J θ (ξ)=r1+...+γ T-1 r T ; Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,……,s T , a T , r T ), θ is the policy network parameter, and T is the length of the trajectory sequence.

3. The computing task scheduling method taking into account the uncertainty risk of data centers according to claim 2 is characterized in that: The task scheduling model uses a policy network to represent the decision-making process of task scheduling. It measures the risk of excessive operating costs based on the conditional value at risk (CVR). The CVR is embedded as a constraint in the reinforcement learning framework to optimize data center operating costs while ensuring that risks are controllable. This includes: A policy network is used to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of each action being executed. The task scheduling model selects the action to be executed according to the probability. The goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the operating costs of the data center, that is: min e J θ (ξ)subject to CVaR α (J θ (ξ))≤β; Where β is the maximum acceptable operating cost, CVaR. It is defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions. α (J θ (ξ)): Among them, (J θ (ξ)-v) + =max(J θ (ξ)-v, 0), represents the part greater than 0; v is the VaR parameter, which is used to calculate the risk threshold of CVaR, E is the expected calculation, and α is the confidence level; Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem: Where λ is the Lagrange multiplier and L(v, θ, λ) is the loss function of the task scheduling model.

4. The computing task scheduling method taking into account the uncertainty risk of data centers according to claim 3 is characterized in that: The improved policy gradient algorithm that integrates Monte Carlo sampling is used to solve the task scheduling policy. The improved policy gradient algorithm that integrates Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradient required for the policy gradient algorithm to solve the problem, including: During each iteration, N Monte Carlo samplings are performed. The task scheduling model simulates the data center task scheduling process based on the current policy network, obtains the operation trajectories of N data centers, and uses the following formula to make an unbiased estimate of the gradient of L(v, θ, λ) and update the parameters: in, Represents the generated trajectory ξ j，k The probability of Represent the derivatives of the loss function in the direction of parameters v, θ, and λ respectively, j is the actual number of sampling times, k is the actual number of training times, and λ k is the Lagrange multiplier of the kth training, v k is the VaR parameter of the kth training.

5. The computing task scheduling method taking into account the uncertainty risk of data centers according to claim 4 is characterized in that: The training of the task scheduling model includes: S1, input confidence level α, maximum acceptable running cost β, number of training times K and update step size η1, η2, η3; S2, initialize the strategy network parameters θ, Lagrange multiplier λ, VaR parameter V and Monte Carlo sampling scale N; S3. If k < K, go to S4; otherwise the algorithm ends; S4. If i < N, proceed to S5; otherwise proceed to S6; S5. Form a trajectory according to the strategy, set j = j + 1, and return to S4; S6. Estimating Gradients S7. Update the model parameters of each data center: Among them, θ k is the policy network parameter of the k-th training; S8. Set k=k+1 and return to S3.

6. A computing task scheduling system taking into account the uncertainty risk of data centers, characterized in that: Includes task scheduling model building module, optimization module, gradient estimation module and training scheduling module; The task scheduling model building module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity price are used as uncertain parameters; The optimization module is used to use a policy network to represent the decision-making process of task scheduling, measure the risk of excessive operating costs based on conditional value at risk, and embed the conditional value at risk as a constraint into the reinforcement learning framework to optimize the operating costs of the data center while ensuring that the risks are controllable; The gradient estimation module is used to solve the task scheduling policy by integrating the improved policy gradient algorithm with Monte Carlo sampling. The improved policy gradient algorithm with Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling policies and estimates the gradient required for the policy gradient algorithm to solve the problem. The training scheduling module is used to train the task scheduling model and output the task scheduling strategy of the data center through the trained task scheduling model.

7. The computing task scheduling system taking into account the uncertainty risk of data centers according to claim 6, characterized in that: The task scheduling model includes a state space, an action space, a reward function and a value function; The state space S includes the execution order of tasks, CPU resource requirements, memory resource requirements and execution time, CPU resource utilization rate of each server, memory resource utilization rate, current time, and current electricity price; The action space A includes task execution actions and empty actions; the task execution action a i : Task execution action a i Indicates that the task Assign to server m Execute on; The empty action φ indicates that the optimal action at this time is not to assign any tasks; The reward function r i is the sum of the energy consumption cost function and the service quality cost function; the energy consumption cost function is obtained by multiplying the energy consumption caused by executing the task by the time-varying electricity price; the service quality cost function is determined by the difference between the task completion time and the user submission time; The value function J θ (ξ) is: J θ (ξ)=r1+...+γ T-1 r T ; Where γ∈[0,1] represents the trade-off between future costs and current costs. The larger the γ, the more the agent values future benefits. ξ is the trajectory, ξ=(s1,a1,r1,……,s T , a T , r T ), θ is the policy network parameter, and T is the length of the trajectory sequence.

8. The computing task scheduling system taking into account the uncertainty risk of data centers according to claim 7, characterized in that: The strategy network is used to represent the decision-making process of task scheduling. The risk of excessive operating costs is measured based on the conditional value at risk. The conditional value at risk is embedded as a constraint in the reinforcement learning framework to optimize the data center operating costs while ensuring that the risks are controllable. This includes: A policy network is used to represent the decision-making process of task scheduling. The input of the policy network is the data included in the state space, and the output is the probability of each action being executed. The agent selects the action to be executed according to the probability. The goal of the optimization problem is to optimize the strategic network parameters θ under the premise of controllable risks to minimize the operating costs of the data center, that is: min θ J θ (ξ)subject to CVaR α (J θ (ξ))≤β; Where β is the maximum acceptable operating cost, CVaR. It is defined as the tail expected value of the distribution, reflecting the expected operating cost CVaR of the data center under extreme conditions. α (J θ (ξ)): Among them, (J θ (ξ)-v) + =max(J θ (ξ)-v0), represents the part greater than 0; v is the VaR parameter, which is used to calculate the risk threshold of CVaR, E is the expected calculation, and α is the confidence level; Based on the Lagrangian relaxation method, the optimization problem is transformed into an unconstrained optimization problem: Where λ is the Lagrange multiplier and L(v, θ, λ) is the loss function of the task scheduling model.

9. The computing task scheduling system taking into account the uncertainty risk of data centers according to claim 8, characterized in that: The improved policy gradient algorithm integrating Monte Carlo sampling uses the Monte Carlo sampling method to obtain trajectories under different task scheduling strategies and estimates the gradient required for solving the policy gradient algorithm, including: In each iteration, N Monte Carlo samplings are performed. The agent simulates the data center task scheduling process based on the current policy network, obtains the operation trajectories of N data centers, and makes an unbiased estimate of the gradient of L(v, θ, λ) using the following formula and updates the parameters: in, Represents the generated trajectory ξ j，k The probability of Represent the derivatives of the loss function in the direction of parameters v, θ, and λ respectively, j is the actual number of sampling times, k is the actual number of training times, and λ k is the Lagrange multiplier of the kth training, v k is the VaR parameter of the kth training.

10. The computing task scheduling system taking into account the uncertainty risk of data centers according to claim 9, characterized in that: The improved policy gradient algorithm integrated with Monte Carlo sampling is used to solve the task scheduling strategy, including: S1, input confidence level α, maximum acceptable running cost β, number of training times K and update step size η1, η2, η3; S2, initialize the strategy network parameters θ, Lagrange multiplier λ, VaR parameter v and Monte Carlo sampling scale N; S3. If k < K, go to S4; otherwise the algorithm ends; S4. If j < N, go to S5; otherwise go to S6; S5. Form a trajectory according to the strategy, set j = j + 1, and return to S4; S6. Estimating Gradients S7. Update the model parameters of each data center: Among them, θ k is the policy network parameter of the k-th training; S8. Set k=k+1 and return to S3.

Images