A data center task scheduling method and system under uncertain environment

Abstract

The present invention relates to the technical field of data center energy management, and particularly relates to a task scheduling method and system for a data center under an uncertain environment; the method of the present invention includes modeling the task scheduling process of the data center based on a Markov decision process to obtain a task scheduling model. In the task scheduling model, the information of tasks and electricity prices are used as uncertain parameters. The quantile regression reinforcement learning algorithm is adopted to select task scheduling actions, calculate the rewards after the data center executes the tasks, and store the state, task scheduling actions, rewards, and next state after executing the tasks in a replay buffer, thereby obtaining different task scheduling distribution strategy data. To maximize the long-term cumulative discounted rewards, the optimal task scheduling distribution strategy of the task scheduling model is determined; through the method of the present invention, an uncertainty-aware task scheduling strategy can be realized, which can effectively improve the profit and completion rate of the data center.

Description

Technical Field

[0001] The present invention relates to the technical field of data center energy management, and particularly to a data center task scheduling method and system under an uncertain environment. Background Art

[0002] With the rapid development of cloud computing, big data analysis, and machine learning, the power consumption of data centers has shown a sharp growth trend, attracting wide attention. The main component of the energy consumption of data centers is the energy consumption of computing tasks submitted by users, which has occupied a considerable proportion of the total energy consumption. In this context, we notice the latency tolerance of data center tasks. Latency tolerance means that some tasks with high latency tolerance can be postponed to a future period without violating the task deadline. This unique operating property provides flexibility in the time dimension for task execution. Specifically, by scheduling tasks to be executed during periods with lower electricity prices, data centers can flexibly respond to grid electricity price signals, thereby reducing the energy consumption cost of task computing. However, in this process, data centers cannot know in advance the future task arrival situation and electricity prices. In this uncertain environment, the task scheduling decision of data centers becomes very difficult.

[0003] To solve this problem, CN112949195A proposes a multi-objective task scheduling method for power cloud data centers to uniformly coordinate the efficiency and energy consumption of task scheduling in power cloud data centers. However, this method does not consider the risks that task heterogeneity and arrival uncertainty may bring to task scheduling. On the other hand, CN117076100A is a mathematical model for hybrid data center task scheduling based on queuing theory. By analyzing the delays and energy consumption of each node, it can perform load distribution more effectively, thereby reducing energy consumption and delay. However, this method only optimizes the energy consumption and delay of servers and does not consider the impact of electricity price uncertainty during task scheduling on the energy consumption cost of data centers. Summary of the Invention

[0004] Aiming at the above deficiencies of the prior art, the present invention aims to provide a data center task scheduling method and system under an uncertain environment to solve the problem of energy waste in data centers caused by the existing data center task scheduling not considering uncertain task arrivals and electricity price uncertainty.

[0005] To solve the above problems, the present invention adopts the following technical solutions:

[0006] An energy-aware data center task scheduling method under an uncertain environment, comprising:

[0007] Model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of the task and the electricity price are used as uncertain parameters;

[0008] Adopt the quantile regression reinforcement learning algorithm to select the task scheduling action, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward, and next state after executing the task in the replay buffer, so as to obtain different task scheduling distribution strategy data, and maximize the long-term cumulative discounted reward to determine the optimal task scheduling distribution strategy of the task scheduling model.

[0009] Furthermore, the modeling of the task scheduling process of the data center based on the Markov decision process according to the tasks received by the data center to obtain a task scheduling model includes:

[0010] The task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set;

[0011] Among them, the state set is .

[0012] Among them, is the information of task n, , N is the total number of tasks processed by an M-server data center within T time steps, are respectively the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task, is the resource occupancy of M servers, , is the resource occupancy of server m, , is the predicted electricity price for each time step, , after the current task scheduling is completed, the state transitions from to , the task information is transferred to the next task information , and the resource occupancy of the server will change accordingly according to the execution of task n.

[0013] The action set is .

[0014] Among them, is the execution time and is the execution server, and .

[0015] The reward function set is: .

[0016] Among them, is the energy consumption cost of task n, , is the IT device energy consumption of task n per time step, , and are the power of server m in the idle state and full load state respectively, is the penalty of task n.

[0017] The discount factor set is .

[0018] Furthermore, the long-term cumulative discounted reward .

[0019] Furthermore, to maximize the long-term cumulative discounted reward and determine the optimal distribution strategy of the task scheduling model for task scheduling, it includes: the state-action distribution corresponding to each task scheduling distribution strategy is:

[0020] .

[0021] Among them, is the value of taking action in state , is the expected value.

[0022] That is, the optimal distribution strategy satisfies:

[0023] .

[0024] Furthermore, use distributed reinforcement learning to calculate the state-action value distribution corresponding to each task scheduling distribution strategy, and its expected value is :

[0025] .

[0026] Furthermore, use the quantile regression temporal difference learning algorithm to characterize the state-action value distribution by learning the quantiles corresponding to quantiles:

[0027] .

[0028] Among them is the inverse function of the cumulative distribution function of the state-action value distribution , represents the midpoint of the quantiles and, and , the state-action value distribution Updated according to the following formula:

[0029] .

[0030] Where is an indicator function.

[0031] Furthermore, the loss function of the quantile regression time series difference learning algorithm is:[[]]END]

[0032] .

[0033] Where k is a constant, which is the quantile Huber loss threshold, is the TD error.

[0034] Furthermore, the process of determining the optimal distribution strategy of task scheduling for the task scheduling model by using the quantile regression reinforcement learning algorithm includes:

[0035] S201. Initialize the online network, target network, environment, number of quantile samples , number of experiences , batch size , and the target network update frequency F.

[0036] S202. Obtain the state based on the task received by the data center, and use it as the input of the neural network.

[0037] S203. Select a scheduling action and execute it.

[0038] S204. The state transfers to .

[0039] S205. Calculate the reward .

[0040] S206. Store the state, scheduling action, reward, and next state after executing the task in the replay buffer.

[0041] S207. When the number of stored data in the replay buffer is greater than , execute S208 - S209, otherwise return to S202 - S206.

[0042] S208. Sample from the replay buffer, calculate the loss value of the sample, and update the network.

[0043] S209. When the number of iterations is greater than the target network update frequency F, update the target network to obtain the optimal distribution strategy of task scheduling for the task scheduling model.

[0044] An energy-aware data center task scheduling system under uncertain environment, including a task scheduling model modeling module and a task scheduling optimal distribution strategy determination module.

[0045] The task scheduling model modeling module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of the task and the electricity price are used as uncertain parameters.

[0046] The task scheduling optimal distribution strategy determination module is used to adopt the quantile regression reinforcement learning algorithm to select task scheduling actions, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward, and next state after executing the task in the replay buffer, so as to obtain different task scheduling distribution strategy data, and determine the task scheduling optimal distribution strategy of the task scheduling model by maximizing the long-term cumulative discounted reward.

[0047] Furthermore, the task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set.

[0048] Among them, the state set is .

[0049] Among them, is the information of task n, , N is the total number of tasks processed by the data center with M servers within T time steps, are respectively the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task, is the resource occupancy of M servers, , is the resource occupancy of server m, , is the predicted electricity price for each time step, , after the current task scheduling is completed, the state transitions from to , the task information is transferred to the next task information , and the resource occupancy of the server will change accordingly according to the execution of task n.

[0050] The action set is .

[0051] Among them, is the execution time and is the execution server, and .

[0052] The reward function set is: .

[0053] Among them, is the energy consumption cost of task n, , is the energy consumption of IT equipment per time step for task n, , and are the power of server m in the idle state and full load state respectively, is the penalty for task n.

[0054] The discount factor set is .

[0055] The beneficial effects of the present invention are as follows: The present invention models the energy-aware data center task scheduling process as a Markov decision process. In the proposed model, the present invention incorporates the uncertainties of task arrival and electricity price into the proposed model, and develops a distributed reinforcement learning scheme based on the QR-DQN method to solve the proposed model. By using the quantile regression reinforcement learning algorithm, the probability characteristics of the state-action value can be explicitly captured, thus realizing an uncertainty-aware task scheduling strategy; through the method of the present invention, the profit and completion rate of the data center can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flowchart of an energy-aware data center task scheduling method under an uncertain environment in Embodiment 1 of the present invention.

[0057] Figure 2 is a schematic diagram of an energy-aware data center task scheduling system under an uncertain environment in Embodiment 1 of the present invention.

[0058] Figure 3 is a profit comparison diagram of the data center under different scheduling strategies in Embodiment 2 of the present invention.

[0059] Figure 4 is a task completion rate comparison diagram of the data center under different scheduling strategies in Embodiment 2 of the present invention.

[0060] Figure 5 is a resource utilization rate display diagram of the FIFO scheduling strategy under electricity price fluctuations in Embodiment 2 of the present invention.

[0061] Figure 6 is a resource utilization rate display diagram of the scheduling strategy of the present invention under electricity price fluctuations in Embodiment 2 of the present invention.

[0062] Figure 7 is a resource utilization rate display diagram of the FIFO scheduling strategy under electricity price fluctuations in Embodiment 2 of the present invention.

[0063] Figure 8 This is a diagram showing the resource utilization rate of the scheduling strategy of the present invention under electricity price fluctuations in Embodiment 2 of the present invention. Detailed implementation manners

[0064] The present invention will be further described in detail below in conjunction with specific embodiments.

[0065] It should be noted that these embodiments are only used to illustrate the present invention, rather than limiting the present invention. Any simple improvement of the method under the premise of the concept of the present invention belongs to the scope protected by the present invention.

[0066] Considering the uncertainty of task arrival and electricity price, the task scheduling strategy needs to be adjusted in real time to achieve the minimum energy cost. Specifically, when the task arrival rate is low, the data center can schedule some of the tasks that arrived in the past to this time, avoiding the situation where tasks cannot be executed due to resource limitations when the past task rate was high. When the price is high, the data center can schedule tasks to be executed at a lower electricity price in the future to reduce the energy consumption cost. Therefore, the task scheduling model needs to further consider the uncertainty of task arrival and the fluctuation of electricity price.

[0067] More specifically, when a task arrives at the data center, the data center simultaneously obtains various information of the task, including arrival time, resource utilization rate, execution time, deadline, and computing income. Based on this information, the data center assigns the task to a server and executes the task at the specified time. By combining the energy consumption of task execution with the corresponding electricity price, the data center can calculate the profit generated by task execution. After all tasks are completed, the data center obtains the total income.

[0068] Based on the task information obtained by the data center, the server occupancy situation, and the predicted electricity price, task scheduling actions are selected. After the data center executes the scheduling action, the income is fed back to the agent as the immediate reward after task execution. The agent stores the current state, action, reward, and next state in the replay buffer. Subsequently, the agent uses the stored data to iteratively update the neural network data to optimize the scheduling strategy. In the above process, since users can submit computing requests at any time, the task arrival situation is regarded as an uncertain parameter. In addition to the arrival time, during the execution of the current task, other future task information is also uncertain. At the same time, considering the widely adopted real-time pricing after the event, the electricity price is also regarded as an uncertain parameter.

[0069] Embodiment 1

[0070] Refer to Figure 1 , which is a method for task scheduling in an energy-aware data center under an uncertain environment, including:

[0071] S100. Based on the tasks received by the data center, model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model. In the task scheduling model, the information of the tasks and the electricity price are used as uncertain parameters.

[0072] As an implementable approach, modeling the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model includes:

[0073] The task scheduling model includes an MDP framework, including a state set S, an action set A, a reward function set R, and a discount factor set .

[0074] Among them, the state set is .

[0075] Among them, is the information of task n, , N is the total number of tasks processed by the data center with M servers within T time steps, are respectively the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task, is the resource occupancy of M servers, , is the resource occupancy of server m, , is the predicted electricity price for each time step, , after the current task scheduling is completed, the state transitions from to , the task information is transferred to the next task information , and the resource occupancy of the server will change accordingly according to the execution situation of task n.

[0076] The action set is .

[0077] Among them, is the execution time and is the execution server. Considering the actual situation, the execution time must be after the arrival time of the task, that is .

[0078] The action is a discrete vector, and the execution time and the execution server are both discrete variables.

[0079] The server selection at any time step should not cause a violation of its resource limit. The reward is the direct reward, which is the generated revenue.

[0080] That is, the reward function set is: .

[0081] Among them, is the energy consumption cost of task n, , is the IT device energy consumption of task n per time step, , and are the powers of server m in the idle state and full load state respectively. If task n is completed after the deadline, that is , is the penalty for task n.

[0082] The discount factor set is .

[0083] The ultimate goal is to maximize the total revenue from executing all tasks, that is, the long-term cumulative discounted reward to be maximized.

[0084] S200. Adopt the quantile regression reinforcement learning algorithm to select task scheduling actions, calculate the rewards after the data center executes the tasks, and store the states, task scheduling actions, rewards, and next states after executing the tasks in the replay buffer, so as to obtain different task scheduling distribution strategy data. Maximize the long-term cumulative discounted reward to determine the optimal distribution strategy of the task scheduling model for task scheduling.

[0085] As an implementable manner, the step of maximizing the long-term cumulative discounted reward to determine the optimal distribution strategy of the task scheduling model includes: The state-action distribution corresponding to each task scheduling distribution strategy is:

[0086] .

[0087] Among them, is the value of taking action in state , is the expected value.

[0088] That is, the optimal distribution strategy satisfies:

[0089] .

[0090] is the policy function, that is, the corresponding to be taken in each state , is to select the action to make the value the largest in state policy function.

[0091] As an implementable manner, the present invention adopts distributed reinforcement learning to calculate the state-action value distribution corresponding to each task scheduling distribution policy , and its expected value is :

[0092] .

[0093] The distribution of Z in the distributed Bellman equation is determined by the interaction of three random variables: the reward R, the next state-action , and the stochastic return distribution of its state-action value distribution .

[0094] As an implementable manner, the present invention adopts the quantile regression temporal difference learning algorithm to characterize the state-action value distribution by learning the quantiles corresponding to quantiles, then :

[0095] .

[0096] Where is the inverse function of the cumulative distribution function of the state-action value distribution , represents the midpoint of the quantiles and, and , the state-action value distribution is updated according to the following formula:

[0097] .

[0098] Where is the indicator function.

[0099] The quantile regression temporal difference learning algorithm can more accurately capture the diverse performance of state-action pairs in different environments by learning the quantiles corresponding to Nq quantiles to characterize the value distribution.

[0100] As an implementable manner, the present invention uses an extended version of the Huber loss Lk(u) as the quantile Huber loss and uses it as the loss function.

[0101] The loss function of the quantile regression temporal difference learning algorithm is:[[]]

[0102] .

[0103] Where k is a constant, is the quantile Huber loss threshold,[[]] is the TD error.

[0104] ​​As an implementable manner, the process of determining the optimal distribution strategy of task scheduling for the task scheduling model by using the quantile regression reinforcement learning algorithm includes:

[0105] S201. Initialize the online network, target network, environment, number of quantile samples , number of experiences , batch size , and target network update frequency F.

[0106] S202. Obtain the state from the environment (that is, according to the tasks received by the data center, the task information, server resource occupancy rate, and predicted electricity price can be used to generate the state ), and use it as the input of the neural network.

[0107] S203. Select a scheduling action and execute it.

[0108] S204. The state is transferred to .

[0109] S205. Calculate the reward after the data center executes the task .

[0110] S206. Store the state, scheduling action, reward, and next state after executing the task in the replay buffer.

[0111] S207. When the number of stored data in the replay buffer is greater than , execute S208 - S209; otherwise, return to S202 - S206.

[0112] S208. Sample from the replay buffer, calculate the loss value of the sample, and update the network.

[0113] S209. When the number of iterations is greater than the target network update frequency F, update the target network to obtain the optimal distribution strategy of task scheduling for the task scheduling model.

[0114] S300. Schedule the tasks received by the data center through the optimal distribution strategy of task scheduling.

[0115] The present invention takes into account the task uncertainty and electricity price fluctuations in the task scheduling process, aiming to minimize the energy consumption cost. The present invention focuses on the time, resources, and revenue information of tasks, and the execution time of tasks affects the corresponding electricity price, having a coupling relationship with the energy consumption cost. The impact brought by this coupling will change with the uncertainty of task information and electricity price fluctuations. Therefore, considering the energy consumption cost is more difficult than simply considering energy consumption and completion time. For this reason, the present invention adopts the QR-DQN algorithm, whose output is not a single value but a distribution of values. This distribution property enables the training process to better face the uncertainty of tasks and electricity prices, reduce variance, and improve stability. By modeling the distribution, the system can more flexibly respond to environmental changes, providing a more reliable basis for achieving the minimum energy consumption cost.

[0116] See Figure 2 , which is an energy-aware data center task scheduling system under uncertain environment, including a task scheduling model modeling module 100 and a task scheduling optimal distribution strategy determination module 200.

[0117] The task scheduling model modeling module 100 is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of tasks and electricity prices are used as uncertain parameters.

[0118] The task scheduling optimal distribution strategy determination module 200 is used to adopt the quantile regression reinforcement learning algorithm, select task scheduling actions, calculate the rewards after the data center executes the tasks, and store the states, task scheduling actions, rewards, and next states after executing the tasks in the replay buffer, so as to obtain different task scheduling distribution strategy data, and determine the task scheduling optimal distribution strategy of the task scheduling model by maximizing the long-term cumulative discounted rewards.

[0119] As an implementable manner, the task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set.

[0120] Among them, the state set is .

[0121] Among them, is the information of task n, , N is the total number of tasks processed by an M-server data center within T time steps, are respectively the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task, is the resource occupancy situation of M servers, , is the resource occupancy situation of server m, , The predicted electricity price for each time step After the current task scheduling is completed, the state changes from to The task information is transferred to the next task information The resource occupancy of the server will change accordingly according to the execution of task n.

[0122] The action set is .

[0123] Among them, is the execution time and is the execution server, and .

[0124] The reward function set is: .

[0125] Among them, is the energy consumption cost of task n, , is the IT equipment energy consumption of task n per time step, , and are the powers of server m in the idle state and full load state respectively, is the penalty for task n.

[0126] The discount factor set is .

[0127] Embodiment 2

[0128] The present invention compares the profit and completion rate of the data center under the existing first-in-first-out (FIFO) strategy. It can be seen from Figure 3 that when the number of tasks processed by the data center ranges from 30 to 70, the strategy of the present invention is superior to the comparison strategy in optimizing profit. Figure 4 It can be seen that when the number of tasks gradually increases to 170, the task completion rate of the comparison strategy begins to decline, while the task completion rate under the strategy of the present invention remains at 100%.

[0129] The present invention compares the resource utilization rates of different scheduling strategies under different electricity price scenarios. Figure 5 、 Figure 6 It can be seen that the strategy of the present invention can perform effective task scheduling well based on electricity price fluctuations. During high electricity price periods, some tasks are transferred to time steps with lower prices, reducing the energy consumption cost while enhancing the flexibility of the data center's energy consumption. In contrast, the FIFO strategy cannot adjust the energy consumption according to the electricity price, resulting in a decrease in profit. In addition, in Figure 7 、 Figure 8In [the situation], after the number of tasks increases from 60 to 70, the strategy of the present invention can still effectively sense price fluctuations, indicating that the strategy of the present invention also has advantages under the condition of uncertain task arrival.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described by referring to the preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. An energy-aware data center task scheduling method in an uncertain environment, characterized in that, Including: Modeling the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; In the task scheduling model, the information of tasks and electricity prices are used as uncertain parameters; Using the quantile regression reinforcement learning algorithm, selecting task scheduling actions, calculating the rewards after the data center executes the tasks, and storing the state, task scheduling actions, rewards, and the next state after executing the tasks in the replay buffer, so as to obtain different task scheduling distribution strategy data, and maximizing the long-term cumulative discounted rewards to determine the optimal task scheduling distribution strategy of the task scheduling model; The task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set; The set of reward functions is: r n = e n - c n - b n , Among them, c n is the energy consumption cost of task n, c n = ∑ t∈S (p t * energy n ), energy n is the IT device energy consumption of task n per time step, and are the powers of server m in the idle state and full load state respectively, b n is the penalty of task n, t is time, S is the execution time period of task n, p t is the electricity price at time t; The quantile regression reinforcement learning algorithm includes a quantile regression temporal difference learning algorithm. By using the quantile regression temporal difference learning algorithm, N q parameters are used to approximate the state-action value distribution Among them, F Z -1 is the inverse function of the cumulative distribution function of the state-action value , and quantile j is the quantile label, and τ0 = 0. The state-action value distribution is updated according to the following formula: Among them, is an indicator function.

2. The method for task scheduling of an energy-aware data center in an uncertain environment according to claim 1, characterized in that The state set is s n =(job n , Ser, P′), Among them, job n is the information of task n, and job n =(u n , ta n , te n , td n , e n ), n ∈ 1, 2,..., N, where N is the total number of tasks processed by the data centers of M servers within T time steps. u n , ta n , te n , td n , e n are the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task respectively. Ser is the resource occupancy situation of M servers, Ser = (ser1,..., ser m ,..., ser M ), and ser m is the resource occupancy situation of server m, ser m =(ser m1 , ser m1 ,..., ser mT ). P' is the predicted electricity price for each time step, P' = (p′1, p′2,..., p′ T ). After the current task scheduling is completed, the state transitions from s n to s n+1 , the task information job n is transferred to the next task information job n+1 , and the resource occupancy situation Ser of the server will change accordingly according to the execution situation of task n; The action set is a n =(et n , se n ), Among them, et n is the execution time and se n is the execution server, and et n > ta n ; The discount factor set is γ ∈ [0, 1].

3. The energy-aware data center task scheduling method under uncertain environment according to claim 2, characterized in that, The long-term cumulative discounted reward R = r1 + γr2 +... γ N-1 r N .

4. The energy-aware data center task scheduling method under uncertain environment according to claim 3, wherein The process of maximizing the long-term cumulative discounted rewards to determine the optimal task scheduling distribution strategy of the task scheduling model includes: The state-action distribution corresponding to each task scheduling distribution strategy is: Q(s n ,a n ) = E(r n + γQ(s n+1 ,a n+1 )), where Q is the value of state s n when taking action a n and E is the expected value; That is, the optimal distribution strategy satisfies: where π is the policy function, i.e., the action a taken corresponding to each state S n below n .

5. The energy-aware data center task scheduling method under uncertain environment according to claim 4, wherein Use distributed reinforcement learning to calculate the state-action value distribution Z(s n , a n ), and its expected value is Q(s n , a n ): Q(s n ,a n ) = E(Z(s n ,a n )) 6. The task scheduling method for an energy-aware data center in an uncertain environment according to claim 1, wherein The loss function of the quantile regression temporal difference learning algorithm is: where k is a constant, which is the quantile Huber loss threshold, and u = r n +γZ(s n+1 , a n+1 ) - Z(s n , a n ) is the temporal difference error.

7. The task scheduling method for an energy-aware data center in an uncertain environment according to claim 6, wherein The process of using the quantile regression reinforcement learning algorithm to determine the optimal task scheduling distribution strategy of the task scheduling model includes: S201. Initialize the online network, target network, environment, the number of quantile samples N q , the number of experiences, batch size N b , and the target network update frequency F; S202. Obtain the status s according to the task received by the data center n and use it as the input of the neural network; S203. Select a scheduling action a n and execute it; S204, state s n Transfer to s n+1 ; S205. Calculate the reward r after the data center executes this task n ; S206. Store the state, scheduling actions, rewards, and the next state after executing the tasks in the replay buffer; S207. When the number of data stored in the playback buffer is greater than N b then execute S208 - S209, otherwise return to S202 - S206; S208. Sample from the replay buffer, calculate the loss value of the sample and update the network; S209. Until the number of iterations is greater than the target network update frequency F, update the target network to obtain the optimal task scheduling distribution strategy of the task scheduling model.

8. An energy-aware data center task scheduling system in an uncertain environment, characterized in that, Including a task scheduling model modeling module and an optimal task scheduling distribution strategy determination module; The task scheduling model modeling module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, the information of tasks and electricity prices are used as uncertain parameters; The optimal task scheduling distribution strategy determination module is used to use the quantile regression reinforcement learning algorithm to select task scheduling actions, calculate the rewards after the data center executes the tasks, and store the state, task scheduling actions, rewards, and the next state after executing the tasks in the replay buffer, so as to obtain different task scheduling distribution strategy data, and maximizing the long-term cumulative discounted rewards to determine the optimal task scheduling distribution strategy of the task scheduling model; The task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set; The set of reward functions is: r n = e n - c n - b n , Among them, c n is the energy consumption cost of task n, c n = ∑ t∈S (p t * energy n ), energy n is the IT device energy consumption of task n per time step. and are the powers of server m in the idle state and full load state respectively, b n is the penalty of task n, t is the time, S is the execution time period of task n, p t is the electricity price at time t; The quantile regression reinforcement learning algorithm includes a quantile regression temporal difference learning algorithm. By using the quantile regression temporal difference learning algorithm, N q parameters are used to approximate the state-action value distribution Among them, F Z -1 is the inverse function of the cumulative distribution function of the state-action value , quantile j is the quantile label, and τ0 = 0, the state-action value distribution is updated according to the following formula: Among them, is an indicator function.

9. The energy-aware data center task scheduling system under uncertain environment according to claim 8, characterized in that, The state set is s n =(job n , Ser, P′), Among them, job n is the information of task n, and job n =(u n , ta n , te n , td n , e n ), where n ∈ 1, 2,..., N, and N is the total number of tasks processed by the data centers of M servers within T time steps. u n , ta n , te n , td n , e n are respectively the resource occupancy rate, arrival time, execution time, deadline, and computing revenue of the task. Ser is the resource occupancy situation of M servers, Ser=(ser1,..., ser m ,..., ser M ), and ser m is the resource occupancy situation of server m, ser m =(ser m1 , ser m1 ,..., ser mT ). P’ is the predicted electricity price for each time step, P′=(p′1, p′2,..., p′ T ). After the current task scheduling is completed, the state transitions from s n to s n+1 , the task information job n is transferred to the next task information job n+1 , and the resource occupancy situation Ser of the server will change accordingly according to the execution situation of task n; The action set is a n =(et n , se n ), Among them, et n is the execution time and se n is the execution server, and et n > ta n ; The discount factor set is γ ∈ [0, 1].

Images