Data center task scheduling method and system in uncertain environment

By using Markov decision-making process and quantile regression reinforcement learning algorithm in the data center, the task scheduling process is modeled, and the energy waste problem of task scheduling in uncertain environments is solved, achieving higher profits and task completion rates.

CN118964032AActive Publication Date: 2024-11-15NORTH CHINA ELECTRIC POWER UNIV +2

Patent Information

Application Number
CN202411109939.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-15
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The prior art cannot effectively consider task arrival uncertainty and electricity price uncertainty in uncertain environments, resulting in the problem of energy waste in data centers.

Method used

The Markov decision-making process model is used to combine quantile regression reinforcement learning algorithm to model the task scheduling process, and calculate the optimal distribution strategy of task scheduling through quantile regression timing difference learning algorithm, maximize long-term cumulative discount rewards, and optimize task scheduling decision making.

Benefits of technology

It significantly improves the profit and task completion rate of data centers, effectively reduces energy costs, and improves adaptability to uncertain environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964032A_ABST
    Figure CN118964032A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of data center energy management, in particular to a data center task scheduling method and system in an uncertain environment. The method comprises the steps of modeling a task scheduling process of a data center based on a Markov decision process to obtain a task scheduling model, and selecting task scheduling actions by using task information and electricity price as uncertain parameters and adopting a quantile regression reinforcement learning algorithm in the task scheduling model to obtain a task scheduling result. Calculating a reward after the data center executes the task, and storing a state after the task is executed, a task scheduling action, the reward and a next state in a playback buffer area, thereby obtaining different task scheduling distribution strategy data, and determining a task scheduling optimal distribution strategy of the task scheduling model by long-term accumulation discount reward maximization; through the method, an uncertainty-perceived task scheduling strategy can be realized, and the profit and the completion rate of a data center can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data center energy management, and in particular to a data center task scheduling method and system under an uncertain environment. Background Art

[0002] With the rapid development of cloud computing, big data analysis and machine learning, the power consumption of data centers has shown a sharp growth trend, which has attracted widespread attention. The main component of data center energy consumption is the energy consumption of computing tasks submitted by users, which has accounted for a considerable proportion of the total energy consumption. In this context, we note the delay tolerance of data center tasks. Delay tolerance means that some tasks with high delay tolerance can be postponed to a certain period in the future without violating the task deadline. This unique operating property provides flexibility in the time dimension for task execution. Specifically, by scheduling tasks to execute during periods with lower electricity prices, data centers can flexibly respond to grid electricity price signals, thereby reducing the energy consumption cost of task computing. However, in this process, data centers cannot know the arrival of future tasks and electricity prices in advance. In this uncertain environment, task scheduling decisions in data centers become very difficult.

[0003] In order to solve this problem, CN112949195A proposed a multi-objective task scheduling method for power cloud data centers to unify and coordinate the efficiency and energy consumption of task scheduling in power cloud data centers. However, this method does not consider the risks that task heterogeneity and arrival uncertainty may bring to task scheduling. On the other hand, CN117076100A proposed a hybrid data center task scheduling mathematical model based on queuing theory, which analyzes the delay and energy consumption of each node to more effectively distribute the load, thereby reducing energy consumption and delay. However, this method only optimizes the energy consumption and delay of the server, and does not consider the impact of electricity price uncertainty on the energy consumption cost of the data center during task scheduling. Summary of the invention

[0004] In view of the above-mentioned deficiencies of the prior art, the present invention aims to provide a data center task scheduling method and system in an uncertain environment, so as to solve the problem of data center energy waste caused by the existing data center task scheduling not taking into account the uncertain task arrival and the uncertainty of electricity prices.

[0005] In order to solve the above problems, the present invention adopts the following technical solutions:

[0006] An energy-aware data center task scheduling method in an uncertain environment includes:

[0007] The task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model. In the task scheduling model, task information and electricity prices are used as uncertain parameters.

[0008] The quantile regression reinforcement learning algorithm is used to select task scheduling actions, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward and next state after executing the task in the playback buffer, so as to obtain data of different task scheduling distribution strategies, maximize the long-term cumulative discounted reward, and determine the optimal task scheduling distribution strategy of the task scheduling model.

[0009] Furthermore, according to the tasks received by the data center, the task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model, including:

[0010] The task scheduling model includes an MDP framework, which includes a state set, an action set, a reward function set and a discount factor set;

[0011] The state set is .

[0012] in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n.

[0013] The action set is .

[0014] in, For execution time and is the execution server, and .

[0015] The reward function set is: .

[0016] in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively, is the penalty for task n.

[0017] The set of discount factors is .

[0018] Furthermore, the long-term cumulative discount reward .

[0019] Furthermore, the task scheduling optimal distribution strategy of the task scheduling model is determined by maximizing the long-term cumulative discounted reward, including: the state-action distribution corresponding to each task scheduling distribution strategy is:

[0020] .

[0021] in, Status Take action The value of is the expected value.

[0022] That is, the optimal distribution strategy satisfies:

[0023] .

[0024] Furthermore, distributed reinforcement learning is used to calculate the state-action value distribution corresponding to each task scheduling distribution strategy , and its expected value is :

[0025] .

[0026] Furthermore, the quantile regression time series difference learning algorithm is used to learn the corresponding The quantile of the quantiles is used to characterize the state-action value distribution :

[0027] .

[0028] in is the state-action value distribution The inverse function of the cumulative distribution function is Represents quantile and the midpoint of , state-action value distribution Update according to the following formula:

[0029] .

[0030] in is the indicator function.

[0031] Furthermore, the loss function of the quantile regression time series difference learning algorithm is:

[0032] .

[0033] Where k is a constant, is the quantile Huber loss threshold, is the TD error.

[0034] Furthermore, the process of using the quantile regression reinforcement learning algorithm to determine the optimal distribution strategy for task scheduling of the task scheduling model includes:

[0035] S201, initialize the online network, target network, environment, and number of quantile samples , number of experiences , batch size , target network update frequency F.

[0036] S202: Obtaining status based on tasks received by the data center , and used as the input of the neural network.

[0037] S203, select a scheduling action and execute.

[0038] S204, Status Transfer to .

[0039] S205: Calculate the reward for the data center after executing the task .

[0040] S206: Store the state, scheduled action, reward and next state after executing the task in the playback buffer.

[0041] S207, when the amount of data stored in the playback buffer is greater than Then, execute S208-S209, otherwise return to S202-S206.

[0042] S208, sample from the playback buffer, calculate the loss value of the sample and update the network.

[0043] S209: When the number of iterations is greater than the target network update frequency F, the target network is updated to obtain the optimal distribution strategy for task scheduling of the task scheduling model.

[0044] An energy-aware data center task scheduling system in an uncertain environment includes a task scheduling model modeling module and a task scheduling optimal distribution strategy determination module.

[0045] The task scheduling model modeling module is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity prices are used as uncertain parameters.

[0046] The task scheduling optimal distribution strategy determination module is used to adopt the quantile regression reinforcement learning algorithm, select the task scheduling action, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward and next state after executing the task in the playback buffer, so as to obtain different task scheduling distribution strategy data, maximize the long-term cumulative discounted reward, and determine the task scheduling optimal distribution strategy of the task scheduling model.

[0047] Furthermore, the task scheduling model includes an MDP framework, which includes a state set, an action set, a reward function set and a discount factor set.

[0048] The state set is .

[0049] in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n.

[0050] The action set is .

[0051] in, For execution time and is the execution server, and .

[0052] The reward function set is: .

[0053] in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively, is the penalty for task n.

[0054] The set of discount factors is .

[0055] The beneficial effects of the present invention are as follows: the present invention models the energy-aware data center task scheduling process as a Markov decision process. In the proposed model, the present invention incorporates the uncertainty of task arrival and electricity price into the proposed model, and develops a distributed reinforcement learning scheme based on the QR-DQN method to solve the proposed model. The quantile regression reinforcement learning algorithm can explicitly capture the probability characteristics of the state-action value, thereby realizing the uncertainty-aware task scheduling strategy; through the method of the present invention, the profit and completion rate of the data center can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a method for scheduling energy-aware data center tasks in an uncertain environment in Example 1 of the present invention.

[0057] Figure 2 This is a schematic diagram of an energy-aware data center task scheduling system in an uncertain environment in Example 1 of the present invention.

[0058] Figure 3 This is a profit comparison chart of a data center under different scheduling strategies in Example 2 of the present invention.

[0059] Figure 4 This is a comparison chart of task completion rates of a data center under different scheduling strategies in Example 2 of the present invention.

[0060] Figure 5 This is a diagram showing resource utilization of the FIFO scheduling strategy under electricity price fluctuations in Example 2 of the present invention.

[0061] Figure 6 This is a diagram showing resource utilization of the scheduling strategy of the present invention under electricity price fluctuations in Example 2 of the present invention.

[0062] Figure 7 This is a diagram showing resource utilization of the FIFO scheduling strategy under electricity price fluctuations in Example 2 of the present invention.

[0063] Figure 8 This is a diagram showing resource utilization of the scheduling strategy of the present invention under electricity price fluctuations in Example 2 of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below in conjunction with specific embodiments.

[0065] It should be noted that these embodiments are only used to illustrate the present invention rather than to limit the present invention. Simple improvements to the method based on the concept of the present invention all fall within the scope of protection claimed by the present invention.

[0066] Considering the uncertainty of task arrival and electricity price, the task scheduling strategy needs to be adjusted in real time to achieve the minimum energy cost. Specifically, when the task arrival rate is low, the data center can schedule some of the tasks that arrived in the past to this point, avoiding the inability to execute tasks due to resource limitations when the task rate was high in the past. When the price is high, the data center can schedule tasks to be executed in the future when the electricity price is lower, reducing energy consumption costs. Therefore, the task scheduling model needs to further consider the uncertainty of task arrival and the fluctuation of electricity prices.

[0067] More specifically, when a task arrives at the data center, the data center also obtains various information about the task, including arrival time, resource utilization, execution time, deadline, and computing revenue. Based on this information, the data center assigns the task to the server and executes the task at the specified time. Combining the energy consumption of task execution with the corresponding electricity price, the data center can calculate the profit generated by task execution. After completing all tasks, the data center obtains the total revenue.

[0068] According to the task information, server occupancy and predicted electricity price obtained by the data center, the task scheduling action is selected. After the data center executes the scheduling action, the income is fed back to the agent as an immediate reward after the task is executed. The agent stores the current state, action, reward and next state in the replay buffer. Subsequently, the agent uses the stored data to iteratively update the neural network data to optimize the scheduling strategy. In the above process, since users can submit computing requests at any time, the arrival of the task is regarded as an uncertain parameter. In addition to the arrival time, other future task information is also uncertain during the execution of the current task. At the same time, considering the widely adopted ex post real-time pricing, the electricity price is also regarded as an uncertain parameter.

[0069] Example 1

[0070] See also Figure 1 , is an energy-aware data center task scheduling method in an uncertain environment, including:

[0071] S100. According to the tasks received by the data center, the task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity prices are used as uncertain parameters.

[0072] As an implementable method, the task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model, including:

[0073] The task scheduling model includes an MDP framework, including a state set S, an action set A, a reward function set R and a discount factor set .

[0074] The state set is .

[0075] in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n.

[0076] Action set is .

[0077] in, For execution time and For the execution server, considering the actual situation, the execution time must be after the task arrival time, that is, .

[0078] action is a discrete vector, and the execution time and execution server are all discrete variables.

[0079] Server selection should not lead to violation of its resource limits at any time step. Rewards are direct rewards generated.

[0080] That is, the reward function set is: .

[0081] in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively. If task n is completed after the deadline, that is, , is the penalty for task n.

[0082] The discount factor set is .

[0083] The ultimate goal is to make the total revenue of all tasks, that is, the long-term cumulative discounted reward maximum.

[0084] S200, using the quantile regression reinforcement learning algorithm, select the task scheduling action, calculate the reward after the data center executes the task, store the state after executing the task, the task scheduling action, the reward and the next state in the playback buffer, so as to obtain different task scheduling distribution strategy data, maximize the long-term cumulative discounted reward, and determine the optimal task scheduling distribution strategy of the task scheduling model.

[0085] As an implementable method, the task scheduling optimal distribution strategy of the task scheduling model is determined by maximizing the long-term cumulative discounted reward, including: the state-action distribution corresponding to each task scheduling distribution strategy is:

[0086] .

[0087] in, Status Take action The value of is the expected value.

[0088] That is, the optimal distribution strategy satisfies:

[0089] .

[0090] is the policy function, that is, in each state The following corresponding measures , To select the state Give in The action with the largest value The strategy function.

[0091] As an implementation method, the present invention uses distributed reinforcement learning to calculate the state-action value distribution corresponding to each task scheduling distribution strategy , and its expected value is :

[0092] .

[0093] The distribution of Z in the distributed Bellman equation is determined by the interaction of three random variables: reward R, next state-action , and the random return distribution of its state-action value distribution .

[0094] As an implementation method, the present invention adopts a quantile regression time series difference learning algorithm to learn the corresponding The quantile of the quantiles is used to characterize the state-action value distribution, then :

[0095] .

[0096] in is the state-action value distribution The inverse function of the cumulative distribution function is Represents quantile and the midpoint of , state-action value distribution Update according to the following formula:

[0097] .

[0098] in is the indicator function.

[0099] The quantile regression time series difference learning algorithm characterizes the distribution of values ​​by learning the quantiles corresponding to Nq quantiles, so as to more accurately capture the diverse performance of state-action pairs in different environments.

[0100] As an implementable embodiment, the present invention uses an extended version of the Huber loss Lk(u) as the quantile Huber loss, which is used as the loss function.

[0101] The loss function of the quantile regression temporal difference learning algorithm is:

[0102] .

[0103] Where k is a constant, is the quantile Huber loss threshold, is the TD error.

[0104] As an implementable method, the process of determining the optimal distribution strategy for task scheduling of the task scheduling model using the quantile regression reinforcement learning algorithm includes:

[0105] S201, initialize the online network, target network, environment, and number of quantile samples , number of experiences , batch size , target network update frequency F.

[0106] S202, from the environment (i.e., based on the tasks received from the data center, task information, server resource occupancy rate, and predicted electricity price generation status can be obtained ) Get status , and used as the input of the neural network.

[0107] S203, select a scheduling action and execute.

[0108] S204, Status Transfer to .

[0109] S205: Calculate the reward for the data center after executing the task .

[0110] S206: Store the state, scheduled action, reward and next state after executing the task in the playback buffer.

[0111] S207, when the amount of data stored in the playback buffer is greater than Then, execute S208-S209, otherwise return to S202-S206.

[0112] S208, sample from the playback buffer, calculate the loss value of the sample and update the network.

[0113] S209: When the number of iterations is greater than the target network update frequency F, the target network is updated to obtain the optimal distribution strategy for task scheduling of the task scheduling model.

[0114] S300: Schedule the tasks received by the data center using the optimal distribution strategy for task scheduling.

[0115] The present invention takes into account the uncertainty of tasks and the fluctuation of electricity prices in the task scheduling process, aiming to minimize energy consumption costs. The present invention focuses on the time, resources, and revenue information of tasks, and the execution time of tasks and the corresponding electricity prices are coupled with the energy consumption costs, and the impact of this coupling will change with the uncertainty of task information and the fluctuation of electricity prices. Therefore, considering energy consumption costs is more difficult than considering only energy consumption and completion time. To this end, the present invention adopts the QR-DQN algorithm, whose output is not a single value, but a distribution of values. This distribution property enables the training process to better deal with the uncertainty of tasks and electricity prices, reduce variance, and improve stability. By modeling the distribution, the system can respond to environmental changes more flexibly, providing a more reliable basis for minimizing energy consumption costs.

[0116] See also Figure 2 , which is an energy-aware data center task scheduling system in an uncertain environment, including a task scheduling model modeling module 100 and a task scheduling optimal distribution strategy determination module 200.

[0117] The task scheduling model modeling module 100 is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity prices are used as uncertain parameters.

[0118] The task scheduling optimal distribution strategy determination module 200 is used to adopt the quantile regression reinforcement learning algorithm, select the task scheduling action, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward and next state after executing the task in the playback buffer, so as to obtain different task scheduling distribution strategy data, maximize the long-term cumulative discounted reward, and determine the task scheduling optimal distribution strategy of the task scheduling model.

[0119] As an implementable method, the task scheduling model includes an MDP framework, and the MDP framework includes a state set, an action set, a reward function set, and a discount factor set.

[0120] The state set is .

[0121] in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n.

[0122] Action set is .

[0123] in, For execution time and is the execution server, and .

[0124] The reward function set is: .

[0125] in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively, is the penalty for task n.

[0126] The discount factor set is .

[0127] Example 2

[0128] The present invention compares the profit and completion rate of the data center under the existing first-in-first-out (FIFO) strategy. Figure 3 It can be seen that when the number of tasks processed by the data center increases from 30 to 70, the strategy of the present invention is better than the comparison strategy in optimizing profits. Figure 4 It can be seen that when the task volume gradually increases to 170, the task completion rate of the comparison strategy begins to decline, while the task completion rate under the strategy of the present invention remains at 100%.

[0129] The present invention compares resource utilization rates of different dispatching strategies under different electricity price scenarios. Figure 5 , Figure 6 It can be seen that the strategy of the present invention can effectively schedule tasks based on electricity price fluctuations. During periods of high electricity prices, some tasks are transferred to time steps with lower prices, reducing energy costs while enhancing the flexibility of data center energy consumption. In contrast, the FIFO strategy cannot adjust energy consumption according to electricity prices, resulting in reduced profits. In addition, Figure 7 , Figure 8In the figure, when the number of tasks increases from 60 to 70, the strategy of the present invention can still effectively perceive price fluctuations, indicating that the strategy of the present invention also has advantages when the arrival of tasks is uncertain.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. An energy-aware data center task scheduling method under uncertain environment, It is characterized in that include: The task scheduling process of the data center is modeled based on the Markov decision process to obtain the task scheduling model; In the task scheduling model, task information and electricity prices are used as uncertain parameters; The quantile regression reinforcement learning algorithm is used to select task scheduling actions, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward and next state after executing the task in the playback buffer, so as to obtain data of different task scheduling distribution strategies, maximize the long-term cumulative discounted reward, and determine the optimal task scheduling distribution strategy of the task scheduling model.

2. The energy-aware data center task scheduling method under uncertain environment according to claim 1, It is characterized in that According to the tasks received by the data center, the task scheduling process of the data center is modeled based on the Markov decision process to obtain a task scheduling model, including: The task scheduling model includes an MDP framework, which includes a state set, an action set, a reward function set and a discount factor set; The state set is , in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n; The action set is , in, For execution time and is the execution server, and ; The reward function set is: , in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively, is the penalty for task n; The discount factor set is .

3. The energy-aware data center task scheduling method under uncertain environment according to claim 2, It is characterized in that The long-term cumulative discount reward .

4. The energy-aware data center task scheduling method under uncertain environment according to claim 3, It is characterized in that The method of maximizing the long-term cumulative discounted reward to determine the optimal distribution strategy for task scheduling of the task scheduling model includes: the state-action distribution corresponding to each task scheduling distribution strategy is: in, Status Take action The value of is the expected value; That is, the optimal distribution strategy satisfies: 。 5. The energy-aware data center task scheduling method under uncertain environment according to claim 4, It is characterized in that Distributed reinforcement learning is used to calculate the state-action value distribution corresponding to each task scheduling distribution strategy , and its expected value is : 。 6. The energy-aware data center task scheduling method in an uncertain environment according to claim 5, It is characterized in that The quantile regression time series difference learning algorithm is used to learn the corresponding The quantile of the quantiles is used to characterize the state-action value distribution : in is the state-action value The inverse function of the cumulative distribution function is Represents quantile and the midpoint of , state-action value distribution Update according to the following formula: in is the indicator function.

7. The energy-aware data center task scheduling method in an uncertain environment according to claim 6, It is characterized in that The loss function of the quantile regression time series difference learning algorithm is: Where k is a constant, is the quantile Huber loss threshold, is the TD error.

8. The energy-aware data center task scheduling method under uncertain environment according to claim 7, It is characterized in that The process of using the quantile regression reinforcement learning algorithm to determine the optimal distribution strategy for task scheduling of the task scheduling model includes: S201, initialize the online network, target network, environment, and number of quantile samples , number of experiences, batch size , target network update frequency F; S202: Obtaining status based on tasks received by the data center , and used as the input of the neural network; S203, select a scheduling action and execute; S204, Status Transfer to ; S205: Calculate the reward for the data center after executing the task ; S206, storing the state, scheduling action, reward and next state after executing the task in the playback buffer; S207, when the amount of data stored in the playback buffer is greater than Then, execute S208-S209, otherwise return to S202-S206; S208, sampling from the playback buffer, calculating the loss value of the sample and updating the network; S209, until the number of iterations is greater than the target network update frequency F, update the target network to obtain the task scheduling optimal distribution strategy of the task scheduling model.

9. An energy-aware data center task scheduling system in uncertain environments, It is characterized in that It includes a task scheduling model building module 100 and a task scheduling optimal distribution strategy determination module 200; The task scheduling model modeling module 100 is used to model the task scheduling process of the data center based on the Markov decision process to obtain a task scheduling model; in the task scheduling model, task information and electricity prices are used as uncertain parameters; The task scheduling optimal distribution strategy determination module 200 is used to adopt the quantile regression reinforcement learning algorithm, select the task scheduling action, calculate the reward after the data center executes the task, and store the state, task scheduling action, reward and next state after executing the task in the playback buffer, so as to obtain different task scheduling distribution strategy data, maximize the long-term cumulative discounted reward, and determine the task scheduling optimal distribution strategy of the task scheduling model.

10. The energy-aware data center task scheduling system in an uncertain environment according to claim 9, It is characterized in that The task scheduling model includes an MDP framework, which includes a state set, an action set, a reward function set and a discount factor set; The state set is , in, is the information of task n, , N is the total number of tasks processed by the data center with M servers in T time steps, They are the resource occupancy rate, arrival time, execution time, deadline and calculation benefit of the task respectively. is the resource usage of M servers, , is the resource usage of server m, , For each time step, the predicted electricity price is After the current task is scheduled, the status changes from Transition to , task information Transferred to next task information , server resource usage It will change accordingly according to the execution status of task n; The action set is , in, For execution time and is the execution server, and ; The reward function set is: , in, is the energy cost of task n, , is the IT equipment energy consumption per time step of task n, , and are the power of server m in idle state and full load state respectively, is the penalty for task n; The set of discount factors is .

Citation Information

Patent Citations

  • Intelligent computing network scheduling method for computing and communication fusion of large model task

    CN117667360A

  • Time-varying task scheduling method and system based on constraint near-end strategy optimization

    CN117851056A

  • Graph neural network and reinforcement learning techniques for connection management

    US20220124543A1

  • Systems and methods for joint inter- and intra-slice orchestration using reinforcement learning

    WO2024151189A1

Cited By

  • Computing task scheduling method and system considering uncertainty risk of data center

    CN120448067A

  • Task scheduling method and system considering data center uncertainty risk

    CN120448067B

  • Adaptive power capping method and system considering uncertainty of cloud data center

    CN120494873A

  • An Adaptive Power Capping Method and System Considering Uncertainties in Cloud Data Centers

    CN120494873B

  • Operating system self-adaptive task scheduling method and device and computer equipment

    CN120704845A