A data center task scheduling decision method and system for power grid frequency modulation

Abstract

The present invention discloses a data center task scheduling decision method and system for power grid frequency modulation, which decides the execution time of data center workload and server resource allocation at the task scheduling level, optimizes the energy cost of the data center and accurately tracks the power grid frequency modulation signal under the premise of ensuring service quality, and proposes a data center task scheduling method, which not only considers the delay of workload on the time scale, but also considers the allocation on the server resource scale, and transforms the traditional task scheduling problem into a resource allocation problem for solution, models the data center task scheduling decision process, including the setting of state space, action space, and reward function; and ensures the minimization of data center energy consumption and accurate response to frequency modulation signals under the premise of ensuring service quality.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a data center task scheduling decision method and system for power grid frequency modulation. Background Art

[0002] In the era of digital economy, digital technologies such as cloud computing, big data, and artificial intelligence are developing rapidly. In particular, the advent of ChatGPT in 2022 has significantly driven the growth of demand for intelligent computing power, and computing power is gradually becoming a new productivity. Data centers are the physical carrier of computing power and are key infrastructure for digital development. With the continuous expansion of the number and scale of data centers around the world, the massive computing power demand will inevitably cause a sharp increase in data center electricity consumption. The massive computing power demand and the time-shifting characteristics of data center workloads give the center huge adjustable load potential, and data centers have become an emerging adjustable load resource on the demand side of the power grid. On the other hand, in the context of a high proportion of renewable energy power systems, the problem of power grid frequency regulation has become an important aspect that threatens the safe and stable operation of the power system. Simply relying on the secondary frequency regulation of traditional generators can no longer meet the requirements of power system frequency stability, and there is an urgent need to improve the regulation capacity of the power system. Exploring the adjustable load resources on the demand side of the power grid is one of the important ways to improve the system regulation capacity. In fact, in recent years, data centers have become a highly anticipated adjustable load resource that provides auxiliary services for power system frequency regulation by virtue of their dynamic regulation capabilities of power loads. By using data center task scheduling to transfer workloads on a time scale, the data center can achieve precise control of the power load, thereby enabling the data center to accurately respond to the power grid frequency modulation signals, helping to reduce the energy costs of the data center and obtain frequency modulation economic compensation by providing power frequency modulation auxiliary services.

[0003] CN112949195A proposes a real-time task scheduling method for a data center based on market electricity prices, which can solve the problem of high energy consumption costs in data centers, but does not consider the optimization of energy consumption costs in data centers from the perspective of providing frequency modulation auxiliary services. A real-time scheduling technology for data centers based on server operating condition management is proposed, which enables the data center to provide frequency regulation backup services, but only considers the impact of server active, closed and other operating states on the operating load of the data center at the server level, and does not propose a scheduling strategy at the task scheduling level.

[0004] At present, the main method to realize data center frequency modulation signal tracking is to realize data center power load regulation under the guidance of frequency modulation signal at the server level:

[0005] The first is to use dynamic voltage and frequency adjustment technology to adjust the server CPU operating frequency and operating voltage;

[0006] The second is to use server operation status management technology to adjust the number of active servers and idle servers.

[0007] There are great limitations in adjusting the server operating conditions. On the one hand, frequent adjustments to the server operating conditions will affect the performance of the server running applications. DVFS technology reduces data center power consumption at the cost of extending task execution time. Increased load execution time may cause the load to not be completed before the deadline, reducing the service quality of the data center. On the other hand, the arrival of data center task scheduling is random. In the power grid frequency modulation scenario, the data center needs to adjust the power load of the data center within a time scale of seconds. Compared with power control at the server level, the precise response to the frequency modulation signal can be achieved more quickly and accurately through data center task scheduling. Summary of the invention

[0008] In view of the above-mentioned deficiencies in the prior art, the present application provides a data center task scheduling decision method and system for power grid frequency modulation.

[0009] In a first aspect, the present application proposes a data center task scheduling decision method for power grid frequency modulation, comprising the following steps:

[0010] The data center task scheduling for grid frequency regulation is modeled as a Markov decision model;

[0011] Constructing a data center task scheduling algorithm based on policy gradient, and using the constructed algorithm to solve the Markov decision model to obtain the optimal task scheduling strategy;

[0012] The real data center workload arrival trajectory is sampled, and the data center task scheduler is guided to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

[0013] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes configuring an intelligent agent;

[0014] The configuration agent includes: setting the data center task scheduler as an agent for task scheduling strategy learning, obtaining state information of the power market environment at each moment through the agent, and obtaining the environment state , and at the same time, the agent observes the internal workload and server status information of the data center, and the agent obtains the final state based on the environmental state, the internal workload and server status information of the data center .

[0015] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes defining a state space;

[0016] The state space definition includes: defining the server resource utilization rate at time step t as , the workload queue status is The electricity price is , the FM signal is , the CPU resources required for each time step of the workload are , the execution time is , the CPU resources that have been allocated are , the execution time is .

[0017] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes defining an action space;

[0018] The action space is defined as follows:

[0019]

[0020] in, It represents the upper limit of CPU resources allocated by the task scheduler to the workload. At each time step, the server resources allocated by the task scheduler to the workload do not exceed the total resources provided by the server. It is expressed as:

[0021]

[0022] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes setting a reward function;

[0023] Setting the reward function includes setting the electricity fee reward, setting the deviation reward, and setting the workload reward:

[0024] Set electricity fee reward: The electricity fee spent by the data center at time step t is used to represent the energy consumption cost of the data center, and negative electricity fee expenditure is used as the electricity fee reward ;

[0025] Setting Deviation Rewards: Introducing a FM Deviation Price and Combining Power Deviations at Each Time Step Get the FM signal tracking deviation cost, and use the negative cost as the deviation reward ;

[0026] Set workload reward: All workloads in the waiting state in the data center at time step t will incur workload cost, and the negative value of the sum of all workload costs is used as the workload reward .

[0027] In some embodiments, the step of constructing a data center task scheduling algorithm based on a policy gradient includes:

[0028] The objective function of the construction strategy is:

[0029]

[0030] Constructing the policy gradient is:

[0031]

[0032] The formula for updating the parameters of the constructed strategy network is:

[0033]

[0034] The error function for constructing the state value estimate is:

[0035]

[0036] The update formula of the build state value parameter is:

[0037]

[0038] in, represents the state value function of the task scheduler at time t, represents the action-state value function of the task scheduler at time t, represents the policy network parameters, Indicates the state value parameter represents an unbiased estimate of the action-state value function, represents the update step size of the policy network parameters, represents the error function of the state value estimate, Indicates the update step size of the state value parameter.

[0039] In some embodiments, solving the Markov decision model using the constructed algorithm to obtain the optimal task scheduling strategy includes:

[0040] Step 1: Initialize the strategy , state value estimation , number of iterations N, number of rounds E and update step size ;

[0041] Step 2: Sample the workload arrival trajectory, and decide on resource allocation actions in turn based on the workload trajectory obtained by sampling ;

[0042] Step 3: After executing the action, obtain the electricity reward, deviation reward and workload reward of the environment feedback;

[0043] Step 4: After the task scheduler completes the scheduling of a workload trajectory, it indicates the end of a round and updates the policy network parameters and state value parameters;

[0044] Step 5: When the number of iterations is less than N, return to step 2 and start iterating again until the iteration is completed and the training result is the optimal strategy parameter. , thus obtaining the optimal task scheduling strategy.

[0045] In the second aspect, the present application proposes a data center task scheduling decision system for power grid frequency regulation, including a model building module, a strategy acquisition module and a scheduling execution module;

[0046] The model building module is used to model the data center task scheduling for power grid frequency regulation as a Markov decision model;

[0047] The strategy acquisition module is used to construct a data center task scheduling algorithm based on policy gradient, and solve the Markov decision model using the constructed algorithm to obtain the optimal task scheduling strategy;

[0048] The scheduling execution module is used to sample the actual data center workload arrival trajectory and guide the data center task scheduler to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

[0049] In a third aspect, the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0050] In a fourth aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0051] Beneficial effects of the present invention:

[0052] The present invention optimizes the energy cost of the data center and accurately tracks the power grid frequency modulation signal under the premise of ensuring service quality, and models the task scheduling decision process of the data center, including the setting of state space, action space, and reward function; a data center task scheduling method is proposed, which not only considers the delay of workload on the time scale, but also considers the allocation on the server resource scale, and converts the traditional task scheduling problem into a resource allocation problem for solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is the overall flow chart of the present invention.

[0054] Figure 2 It is a system principle block diagram of the present invention. DETAILED DESCRIPTION

[0055] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein; on the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0056] Workload can be represented by CPU resource demand and duration. A rectangle can be used to represent the workload of a data center. The length of the rectangle represents the CPU resources required by the workload at each moment, the width of the rectangle represents the duration of the workload, and the area of ​​the rectangle represents the total amount of CPU resources required during the execution of the workload. To change the processing of workloads in data center servers through task scheduling, one way is to postpone the workload in the time dimension without changing the shape of the workload; the second way is to change the shape of the workload without delaying the start execution time of the workload, that is, to change the number of CPU resources allocated to the workload by the data center at each moment. Considering the impact of the number of resources occupied by the workload on the processing speed, by reducing or increasing the CPU resources allocated to the workload at that moment, the processing speed of the workload will be slowed down or accelerated, thereby increasing or reducing the completion time of the workload; the third way is a combination of the first and second ways, and at the same time changing the start execution time of the workload and the number of CPU resources allocated to it at each moment. In the above three cases, it is necessary to ensure that the CPU resources allocated to the workload by the data center during the execution of the workload are not less than the total amount of CPU resources required by the workload.

[0057] The implementation methods of data center task scheduling proposed in this solution include the above three methods. Task scheduling needs to simultaneously decide the execution time of the workload and the amount of CPU resources allocated to it during the execution of the workload. It can change the load situation of the data center service and realize precise regulation of the data center load.

[0058] Specifically, after users submit requests on the cloud service platform, they become workloads of the data center. Workloads are queued in the workload queue according to the order of arrival and wait for execution. The data center makes scheduling decisions for the workloads in the queue based on the operating power, electricity price information, and frequency modulation signals at each time step.

[0059] Among them, the data center task scheduling process needs to decide whether the workload starts to execute at each time step, and allocate server computing resources (CPU resources) to the workload that has been executed or decided to be executed.

[0060] Based on this, in the first aspect, this application proposes a data center task scheduling decision method for power grid frequency modulation, such as Figure 1 As shown, the following steps are included:

[0061] S100: Modeling data center task scheduling for grid frequency regulation as a Markov decision model;

[0062] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes configuring an intelligent agent;

[0063] The configuration agent includes: setting the data center task scheduler as an agent for task scheduling strategy learning, obtaining state information of the power market environment at each moment through the agent, and obtaining the environment state , and at the same time, the agent observes the internal workload and server status information of the data center, and the agent obtains the final state based on the environmental state, the internal workload and server status information of the data center .

[0064] Among them, the intelligent agent plays the role of action strategy learner in the Markov decision process, and the data center task scheduler is set as the intelligent agent for task scheduling strategy learning, which is responsible for interacting with the environment.

[0065] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes defining a state space;

[0066] The state space definition includes: defining the server resource utilization rate at time step t as , the workload queue status is The electricity price is , the FM signal is , the CPU resources required for each time step of the workload are , the execution time is , the CPU resources that have been allocated are , the execution time is .

[0067] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes defining an action space;

[0068] The action space is defined as follows:

[0069]

[0070] in, It represents the upper limit of CPU resources allocated by the task scheduler to the workload. At each time step, the server resources allocated by the task scheduler to the workload do not exceed the total resources provided by the server. It is expressed as:

[0071]

[0072] In some embodiments, the modeling of data center task scheduling for power grid frequency regulation as a Markov decision model includes setting a reward function;

[0073] Setting the reward function includes setting the electricity fee reward, setting the deviation reward, and setting the workload reward:

[0074] Set electricity fee reward: The electricity fee spent by the data center at time step t is used to represent the energy consumption cost of the data center, and negative electricity fee expenditure is used as the electricity fee reward ;

[0075] Setting Deviation Rewards: Introducing a FM Deviation Price and Combining Power Deviations at Each Time Step Get the FM signal tracking deviation cost, and use the negative cost as the deviation reward ;

[0076] Set workload reward: All workloads in the waiting state in the data center at time step t will incur workload cost, and the negative value of the sum of all workload costs is used as the workload reward ;

[0077] Among them, the data center is used The electricity bill at any time is used to represent the energy cost of the data center, and negative electricity bill expenditure is used as the reward function The first item is the electricity fee reward Here, consider Get electricity fee rewards based on the electricity price and electricity consumption at the moment The definition is as follows:

[0078]

[0079] To measure the data center's ability to respond to FM signals, the power deviation at each moment is used. To measure, and introduce a frequency modulation deviation price to transform it into a frequency modulation signal tracking deviation cost, with negative cost as the reward function The second item, deviation reward .

[0080]

[0081] Before providing services to customers, data centers will sign a service level agreement to stipulate the service type, service quality, etc. Workload QoS is one of the important contents. However, the task scheduler's goal of minimizing energy costs and accurately tracking frequency modulation signals is often contrary to ensuring service quality. For example, a small frequency modulation signal will limit the operating power of the data center, which may cause the workload to be delayed and affect the service quality of the data center. The difference between the workload completion time and the submission time or the sum of the task execution waiting time and the actual execution time is used to calculate the delayed execution time of the workload, and the delay time is calculated by the task scheduler. To evaluate the workload Delay. By introducing the QoS penalty rate, the workload task delay time is converted into the workload delay execution cost, where the QoS penalty rate can be understood as the price of using a unit of server resources during the workload execution process. Therefore, all workloads in the waiting state in the data center at time t will incur QoS costs, and the negative value of the sum of all QoS costs is used as the reward function The third item is workload QoS reward , defined as follows:

[0082]

[0083] In the formula, Indicates the price of unit server CPU resource utilization; Representing Workload QoS evaluation indicators are called QoS violations; represents the workload waiting time; Represents the actual execution time of the workload. As can be seen from the above formula, the longer the waiting time and actual execution time of the workload, the greater the workload QoS cost it generates, and the greater the workload delay evaluation index. By setting the QoS reward item in the reward function, it is equivalent to setting a priority for the workload. When the QoS cost of the workload is large at a certain moment, the task scheduler will give priority to executing these workloads and allocate more CPU resources in order to avoid a large penalty. In this way, the task scheduler can learn a task scheduling strategy that can ensure the service quality of the workload.

[0084] The reward function is the sum of the three negative costs, and the formula is as follows:

[0085] .

[0086] Among them, we already know the reward function after definition The components of , defined as the instantaneous income of the data center at time step t, and represented by negative costs. The total income obtained within the range is the sum of the instant rewards at each time step in the entire task scheduling process, and the reward is express. is a discount factor, which indicates how much the agent values ​​the future.

[0087] S200: constructing a data center task scheduling algorithm based on policy gradient, and solving the Markov decision model using the constructed algorithm to obtain an optimal task scheduling strategy;

[0088] In some embodiments, the step of constructing a data center task scheduling algorithm based on a policy gradient includes:

[0089] The objective function of the construction strategy is:

[0090]

[0091] Constructing the policy gradient is:

[0092]

[0093] The formula for updating the parameters of the constructed strategy network is:

[0094]

[0095] The error function for constructing the state value estimate is:

[0096]

[0097] The update formula of the build state value parameter is:

[0098]

[0099] in, represents the state value function of the task scheduler at time t, represents the action-state value function of the task scheduler at time t, represents the policy network parameters, Indicates the state value parameter represents an unbiased estimate of the action-state value function, represents the update step size of the policy network parameters, represents the error function of the state value estimate, Indicates the update step size of the state value parameter.

[0100] After the algorithm is built, the deep reinforcement learning method based on policy gradient is used to solve it:

[0101] The policy network can output the probability distribution of possible actions under given states and parameters, and the sum of the probabilities of all possible actions is 1. When this policy is actually applied, the agent chooses the action corresponding to the maximum probability. A good policy can guide the agent to maximize the objective function. The policy objective function can characterize the quality of the policy network, and the policy network parameters directly determine the differences in the policy network. At this point, solving the optimal policy problem of the task scheduler is transformed into solving the parameter problem corresponding to the optimal policy.

[0102] The input of the policy network is the state variables in the state space, and the output is the resource allocation action of the task scheduling. For a given state and specific policy network parameters, the policy network can output the probability distribution of possible actions. When the policy is actually applied, the task scheduler selects the action corresponding to the maximum probability.

[0103] In some embodiments, solving the Markov decision model using the constructed algorithm to obtain the optimal task scheduling strategy includes:

[0104] Step 1: Initialize the strategy , state value estimation , number of iterations N, number of rounds E and update step size ;

[0105] Step 2: Sample the workload arrival trajectory, and decide on resource allocation actions in turn based on the workload trajectory obtained by sampling ;

[0106] Step 3: After executing the action, obtain the electricity reward, deviation reward and workload reward of the environment feedback;

[0107] Step 4: After the task scheduler completes the scheduling of a workload trajectory, it indicates the end of a round and updates the policy network parameters and state value parameters;

[0108] Step 5: When the number of iterations is less than N, return to step 2 and start iterating again until the iteration is completed and the training result is the optimal strategy parameter. , thus obtaining the optimal task scheduling strategy.

[0109] S300: Sample the actual arrival trajectory of data center workloads, and guide the data center task scheduler to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

[0110] Compared with best-tracking (data center frequency modulation algorithm that only tracks frequency modulation signals), the algorithm proposed in the present invention performs better in terms of workload delay and reducing the total cost of the data center. This is because the best-tracking method sacrifices more workload QoS and energy consumption utilization to obtain accurate tracking of frequency modulation signals in turn. In addition, compared with EnergyQARE (energy and service quality-aware data center frequency modulation algorithm), the algorithm proposed in the present invention has obvious improvements in minimizing total cost, tracking frequency modulation signals, and workload delay. This is because compared with data center energy management at the server level, the task scheduling method is more refined, and the penalty for frequency modulation signal deviation and workload delay is set in the model of the present invention. In order to achieve the goal, the task scheduler will learn an optimal strategy to avoid frequency modulation signal tracking deviation and workload delay. The algorithm proposed in the present invention converges in about 30 episodes, and compared with the other two methods, the total cost of the final optimization result is the smallest.

[0111] In the second aspect, the present application proposes a data center task scheduling decision system for power grid frequency modulation, such as Figure 2 As shown, it includes a model building module, a strategy acquisition module and a scheduling execution module;

[0112] The model building module is used to model the data center task scheduling for power grid frequency regulation as a Markov decision model;

[0113] The strategy acquisition module is used to construct a data center task scheduling algorithm based on policy gradient, and solve the Markov decision model using the constructed algorithm to obtain the optimal task scheduling strategy;

[0114] The scheduling execution module is used to sample the actual data center workload arrival trajectory and guide the data center task scheduler to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

[0115] In a third aspect, the present application proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0116] In a fourth aspect, the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0117] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A data center task scheduling decision method for power grid frequency modulation, characterized by: The following steps are involved: The data center task scheduling for grid frequency regulation is modeled as a Markov decision model, including configuring intelligent agents; The configuration agent includes: setting the data center task scheduler as an agent for task scheduling strategy learning, obtaining state information of the power market environment at each moment through the agent, and obtaining the environment state , and at the same time, the agent observes the internal workload and server status information of the data center, and the agent obtains the final state based on the environmental state, the internal workload and server status information of the data center ; It also includes setting up a reward function; Setting the reward function includes setting the electricity fee reward, setting the deviation reward, and setting the workload reward: Set electricity fee reward: The electricity fee spent by the data center at time step t is used to represent the energy consumption cost of the data center, and negative electricity fee expenditure is used as the electricity fee reward ; Setting Deviation Rewards: Introducing a FM Deviation Price and Combining Power Deviations at Each Time Step Get the FM signal tracking deviation cost, and use the negative cost as the deviation reward , defined as follows: Set workload reward: All workloads in the waiting state in the data center at time step t will incur workload cost, and the negative value of the sum of all workload costs is used as the workload reward , defined as follows: In the formula, the CPU resources required for each time step of the workload are , the execution time is , Indicates the price of unit server CPU resource utilization; Representing Workload QoS evaluation indicators are called QoS violations; represents the workload waiting time; Indicates the actual execution time of the workload; Build a data center task scheduling algorithm based on policy gradient, including: The objective function of the construction strategy is: Constructing the policy gradient is: The formula for updating the parameters of the constructed strategy network is: The error function for constructing the state value estimate is: The update formula of the build state value parameter is: in, represents the state value function of the task scheduler at time t, represents the action-state value function of the task scheduler at time t, represents the policy network parameters, Indicates the state value parameter represents an unbiased estimate of the action-state value function, represents the update step size of the policy network parameters, represents the error function of the state value estimate, The update step size of the state value parameter is represented, and the Markov decision model is solved by using the constructed algorithm to obtain the optimal task scheduling strategy; The real data center workload arrival trajectory is sampled, and the data center task scheduler is guided to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

2. The method according to claim 1, characterized in that: The data center task scheduling for power grid frequency regulation is modeled as a Markov decision model, including defining a state space; The state space definition includes: defining the server resource utilization rate at time step t as , the workload queue status is The electricity price is , the FM signal is , the CPU resources required for each time step of the workload are , the execution time is , the CPU resources that have been allocated are , the execution time is .

3. The method according to claim 2, characterized in that: The data center task scheduling for power grid frequency regulation is modeled as a Markov decision model, including defining an action space; The action space is defined as follows: in, It represents the upper limit of CPU resources allocated by the task scheduler to the workload. At each time step, the server resources allocated by the task scheduler to the workload do not exceed the total resources provided by the server. It is expressed as: 。 4. The method according to claim 3, characterized in that: The Markov decision model is solved by the constructed algorithm to obtain the optimal task scheduling strategy, including: Step 1: Initialize the strategy , state value estimation , number of iterations N, number of rounds E and update step size ; Step 2: Sample the workload arrival trajectory, and decide on resource allocation actions in turn based on the workload trajectory obtained by sampling ; Step 3: After executing the action, obtain the electricity reward, deviation reward and workload reward of the environment feedback; Step 4: After the task scheduler completes the scheduling of a workload trajectory, it indicates the end of a round and updates the policy network parameters and state value parameters; Step 5: When the number of iterations is less than N, return to step 2 and start iterating again until the iteration is completed and the training result is the optimal strategy parameter. , thus obtaining the optimal task scheduling strategy.

5. A data center task scheduling decision system for power grid frequency modulation, characterized by: It includes model building module, strategy acquisition module and scheduling execution module; The model building module is used to model the data center task scheduling for power grid frequency regulation as a Markov decision model, including configuring an intelligent agent; The configuration agent includes: setting the data center task scheduler as an agent for task scheduling strategy learning, obtaining state information of the power market environment at each moment through the agent, and obtaining the environment state , and at the same time, the agent observes the internal workload and server status information of the data center, and the agent obtains the final state based on the environmental state, the internal workload and server status information of the data center ; It also includes setting up a reward function; Setting the reward function includes setting the electricity fee reward, setting the deviation reward, and setting the workload reward: Set electricity fee reward: The electricity fee spent by the data center at time step t is used to represent the energy consumption cost of the data center, and negative electricity fee expenditure is used as the electricity fee reward ; Setting Deviation Rewards: Introducing a FM Deviation Price and Combining Power Deviations at Each Time Step Get the FM signal tracking deviation cost, and use the negative cost as the deviation reward , defined as follows: Set workload reward: All workloads in the waiting state in the data center at time step t will incur workload cost, and the negative value of the sum of all workload costs is used as the workload reward , defined as follows: In the formula, Indicates the price of unit server CPU resource utilization; Representing Workload QoS evaluation indicators are called QoS violations; represents the workload waiting time; Indicates the actual execution time of the workload; The policy acquisition module is used to construct a data center task scheduling algorithm based on policy gradient, and includes: The objective function of the construction strategy is: Constructing the policy gradient is: The formula for updating the parameters of the constructed strategy network is: The error function for constructing the state value estimate is: The update formula of the build state value parameter is: in, represents the state value function of the task scheduler at time t, represents the action-state value function of the task scheduler at time t, represents the policy network parameters, Indicates the state value parameter represents an unbiased estimate of the action-state value function, represents the update step size of the policy network parameters, represents the error function of the state value estimate, The update step size of the state value parameter is represented, and the Markov decision model is solved by using the constructed algorithm to obtain the optimal task scheduling strategy; The scheduling execution module is used to sample the actual data center workload arrival trajectory and guide the data center task scheduler to schedule the workload according to the optimal task scheduling strategy. The optimal task scheduling strategy represents the probability of outputting different actions in each state, and the task scheduler selects the action with the highest probability for execution.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images