A data center cluster online scheduling method and system

Abstract

The application discloses a kind of data center cluster online scheduling method and system, by analyzing the energy use characteristics of work load in data center work load is divided into interactive work load and batch processing work load and respectively establishes corresponding energy consumption model and work load migration model, for the distributed equipment contained in data center, the corresponding operation model is established, and the data center cluster optimization scheduling model considering the purchase electricity cost, equipment operation cost and load migration cost is constructed, for the data center cluster online optimization scheduling problem, the data center cluster optimization scheduling model constructed above is expressed to Markov decision process framework, and the reward function, state space, action space and training environment of data center cluster agent are designed.The application can realize the online optimization scheduling of data center cluster, and has a vital role for promoting energy structure transformation and accelerating renewable energy consumption.

Description

Technical Field

[0001] This invention belongs to the field of power distribution network operation optimization technology, specifically relating to an online scheduling method and system for data center clusters. Background Technology

[0002] With the surging global demand for cloud computing services, cloud service operators have established large-scale, geographically dispersed data centers worldwide to analyze and process massive workloads. This has led to high energy consumption and environmental pollution problems in data centers. Statistics show that large-scale data centers consume 1.3% of the world's total electricity consumption annually and 2% of the total electricity consumption in the United States. Furthermore, the irrational allocation of cloud resources further increases energy consumption and maintenance costs; in Google's data centers, the average server utilization rate is only around 20%. Therefore, designing an adaptive and efficient solution for data center resource allocation is of great significance.

[0003] Furthermore, with the widespread integration of large-scale distributed generation globally, the construction scale of some new energy power plants is mismatched with local load demand, hindering the effective utilization of large-scale distributed new energy in multiple regions and resulting in severe wind and solar curtailment. Globally geographically distributed data centers, as important flexible resources, can effectively enhance their ability to locally absorb distributed renewable energy through proper scheduling.

[0004] Currently, many scholars have conducted research on data center energy consumption, but most studies are limited to the optimization and management of individual data centers, neglecting the coordinated optimization among the rapidly developing global data center clusters, and primarily focusing on data centers operating in deterministic environments. However, in actual data center operation scenarios, scheduling processes should typically be completed within a short time. Furthermore, due to the uncertainty of workload and renewable energy sources, the optimal decision derived in a deterministic environment may no longer be the optimal decision in actual operation. This severely impacts the security and economy of real-time data center operation. Therefore, research considering real-time energy management of data centers under uncertain environments is essential. Traditional real-time energy management for data centers is mainly based on rules, heuristics, and control theory. These schemes can solve resource allocation problems to some extent, but they typically utilize prior knowledge of cloud systems to formulate corresponding short-term resource allocation strategies, failing to adaptively meet the dynamic demands of user workloads in the long term. Moreover, when considering both time and spatial load migration, the mathematical models constructed become difficult to solve, and traditional solution algorithms cannot guarantee real-time response.

[0005] Deep reinforcement learning, a method that approximates optimal rewards by iteratively obtaining feedback from historical decisions, has become a promising approach for handling resource allocation problems with high adaptability and low complexity. Currently, research on online scheduling of geographically distributed data center clusters is limited. In general, online scheduling of data center clusters is still in its early stages, lacking depth and comprehensiveness. Furthermore, there is a dearth of research on maximizing the absorption of geographically distributed renewable energy and reducing data center energy costs through online scheduling of multiple data centers. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an online scheduling method and system for data center clusters, which addresses the shortcomings of the prior art and solves the technical problem of adaptively managing the energy consumption of data centers across regions and dynamically in an uncertain environment, thereby realizing online optimized scheduling of data center clusters.

[0007] The present invention adopts the following technical solution: A data center cluster online scheduling method includes the following steps: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model for data center workloads and a data center energy consumption model are constructed. Based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, a data center cluster optimization scheduling model is constructed that considers electricity purchase cost, equipment operation cost and load migration cost. The constructed data center cluster optimization scheduling model is expressed as a Markov decision process. By designing the state space, action space, and reward function of the agent, an advanced deep reinforcement learning algorithm is used to realize the online scheduling of the data center cluster.

[0008] Preferably, the spatiotemporal migration model for data center workloads is as follows: Constraints associated with interactive loads:

[0009] in, Indicates time From the terminal server Assigned to data center Interactive workload Indicates time Reaching the terminal server Interactive workload; Constraints associated with batch processing workloads:

[0010] in, Indicates time Arrival at the data center Batch processing workload , Indicates in data center Transfer to time Batch processing workload The amount; The relationship between the minimum active server and the interactive workload being processed can be expressed as follows:

[0011] in, Indicates time Time Data Center The minimum number of active servers for interactive workloads in the middle. Indicates data center The average server service rate, This indicates the tolerance service latency for interactive workloads; The relationship between the lowest level of the active server and the batch workloads processed is expressed as follows:

[0012] in, Indicates time Time Data Center Medium-sized service batch processing workloads The number of redundant active servers; Interactive workloads allocated from a terminal server to a data center and migrated to time slots One of the batch processing workloads and active servers is non-negative, as detailed below:

[0013] in, Indicates time Time Data Center Interactive workloads / batch workloads in the middle of the service The minimum number of active servers, time Time Data Center Interactive workloads / batch workloads in the middle of the service The number of redundant active servers, Indicates service for interactive workloads / batch workloads The number of servers; The total number of active servers cannot exceed the total number of servers:

[0014] in, For data centers The total number of servers in the country; Range of active servers operating at peak power:

[0015] Data center energy consumption model:

[0016] in, Indicates time Used to ensure data center The power consumption of server equipment with the lowest service quality. Indicates time Data Center The power consumption of redundant server equipment used for higher service quality. For the power consumption of other equipment, This represents the power consumption of the cooling system.

[0017] Preferably, in time Used to ensure data center The power consumption of the server equipment with the lowest service quality. for:

[0018] in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server.

[0019] Preferably, in time Data Center Power consumption of redundant server equipment for higher service quality for:

[0020] in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server.

[0021] Preferably, the power consumption of the cooling system for:

[0022] in, This is an empirical constant for the cooling system. The heat dissipated.

[0023] Preferably, the data center cluster optimized scheduling model is as follows:

[0024]

[0025]

[0026] Energy costs include the cost of purchasing electricity from the external electricity market. Operating costs of traditional generators , Indicates time China and Israel The energy market spot price, workload migration cost represents the cost of transferring workloads. The quantity, and It means from Migration cost coefficient for exported workload units.

[0027] Preferably, the operating model of conventional generator sets in the data center is as follows:

[0028]

[0029] in, and express conventional generators in The maximum and minimum power output, and It indicates the generator Whether it is enabled (binary variable) It is the slope coefficient. and yes medium generator The minimum turn-on time and minimum turn-off time. and yes medium generator The actual connection and disconnection times.

[0030] Startup and shutdown costs and operating costs:

[0031] in, The cost factor for generator output. This represents the square of the generator set's output. This indicates the cost of a single start-up and shutdown of the generator set. The generator set starts and stops with 0-1 variable states.

[0032] The operating model of energy storage systems in data centers is as follows:

[0033]

[0034]

[0035] make and It is an energy storage system in The maximum and minimum capacity in, and It is time Storage capacity of energy storage systems.

[0036] Electricity generated during the charging and discharging process of an energy storage system:

[0037] Ensure that the storage level is at the same level as the initial state at the end of the operating day to guarantee that the storage system has the same energy exchange capacity for each day:

[0038] The constraints on the connection between the data center and the upstream power grid are:

[0039] Power generated by renewable energy sources is expressed as:

[0040] The power balance of this system is as follows:

[0041] in, Indicates in time slot The power generated in Indicates in time slot The power consumed in the process.

[0042] Preferably, the constructed data center cluster optimization scheduling model is expressed as a Markov decision process as follows: In any time slot, in order to find the state To action The mapping relationship is used to introduce a strategy. This represents the conditional probability distribution of each action given the current state.

[0043] When known At that time, a reward value related to the objective function can be obtained. This is used to guide the training of the agent and to update the action policy using gradient ascent to maximize the reward value, i.e.:

[0044] Therefore, a state value function is defined. To measure the quality of the parameterized state of a neural network.

[0045]

[0046] The goal of an agent during training is to select the optimal policy and maximize the state value function, i.e.:

[0047] The critic network simulates the real state-value function, while the actor network is used to train a deterministic policy. The update policies of the two networks are expressed as follows:

[0048]

[0049] in, The function representing the current actual state value. Customizable parameters required for the neural network model; Finally, a soft update method was used to update the network parameters in the critic network and actor network. To update, that is:

[0050]

[0051] in, These are the hyperparameters that need to be updated.

[0052] Preferably, the action space of the intelligent agent:

[0053] in, The photovoltaic, wind power, diesel generators and energy storage configured in each data center are respectively t Time-slot scheduling output For data centers to handle interactive workloads t The number of servers that are online during a given time period. for t The processing capacity of time-based batch processing workloads. For each data center t The amount of load transmitted in the fiber optic link during a given period; The state space of an agent:

[0054] in, For data centers t The interactive task load carried by the time period. Energy storage in data centers t State of charge during a period of time For data centers t The remaining batch processing load during the time period. t The current runtime segment in which the data center operates; Training environment of the agent: During offline training, the agent's actor network provides the decision output of each data center based on the state space. The power flow model of the distribution network system, as the environment, provides operating costs, network losses, etc. as reward values ​​to the agent after obtaining the injected power of each data center node and moves to the next state. The critic network evaluates this action and guides the agent's action at the next moment. The agent continuously adjusts its actions through training until the training round ends. The agent's reward function:

[0055] in, The penalty coefficient is... Penalty for out-of-bounds state of charge in energy storage Quality of Service (QoS) penalty for interactive load balancing. Penalties for failing to complete batch processing tasks on time. Penalty for exceeding the server limit.

[0056] Secondly, embodiments of the present invention provide an online scheduling system for data center clusters, comprising: The module constructs a data center workload spatiotemporal migration model and a data center energy consumption model based on the spatiotemporal migration characteristics of workloads in the data center. The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, constructs a data center cluster optimization scheduling model that considers electricity purchase cost, equipment operation cost and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and implements online scheduling of the data center cluster by designing the state space, action space and reward function of the agent and using advanced deep reinforcement learning algorithms.

[0057] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described online scheduling method for data center clusters.

[0058] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described online scheduling method for data center clusters.

[0059] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described online scheduling method for data center clusters.

[0060] Sixthly, embodiments of the present invention provide an electronic device including a computer program, wherein when the computer program is executed by the electronic device, it implements the steps of the above-described online scheduling method for data center clusters.

[0061] Compared with the prior art, the present invention has at least the following beneficial effects: An online scheduling method for data center clusters is proposed. This method analyzes the energy consumption characteristics of workloads in the data center, categorizing workloads into interactive and batch processing workloads, and establishing corresponding energy consumption and workload migration models for each. For the distributed devices within the data center, corresponding operation models are established. A data center cluster optimization scheduling model considering electricity purchase costs, equipment operating costs, and load migration costs is constructed. For the online optimization scheduling problem of data center clusters, the constructed data center cluster optimization scheduling model is expressed using a Markov decision process framework, and the reward function, state space, action space, and training environment of the data center cluster agent are designed. Implemented case studies demonstrate that this method can achieve online optimization scheduling of data center clusters. The constructed data center cluster optimization scheduling model comprehensively considers constraints such as load migration, distributed power output, and service quality, minimizing the operating costs of data center clusters and improving the absorption rate of renewable energy in multiple regions. Furthermore, the advanced deep reinforcement learning method used has low computational complexity, meeting the real-time computational requirements of data centers. Furthermore, the data center workload spatiotemporal migration model and data center energy consumption model based on step S1 can provide accurate foundational data for subsequent optimized scheduling. The construction of these models helps to accurately predict the dynamic changes in data center workloads and their energy consumption characteristics, thereby enabling efficient resource allocation across different times and spaces. The workload spatiotemporal migration model identifies load change patterns by analyzing the geographical and temporal distribution characteristics of the load within the data center. This allows the system to anticipate peak and off-peak periods of load and their potential locations, thus enabling dynamic load migration and optimized allocation. The data center energy consumption model matches energy consumption with load, accurately estimating energy demand for each time period by considering the impact of load fluctuations on energy consumption.

[0062] Furthermore, step S2, based on the data center workload spatiotemporal migration model and energy consumption model constructed in step S1, and combined with the operating characteristics of distributed data center equipment, constructs an optimized data center cluster scheduling model to minimize energy consumption and costs. The optimized scheduling model in step S2 considers various cost factors, including electricity purchase costs, equipment operating costs, and load migration costs. By incorporating these costs into the scheduling model, different overheads can be effectively balanced, ultimately minimizing the overall cost of the data center cluster. By constructing the optimized scheduling model and combining it with deep reinforcement learning algorithms, the system can automatically adjust and optimize resource allocation during actual operation. Intelligent real-time scheduling enables the system to cope with dynamic changes in workload, improving scheduling flexibility and adaptability.

[0063] Furthermore, step S3 describes the data center cluster optimization scheduling model as a Markov decision process. The agent's state space includes key parameters such as data center load, energy storage status, and server status, reflecting the current operational status of the data center; the action space involves operations on various controllable resources (such as the number of allocated servers and the amount of load scheduled). By designing a reasonable state and action space, the agent can efficiently evaluate the current state, take appropriate actions, and adapt to different workloads and energy consumption changes. By designing a reasonable reward function, operations that exceed boundaries (such as out-of-bounds energy storage status or service quality degradation) can be penalized, thereby guiding the agent to learn towards the target optimization. This design improves the intelligence level of scheduling, enabling the system to achieve better economic benefits and service levels in the long run. Deep reinforcement learning methods endow the data center cluster scheduling system with self-optimization and learning capabilities, enabling the system to respond to various uncertainties in real time, optimize resource allocation, and improve service quality. This intelligent online scheduling method has significant advantages in reducing operating costs, improving energy efficiency, and enhancing system flexibility.

[0064] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0065] In summary, the online scheduling method and system for data center clusters that considers the spatiotemporal migration characteristics of workloads in this invention significantly reduces computation time, achieves real-time near-optimal matching between workload migration and renewable energy generation, improves the renewable energy absorption rate in multiple regions, and effectively reduces the operating cost of the entire data center cluster. This aligns with the current development direction of power distribution network optimization scheduling and plays a crucial role in promoting energy structure transformation and accelerating renewable energy absorption.

[0066] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method of the present invention; Figure 2 Training graph for deep reinforcement learning algorithms; Figure 3 A comparison chart of scenarios for optimizing scheduling for data center clusters; Figure 4 This is a schematic diagram of a computer device provided according to an embodiment of the present invention.

[0068] Figure 5 This is a block diagram of a chip provided according to an embodiment of the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0071] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0072] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0073] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0074] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0075] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0076] This invention provides an online scheduling method for data center clusters. First, by analyzing the energy consumption characteristics of workloads in the data center, workloads are divided into interactive workloads and batch processing workloads, and corresponding energy consumption models and workload migration models are established for each. Second, for the distributed devices contained in the data center, corresponding operation models are established, and an optimized scheduling model for the data center cluster considering power purchase costs, equipment operating costs, and load migration costs is constructed. Finally, for the online optimized scheduling problem of data center clusters, the constructed optimized scheduling model is expressed as a Markov decision process framework, and the reward function, state space, action space, and training environment of the data center cluster agent are designed. Implemented case studies show that this method can achieve online optimized scheduling of data center clusters. The constructed optimized scheduling model comprehensively considers constraints such as load migration, distributed power output, and service quality, which can minimize the operating costs of data center clusters and improve the absorption rate of new energy sources in multiple regions. Moreover, the advanced deep reinforcement learning method used has low computational complexity, meeting the real-time computational requirements of data centers. Meanwhile, this online scheduling method for data center clusters aligns with the current development trend of power distribution network optimization scheduling, and plays a crucial role in promoting energy structure transformation and accelerating the consumption of renewable energy.

[0077] Please see Figure 1 This invention discloses an online scheduling method for data center clusters, targeting a power distribution network system containing multiple data centers, comprising the following steps: S1. Based on the spatiotemporal migration characteristics of workloads in data centers, construct a spatiotemporal migration model for data center workloads and a data center energy consumption model. 1. The data center energy consumption model is constructed as follows: The power consumption of a data center consists of the following components:

[0078] in, Indicates time Used to ensure data center The power consumption of server equipment with the lowest service quality. Indicates time Data Center The power consumption of redundant server equipment used for higher service quality. For the electricity consumption of other equipment (e.g., lighting and electrical facilities), The power consumption of the cooling system.

[0079]

[0080] in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server.

[0081] The above formula shows the first part of the server device's power consumption, which increases with the activity of the server and the workload it handles.

[0082]

[0083] The above formula shows the second part of the server equipment's power consumption, which includes two terms. The first term accounts for the increased power consumption of the server equipment due to the additional use of active servers. The second term accounts for the increased power consumption of the server equipment considering that some active servers will run at a service rate higher than their minimum required service rate. This formula indicates that active servers always operate at their peak power, regardless of the workload they are handling.

[0084] For simplicity, assume that the servers in each data center are homogeneous.

[0085]

[0086] The above equation shows the power consumption of the cooling system, which is defined as a linear function of thermal cooling power generation.

[0087]

[0088] in, For the cooling system, This represents the upper limit of power for cooling systems in data centers.

[0089] 2. The spatiotemporal migration model for data center workloads is constructed as follows:

[0090] The above formula is a constraint associated with interactive workloads, indicating that the total workload distributed from one terminal server to multiple data centers equals the workload arriving at that terminal server. Indicates time From the terminal server Assigned to data center Interactive workload Indicates time Reaching the terminal server Interactive workload.

[0091]

[0092] The above formula represents the constraints associated with batch processing workloads, ensuring that within time slots... Total batch workload arriving during the period If it is possible in a time slot If the service is provided during the period, it can be provided before its expiration date. Indicates time Arrival at the data center Batch processing workload Indicates in data center Transfer to time Batch processing workload The amount.

[0093] To ensure that interactive workloads are served with a certain probability before their deadlines, the following approach is adopted: Using a queue model to consider response time, the relationship between the minimum active server and the interactive workload being processed is expressed as follows:

[0094] in, Indicates time Time Data Center The minimum number of active servers for interactive workloads in the middle. Indicates data center The average server service rate, This indicates the tolerance service latency for interactive workloads.

[0095] To ensure that batch workloads can be handled, the relationship between the lowest level of the active server and the batch workloads already processed is expressed as follows:

[0096] in, Indicates time Time Data Center Medium-sized service batch processing workloads The number of redundant active servers.

[0097]

[0098] The above formula represents the transfer of interactive workloads from a terminal server to a data center to time slots. One of the batch processing workloads and active servers is non-negative.

[0099] in, Indicates time Time Data Center Interactive workloads / batch workloads in the middle of the service The minimum number of active servers, time Time Data Center Interactive workloads / batch workloads in the middle of the service The number of redundant active servers, Indicates service for interactive workloads / batch workloads The number of servers. Servers operate at their peak power, regardless of their time. Time Data Center The workload in the middle.

[0100]

[0101] The above formula indicates that the total number of active servers cannot exceed the total number of servers. For data centers The total number of servers in China.

[0102]

[0103] The above formula represents the range of active servers operating at peak power, which cannot exceed the number of active servers.

[0104] S2. Based on the data center workload spatiotemporal migration model and data center energy consumption model constructed in step S1, and combined with the operation model of distributed devices in the data center, a data center cluster optimization scheduling model considering electricity purchase cost, equipment operation cost and load migration cost is constructed. 1. The operational model of distributed devices in a data center is constructed as follows: The operating model of conventional generator sets in a data center is as follows:

[0105]

[0106] Existing research has provided a good explanation of the power output model of conventional generator sets, including upper and lower limits of output, ramping constraints, and start-stop time constraints.

[0107] in, and express conventional generators in The maximum and minimum power output, and It indicates the generator Whether it is enabled (binary variable) It is the slope coefficient. and yes medium generator The minimum turn-on time and minimum turn-off time. and yes medium generator The actual connection and disconnection times.

[0108] The operating cost of a conventional generator set depends on the generator set's output and typically includes start-up and shutdown costs as well as operating costs.

[0109] in, The cost factor for generator output. This represents the square of the generator set's output. This indicates the cost of a single start-up and shutdown of the generator set. The generator set starts and stops with 0-1 variable states.

[0110] The operating model of energy storage systems in data centers is as follows: The charging power and discharging power of the energy storage system in the figure are defined as follows: and ,and and It is a binary variable that indicates whether the system is charging or discharging.

[0111] Moreover, batteries generally cannot be charged or discharged at the same time.

[0112]

[0113]

[0114]

[0115] make and It is an energy storage system in The maximum and minimum capacity in, and It is time Storage capacity of energy storage systems.

[0116]

[0117] The above formula is the formula for the amount of electricity generated during the charging and discharging process of an energy storage system.

[0118]

[0119] The above formula means that the storage level at the end of the operating day should be the same as the initial state to ensure that the storage system has the same energy exchange capacity for each day.

[0120] The constraints on the connection between the data center and the upstream power grid are:

[0121] The above formula indicates that to ensure the safe transmission of electricity without exceeding the heat, voltage, and stable capacity of the transmission line, in order to meet system reliability requirements.

[0122] Power generated by renewable energy sources is expressed as:

[0123] The power balance of this system is as follows:

[0124] As mentioned above, in this model, electricity demand is supplied by the public grid, micro gas turbines, renewable energy sources, and energy storage. Indicates in time slot The power generated in Indicates in time slot 1. Power consumed in the process. 2. The optimal scheduling model for a data center cluster, considering electricity purchase costs, equipment operating costs, and load migration costs, is constructed as follows: The data center cluster optimization scheduling model is considered a multi-objective optimization problem. To avoid subjective coefficient selection in the multi-objective optimization solution, objective monetization is achieved by introducing different cost factors into the proposed model. The total cost of the model then includes energy costs, workload migration and computing service demand costs, and system operation demand costs.

[0125] Energy costs include the cost of purchasing electricity from external electricity markets, of which, Indicates time China and Israel The energy market spot price. Workload migration cost represents the cost of transferring workloads. The quantity, and It means from The migration cost factor for the exported workload unit is then routed to...

[0126]

[0127]

[0128] Workload migration cost represents the cost of transferring workloads. The quantity, and It means from The migration cost factor for the exported workload unit is then routed to... .

[0129]

[0130] S3. The data center cluster optimization scheduling model constructed in step S2 is expressed as a Markov decision process. By designing the state space, action space and reward function of the agent, the online scheduling of the data center cluster is realized by using an advanced deep reinforcement learning algorithm.

[0131] 1. The Markov decision process described in the data center cluster optimization scheduling model is as follows: Since the data center cluster optimization scheduling model established above contains nonlinear constraints, and distributed new energy sources and loads have random and fluctuating characteristics, traditional optimization algorithms are difficult to achieve satisfactory results in solving this problem. However, deep reinforcement learning algorithms have excellent performance in the face of uncertain environments, and models trained offline can be directly applied to online economic scheduling, making them suitable for formulating economic scheduling schemes for data center clusters in a short period of time. Therefore, this paper adopts the DDPG algorithm to solve the problem.

[0132] In the implementation of deep reinforcement learning algorithms, the data center cluster scheduling center is treated as the agent, and the actual system is considered as the external environment. The agent needs to make decisions in an unknown environment and maximize the accumulated expected reward, which can be achieved using Markov decision processes. This is used to represent [the process]. Among them... For a finite state space, For limited space of motion, For the limited rewards of the environment, It is the decay coefficient for future rewards. It is the state transition matrix The current state-action pair can be determined through the state transition matrix. Mapping to the next state The probability distribution on.

[0133] 2. Design the state space, action space, and reward function of the intelligent agent, and implement online scheduling of the data center cluster using advanced deep reinforcement learning algorithms: In any time slot, in order to find the state To action The mapping relationship is used to introduce a strategy. This represents the conditional probability distribution of each action given the current state. At that time, a reward value related to the objective function can be obtained. This is used to guide the training of the agent and to update the action policy using gradient ascent to maximize the reward value, i.e.:

[0134] Therefore, a state value function is defined. To measure the quality of the parameterized state of a neural network.

[0135]

[0136] The goal of an agent during training is to select the optimal policy and maximize the state value function, i.e.:

[0137] The critic network simulates the real state-value function, while the actor network is used to train a deterministic policy. The update policies of the two networks are expressed as follows:

[0138]

[0139] in, The function representing the current actual state value. These are the custom parameters required for the neural network model.

[0140] Finally, a soft update method was used to update the network parameters in the critic network and actor network. To update, that is:

[0141]

[0142] in, These are the hyperparameters that need to be updated.

[0143] To solve the online scheduling problem of data center clusters using reinforcement learning methods, the state space, action space, environment, and reward function of the agent are defined as follows: Action Space: A data center cluster contains multiple data centers, each containing multiple controllable devices such as diesel generators, wind power, photovoltaics, and energy storage. At each time interval, the agent needs to interact with the environment and select its action based on the current state of the environment. Its action space can be represented as:

[0144] in, The photovoltaic, wind power, diesel generators and energy storage configured in each data center are respectively t Time-slot scheduling output For data centers to handle interactive workloads tThe number of servers that are online during a given time period. for t The processing capacity of time-based batch processing workloads. For each data center t The amount of load transmitted in the fiber optic link during a given period; State Space: The intelligent agent of a data center cluster makes optimal scheduling decisions for the output of each device in the system by observing its current environmental state. Its observed state includes the renewable energy output, electricity price, load of interactive loads, state of charge of energy storage, remaining load of batch processing loads, and the current runtime segment of each data center location. Its state space can be represented as:

[0145] in, For data centers t The interactive task load carried by the time period. Energy storage in data centers t State of charge during a period of time For data centers t The remaining batch processing load during the time period. t This refers to the current runtime segment of the data center.

[0146] To provide heterogeneous service latency guarantees for data center batch processing workloads and maintain the stable operation of energy storage devices, it is necessary to decouple the coupling constraints between time periods and transform them into single-time period optimization problems. Therefore, this invention defines the energy storage charging state and the remaining load of the batch processing load in the state space. The energy storage charging and discharging power and the processing load in the action space will affect these states and cause them to transition. Then, by applying penalties for out-of-bounds charging states and penalties for incomplete batch processing, the expected total reward based on the Markov decision process is maximized.

[0147] Training environment: During offline training, the agent's actor network will provide the decision output of each data center based on the state space. The power flow model of the distribution network system, as the environment, will provide operating costs, network losses, etc. as reward values ​​to the agent after obtaining the injected power of each data center node and move to the next state. The critic network will evaluate this action and guide the agent's action in the next moment. The agent will then continuously adjust its actions through training, and so on, until the training round ends.

[0148] Reward Value: The objective function in the data center cluster load balancing model is designed as the immediate reward in reinforcement learning, and the constraints are incorporated into the immediate reward in the form of penalty functions. Considering that too many penalty functions will affect the convergence and stability of the algorithm, this paper takes the upper and lower bound constraints into account when defining the action space.

[0149]

[0150] in, The penalty coefficient is... Penalty for out-of-bounds state of charge in energy storage Quality of Service (QoS) penalty for interactive load balancing. Penalties for failing to complete batch processing tasks on time. Penalty for exceeding the server limit.

[0151]

[0152]

[0153]

[0154]

[0155] in, express For the actual completion time of the batch processing load in data center j, This refers to the latest processing time required for batch processing loads. In another embodiment of the present invention, an online scheduling system for data center clusters is provided. This system can be used to implement the above-mentioned online scheduling method for data center clusters. Specifically, the online scheduling system for data center clusters includes a construction module, a scheduling module, and an output module.

[0156] Among them, the construction module constructs a data center workload spatiotemporal migration model and a data center energy consumption model based on the spatiotemporal migration characteristics of workloads in the data center. The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, constructs a data center cluster optimization scheduling model that considers electricity purchase cost, equipment operation cost and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and implements online scheduling of the data center cluster by designing the state space, action space and reward function of the agent and using advanced deep reinforcement learning algorithms.

[0157] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of an online scheduling method for data center clusters, including: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model for data center workloads and a data center energy consumption model are constructed. Based on the constructed spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in data centers, an optimal scheduling model for data center clusters considering electricity purchase costs, equipment operation costs, and load migration costs is constructed. The constructed optimal scheduling model for data center clusters is expressed as a Markov decision process, and by designing the state space, action space, and reward function of the intelligent agent, an advanced deep reinforcement learning algorithm is used to achieve online scheduling of the data center cluster.

[0158] Please see Figure 4 The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the fluid composition calculation method in the reservoir stimulation wellbore of this embodiment. To avoid repetition, details are omitted here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the online scheduling system of the data center cluster of this embodiment. To avoid repetition, details are omitted here.

[0159] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 4This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0160] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0161] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or RAM of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the computer device 60.

[0162] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0163] Please see Figure 5 The terminal device is a chip. In this embodiment, the chip 600 includes a processor 622, which may be one or more, and a memory 632 for storing computer programs executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 622 may be configured to execute the computer program to perform the aforementioned online scheduling method for data center clusters.

[0164] Additionally, chip 600 may also include a power supply component 626 and a communication component 650. The power supply component 626 can be configured to perform power management of chip 600, and the communication component 650 can be configured to enable communication of chip 600, such as wired or wireless communication. Furthermore, chip 600 may also include an input / output interface 658. Chip 600 can operate on an operating system stored in memory 632.

[0165] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor; these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0166] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the online scheduling method for data center clusters in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model for data center workloads and a data center energy consumption model are constructed. Based on the constructed spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in data centers, an optimal scheduling model for data center clusters considering electricity purchase costs, equipment operation costs, and load migration costs is constructed. The constructed optimal scheduling model for data center clusters is expressed as a Markov decision process, and by designing the state space, action space, and reward function of the intelligent agent, an advanced deep reinforcement learning algorithm is used to achieve online scheduling of the data center cluster.

[0167] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0168] A data center workload spatiotemporal migration model and a data center energy consumption model are constructed using standard mathematical optimization models. These models include: interactive workload service quality constraints, interactive workload balancing constraints, migration task bandwidth constraints, interactive workload processing constraints, batch workload service quality constraints, batch workload balancing constraints, interactive workload server quantity constraints, batch workload server quantity constraints, and active server quantity constraints. Based on a standard reinforcement learning training framework, the data center workload spatiotemporal migration model and data center energy consumption model constructed by the model building module are converted into Markov decision processes. The state space, action space, reward function, and training environment of the agent are designed, and the advanced DDPG deep reinforcement learning algorithm is used for offline training of the data center cluster agent. Based on the data center cluster agent neural network model stored in the offline training module, the data center cluster completes online optimization scheduling based on the currently collected real-time state information during real-time operation, including real-time data of new energy sources and loads in the data center, real-time state of charge of energy storage, electricity purchase price of the upstream power grid, remaining batch processing load, and the current time, thus obtaining the online optimization scheduling result of the data center cluster.

[0169] The online scheduling method for data center clusters proposed in this invention, which considers the spatiotemporal migration characteristics of workloads, has been applied in actual data center cluster operation scenarios. To highlight the compatibility and scalability of the proposed method, a case study was designed. The test case is a modified IEEE-33 node system, which contains three data centers forming a data center cluster. The distributed equipment configured in each data center includes: photovoltaic units, wind turbines, energy storage systems, and micro gas turbines.

[0170] Figure 2This is a round-based reward graph of the data center cluster agent during training. Through continuous interaction and policy improvement, the agent converges after 200 rounds, exhibiting slight fluctuations within a certain range, indicating that the agent has obtained a near-optimal policy during training. Under a stochastic policy, since the agent is initially unfamiliar with the environment, it will take random actions to explore it. It is worth noting that the more complex the environment, the more explorations are required, which is a drawback of traditional deep reinforcement learning algorithms. The initial random exploration takes a long time, during which the total reward is typically low.

[0171] To verify the effectiveness of the deep reinforcement learning algorithm used in improving the renewable energy absorption rate, reducing system network losses and data center energy consumption, four different scenarios were designed to evaluate the impact of workload spatiotemporal migration.

[0172] Scenario 1: Simultaneously consider the spatial and temporal migration of two types of workloads.

[0173] Scenario 2: Only consider time migration of batch processing workloads.

[0174] Scenario 3: Considering only spatial migration of interactive workloads.

[0175] Scenario 4: Ignore spatial migration of interactive workloads while handling batch workloads with maximum computing power.

[0176] Figure 3 Comparing the test results of the four scenarios, compared with scenario 1, the total operating costs of scenarios 2, 3 and 4 increased by 48.87%, 18.56% and 58.58% respectively, indicating that the spatiotemporal migration of workloads can significantly reduce the total operating cost of data center clusters, and the spatial migration of interactive workloads is more effective in reducing costs.

[0177] Furthermore, scenarios 2 and 4 both involve significant renewable energy reduction costs, while scenarios 1 and 3 significantly reduce renewable energy reduction costs through spatial migration of interactive workloads.

[0178] Among them, the superposition effect of spatiotemporal migration in Scenario 1 has a significant effect on improving the renewable energy absorption rate, while Scenario 4 does not consider spatiotemporal migration and cannot adjust the load level of the data center cluster, resulting in very high renewable energy reduction costs and data center cluster energy consumption costs.

[0179] Overall, the spatiotemporal migration of data center cluster workloads can significantly improve the utilization rate of renewable energy and reduce the energy consumption cost of data center clusters. Based on the real-time output of renewable energy, the flexible transfer of workloads within a data center cluster through interactive workload spatial migration can greatly improve the renewable energy utilization rate. Furthermore, the temporal migration of batch processing workloads can respond to changes in electricity prices, further reducing the energy consumption cost of data center clusters.

[0180] In summary, the online scheduling method and system for data center clusters of this invention significantly reduces computation time, achieves real-time near-optimal matching of workload migration and renewable energy generation, improves the renewable energy absorption rate in multiple regions, and effectively reduces the operating cost of the entire data center cluster. It conforms to the current development direction of power distribution network optimization scheduling and plays a vital role in promoting energy structure transformation and accelerating renewable energy absorption.

[0181] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0182] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0183] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0184] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0185] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0187] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random-access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0188] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0189] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0190] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0191] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for online scheduling of a data center cluster, characterized in that, Includes the following steps: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model for data center workloads and a data center energy consumption model are constructed, specifically as follows: Constraints associated with interactive loads: in, Indicates time From the terminal server Assigned to data center Interactive workload Indicates time Reaching the terminal server Interactive workload; Constraints associated with batch processing workloads: in, Indicates time Arrival at the data center Batch processing workload , Indicates in data center Lieutenant General Batch Processing Workload Transfer to time The workload being processed; The relationship between the minimum active server and the interactive workload being processed can be expressed as follows: in, Indicates time Time Data Center The minimum number of active servers for interactive workloads in the middle. Indicates data center The average server service rate, This indicates the tolerance service latency for interactive workloads; The relationship between the lowest level of the active server and the batch workloads processed is expressed as follows: Interactive workloads allocated from a terminal server to a data center and migrated to time slots One of the batch processing workloads and active servers is non-negative, as detailed below: in, Indicates time Time Data Center Interactive workloads / batch workloads in the middle of the service The minimum number of active servers, time Time Data Center Interactive workloads / batch workloads in the middle of the service The number of redundant active servers, Indicates service for interactive workloads / batch workloads The number of servers; The total number of active servers cannot exceed the total number of servers: in, For data centers The total number of servers in the country; Range of active servers operating at peak power: Data center energy consumption model: in, Indicates time Used to ensure data center The power consumption of server equipment with the lowest service quality. Indicates time Data Center The power consumption of redundant server equipment used for higher service quality. For the power consumption of other equipment, This refers to the power consumption of the cooling system; In time Used to ensure data center The power consumption of the server equipment with the lowest service quality. for: in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server; In time Data Center Power consumption for redundant active server equipment for: in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server; Power consumption of the cooling system for: in, This is an empirical constant for the cooling system. For the heat dissipated Based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, a data center cluster optimization scheduling model is constructed that considers electricity purchase cost, equipment operation cost and load migration cost. The constructed data center cluster optimization scheduling model is expressed as a Markov decision process. By designing the state space, action space, and reward function of the agent, the online scheduling of the data center cluster is achieved using a deep deterministic policy gradient algorithm.

2. The online scheduling method for data center clusters according to claim 1, characterized in that, The data center cluster optimization scheduling model aims to minimize the total scheduling cost of the data center cluster. The total scheduling cost includes the cost of purchasing electricity from the external power market, the operating cost of conventional generator sets, the cost of migrating workloads, and the cost of system operation requirements.

3. The online scheduling method for data center clusters according to claim 1, characterized in that, The Markov decision process includes a state space, an action space, a reward function, state transition relationships, and a future reward decay coefficient. During offline training, the actor network outputs scheduled actions based on the current state, the critic network evaluates the scheduled actions based on the reward function, and updates the network parameters of the actor network and the critic network through a soft update method.

4. The online scheduling method for data center clusters according to claim 3, characterized in that, The reward function of the intelligent agent is determined based on the operating cost of the data center cluster and the constraint violation penalty items; the constraint violation penalty items include the energy storage charging state exceeding the limit penalty, the interactive workload service quality penalty, the batch processing workload not being completed in time penalty, and the server number exceeding the limit penalty.

5. An online scheduling system for a data center cluster, characterized in that, include: The construction module, based on the spatiotemporal migration characteristics of workloads in data centers, constructs a spatiotemporal migration model for data center workloads and a data center energy consumption model, specifically as follows: Constraints associated with interactive loads: in, Indicates time From the terminal server Assigned to data center Interactive workload Indicates time Reaching the terminal server Interactive workload; Constraints associated with batch processing workloads: in, Indicates time Arrival at the data center Batch processing workload , Indicates in data center Lieutenant General Batch Processing Workload Transfer to time The workload being performed; The relationship between the minimum active server and the interactive workload being processed can be expressed as follows: in, Indicates time Time Data Center The minimum number of active servers for interactive workloads in the middle. Indicates data center The average server service rate, This indicates the tolerance service latency for interactive workloads; The relationship between the lowest level of the active server and the batch workloads processed is expressed as follows: Interactive workloads allocated from a terminal server to a data center and migrated to time slots One of the batch processing workloads and active servers is non-negative, as detailed below: in, Indicates time Time Data Center Interactive workloads / batch workloads in the middle of the service The minimum number of active servers, time Time Data Center Interactive workloads / batch workloads in the middle of the service The number of redundant active servers, Indicates service for interactive workloads / batch workloads The number of servers; The total number of active servers cannot exceed the total number of servers: in, For data centers The total number of servers in the country; Range of active servers operating at peak power: Data center energy consumption model: in, Indicates time Used to ensure data center The power consumption of server equipment with the lowest service quality. Indicates time Data Center The power consumption of redundant server equipment used for higher service quality. For the power consumption of other equipment, This refers to the power consumption of the cooling system; In time Data Center Power consumption for redundant active server equipment for: in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server; In time Data Center Power consumption of redundant server equipment for higher service quality for: in, The rated power consumption of each active edge switch / aggregator switch / core switch. For the number of edge switches / aggregator switches / core switches / servers, Idle / peak power for each active server; Power consumption of the cooling system for: in, This is an empirical constant for the cooling system. For the heat dissipated Based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, a data center cluster optimization scheduling model is constructed that considers electricity purchase cost, equipment operation cost and load migration cost. The constructed data center cluster optimization scheduling model is expressed as a Markov decision process, and the online scheduling of the data center cluster is achieved by designing the state space, action space and reward function of the agent and using a deep deterministic policy gradient algorithm. The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed devices in the data center, constructs a data center cluster optimization scheduling model that considers electricity purchase cost, equipment operation cost and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and implements online scheduling of the data center cluster by designing the state space, action space and reward function of the agent and using a deep reinforcement learning algorithm.

Images