Data center cluster online scheduling method and system

Abstract

The invention discloses a data center cluster online scheduling method and system, and the method comprises the steps: dividing workloads into interactive workloads and batch processing workloads through analyzing the energy consumption characteristics of the workloads in a data center, and respectively building a corresponding energy consumption model and a workload migration model; for distributed devices contained in a data center, corresponding operation models are respectively established, a data center cluster optimization scheduling model considering the electricity purchase cost, the device operation cost and the load migration cost is constructed, and for a data center cluster online optimization scheduling problem, a data center cluster optimization scheduling model considering the electricity purchase cost, the device operation cost and the load migration cost is established. The established data center cluster optimization scheduling model is expressed as a Markov decision process framework, and a reward function, a state space, an action space and a training environment of a data center cluster agent are designed. According to the method, online optimal scheduling of the data center cluster can be realized, and the method plays a crucial role in promoting energy structure transformation and accelerating renewable energy consumption.

Description

Technical Field

[0001] The present invention belongs to the technical field of distribution network operation optimization, and in particular relates to a data center cluster online scheduling method and system. Background Art

[0002] As the global demand for cloud computing services continues to surge, cloud service operators have established large-scale geographically dispersed data centers around the world to analyze and process a large number of workloads, which has led to high energy consumption and environmental pollution in data centers. According to statistics, the annual energy consumption of large-scale data centers accounts for 1.3% of the world's total electricity consumption and 2% of the total electricity consumption in the United States. At the same time, the unreasonable allocation of cloud resources further increases energy consumption and maintenance costs. In Google data centers, the average utilization rate of servers is only about 20%. Therefore, it is of great significance to design an adaptive and efficient solution for resource allocation in data centers.

[0003] In addition, with the widespread access to large-scale distributed power generation around the world, the construction scale of some new energy power stations does not match the local load demand, which hinders the effective use of large-scale distributed new energy in multiple regions and leads to serious wind and solar power abandonment. As an important flexible resource, the global geographically distributed data center can effectively improve its local consumption capacity of distributed renewable energy through reasonable scheduling.

[0004] At present, many scholars have carried out certain research work on the energy consumption problem of data centers, but most of them are limited to the optimization management of a single data center, without considering the coordinated optimization between the data center clusters that are currently developing rapidly around the world, and are mainly aimed at data centers running in a certain environment. However, in the actual operation scenario of the data center, the scheduling process should usually be completed in a short time, and due to the uncertainty of workload and new energy, the optimal decision made in a certain environment is no longer the optimal decision in actual operation, which seriously affects the safety and economy of the real-time operation of the data center. Therefore, it is very necessary to study the real-time energy management of data centers in uncertain environments. Traditional data center real-time energy management is mainly based on rules, heuristics and control theory. These solutions can solve the resource allocation problem to a certain extent, but they usually use the prior knowledge of the cloud system to formulate the corresponding short-term resource allocation strategy, and cannot adaptively meet the dynamic needs of user workloads from a long-term perspective. Moreover, when the load time and space migration are considered at the same time, the mathematical model constructed is difficult to solve, and the traditional solution algorithm cannot guarantee real-time response.

[0005] Deep reinforcement learning is a method that approximates the optimal reward by iteratively obtaining feedback from historical decisions. It has become a promising method for dealing with high-adaptability and low-complexity resource allocation problems. Currently, there are few studies on online scheduling of geographically distributed data center clusters. In general, the online scheduling of data center clusters has just started and is not in-depth and comprehensive enough. At present, there are few studies on maximizing the consumption of geographically distributed new energy and reducing the energy cost of data centers through online scheduling of multiple data centers. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a data center cluster online scheduling method and system in view of the deficiencies in the above-mentioned prior art, which is used to solve the technical problem of adaptively managing the energy consumption of data centers across regions and dynamically under uncertain environments, and to realize online optimization scheduling of data center clusters.

[0007] The present invention adopts the following technical solutions: A data center cluster online scheduling method comprises the following steps: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model of data center workloads and a data center energy consumption model are constructed; Based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed equipment in the data center, a data center cluster optimization scheduling model is constructed that takes into account the electricity purchase cost, equipment operation cost and load migration cost; The constructed data center cluster optimization scheduling model is expressed as a Markov decision process, and by designing the state space, action space and reward function of the intelligent agent, the advanced deep reinforcement learning algorithm is used to realize the online scheduling of the data center cluster.

[0008] Preferably, the data center workload spatiotemporal migration model: Constraints associated with interactive loads:

[0009] in, Indicates at time From a terminal server Assign to data center interactive workloads, Indicates at time Reaching the Terminal Server The amount of interactive workload; Constraints associated with batch workloads:

[0010] in, Indicates at time Arrival at the data center Batch workload volume , In the data center Transfer to time Batch processing workload The amount of The relationship between the minimum active servers and the interactive workload processed is expressed as:

[0011] in, Indicates at time Time Data Center The minimum number of active servers serving interactive workloads in Indicates data center The average service rate of the server, represents the tolerable service delay for interactive workloads; The relationship between the lowest level of active servers and the processed batch workload is expressed as:

[0012] in, Indicates at time Time Data Center Serving batch workloads The number of redundant active servers; Distribute interactive workloads from a terminal server to a data center and shift them to time slots One of the batch workloads and the active server is non-negative, as follows:

[0013] in, Indicates time Time Data Center Medium service interactive workload / batch workload The minimum number of active servers, time Time Data Center Medium service interactive workload / batch workload The number of redundant active servers, Indicates serving interactive workloads / batch workloads The number of servers; The total number of active servers cannot exceed the total number of servers:

[0014] in, For Data Center The total number of servers in; Range of active servers operating at peak power:

[0015] Data center energy consumption model:

[0016] in, Indicates at time Used to ensure data center The power consumption of the server equipment with the lowest quality of service in Indicates at time Data Center The power consumption of redundant server equipment for higher quality of service, For the power consumption of other devices, is the power consumption of the cooling system.

[0017] Preferably, at time Used to ensure data center Power consumption of server equipment with the lowest quality of service in for:

[0018] in, is the rated power consumption of each active edge switch / aggregation switch / core switch, is the number of edge switches / aggregation switches / core switches / servers, Idle / peak power for each active server.

[0019] Preferably, at time Data Center Power consumption of redundant server equipment for higher quality of service for:

[0020] in, is the rated power consumption of each active edge switch / aggregation switch / core switch, is the number of edge switches / aggregation switches / core switches / servers, Idle / peak power for each active server.

[0021] Preferably, the power consumption of the cooling system for:

[0022] in, is the empirical constant of the cooling system, To dissipate heat.

[0023] Preferably, the data center cluster optimization scheduling model is as follows:

[0024]

[0025]

[0026] The energy cost includes the electricity purchase cost from the external electricity market. and traditional generator operating costs , Indicates at time China-Israel The spot price of the energy market is calculated, and the workload migration cost represents the cost of transferring the workload. The number of It means from The migration cost factor for the exported workload unit.

[0027] Preferably, the operation model of the conventional generator set in the data center is:

[0028]

[0029] in, and express Conventional generators in The maximum and minimum power outputs, and It is an indicator generator A binary variable indicating whether it is enabled. is the slope coefficient. and yes Medium generator The minimum on-time and minimum off-time, and yes Medium generator The actual on-time and off-time.

[0030] Start-up and shutdown costs and operating costs:

[0031] in, is the cost coefficient of the generator output, It represents the square of the generator output. represents the single start-stop cost of the generator set, It is the 0-1 variable status of the generator set start and stop.

[0032] The operation model of the energy storage system in the data center is:

[0033]

[0034]

[0035] make and The energy storage system The maximum and minimum capacity in It's time The storage capacity of the energy storage system.

[0036] The amount of electricity during the charging and discharging process of the energy storage system:

[0037] Make sure that the storage level at the end of the operating day is at the same level as at the beginning, to guarantee that the storage system has the same energy exchange capacity for each day:

[0038] The interconnection line constraints between the data center and the upper power grid are:

[0039] The power generated by renewable energy is expressed as:

[0040] The power balance of the system is as follows:

[0041] in, Indicates time slot The power generated in Indicates time slot The power consumed in.

[0042] Preferably, the constructed data center cluster optimization scheduling model is expressed as a Markov decision process as follows: In any time slot, in order to find the state To Action The mapping relationship of Represents the conditional probability distribution of each action given the current state.

[0043] When known When , the reward value related to the objective function can be obtained , which is used to guide the training of the intelligent agent, and the gradient ascent method is used to update the action strategy to maximize the reward value, that is:

[0044] To do this, define the state value function To measure the quality of the neural network after parameterization.

[0045]

[0046] The goal of the agent during training is to select the best strategy and maximize the state value function, namely:

[0047] Among them, the critic network is used to simulate the real state value function, and the actor network is used to finally train a deterministic strategy. The update strategies of the two networks are expressed as follows:

[0048]

[0049] in, Represents the current actual state value function, Set the parameters required for the neural network model; Finally, the network parameters in the critic network and actor network are adjusted by soft update. To update, that is:

[0050]

[0051] in, is a hyperparameter that updates the desired setting.

[0052] Preferably, the action space of the agent:

[0053] in, The photovoltaic, wind power, diesel generators and energy storage configured in each data center are t The dispatch output of the time period, Processing interactive workloads in the data center t The number of servers powered on during the time period. for t The processing capacity of batch processing workloads during the period, For each data center t The amount of load transmitted in the optical fiber link during the time period; The state space of the agent:

[0054] in, For Data Center t The interactive task load carried by the time period, Energy storage in data centers t The state of charge during the time period, For data centers t The amount of batch load remaining in the period, t The current operation period of the data center; Training environment of the agent: During offline training, the agent's actor network gives the decision output of each data center based on the state space. The power flow model of the distribution network system, as the environment, gives the operating cost, network loss, etc. as reward values ​​after obtaining the injected power of the nodes where each data center is located, and then feeds them back to the agent and transfers to the next state. The critic network evaluates this action and guides the agent's action at the next moment. The agent continuously adjusts its actions through training until the end of the training round. The agent’s reward function:

[0055] in, is the penalty coefficient, Penalty for energy storage state of charge out of bounds, Quality of service penalty for interactive loads, Penalty for batch tasks not completed in time, Penalty for exceeding the limit of server number.

[0056] In a second aspect, an embodiment of the present invention provides a data center cluster online scheduling system, including: A construction module, based on the spatiotemporal migration characteristics of workloads in the data center, constructs a spatiotemporal migration model of data center workloads and a data center energy consumption model; The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combines the operation model of distributed equipment in the data center, and constructs a data center cluster optimization scheduling model that takes into account the power purchase cost, equipment operation cost, and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and realizes the online scheduling of the data center cluster by designing the state space, action space and reward function of the intelligent agent and adopting the advanced deep reinforcement learning algorithm.

[0057] In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data center cluster online scheduling method when executing the computer program.

[0058] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned data center cluster online scheduling method.

[0059] In a fifth aspect, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data center cluster online scheduling method when executing the computer program.

[0060] In a sixth aspect, an embodiment of the present invention provides an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned data center cluster online scheduling method.

[0061] Compared with the prior art, the present invention has at least the following beneficial effects: A method for online scheduling of data center clusters, by analyzing the energy consumption characteristics of workloads in the data center, divides the workloads into interactive workloads and batch workloads, and establishes corresponding energy consumption models and workload migration models respectively. For the distributed equipment contained in the data center, corresponding operation models are established respectively, and a data center cluster optimization scheduling model considering electricity purchase costs, equipment operation costs and load migration costs is constructed. For the problem of online optimization scheduling of data center clusters, the above-constructed data center cluster optimization scheduling model is expressed as a Markov decision process framework, and the reward function, state space, action space and training environment of the data center cluster intelligent agent are designed. The implemented case study shows that the method of the present invention can realize the online optimization scheduling of data center clusters. The constructed data center cluster optimization scheduling model comprehensively considers constraints such as load migration, distributed power output and service quality, which can minimize the operating cost of the data center cluster and improve the absorption rate of new energy in multiple regions. Moreover, the advanced deep reinforcement learning method adopted has low computational complexity and meets the real-time computing requirements of the data center. Furthermore, the data center workload spatiotemporal migration model and data center energy consumption model based on step S1 can provide accurate basic data for subsequent optimization scheduling. The construction of these models helps to accurately predict the dynamic changes of data center workloads and their energy consumption characteristics, so that resources can be effectively allocated at different times and spaces. The workload spatiotemporal migration model identifies the pattern of load changes by analyzing the geographical and temporal distribution characteristics of the load in the data center. This allows the system to foresee the peak and trough time periods of the load and their possible locations, and then perform dynamic load migration and optimal allocation. The data center energy consumption model matches energy consumption with load, and accurately estimates the energy consumption requirements for each time period by considering the impact of load fluctuations on energy consumption.

[0062] Furthermore, step S2 constructs an optimized data center cluster scheduling model based on the data center workload spatiotemporal migration model and energy consumption model constructed in step S1, combined with the operating characteristics of the data center distributed equipment, to minimize energy consumption and cost. The optimized scheduling model of step S2 takes into account a variety of cost factors, including electricity purchase costs, equipment operating costs, and load migration costs. By incorporating these costs into the scheduling model, different expenses can be effectively balanced, and ultimately the overall cost of the data center cluster can be minimized. By constructing an optimized scheduling model and combining it with a deep reinforcement learning algorithm, the system can automatically adjust and optimize resource allocation during actual operation. Intelligent real-time scheduling enables the system to cope with dynamic changes in workloads and improve the flexibility and adaptability of scheduling.

[0063] Further, step S3 expresses the data center cluster optimization scheduling model as a Markov decision process. The state space of the agent includes key parameters such as data center load, energy storage state and server state, which reflect the current operation status of the data center; the action space involves the operation of various controllable resources (such as the number of allocated servers, the amount of scheduled load, etc.). By designing a reasonable state and action space, the agent can efficiently evaluate the current state, take appropriate actions and adapt to different workloads and energy consumption changes. By designing a reasonable reward function, operations that exceed the boundary (such as energy storage state crossing the boundary, service quality degradation, etc.) can be punished, thereby guiding the agent to learn in the direction of target optimization. This design improves the intelligence level of scheduling and enables the system to obtain better economic benefits and service levels in the long run. The deep reinforcement learning method gives the data center cluster scheduling system the ability to self-optimize and learn, so that the system can respond to various uncertainties in real time, optimize resource allocation and improve service quality. This intelligent online scheduling method has significant advantages in reducing operating costs, improving energy efficiency and enhancing system flexibility.

[0064] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0065] In summary, the online scheduling method and system for data center clusters of the present invention, which take into account the temporal and spatial migration characteristics of workloads, significantly reduces the calculation time, realizes real-time approximate optimal matching of workload migration and renewable energy power generation, improves the new energy consumption rate in multiple regions, and effectively reduces the operating cost of the entire data center cluster. It is in line with the current development direction of distribution network optimization and scheduling, and plays a vital role in promoting the transformation of energy structure and accelerating the consumption of renewable energy.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a flow chart of the method of the present invention; Figure 2 A training graph for a deep reinforcement learning algorithm; Figure 3 Comparison chart of scenarios for optimizing scheduling for data center clusters; Figure 4 A schematic diagram of a computer device provided by an embodiment of the present invention.

[0068] Figure 5 The block diagram of a chip provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0071] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0072] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0073] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0074] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0075] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0076] The present invention provides a data center cluster online scheduling method. First, by analyzing the energy consumption characteristics of the workload in the data center, the workload is divided into interactive workload and batch workload, and the corresponding energy consumption model and workload migration model are established respectively; secondly, for the distributed equipment contained in the data center, the corresponding operation model is established respectively, and the data center cluster optimization scheduling model considering the power purchase cost, equipment operation cost and load migration cost is constructed; finally, for the problem of online optimization scheduling of the data center cluster, the above-constructed data center cluster optimization scheduling model is expressed as a Markov decision process framework, and the reward function, state space, action space and training environment of the data center cluster intelligent agent are designed. The implemented case study shows that this method can realize the online optimization scheduling of the data center cluster. The constructed data center cluster optimization scheduling model comprehensively considers the constraints such as load migration, distributed power output and service quality, which can minimize the operation cost of the data center cluster and improve the absorption rate of multi-regional new energy. Moreover, the advanced deep reinforcement learning method adopted has low computational complexity and meets the real-time computing requirements of the data center. At the same time, the online scheduling method of the data center cluster is in line with the current development direction of distribution network optimization scheduling, and plays a vital role in promoting the transformation of energy structure and accelerating the consumption of renewable energy.

[0077] See also Figure 1 The present invention provides a data center cluster online scheduling method, for a distribution network system containing multiple data centers, comprising the following steps: S1. Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model of data center workloads and a data center energy consumption model are constructed; 1. The data center energy consumption model is constructed as follows: The load power consumption of a data center consists of the following components:

[0078] in, Indicates at time Used to ensure data center The power consumption of the server equipment with the lowest quality of service in Indicates at time Data Center The power consumption of redundant server equipment for higher quality of service, For electricity consumption of other equipment (e.g., lighting and electrical utilities), The power consumption for the cooling system.

[0079]

[0080] in, is the rated power consumption of each active edge switch / aggregation switch / core switch, is the number of edge switches / aggregation switches / core switches / servers, Idle / peak power for each active server.

[0081] The above equation shows the first part of the power consumption of server equipment, which increases with the active servers and the workload processed.

[0082]

[0083] The above formula shows the second part of the power consumption of the server equipment, which includes two terms. The first term is the increase in the power consumption of the server equipment taking into account the additional use of the active servers. The second term is the increase in the power consumption of the server equipment taking into account that some active servers will run at a service rate higher than their minimum requirement. This formula means that the active servers always run at their peak power, regardless of the workload they are processing.

[0084] For simplicity, we assume that the servers in each data center are homogeneous.

[0085]

[0086] The above equation shows the power consumption of the cooling system, which is determined as a linear function of the thermal cooling power generation.

[0087]

[0088] in, is the cooling system empirical constant, The power limit for the cooling system in the data center.

[0089] 2. The data center workload spatiotemporal migration model is constructed as follows:

[0090] The above equation is a constraint associated with interactive load, which indicates that the total workload distributed from one terminal server to multiple data centers is equal to the workload reaching the terminal server, Indicates at time From Terminal Server Assign to data center interactive workloads, Indicates at time Reaching the Terminal Server of interactive workload.

[0091]

[0092] The above equation is a constraint associated with batch workloads, which ensures that Total batch workload that arrived during the period If you can If the term is not served during the period, it may be served before its expiration date. Indicates at time Arrival at the data center Batch workload volume In the data center Transfer to time Batch processing workload The amount.

[0093] To ensure that interactive workloads are served with a certain probability before their deadlines, we use The queue model considers the relationship between response time, minimum active servers and processed interactive workload as:

[0094] in, Indicates at time Time Data Center The minimum number of active servers serving interactive workloads in Indicates data center The average service rate of the server, Indicates the tolerated service delay for interactive workloads.

[0095] To ensure that the batch workload can be processed, the relationship between the lowest level of active servers and the processed batch workload is expressed as:

[0096] in, Indicates at time Time Data Center Serving batch workloads The number of redundant active servers.

[0097]

[0098] The above formula represents the interactive workload assigned from a terminal server to a data center and transferred to the time slot One of the batch workloads and the active server is non-negative.

[0099] in, Indicates time Time Data Center Medium service interactive workload / batch workload The minimum number of active servers, time Time Data Center Medium service interactive workload / batch workload The number of redundant active servers, Indicates serving interactive workloads / batch workloads The servers are running at their peak power, regardless of their time. Time Data Center workloads in .

[0100]

[0101] The above formula means that the total number of active servers cannot exceed the total number of servers. For Data Center The total number of servers in the .

[0102]

[0103] The above equation represents the range of active servers operating at peak power, which cannot exceed the number of active servers.

[0104] S2. Based on the data center workload spatiotemporal migration model and data center energy consumption model constructed in step S1, combined with the operation model of distributed equipment in the data center, a data center cluster optimization scheduling model that considers electricity purchase cost, equipment operation cost and load migration cost is constructed; 1. The operation model of distributed devices in the data center is constructed as follows: The operating model of conventional generator sets in data centers is:

[0105]

[0106] The power output model of conventional generator sets is well explained in existing research, including upper and lower output constraints, climbing constraints and start-stop time constraints.

[0107] in, and express Conventional generators in The maximum and minimum power outputs, and It is an indicator generator A binary variable indicating whether it is enabled. is the slope coefficient. and yes Medium generator The minimum on-time and minimum off-time, and yes Medium generator The actual on-time and off-time.

[0108] The running costs of a conventional generator set depend on the output of the generator set and typically include start-up and shutdown costs as well as operating costs:

[0109] in, is the cost coefficient of the generator output, It represents the square of the generator output. represents the cost of starting and stopping a generator set once, It is the 0-1 variable status of the generator start and stop.

[0110] The operation model of the energy storage system in the data center is: The charging power and discharging power of the energy storage system in are defined as and ,and and is a binary variable indicating whether the system is charging or discharging.

[0111] And generally speaking, batteries cannot be charged or discharged at the same time.

[0112]

[0113]

[0114]

[0115] make and The energy storage system The maximum and minimum capacity in It's time The storage capacity of the energy storage system.

[0116]

[0117] The above formula is the formula for the amount of electricity during the charging and discharging process of the energy storage system.

[0118]

[0119] The above formula means ensuring that the storage level at the end of the operating day is at the same level as at the beginning, so as to guarantee that the storage system has the same energy exchange capacity for each day.

[0120] The tie line constraints between the data center and the upper power grid are:

[0121] The above formula means ensuring that electricity can be safely transmitted without exceeding the thermal, voltage and stability capacity of the transmission line to meet system reliability.

[0122] The power generated by renewable energy is expressed as:

[0123] The power balance of the system is as follows:

[0124] As mentioned above, in this model, the electricity demand is supplied by the utility grid, microturbines, renewable energy, and energy storage. Indicates time slot The power generated in Indicates time slot 2. The data center cluster optimization scheduling model considering electricity purchase cost, equipment operation cost and load migration cost is constructed as follows: The data center cluster optimization scheduling model is considered as a multi-objective optimization problem. In order to avoid subjective coefficient selection in multi-objective optimization solutions, the objectives are monetized by introducing different cost factors in the proposed model. Then, the total cost of the model includes energy cost, demand cost of workload migration and computing services, and demand cost of system operation.

[0125] Energy costs include the cost of purchasing electricity from the external electricity market, of which Indicates at time China-Israel The workload migration cost represents the cost of transferring the workload. The number of It means from The migration cost coefficient of the workload unit is derived and then routed to

[0126]

[0127]

[0128] Workload migration cost represents the cost of transferring workload The number of It means from The migration cost coefficient of the workload unit is derived and then routed to .

[0129]

[0130] S3. The data center cluster optimization scheduling model constructed in step S2 is expressed as a Markov decision process, and the online scheduling of the data center cluster is realized by designing the state space, action space and reward function of the intelligent agent and using the advanced deep reinforcement learning algorithm.

[0131] 1. The Markov decision process expressed by the data center cluster optimization scheduling model is: Since the data center cluster optimization scheduling model established above contains nonlinear constraints, and distributed new energy and loads have the characteristics of randomness and volatility, it is difficult for traditional optimization algorithms to achieve satisfactory results in solving this problem. The deep reinforcement learning algorithm has excellent performance in the face of uncertain environments, and the offline trained model can be directly applied to online economic scheduling, which is suitable for the formulation of economic scheduling plans for data center clusters in a short period of time. Therefore, this paper adopts the DDPG algorithm to solve the problem.

[0132] In the process of implementing the deep reinforcement learning algorithm, the data center cluster scheduling center is used as the intelligent agent of deep reinforcement learning, and the actual system is regarded as the external environment. The intelligent agent needs to make decisions in an unknown environment and maximize the cumulative expected reward, which can be expressed by the Markov decision process. In which is a finite state space, For a limited action space, For environmentally limited rewards, is the decay coefficient for future rewards, is the state transition matrix , the current state-action pair can be known through the state transfer matrix Mapping to the next state The probability distribution on .

[0133] 2. Design the state space, action space and reward function of the intelligent agent and use advanced deep reinforcement learning algorithms to achieve online scheduling of data center clusters: In any time slot, in order to find the state To Action The mapping relationship of represents the conditional probability distribution of each action when the current state is known. When , the reward value related to the objective function can be obtained It is used to guide the training of the intelligent agent and use the gradient ascent method to update the action strategy to maximize the reward value, that is:

[0134] To do this, define the state value function To measure the quality of the neural network after parameterization.

[0135]

[0136] The goal of the agent during training is to select the best strategy and maximize the state value function, namely:

[0137] Among them, the critic network is used to simulate the real state value function, and the actor network is used to finally train a deterministic strategy. The update strategies of the two networks are expressed as follows:

[0138]

[0139] in, Represents the current actual state value function, Set the parameters required for the neural network model.

[0140] Finally, the network parameters in the critic network and actor network are adjusted by soft update. To update, that is:

[0141]

[0142] in, is a hyperparameter that updates the desired setting.

[0143] Based on the reinforcement learning method, the online scheduling problem of data center cluster is solved. The state space, action space, environment and reward function of the agent are defined as follows: Action space: The data center cluster contains multiple data centers, each of which contains multiple controllable devices such as diesel generators, wind power, photovoltaics and energy storage. At each time interval, the agent needs to interact with the environment and choose the action at this moment according to the current state of the environment; its action space can be expressed as:

[0144] in, The photovoltaic, wind power, diesel generators and energy storage configured in each data center are t The dispatch output of the time period, Processing interactive workloads in the data center tThe number of servers powered on during the time period. for t The processing capacity of batch processing workloads during the period, For each data center t The amount of load transmitted in the optical fiber link during the time period; State space: The data center cluster agent makes the optimal scheduling decision for the output of each device in the system by observing its current environmental state. Its observed state includes the output of new energy sources, electricity purchase price, load of interactive loads, charge state of energy storage, remaining load of batch loads and current operating period at each data center. Its state space can be expressed as:

[0145] in, For Data Center t The interactive task load carried by the time period, Energy storage in data centers t The state of charge during the time period, For data centers t The amount of batch load remaining in the period, t The current operating time period of the data center.

[0146] In order to provide heterogeneous service delay guarantee for batch workloads in data centers and maintain stable operation of energy storage equipment, it is necessary to decouple the coupling constraints between time periods and convert them into single-period optimization problems. Therefore, the present invention defines the energy storage charge state and the remaining load of the batch load in the state space. The energy storage charging and discharging power and the batch load processing in the action space will affect it and cause state transfer, and then achieve the maximum expected value of the total return based on the Markov decision process through subsequent charge state out-of-bounds penalty and batch load incomplete penalty.

[0147] Training environment: During offline training, the agent's actor network will give the decision output of each data center based on the state space. The power flow model of the distribution network system, as the environment, will give the operating cost, network loss, etc. as reward values ​​after obtaining the injected power of the nodes where each data center is located. Feedback to the agent and transfer to the next state, the critic network will evaluate this action and guide the agent's action at the next moment. The agent will then continuously adjust its actions through training, and continue to interact until the end of the training round.

[0148] Reward value: The objective function in the data center cluster load balancing model is designed as an instant reward in reinforcement learning, and the constraints are added to the instant reward in the form of a penalty function. Considering that too many penalty functions will affect the convergence and stability of the algorithm, this paper considers the upper and lower limit constraints when defining the action space.

[0149]

[0150] in, is the penalty coefficient, Penalty for energy storage state of charge out of bounds, Quality of service penalty for interactive loads, Penalty for batch tasks not completed in time, Penalty for exceeding the limit of server number.

[0151]

[0152]

[0153]

[0154]

[0155] in, express is the actual completion period of the batch processing load in data center j, The latest processing time required for batch processing load. In another embodiment of the present invention, a data center cluster online scheduling system is provided, which can be used to implement the above data center cluster online scheduling method. Specifically, the data center cluster online scheduling system includes a construction module, a scheduling module and an output module.

[0156] Among them, the construction module constructs a data center workload spatiotemporal migration model and a data center energy consumption model based on the spatiotemporal migration characteristics of the workload in the data center; The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combines the operation model of distributed equipment in the data center, and constructs a data center cluster optimization scheduling model that takes into account the power purchase cost, equipment operation cost, and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and realizes the online scheduling of the data center cluster by designing the state space, action space and reward function of the intelligent agent and adopting the advanced deep reinforcement learning algorithm.

[0157] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the online scheduling method of the data center cluster, including: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model of data center workloads and a data center energy consumption model are constructed; based on the constructed spatiotemporal migration model of data center workloads and the data center energy consumption model, combined with the operation model of distributed equipment in the data center, a data center cluster optimization scheduling model that considers electricity purchase costs, equipment operation costs, and load migration costs is constructed; the constructed data center cluster optimization scheduling model is expressed as a Markov decision process, and by designing the state space, action space, and reward function of the intelligent agent, an advanced deep reinforcement learning algorithm is used to realize the online scheduling of the data center cluster.

[0158] See also Figure 4 , the terminal device is a computer device. The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the method for calculating the fluid composition in the reservoir transformation wellbore in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the data center cluster online scheduling system in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0159] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 4This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0160] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0161] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 60.

[0162] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0163] See also Figure 5 , the terminal device is a chip, and the chip 600 of this embodiment includes a processor 622, which may be one or more, and a memory 632 for storing a computer program executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 622 may be configured to execute the computer program to execute the above-mentioned data center cluster online scheduling method.

[0164] In addition, the chip 600 may further include a power supply component 626 and a communication component 650, wherein the power supply component 626 may be configured to perform power management of the chip 600, and the communication component 650 may be configured to implement communication, for example, wired or wireless communication, of the chip 600. In addition, the chip 600 may further include an input / output interface 658. The chip 600 may operate based on an operating system stored in the memory 632.

[0165] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs. It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk memory.

[0166] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the online scheduling method for a data center cluster in the above embodiment; the processor may load and execute the following steps: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model of data center workloads and a data center energy consumption model are constructed; based on the constructed spatiotemporal migration model of data center workloads and the data center energy consumption model, combined with the operation model of distributed equipment in the data center, a data center cluster optimization scheduling model that considers electricity purchase costs, equipment operation costs, and load migration costs is constructed; the constructed data center cluster optimization scheduling model is expressed as a Markov decision process, and by designing the state space, action space, and reward function of the intelligent agent, an advanced deep reinforcement learning algorithm is used to realize the online scheduling of the data center cluster.

[0167] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0168] The data center workload spatiotemporal migration model and data center energy consumption model are constructed in the form of standard mathematical optimization models, including: interactive workload service quality constraints, interactive workload balancing constraints, migration task bandwidth constraints, interactive load processing constraints, batch workload service quality constraints, batch workload balancing constraints, interactive workload server quantity constraints, batch workload server quantity constraints and active server quantity constraints; based on the reinforcement learning standard training framework, the data center workload spatiotemporal migration model and data center energy consumption model constructed by the model building module are converted into the form of Markov decision process, the state space, action space, reward function and training environment of the agent are designed, and the advanced DDPG deep reinforcement learning algorithm is used for offline training of data center cluster agents; based on the data center cluster agent neural network model saved by the offline training module, the online optimization scheduling of the data center cluster is completed based on the real-time status information currently collected during real-time operation, including the real-time data of new energy and load in the data center, the real-time charge state of energy storage, the electricity purchase price of the upper power grid, the remaining batch load and the current time, and the online optimization scheduling result of the data center cluster is obtained.

[0169] The online scheduling method of data center cluster considering the time and space migration characteristics of workload proposed in this paper has been applied in the actual data center cluster operation scenario. In order to highlight the compatibility and scalability of the proposed method, a case study is designed. The test case is a modified IEEE-33 node system, which contains 3 data centers to form a data center cluster. The distributed equipment configured in each data center includes: photovoltaic units, wind turbines, energy storage systems and micro gas turbines.

[0170] Figure 2The following is a graph of round rewards for the data center cluster agent during training. After continuous interaction and strategy improvement, the data center cluster agent reached convergence after 200 rounds and fluctuated slightly within a certain range, indicating that the agent obtained a near-optimal strategy during training. Under the random strategy, since the vector is initially unfamiliar with the environment, it will take random actions to explore the environment. It is worth noting that the more complex the environment, the more explorations are required, which is also a shortcoming of traditional deep reinforcement learning algorithms. The initial random exploration takes a long time, and the total reward is usually low during this process.

[0171] In order to verify the effectiveness of the adopted deep reinforcement learning algorithm in improving the new energy consumption rate and reducing system network losses and data center energy consumption, four different scenarios were designed to evaluate the impact of workload spatiotemporal migration.

[0172] Scenario 1: Considering both spatial and temporal migration of two types of workloads.

[0173] Scenario 2: Consider only the time migration of batch workloads.

[0174] Scenario 3: Consider only the space migration of interactive workloads.

[0175] Scenario 4: Ignore the spatial migration of interactive workloads and process batch workloads with maximum computing power.

[0176] Figure 3 The test results of the four scenarios are compared. Compared with scenario 1, the total operating costs of scenario 2, scenario 3 and scenario 4 increased by 48.87%, 18.56% and 58.58% respectively, indicating that the spatiotemporal migration of workloads can significantly reduce the total operating costs of data center clusters, and the spatial migration of interactive workloads is more effective in reducing costs.

[0177] In addition, both scenarios 2 and 4 have serious renewable energy curtailment costs, while scenarios 1 and 3 significantly reduce renewable energy curtailment costs through spatial migration of interactive workloads.

[0178] Among them, the superimposed effect of time and space migration in scenario 1 is effective in improving the renewable energy consumption rate, while scenario 4 does not consider time and space migration and cannot adjust the load level of the data center cluster, resulting in very high renewable energy reduction costs and energy consumption costs of the data center cluster.

[0179] In general, the spatial and temporal migration of workloads in data center clusters can significantly improve the renewable energy consumption rate and reduce the energy consumption cost of data center clusters. According to the real-time output of renewable energy, the flexible transfer of workloads in data center clusters can be achieved through the spatial migration of interactive workloads, which can greatly improve the renewable energy consumption rate of data center clusters. In addition, the temporal migration of batch workloads can respond to changes in electricity prices and reduce the energy consumption cost of data center clusters.

[0180] In summary, the online scheduling method and system of a data center cluster in the present invention significantly reduces the calculation time, realizes real-time approximate optimal matching of workload migration and renewable energy power generation, improves the new energy consumption rate in multiple regions, and effectively reduces the operating cost of the entire data center cluster. It is in line with the current development direction of distribution network optimization and scheduling, and plays a vital role in promoting the transformation of energy structure and accelerating the consumption of renewable energy.

[0181] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0182] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0183] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0184] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0187] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A data center cluster online scheduling method, characterized in that: The following steps are involved: Based on the spatiotemporal migration characteristics of workloads in data centers, a spatiotemporal migration model of data center workloads and a data center energy consumption model are constructed; Based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combined with the operation model of distributed equipment in the data center, a data center cluster optimization scheduling model is constructed that takes into account the electricity purchase cost, equipment operation cost and load migration cost; The constructed data center cluster optimization scheduling model is expressed as a Markov decision process, and by designing the state space, action space and reward function of the intelligent agent, the advanced deep reinforcement learning algorithm is used to realize the online scheduling of the data center cluster.

2. The data center cluster online scheduling method according to claim 1, characterized in that: Data center workload time and space migration model: Constraints associated with interactive loads: in, Indicates at time From Terminal Server Assign to data center interactive workloads, Indicates at time Reaching the Terminal Server The amount of interactive workload; Constraints associated with batch workloads: in, Indicates at time Arrival at the data center Batch workload volume , In the data center Transfer to time Batch processing workload The amount of The relationship between the minimum active servers and the interactive workload processed is expressed as: in, Indicates at time Time Data Center The minimum number of active servers serving interactive workloads in Indicates data center The average service rate of the server, represents the tolerable service delay for interactive workloads; The relationship between the lowest level of active servers and the processed batch workload is expressed as: in, Indicates at time Time Data Center Serving batch workloads The number of redundant active servers; Distribute interactive workloads from a terminal server to a data center and shift them to time slots One of the batch workloads and the active server is non-negative, as follows: in, Indicates time Time Data Center Medium service interactive workload / batch workload The minimum number of active servers, time Time Data Center Medium service interactive workload / batch workload The number of redundant active servers, Indicates serving interactive workloads / batch workloads The number of servers; The total number of active servers cannot exceed the total number of servers: in, For Data Center The total number of servers in; Range of active servers operating at peak power: Data center energy consumption model: in, Indicates at time Used to ensure data center The power consumption of the server equipment with the lowest quality of service in Indicates at time Data Center The power consumption of redundant server equipment for higher quality of service, For the power consumption of other devices, is the power consumption of the cooling system.

3. The data center cluster online scheduling method according to claim 2, characterized in that: In time Used to ensure data center Power consumption of server equipment with the lowest quality of service in for: in, is the rated power consumption of each active edge switch / aggregation switch / core switch, is the number of edge switches / aggregation switches / core switches / servers, Idle / peak power for each active server.

4. The data center cluster online scheduling method according to claim 2, characterized in that: In time Data Center Power consumption of redundant server equipment for higher quality of service for: in, is the rated power consumption of each active edge switch / aggregation switch / core switch, is the number of edge switches / aggregation switches / core switches / servers, Idle / peak power for each active server.

5. The data center cluster online scheduling method according to claim 2, characterized in that: Power consumption of cooling system for: in, is the empirical constant of the cooling system, To dissipate heat.

6. The data center cluster online scheduling method according to claim 1, characterized in that: The data center cluster optimization scheduling model is as follows: The energy cost includes the electricity purchase cost from the external electricity market. and traditional generator operating costs , Indicates at time China-Israel The spot price of the energy market is calculated, and the workload migration cost represents the cost of transferring the workload. The number of It means from The migration cost factor for the exported workload unit.

7. The data center cluster online scheduling method according to claim 6, characterized in that: The operating model of conventional generator sets in data centers is: in, and express Conventional generators in The maximum and minimum power outputs, and It is an indicator generator A binary variable indicating whether it is enabled. is the slope coefficient; and yes Medium generator The minimum on-time and minimum off-time, and yes Medium generator The actual on-time and off-time of Start-up and shutdown costs and operating costs: in, is the cost coefficient of the generator output, It represents the square of the generator output. represents the cost of starting and stopping a generator set once, It is the 0-1 variable status of the generator start and stop; The operation model of the energy storage system in the data center is: make and The energy storage system The maximum and minimum capacity in It's time The storage capacity of the energy storage system; The amount of electricity during the charging and discharging process of the energy storage system: Make sure that the storage level at the end of the operating day is at the same level as at the beginning, to guarantee that the storage system has the same energy exchange capacity for each day: The interconnection line constraints between the data center and the upper power grid are: The power generated by renewable energy is expressed as: The power balance of the system is as follows: in, Indicates time slot The power generated in Indicates time slot The power consumed in.

8. The data center cluster online scheduling method according to claim 1, characterized in that: The constructed data center cluster optimization scheduling model is expressed as a Markov decision process as follows: In any time slot, in order to find the state To Action The mapping relationship of Represents the conditional probability distribution of each action given the current state; When known When , the reward value related to the objective function can be obtained , which is used to guide the training of the intelligent agent, and the gradient ascent method is used to update the action strategy to maximize the reward value, that is: To do this, define the state value function To measure the quality of the neural network after parameterization; The goal of the agent during training is to select the best strategy and maximize the state value function, namely: Among them, the critic network is used to simulate the real state value function, and the actor network is used to finally train a deterministic strategy. The update strategies of the two networks are expressed as follows: in, Represents the current actual state value function, Set the parameters required for the neural network model; Finally, the network parameters in the critic network and actor network are adjusted by soft update. To update, that is: in, is a hyperparameter that updates the desired setting.

9. The data center cluster online scheduling method according to claim 8, characterized in that: The action space of the agent: in, The photovoltaic, wind power, diesel generators and energy storage configured in each data center are t The dispatch output of the time period, Processing interactive workloads in the data center t The number of servers powered on during the time period. for t The processing capacity of batch processing workloads during the period, For each data center t The amount of load transmitted in the optical fiber link during the time period; The state space of the agent: in, For Data Center t The interactive task load carried by the time period, Energy storage in data centers t The state of charge during the time period, For data centers t The amount of batch load remaining in the period, t The current operation period of the data center; Training environment of the agent: During offline training, the agent's actor network gives the decision output of each data center based on the state space. The power flow model of the distribution network system, as the environment, gives the operating cost, network loss, etc. as reward values ​​after obtaining the injected power of the nodes where each data center is located, and then feeds them back to the agent and transfers to the next state. The critic network evaluates this action and guides the agent's action at the next moment. The agent continuously adjusts its actions through training until the end of the training round. The agent’s reward function: in, is the penalty coefficient, Penalty for energy storage state of charge out of bounds, Quality of service penalty for interactive loads, Penalty for batch tasks not completed in time, Penalty for exceeding the limit of server number.

10. A data center cluster online scheduling system, characterized in that: include: A construction module, based on the spatiotemporal migration characteristics of workloads in the data center, constructs a spatiotemporal migration model of data center workloads and a data center energy consumption model; The scheduling module, based on the constructed data center workload spatiotemporal migration model and data center energy consumption model, combines the operation model of distributed equipment in the data center, and constructs a data center cluster optimization scheduling model that takes into account the power purchase cost, equipment operation cost, and load migration cost; The output module expresses the constructed data center cluster optimization scheduling model as a Markov decision process, and realizes the online scheduling of the data center cluster by designing the state space, action space and reward function of the intelligent agent and adopting the advanced deep reinforcement learning algorithm.

Images