Energy scheduling evaluation method based on data center source network load storage integration

Abstract

The invention discloses an energy scheduling evaluation method based on source-network-load-storage integration of a data center. The method comprises the following steps: step 1, selecting key indexes of each unit according to a target to be realized of source-network-load-storage integration; step 2, acquiring key features of a source-grid-load-storage integrated project according to the constructed index system so as to research core contents of a micro-grid dispatching optimization model in a data center; 3, establishing a data center micro-grid dispatching model based on a Markov decision process; and step 4, based on the improved TD3 algorithm, performing cyclic solution on the data center source network load storage integrated economic dispatching method to achieve an optimal strategy. According to the method provided by the invention, the influence of multiple factors of the source-network-load-storage integrated project is comprehensively considered, the economic characteristics are mainly analyzed, the effective analysis of the source-network-load-storage integrated project can be comprehensively realized, and technical guidance can be provided for the construction and development of the source-network-load-storage integrated project of the data center.

Description

Technical Field

[0001] The present invention belongs to the technical field of source-network project evaluation, and relates to an energy scheduling evaluation method based on the integration of source, network, load, and storage in a data center. Background Art

[0002] With the continuous growth of global energy demand, the energy production and consumption patterns are facing profound changes. The wide application of renewable energy has brought unprecedented challenges to the traditional power system. As an important infrastructure in modern information society, the energy consumption of data centers is increasing day by day. How to efficiently and stably manage energy has become a difficult problem to be solved. For this reason, the integration of source, network, load, and storage has emerged as the times require and has become an important means to optimize energy utilization and improve system operation efficiency.

[0003] The data center microgrid, as an independent power system operating within a local area, can achieve efficient energy utilization, and can also provide load regulation during peak power demand and balance through energy storage devices when power supply is insufficient. Therefore, the application of microgrid technology in data center energy management has important prospects. However, the coordinated scheduling of various types of energy in the data center microgrid, how to maximize economic benefits while ensuring system stability, remains a technical problem. Summary of the Invention

[0004] To solve the above technical problems, the object of the present invention is to provide an energy scheduling evaluation method based on the integration of source, network, load, and storage in a data center.

[0005] The present invention provides an energy scheduling evaluation method based on the integration of source, network, load, and storage in a data center, including:

[0006] Step 1: Establish a mathematical model of distributed units in the data center microgrid;

[0007] Step 2: Establish an objective function to be optimized and constraint conditions;

[0008] Step 3: Establish a data center microgrid scheduling model based on the Markov decision process;

[0009] Step 4: Perform iterative solution on the data center microgrid scheduling model based on the improved TD3 algorithm to achieve the optimal strategy.

[0010] An energy scheduling evaluation method based on the integration of power generation, grid, load, and energy storage in a data center constructs a system for the efficient energy conservation and low cost of the integration project of power generation, grid, load, and energy storage. Based on the constructed system, the method further expands and optimizes the solution of the power generation, grid, load, and energy storage project strategy based on the RC-TD3 algorithm. The method provided by the present invention comprehensively considers the influence of multiple factors in the integration project of power generation, grid, load, and energy storage, focuses on the analysis of economic characteristics, can comprehensively realize the effective analysis of the integration project of power generation, grid, load, and energy storage, and has a certain degree of integrity and feasibility. The system has strong logic and can provide technical guidance for the construction and development of the integration of power generation, grid, load, and energy storage in the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic flowchart of the energy scheduling evaluation method for the integration of power generation, grid, load, and energy storage in a data center according to an embodiment of the present invention;

[0012] Figure 2 It is a flowchart for solving the data center microgrid scheduling model based on the improved TD3 algorithm. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0013] As Figure 1 shown, an energy scheduling evaluation method based on the integration of power generation, grid, load, and energy storage in a data center according to the present invention includes:

[0014] Step 1: Establish a mathematical model of distributed units in the data center microgrid.

[0015] Specifically, the distributed units of the data center microgrid include: a photovoltaic power generation unit, a wind turbine power generation unit, an energy storage unit, and a shiftable load unit.

[0016] Photovoltaic power generation is a technology that directly converts light energy into electrical energy through the photovoltaic effect of semiconductors, mainly composed of three parts: a photovoltaic panel, a controller, and an inverter. Due to its long service life and clean and pollution-free characteristics, photovoltaic power generation has developed rapidly in recent years and has become an indispensable part of the new power system. Among them, the photovoltaic array is the core device of photovoltaic power generation, and the output power of the photovoltaic array shows strong nonlinearity and is closely related to meteorological conditions such as irradiation intensity and ambient temperature.

[0017] (1) The mathematical model of the photovoltaic power generation unit is:

[0018] T c = T a + 0.0138[1 + 0.031T a (1 - 0.042V w )G ING

[0019] In the formula: T c is the surface temperature of the photovoltaic cell, Ta is the ambient temperature, V W is the wind speed in the environment, G ING is the irradiance intensity on the surface of the photovoltaic cell.

[0020]

[0021] In the formula: P PV is the output power of the photovoltaic cell when the irradiance intensity is G ING ; G STC is the irradiance intensity under standard test conditions; P STC is the maximum output power of the module under standard test conditions; k is the power temperature coefficient; T c is the surface temperature of the photovoltaic cell; T r is the reference temperature.

[0022] Wind power generation technology mainly uses wind energy to drive the blades of a wind turbine to rotate, and then drives a generator to convert mechanical energy into electrical energy.

[0023] (2) The mathematical model of the wind turbine generator unit is:

[0024]

[0025] In the formula: v in is the cut-in wind speed, v r is the rated wind speed, v out is the cut-out wind speed, P r is the rated power, P WT is the output power of the wind turbine.

[0026] In recent years, the penetration rate of renewable energy in the power grid has been continuously increasing. However, the output of renewable energy has strong intermittency and uncertainty. Therefore, it is very necessary to use energy storage devices as response resources. When the output of renewable energy is greater than the load demand, the energy storage is charged to absorb renewable energy; when the output of renewable energy is less than the load demand, the energy storage discharges to support the load power consumption. At present, the battery is the energy storage device with the most mature technology. During the demand response and the daily operation of the data center, there will inevitably be situations of load surplus or shortage. The battery plays a buffering role for the data center and is an emergency electrical energy storage device.

[0027] (3) The mathematical model of the energy storage unit is:

[0028]

[0029] Among them, SOC t is the state of charge of the energy storage unit at time t, SOC t-1 is the state of charge of the energy storage unit at time t-1, σ is the weight coefficient, P ch(t) is the charging power at time t, ξ ch is the efficiency of electric energy conversion during charging, P dis (t) is the discharging power at time t, ξ dis is the efficiency of electric energy conversion during discharging, E es is the total energy storage capacity;

[0030] In addition, to improve the generalization ability of the network model obtained in this study, when training the network model, the initial state of charge of the energy storage unit satisfies the expectation of μ soc , and the variance is σ soc , and the probability density function is:

[0031]

[0032] where x represents the state of charge of the energy storage unit; μ soc = 0.1, σ soc = 0.5.

[0033] In traditional power system dispatching, when there is a power shortage, system dispatchers generally directly cut off some loads to ensure the stable operation of the power system. Although this processing method can quickly alleviate the power shortage problem, it does not take into account the importance of electricity consumption on the demand side and often causes losses on the demand side. Shiftable loads refer to loads that can obey system dispatching and change the electricity consumption time when there is a peak electricity consumption or an emergency in the power system. Shiftable loads are important dispatching units in the current economic dispatching of microgrids. Reasonably arranging the electricity consumption time of shiftable loads can reduce the electricity purchase cost of microgrids and achieve "peak shaving and valley filling" of the net load of microgrids.

[0034] (4) The mathematical model of the shiftable load unit is:

[0035]

[0036] In the formula: t start , t end respectively represent the start time and end time of dispatching, ξ(t) represents the state of the shiftable load at time t, 0 represents no response delay, and 1 represents no response delay; represents the electricity consumption power of the shiftable load at time t, represents the total electricity demand of the shiftable load within the entire dispatching time range.

[0037] Step 2: The main objective of the present invention is to utilize the precise control of the charging and discharging power of the energy storage unit and the electricity consumption time of the shiftable load on the demand side in the distributed economic unit dispatching strategy of the data center microgrid to achieve the maximum consumption of renewable energy generation and reduce the daily operating cost of the data center microgrid. Therefore, an objective function to be optimized and constraint conditions are established, specifically as follows:

[0038] Step 2.1: Establish the following optimization objective function:

[0039] f = min(f 1 , f 2 )

[0040] Where: f 1 is the daily operating cost function of the data center microgrid, and f 2 is the renewable energy output consumption function within the data center microgrid.

[0041] The daily operating cost of the data center microgrid mainly includes the operating cost of distributed units and the daily power purchase cost of the microgrid. Therefore, the daily operating cost function of the data center microgrid can be expressed as:

[0042]

[0043] Where: T is the total scheduling duration; N wind , N pv , N bat are the numbers of wind turbines, photovoltaic arrays, and batteries respectively; are the operating costs of the i-th wind turbine, photovoltaic array, and battery at time t respectively; is the electricity quantity purchased by the microgrid from the main grid at time t; price t is the real-time electricity price of the main grid at time t.

[0044] Consuming renewable energy mainly means that when there is still a surplus after the renewable energy output in the data center microgrid is used for fixed loads, it should be consumed through shiftable load power consumption or energy storage charging. The renewable energy output consumption function can be expressed as:

[0045]

[0046]

[0047] Where: is the photovoltaic power generation at time t, is the wind turbine power generation at time t; is the charge / discharge power of the energy storage at time t, greater than 0, energy storage charging; less than 0, energy storage discharging; represents the unconsumed renewable energy within the microgrid system at time t; are the power consumptions of the fixed load and shiftable load at time t respectively.

[0048] Step 2.2: To ensure the safe and stable operation of the data center microgrid model, it is necessary to constrain some physical quantities. During the operation, the power conservation should be maintained at all times, and the following power constraints are established:

[0049]

[0050] The electric power of the energy storage element is maintained below the maximum electric power of the energy storage unit, and the following constraints on the electric power of the energy storage element are established:

[0051]

[0052] In the formula: is the maximum electric power of the energy storage unit.

[0053] To extend the service life of the energy storage unit, the charge of the energy storage unit should be maintained within a reasonable range during use, that is, the upper and lower limits of the energy storage unit are constrained, expressed as:

[0054] SOC min ≤SOC t ≤SOC max

[0055] In the formula: SOC t is the state of charge of the energy storage unit at time t, SOC max 、SOC min are the upper and lower limits of the charge of the energy storage unit respectively.

[0056] To prevent the shiftable load from consuming too much power at the same time and affecting the safe and stable operation of the microgrid, the power consumption of the shiftable load at each moment is constrained, expressed as:

[0057]

[0058] In the formula, is the upper limit of the power consumption of the shiftable load.

[0059] Step 3: Establish a data center microgrid scheduling model based on the Markov decision process. The Markov decision process is a mathematical model that describes the problem of action selection in a finite and observable state space. Markov sequential decision-making means that the decision-maker selects an action to execute from the available action set according to the observed state information at each moment, and the state change of the system at the next moment is random. After the system state changes, the decision-maker continues to observe and selects the next action, and this process is repeated. The Markov sequential decision process can be represented as a five-tuple {S, A, P, R, γ}, where S represents the state set, indicating the possible states of the system; A represents the action set, indicating the actions that the system can select; P represents the state transition matrix, indicating the probability that the system transitions from one state to another; R represents the reward function, indicating the reward value after taking a certain action. γ represents the attenuation factor, indicating the impact of future rewards on the current moment. The Markov decision process can be used to solve most uncertain continuous decision problems. Its core idea is to calculate and select the optimal strategy of the system at each moment through dynamic programming to maximize the overall benefit. The research on the economic scheduling strategy of the data center microgrid is a typical continuous decision problem. Therefore, the present invention transforms the data center microgrid optimal scheduling problem into a Markov sequential decision process for optimal solution.

[0060] Under the framework of the Markov decision process, the design of the state-action space and the reward function are the keys to determining the quality of the final strategy. Therefore, the present invention conducts a detailed design of the state-action space and the reward function for the data center microgrid scheduling model based on the Markov decision process. Specifically:

[0061] Step 3.1: The state space refers to the environmental information where the agent is located, which should include all variable information that may affect the agent's decision-making. The state information provided by the data center to the agent should include the current moment, photovoltaic output, wind turbine output, fixed load power, shiftable load power, real-time electricity price, and the state of charge of the energy storage unit. The state space is defined as follows:

[0062]

[0063] In the formula: t ∈ {1, 2,... 24} is the current moment information.

[0064] Step 3.2: The action space refers to all possible actions that the agent can take. In the present invention, the action information of the agent is the scheduling strategy of the data center. The economic scheduling of the data center microgrid is mainly achieved by adjusting the charging or discharging actions of the energy storage unit and the power consumption time of the shiftable load. The action space is defined as follows:

[0065]

[0066] In the formula: represents the charging power of the energy storage unit at time t; represents the power consumption of the shiftable load at time t.

[0067] Step 3.3: Design the reward function:

[0068] To ensure the feasibility of the present invention, the reward function must be consistent with the predetermined optimization goal of the agent, ensuring that the agent can be correctly guided to achieve the established learning goal. The optimization goals of the economic dispatch of the distributed units in the data center microgrid include absorbing the output of renewable energy and reducing the daily operating cost of the microgrid. Therefore, the reward function of the dispatch should consist of two parts.

[0069] (1) Design the reward function for the optimization goal of absorbing the output of renewable energy. If the power generation power of renewable energy in a certain period is greater than the fixed load power, the energy storage unit should charge to absorb the excess output of renewable energy. At this time, this part of the reward function is expressed as:

[0070]

[0071] If the output of renewable energy in the current period is less than the fixed load power, the energy storage should choose to discharge to supply the fixed load. At this time, this part of the reward function is expressed as:

[0072]

[0073] (2) Optimize the power purchase cost for each period. Then the reward function of the daily power purchase cost is expressed as:

[0074]

[0075] Wherein, is the optimized new power demand.

[0076] (3) To prevent the power of the energy storage unit from exceeding the critical value, construct the reward function as:

[0077]

[0078] Superimpose the reward processes of the three parts into the total reward function, and add the corresponding weight coefficients according to the weight relationship, which is expressed as:

[0079]

[0080] In the formula: α, β, are the weight coefficients of the three reward functions.

[0081] Step 4: Perform iterative solution on the data center microgrid dispatch model based on the improved TD3 algorithm to achieve the optimal strategy.

[0082] In step 4, the TD3 algorithm uses a double critic network. When updating the network, the smaller of the Q-values output by the two networks is used to calculate the target value function. This method can effectively solve the overestimation problem and can be expressed as:

[0083] y t = r(s t , a t ) + γ * min(Q′Q 1 (s t+1 , a t+1 ), Q′ 2 (s t+1 , a t+1 ))

[0084] To increase the stability of the algorithm, the TD3 algorithm adds noise to the output action of the target actor network, which can be expressed as:

[0085] a t+1 = μ′(s t+1 |θ μ ′) + ε, ε ∼ clip(N(0, σ), -c, c)

[0086] The present invention improves the TD3 algorithm and uses the RC-TD3 algorithm to improve the learning efficiency. An additional experience pool is added, and different experience pools store rewards of different importance. Then, a weighted sampling strategy is used to make the experiences of high importance be used more frequently to improve the learning efficiency.

[0087] As Figure 2 shown, the specific process of training a neural network using the RC-TD3 algorithm is as follows:

[0088] Step 4.1: Initialize the network parameters, the number of episodes n, the number of steps t, the state s t and the maximum capacities of the two experience pools;

[0089] Step 4.2: Input the state s t into the actor network, and output the action a t ;

[0090] Step 4.3: After taking the action, interact with the environment to obtain the reward signal r t and enter the next state s t+1 ;

[0091] Step 4.4: According to the reward value, store the experience sample (s t , a t , r t , s t+1 ) in the experience pool;

[0092] Step 4.5: Update the number of steps, and repeat Steps 4.2 - 4.4 until the experience pool is full;

[0093] Step 4.6: Draw a batch of samples with a total amount of M from the two experience pools in proportion;

[0094] Step 4.7: Calculate the objective function f 1 and f 2 ;

[0095] Step 4.8: Calculate the error between the optimization objective and the ideal value, and update the critic network;

[0096] Step 4.9: Calculate the policy gradient and update the actor network; Let be the policy gradient, and r t be the reward signal;

[0097] Step 4.10: Update the target network parameters;

[0098] Step 4.11: Repeat Steps 4.2 - 4.10 until the set maximum number of iteration rounds or steps is reached to obtain the optimal energy scheduling strategy.

[0099] The above are only the preferred embodiments of the present invention and are not intended to limit the idea of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An energy scheduling evaluation method based on data center source-grid-load-storage integration, characterized in that: include: Step 1: Establish a mathematical model of the distributed units in the data center microgrid; Step 2: Establish the objective function to be optimized and the constraints; Step 3: Establish a data center microgrid scheduling model based on Markov decision process; Step 4: Based on the improved TD3 algorithm, the data center microgrid scheduling model is cyclically solved to achieve the optimal energy scheduling strategy.

2. The energy dispatching and evaluation method based on data center source-grid-load-storage integration as claimed in claim 1 is characterized in that: The distributed units of the data center microgrid include: photovoltaic power generation unit, wind turbine power generation unit, energy storage unit and movable load unit; (1) The mathematical model of the photovoltaic power generation unit is: T c =T a +0.0138[1+0.031T a ](1-0.042V w )G ING Where: T c is the surface temperature of the photovoltaic cell, T a is the ambient temperature, V W is the wind speed in the environment, G ING is the irradiance intensity on the surface of the photovoltaic cell; Where: P PV The radiation intensity is G ING When G STC is the irradiation intensity under standard test conditions; P STC is the maximum output power of the component under standard test conditions; k is the power temperature coefficient; T c is the surface temperature of the photovoltaic cell; T r is the reference temperature; (2) The mathematical model of the wind turbine power generation unit is: Where: v in is the cut-in wind speed, v r is the rated wind speed, v out is the cut-out wind speed, P r is the rated power, P WT is the output power of the fan; (3) The mathematical model of the energy storage unit is: Among them, SOC t is the state of charge of the energy storage unit at time t, SOC t-1 is the state of charge of the energy storage unit at time t-1, σ is the weight coefficient, P ch (t) is the charging power at time t, ξ ch is the efficiency of energy conversion during charging, P dis (t) is the discharge power at time t, ξ dis is the efficiency of energy conversion during discharge, E es is the total energy storage capacity; The initial charge state of the energy storage unit meets the expectation of μ soc , with variance σ soc , the probability density function is: Where x represents the state of charge of the energy storage unit; μ soc =0.1,σ soc =0.5; (4) The mathematical model of the translatable load unit is: Where: t start , t end They represent the start time and end time of scheduling, ξ(t) represents the load state that can be translated at time t, 0 represents response delay, and 1 represents no response delay; represents the power consumption of the load that can be shifted at time t, It represents the total electricity demand of the load that can be shifted within the entire scheduling time range.

3. The energy dispatching and evaluation method based on data center source-grid-load-storage integration according to claim 2, wherein step 2 is specifically: Step 2.1: Establish the following optimization objective function: f=min(f1,f2) Where: f1 is the daily operation cost function of the data center microgrid, f2 is the renewable energy output consumption function in the data center microgrid; Where: T is the total scheduling time; N wind 、N pv 、N bat are the number of wind turbines, photovoltaic arrays and batteries respectively; are the operating costs of the i-th wind turbine, photovoltaic array, and battery at time t, respectively; is the amount of electricity purchased by the microgrid from the main grid at time t; price t is the real-time electricity price of the main power grid at time t; Where: is the photovoltaic power generation at time t, is the wind turbine power generation at time t; is the charging / discharging power of energy storage at time t, Greater than 0, energy storage charging; Less than 0, energy storage discharge; It represents the amount of renewable energy not absorbed in the microgrid system at time t; are the power consumption of the fixed load and the movable load at time t respectively. Step 2.2: During operation, power conservation must be maintained at all times and the following power constraints must be established: The electric power of the energy storage element is kept below the maximum electric power of the energy storage unit, and the following constraints on the electric power of the energy storage element are established: Where: is the maximum electrical power of the energy storage unit; In order to extend the service life of the energy storage unit, the power of the energy storage unit should be kept within a reasonable range during use, that is, the upper and lower limits of the energy storage unit should be constrained, expressed as: SOC min ≤SOC t ≤SOC max Where: SOC t is the state of charge of the energy storage unit at time t, SOC max , SOC min They are the upper and lower limits of the energy storage unit’s power respectively; In order to prevent the power consumption of the movable load from being too high at the same time and affecting the safe and stable operation of the microgrid, the power consumption of the movable load at each moment is constrained, which can be expressed as: In the formula, It is the upper limit of the power consumption of the movable load.

4. The energy dispatching and evaluation method based on data center source-grid-load-storage integration according to claim 3, wherein step 3 is specifically: Step 3.1: The state space is defined as follows: Where: t∈{1, 2, ...24} is the current time information; Step 3.2: The action space is defined as follows: Where: Represents the charging power of the energy storage unit at time t; Represents the electrical power of the load that can be translated at time t; Step 3.3: Design the reward function: (1) Design a reward function for the optimization goal of absorbing renewable energy output. If the renewable energy power generation power is greater than the fixed load power in a certain period of time, the energy storage unit should be charged to absorb the excess renewable energy output. At this time, this part of the reward function is expressed as: If the output of renewable energy in the current period is less than the fixed load power, the energy storage should choose to discharge for the fixed load. The reward function of this part is expressed as: (2) Optimizing the electricity purchase cost for each period, the daily electricity purchase cost reward function is expressed as: in, The new power requirement after optimization; (3) In order to prevent the power of the energy storage unit from exceeding the critical value, the reward function is constructed as follows: The three parts of the reward process are superimposed as the total reward function, and the corresponding weight coefficient is added according to the weight relationship, which is expressed as: Where: α, β, are the weight coefficients of the three reward functions.

5. In the energy dispatching and evaluation method based on data center source-grid-load-storage integration as claimed in claim 4, the TD3 algorithm in step 4 adopts a double-layer critic network, and when updating the network, the smaller of the Q values ​​output by the two networks is used to calculate the objective value function, which is specifically: Step 4.1: Initialize network parameters, round number n, step number t, state s t and the maximum capacity of the two experience pools; Step 4.2: Set the state s t Input into the actor network, output action a t ; Step 4.3: After taking an action, interact with the environment to get a reward signal r t And enter the next moment state s t+1 ; Step 4.4: According to the reward value, the experience samples (s t ,a t ,r t ,s t+1 ) is stored in the experience pool; Step 4.5: Update the number of steps and repeat steps 4.2-4.4 until the experience pool is full; Step 4.6: Draw a batch of samples with a total amount of M from the two experience pools in proportion; Step 4.7: Calculate the objective functions f1 and f2; Step 4.8: Calculate the error between the optimization objective and the ideal value and update the critic network; Step 4.9: Calculate policy gradient and updating the actor network; is the policy gradient, r t For reward signals; Step 4.10: Update the target network parameters; Step 4.11: Repeat steps 4.2-4.10 until the maximum number of iterations or steps is reached to obtain the optimal energy scheduling strategy.

Images