A method, system, device and medium for data center energy security scheduling

Abstract

This invention provides a data center energy security scheduling method, system, equipment, and medium, belonging to the field of computer technology. Specifically, it models the problem as a constrained Markov decision process (CMDP), where the agent's objective is to minimize energy costs while adhering to a security constraint to ensure a continuous and sufficient power supply and prevent power shortages. A Lagrange-based constrained policy optimization algorithm is employed to learn a control strategy that conforms to predefined constraints. Under the premise of ensuring sufficient power supply to the data center, it effectively addresses the intermittency of renewable energy and electricity price fluctuations, achieving optimal operating costs and significantly improving the reliability and economy of microgrid operation.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and specifically relates to a data center energy security scheduling method, system, equipment and medium. Background Technology

[0002] As the world's reliance on cloud computing, artificial intelligence, and big data continues to grow, data centers have become a vital component of modern digital infrastructure. However, the expansion of data centers has driven a significant increase in energy consumption. The International Energy Agency (IEA) reports that data centers will account for nearly 1.5% of global electricity demand in 2024, and this figure is expected to rise further as data-intensive technologies continue to evolve. This surge presents a dual challenge: ensuring uninterrupted power supply while minimizing high electricity costs, as even brief outages can be catastrophic.

[0003] Energy dispatching in power system data centers requires the dynamic management of various energy sources to handle fluctuating workloads, minimize costs, and ensure uninterrupted power supply. This task is complicated by volatile electricity prices, intermittent renewable energy generation, and the need for real-time adaptability, necessitating a control framework that minimizes costs while ensuring power supply reliability.

[0004] Traditional methods for data center energy management utilize Model Predictive Control (MPC), which relies on explicit models for prediction and planning. However, its performance depends on prediction accuracy and cannot be adjusted to fluctuations in renewable energy generation or market prices. Deep Reinforcement Learning (DRL) has emerged as a model-free alternative, allowing agents to learn control strategies through interaction with the environment. However, DRL is typically framed as a Markov Decision Process (MDP), using penalties in the reward function to address security issues. When solving parameter optimization problems for multi-device coordinated scheduling in power systems, data centers, as core loads with high energy consumption and high power supply reliability requirements, have strict security requirements for uninterrupted and stable power supply. However, the intermittency of renewable energy and fluctuations in time-of-use electricity prices often introduce scheduling uncertainty, making it difficult to balance economic goals and security constraints in dynamic energy scheduling. Summary of the Invention

[0005] To address the shortcomings of existing deep reinforcement learning methods in data center optimization scheduling, this invention provides a data center energy security scheduling method, system, device, and medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A data center energy security scheduling method includes the following steps: An objective function for energy scheduling of a power system data center is constructed with the goal of minimizing total operating costs. The energy scheduling problem of the data center is modeled as a Conditional Markov Decision Process (CMDP), and the state space, control actions, immediate reward function, and cost function are defined based on the CMDP. A simulation environment for the CMDP problem is constructed based on historical operational data from the power system data center; a security-energy constraint algorithm is constructed based on a deep deterministic policy gradient framework; the security-energy constraint algorithm is used to train a reinforcement learning agent in the simulation environment to solve the CMDP and obtain an energy dispatching policy model. The system acquires real-time operational data from the power system data center, inputs this data into the energy dispatch strategy model, and solves for the optimal dispatch decision that satisfies the security constraints.

[0007] Preferably, the objective function for energy scheduling in the data center is: ; in, Operating costs for distributed generator sets; The financial impact of power exchange with the main grid. The operating costs of the energy storage system, For time steps, The number of distributed generator sets covered by the system. To optimize cycle duration, This is the index value for the distributed generator set. This is the generator set index.

[0008] Preferably, the state space includes the current renewable energy generation, load demand, the main grid interaction power at the previous time, and the current energy storage state of charge. Specifically, the system state at each time step t is represented as follows: , , Indicates time step Internal wind power output Indicates time step Internal photovoltaic power output Indicates time step Data center load power, This represents the power exchanged with the main grid at the previous time step. Indicates time step Electricity price, Indicates time The energy storage state of charge; The control actions include the output power of each distributed generator set, the charging and discharging power of the energy storage system, and the interaction power with the main grid, specifically: at each time step The control action is represented as ,in, Distributed generators during the time period Those who have made contributions Indicates the time period of the energy storage system Power command, Indicates the time period with the main power grid Net exchange power; The instant reward function is specifically as follows: ; The cost function is specifically as follows: ; in, This is the permissible power shortage threshold for data centers. Operating costs for distributed generator sets; The financial impact of power exchange with the main grid. This refers to the operating losses of the energy storage system.

[0009] Preferably, training the reinforcement learning agent using the security-energy constraint algorithm specifically includes the following steps: (1) Initialize the actor network with random weights With the network of critics ;in, Represents all possible states A set; Indicates possible actions gather; For strategy parameters; (2) Initialize the target network and and set , ; (3) Initialize the experience buffer And initialize the Lagrange multipliers =0; (4) For each training round arrive Perform steps (5)-(15): (5) Obtain the initial state based on the actual data trajectory. ; (6) For each training time step arrive Perform steps (7)-(14): (7) Select an action using the actor network. ; (8) Perform the action Observation Rewards Insufficient power generation costs And the next system state ; (9) Transition the state Store in experience buffer ; (10) Randomly select from the experience buffer Randomly select M state transitions ; (11) For each sampled state transition Calculate its target value: ; (12) Using the loss function Perform gradient descent: Update the network of critics ; (13) Using the policy gradient function Perform gradient ascent Update the actor network ; (14) Update the target actor-commentator network and : , ; (15) The calculation view window is Average power shortage: ;in, It is due to insufficient power supply; (16) Update the Lagrange multipliers: ; This is the permissible power shortage threshold for data centers; Training ends when the policy parameters converge with the Lagrange multipliers.

[0010] Preferably, the actor network and the commentator network updates follow a stochastic approximation criterion, i.e., the learning rate. and satisfy: .

[0011] Preferably, the energy scheduling strategy model is input with real-time data center operation data to solve for the optimal scheduling decision that satisfies security constraints, specifically including the following steps: The real-time operational data of the data center is preprocessed to obtain preprocessed state data; the standardized state vector is then input into the energy scheduling strategy model. The actor network infers the initial scheduling action based on the current state and the real-time updated Lagrange multipliers; the initial action is checked for physical constraints, and if the initial action violates the constraints, it is corrected to generate the final executable action; the final executable action is the optimal scheduling decision that satisfies the safety constraints. The DG units, energy storage systems, and main grid interfaces are scheduled based on the final executable actions.

[0012] This invention also provides a data center energy security scheduling system, specifically comprising: The objective construction module constructs an objective function for energy scheduling of the power system data center with the goal of minimizing total operating costs; the energy scheduling problem of the data center is modeled as a Conditional Markov Decision Process (CMDP), and the state space, control actions, immediate reward function, and cost function are defined based on the CMDP.

[0013] The model training module is used to construct a simulation environment for the CMDP problem based on historical operational data of the power system data center; to construct a security-energy constraint algorithm based on a deep deterministic policy gradient framework; and to train a reinforcement learning agent using the security-energy constraint algorithm in the simulation environment to solve the CMDP and obtain an energy dispatch policy model.

[0014] The scheduling module is used to acquire real-time operational data from the power system data center, input the real-time operational data into the energy scheduling strategy model, and solve for the optimal scheduling decision that satisfies the security constraints.

[0015] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps described in the data center energy security scheduling method.

[0016] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute the steps described in the data center energy security scheduling method.

[0017] The data center energy security scheduling method provided by this invention has the following beneficial effects: This invention focuses on maximizing long-term discount rewards under security constraints as its core optimization objective. By combining multi-device collaborative scheduling and time-of-use pricing adaptation, it achieves refined cost optimization, aiming to minimize operating costs while ensuring sufficient power supply. By formulating the problem as a Conditional Markov Decision Process (CMDP) problem, and distinguishing between security constraints and economic objectives through immediate reward and cost functions, it achieves a hard guarantee for data center power supply security. A simulation environment is constructed using real historical microgrid operating data, and an agent is trained based on this simulation data. This allows for accurate learning of the dynamic correlations and changing patterns of the scenario, improving the robustness and real-time adaptability of the scheduling strategy in real-world scenarios. By comprehensively handling security constraints and economic benefits, the invention enhances the effectiveness of data center optimized scheduling. Attached Figure Description

[0018] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a grid-connected data center microgrid model in an embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating the processing of the safety-constrained energy scheduling algorithm in an embodiment of the present invention.

[0021] Figure 3 This is a learning curve showing the percentage of insufficient operating costs and power during policy training in an embodiment of the present invention. Figure 3 (a) is the learning curve for the percentage of insufficient power; Figure 3 (b) is the learning curve for operating costs.

[0022] Figure 4 This is a performance comparison chart of the energy scheduling algorithm in this embodiment of the invention on the test set. Figure 4 (a) shows the average daily power shortage on the test set; Figure 4 (b) shows the average daily operating cost on the test set.

[0023] Figure 5 This invention provides a comparative analysis of the operation on a test day, including the dynamic changes in SOC (State of Charge) and electricity price fluctuations over 24 hours.

[0024] Figure 6 This is a flowchart of a data center energy security scheduling method according to the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0026] Example Based on a grid-connected data center microgrid system, it integrates diverse local energy assets and is connected to the public power grid via bidirectional interconnect lines, such as... Figure 1 As shown, this invention provides a data center energy security scheduling method, which specifically includes the following steps: Step 1: Construct the energy dispatch framework described in this system, aiming to optimize resource allocation within the microgrid, covering distributed generator sets ( ), photovoltaic system ( ), wind turbine ( ), energy storage system ( ), main power grid ( ) and load ( Various facilities, including [list of facilities]. At the beginning of each time step, control measures are implemented to coordinate energy dispatch within the data center microgrid. System development leads to [further details regarding the next time step]. The state is entirely determined by the current time step. The state of the system and the control measures implemented determine its development. This ensures the consistency and predictability of the system's development.

[0027] The primary objective is to minimize the total operating cost over the entire time period. The specific objective function is as follows: ; in, Operating costs for distributed generator sets; The financial impact of power exchange with the main grid. The operating costs of the energy storage system, For time steps, The number of distributed generator sets covered by the system, To optimize cycle duration, This is the index value for the distributed generator set.

[0028] Total operating costs are divided into three parts: operating costs of distributed generation (DG) units, financial impact of power exchange with the main grid, and operating losses of the energy storage system.

[0029] The cost of the DG unit is determined by a factor that depends on its time. power output The costs associated with distributed generation (DG) units are determined by a quadratic function that captures the complex relationship between power generation and expenditure. ; in, For the first Distributed generator sets in time Power output, , , Corresponding to the first The cost coefficients of the secondary, primary, and constant terms of the distributed generator set are used to characterize the quadratic relationship between the unit's output and operating costs.

[0030] This indicates that the cost associated with grid transactions at time t is based on the time-of-use (TOU) electricity price. Indicates the power of exchange. This indicates that the data center purchases electricity from the main grid. This indicates that the data center sells electricity back to the main grid and introduces a constant value between 0 and 1. The financial impacts related to power exchange with the main grid are as follows: ; in, This indicates the operating costs of the energy storage system. This represents the degradation cost coefficient of the energy storage system. The energy flux of the energy storage system within time t is expressed as: The operating cost of the energy storage system is expressed as: The following are specific constraints on data center energy dispatching systems: Demand for power supply gap The constraint (representing the difference between total electricity demand and supply) must always be non-positive to ensure system reliability. Specifically: ; The generation boundary constraints of distributed generator sets are as follows: ; Peak shaving constraints limit the speed of power adjustment to match the actual equipment capacity, specifically: ; ; Constraints on energy storage systems are introduced, emphasizing their dispatch potential, but operating costs are not considered. These include:

[0031] (1) Charging and discharging power constraints, specifically: ; (2) SOC state preservation constraints, specifically: ; (3) SOC limitation constraints to protect battery health and prevent overcharging or deep discharging, specifically: ; Main grid power trading constraints limit maximum imports or exports to within limits that comply with grid infrastructure constraints, specifically: .

[0032] Step 2: Model the data center energy scheduling problem as a Conditional Markov Decision Process (CMDP) problem, and define the optimization objective of minimizing operating costs while ensuring that power shortage costs do not exceed limits. Separate the security constraints of sufficient power supply from maximizing economic benefits. The goal is to obtain the optimal strategy, i.e., within a given time frame. Minimize internal operating costs.

[0033] Define the state space: at each time step The system state is represented as , Represents all possible states A set of. Indicates time step Internal wind power output Indicates time step Internal photovoltaic power output Indicates time step Data center load power, This represents the power exchanged with the main grid at the previous time step. Indicates time step Electricity price, Indicates time The energy storage state of charge.

[0034] Define the control action. Each time step... The control action is represented as , Indicates possible actions gather. Indicates the first Distributed generators during the time period active power, Indicates the time period of the energy storage system Power command, Indicates the time period with the main power grid The net exchange power.

[0035] Define the instant reward function as , denoted as The negative effect reflecting total operating costs is expressed as: ; Define the cost function as follows: ; in, It is due to insufficient power supply. This is the acceptable power shortage threshold for the data center. Setting it to 0 here means that any power shortage event is intolerable. =0 ensures punishment for any power shortage.

[0036] For any given policy π, the long-run discount reward function and long-term discounted electricity shortage cost function Represented as: ; The ultimate goal is to satisfy Under the conditions, find the way Optimal strategy for maximization ,Right now: .

[0037] Step 3: Use security reinforcement learning to solve the CMDP problem.

[0038] S31: Acquire historical operational data, including renewable energy generation, real-time load data, and time-of-use pricing. Construct a simulation environment based on the CMDP problem. The simulation environment simulates the dynamic characteristics of a real microgrid, providing immediate and reliable feedback signals for the agent's decision-making.

[0039] S32: Training is conducted based on a simulation environment. To obtain the optimal control strategy for the CMDP problem, the Lagrange relaxation method is used to transform the CMDP problem into an equivalent unconstrained optimization problem, specifically as shown below. The strategy parameters and Lagrange multiplier parameters are updated simultaneously at two time scales, so that the strategy minimizes the operating cost while ensuring that the power shortage cost does not exceed the limit, thereby providing a convergent strategy for subsequent online deployment. ; in, It is a parameterized strategy, Lagrange multipliers ≥0 is used to enforce the constraint cost function ≤0.

[0040] Constrained deep reinforcement learning methods with iterative primal-dual optimization are used to process CMDP with expectation constraints. Policy parameters It will be updated quickly to maximize... ,and By slowly adjusting the dual gradient to satisfy the constraints, it eventually converges to... The experience replay buffer Ω stores tuples. The cost will be insufficient. With economic benefits Separate. The commentator's update uses the Lagrange correction for the target:

[0041] ; Guided by the policy gradient ascent of the actor network, dual updates are based on the window. Average power generation deficit within : ; To ensure that the original dual update converges to the optimal policy and penalty pair Learning rate and The random approximation criterion must be followed, namely: .

[0042] S33: Based on a validated simulation environment, the agent is trained using a drive-safe-constrained energy scheduling algorithm, such as... Figure 2 As shown.

[0043] (1) Initialize the actor network with random weights With the network of critics .

[0044] (2) Initialize the target network and and set , .

[0045] (3) Initialize the experience buffer And initialize the Lagrange multipliers =0.

[0046] (4) For each training round arrive Execute (5)-(15): (5) Obtain the initial state based on the actual data trajectory. ; (6) For each training time step arrive Execute (7)-(14): (7) Select an action using the actor network. ; (8) Perform the action Observation Rewards Insufficient power generation costs And the next system state ; (9) Transition the state Store in experience buffer ; (10) Randomly select from the experience buffer Randomly select M state transitions ; (11) For each sampled state transition Calculate its target value: ; (12) Using the loss function Perform gradient descent: Update the network of critics ; (13) Using the policy gradient function Perform gradient ascent Update the actor network ; (14) Update the target actor-commentator network and : , ; (15) The calculation view window is Average power shortage ; (16) Update the Lagrange multipliers: .

[0047] The obtained Lagrange multipliers As the dual weights of the power shortage constraint, the constraint is internalized as a reward penalty and converged collaboratively with the policy parameters through primal-dual updates, ensuring the final policy... In satisfying To achieve optimal operating costs under the premise of [missing information].

[0048] S33: After training convergence, the learned policy is evaluated on a separate, unseen test dataset. This yields an energy scheduling policy model that automatically outputs the optimal scheduling decision that satisfies safety constraints based on real-time conditions.

[0049] A data center microgrid model is proposed to optimize energy dispatch. Three distributed generation units are characterized based on the parameters listed in Table 1. The energy storage system is configured with a 100 kW charge / discharge power limit, a 500 kWh rated capacity, and... With a charge / discharge efficiency of 0.9, its state of charge (SOC) is limited to between 20% and 80%. Grid interaction is restricted to a maximum import and export power of 200 kW, and the sales price is set at half the real-time purchase price. =0.5), to promote the use of local renewable energy. The experiment used a one-year dataset with an hourly time resolution, including renewable energy generation, load demand, and electricity price curves.

[0050] Table 1 Generator Unit Information To enable the energy dispatch algorithm to adapt to seasonal variations, the following data partitioning strategy was adopted: the data from the first three weeks of each month constituted the training set, while the remaining week served as the test set. During training, the initial state of charge (SOC) of the energy storage system was randomly initialized at each iteration to improve generalization ability. These algorithms were implemented in Python and PyTorch and underwent 4000 iterations of training, with each iteration containing... =1000 control cycles. The optimization parameters used are a discount factor γ = 0.995 and a learning rate ( , )=(0.01, 0.001), the experience replay pool capacity is 500,000 samples, the mini-batch size is 2048, and the maximum penalty coefficient is... =2000, and soft target update rate =0.005.

[0051] Performance evaluation is based on the Weighted Reward Energy Dispatch (WRED) method, with grid interaction as a decision variable for easy comparison. WRED formulates the energy dispatch problem as a Markov Decision Process (MDP), whose reward function includes a fixed penalty for power shortages (defined as:).

[0052] ; in,( ) is a fixed penalty coefficient. Two WRED variants were tested: WRED(1000), The value is 1000; and WRED(2000), its The value is 2000.

[0053] Strategy training, such as Figure 3 (a) and Figure 3(b) shows the learning curves for operating costs and power deficit percentages, respectively. All trained strategies exhibited high volatility in the initial phase, stabilizing after approximately 500 epochs. In the final 500 epochs, SCED 0 had an average operating cost of only $8901.81, lower than WRED's $9466.13 and $9640.16. Furthermore, SCED had the lowest average deficit rate at 1.18%, better than WRED's 1.90% and 1.57%, indicating a higher degree of compliance with safety constraints.

[0054] Strategy evaluation, such as Figure 4 (a) shows the average daily power shortage on the test set, indicating that SCED is more secure, reducing power shortages by 58.6% and 55.0% compared to WRED(1000) and WRED(2000), respectively. Figure 4 (b) shows the average daily operating costs on the test set, with WRED(1000) having the lowest cost, followed by WRED(2000), and SCED having the highest cost.

[0055] The WRED agent revealed problems with the fixed penalty mechanism: WRED (1000) reduced costs by tolerating more security breaches, while WRED (2000) increased security status to an average daily reduction of 8.1%, but at the cost of 9.5%, reflecting the sensitivity to adjustments. In contrast, SCED enforces security as a strict constraint, operating in a more secure environment, but at a higher cost. This demonstrates SCED's success in optimizing test set performance while prioritizing security, providing a reliable solution for critical energy management.

[0056] Figure 5 The dynamic changes in SOC (State of Charge) and electricity price fluctuations over a 24-hour period are illustrated. Compared to WRED(1000) and WRED(2000), the SCED algorithm has a shorter ESS (Energy Storage System) scheduling cycle, thus reducing battery losses. It also optimizes cost efficiency by matching charging to low-price periods and discharging during high-price periods. WRED(1000) and WRED(2000) exhibit more frequent SOC fluctuations, indicating poorer battery management. SCED achieves a significant improvement in cost-effectiveness by effectively coordinating the interactions between distributed generation, ESS, and the grid, minimizing dependence on expensive grid imports. WRED(1000) and WRED(2000) are more expensive due to suboptimal ESS scheduling and increased grid dependence during peak-price periods.

[0057] In the technical solution of this invention: by constructing the data center energy scheduling problem as a Conditional Markov Decision Process (CMDP) with hard safety constraints, and employing a safety reinforcement learning method combining Lagrange relaxation and primal-dual optimization, the safety constraints and economic objectives are clearly separated. This allows for dynamic adjustment of the constraint strength during training, ultimately obtaining a scheduling strategy that strictly meets power supply safety requirements while minimizing operating costs. Unlike existing technologies that merely add fixed penalties to the reward function and treat safety as a compromise soft constraint, this invention achieves an optimal energy scheduling model with guaranteed safety and direct deployment through dual multiplier updates, separate modeling of constraint costs, and a unique policy training process. This approach and corresponding training process constitute the core innovation of this invention, distinguishing it from traditional MPC methods and existing deep reinforcement learning methods.

[0058] The present invention also provides a data center energy security scheduling system, comprising: The objective construction module constructs an objective function for energy scheduling of power system data centers with the goal of minimizing total operating costs. The energy scheduling problem of data centers is modeled as a Conditional Markov Decision Process (CMDP), and the state space, control actions, immediate reward function, and cost function are defined based on the CMDP.

[0059] The model training module is used to construct a simulation environment for the CMDP problem based on historical operational data from the power system data center; to construct a security-energy constraint algorithm based on a deep deterministic policy gradient framework; and to train a reinforcement learning agent using the security-energy constraint algorithm in the simulation environment to solve the CMDP and obtain an energy dispatch policy model.

[0060] The scheduling module is used to acquire real-time operational data from the power system data center, input the real-time operational data into the energy scheduling strategy model, and solve for the optimal scheduling decision that satisfies the security constraints.

[0061] The modules in the aforementioned data center energy security scheduling system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0062] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a data center energy security scheduling method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0063] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the aforementioned method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a data center energy security scheduling method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0064] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0066] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0068] It should be noted that the above specific embodiments enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A data center energy security scheduling method, characterized in that, Includes the following steps: An objective function for energy scheduling of a power system data center is constructed with the goal of minimizing total operating costs. The energy scheduling problem of the data center is modeled as a Conditional Markov Decision Process (CMDP), and the state space, control actions, immediate reward function, and cost function are defined based on the CMDP. A simulation environment for the CMDP problem is constructed based on historical operational data from the power system data center; a security-energy constraint algorithm is constructed based on a deep deterministic policy gradient framework; the security-energy constraint algorithm is used to train a reinforcement learning agent in the simulation environment to solve the CMDP and obtain an energy dispatching policy model. The system acquires real-time operational data from the power system data center, inputs this data into the energy dispatch strategy model, and solves for the optimal dispatch decision that satisfies the security constraints.

2. The data center energy security scheduling method according to claim 1, characterized in that, The objective function for energy scheduling in the data center is specifically: ； in, Operating costs for distributed generator sets; The financial impact of power exchange with the main grid. The operating costs of the energy storage system, For time steps, The number of distributed generator sets covered by the system. To optimize cycle duration, For distributed generator set index value, This is the generator set index.

3. The data center energy security scheduling method according to claim 1, characterized in that, The state space includes the current renewable energy generation, load demand, the main grid interaction power at the previous time step, and the current energy storage state of charge. Specifically, the system state at each time step t is represented as follows: ,in, Indicates time step Internal wind power output Indicates time step Internal photovoltaic power output Indicates time step Data center load power, This represents the power exchanged with the main grid at the previous time step. Indicates time step Electricity price, Indicates time The energy storage state of charge; The control actions include the output power of each distributed generator set, the charging and discharging power of the energy storage system, and the interaction power with the main grid, specifically: at each time step The control action is represented as ,in, Indicates the first Distributed generators during the time period Those who have made contributions Indicates the time period of the energy storage system Power command, Indicates the time period with the main power grid Net exchange power; The instant reward function is specifically as follows: ; The cost function is specifically as follows: ; in, It is due to insufficient power supply. This is the permissible power shortage threshold for data centers. Operating costs for distributed generator sets; The financial impact of power exchange with the main grid. This refers to the operating losses of the energy storage system.

4. The data center energy security scheduling method according to claim 1, characterized in that, Training a reinforcement learning agent using the aforementioned safety-energy constraint algorithm specifically includes the following steps: (1) Initialize the actor network with random weights With the network of critics ;in, Represents all possible states A set; Indicates possible actions gather; For strategy parameters; (2) Initialize the target network and and set , ; (3) Initialize the experience buffer And initialize the Lagrange multipliers =0; (4) For each training round arrive Perform steps (5)-(15): (5) Obtain the initial state based on the actual data trajectory. ; (6) For each training time step arrive Perform steps (7)-(14): (7) Select an action using the actor network. ; (8) Perform the action Observation Rewards Insufficient power generation costs And the next system state ; (9) Transition the state Store in experience buffer ; (10) Randomly select from the experience buffer Randomly select M state transitions ; (11) For each sampled state transition Calculate its target value: ； (12) Using the loss function Perform gradient descent: Update the network of critics ; (13) Using the policy gradient function Perform gradient ascent Update the actor network ; (14) Update the target actor-commentator network and : , ; (15) The calculation view window is Average power shortage: ;in, It is due to insufficient power supply; (16) Update the Lagrange multipliers: ; This is the permissible power shortage threshold for data centers; Training ends when the policy parameters converge with the Lagrange multipliers.

5. A data center energy security scheduling method according to claim 4, characterized in that, The actor network and critic network updates follow a stochastic approximation criterion, i.e., the learning rate... and satisfy: .

6. A data center energy security scheduling method according to claim 1, characterized in that, The energy scheduling strategy model is input with real-time data center operation data, and the optimal scheduling decision that satisfies security constraints is obtained by solving the model. This process includes the following steps: The real-time operational data of the data center is preprocessed to obtain preprocessed state data; the standardized state vector is then input into the energy scheduling strategy model. The actor network infers the initial scheduling action based on the current state and the real-time updated Lagrange multipliers; the initial action is checked for physical constraints, and if the initial action violates the constraints, it is corrected to generate the final executable action; the final executable action is the optimal scheduling decision that satisfies the safety constraints. The DG units, energy storage systems, and main grid interfaces are scheduled based on the final executable actions.

7. A data center energy security scheduling system, characterized in that, include: The objective building module constructs an objective function for energy scheduling in a power system data center with the goal of minimizing total operating costs. The energy scheduling problem of the data center is modeled as a Conditional Markov Decision Process (CMDP), and the state space, control actions, immediate reward function and cost function are defined based on the CMDP. The model training module is used to construct a simulation environment for the CMDP problem based on historical operating data of the power system data center; to construct a security-energy constraint algorithm based on a deep deterministic policy gradient framework; and to train a reinforcement learning agent using the security-energy constraint algorithm in the simulation environment to solve the CMDP and obtain an energy dispatch policy model. The scheduling module is used to acquire real-time operational data from the power system data center, input the real-time operational data into the energy scheduling strategy model, and solve for the optimal scheduling decision that satisfies the security constraints.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 6.

Images