Energy system scheduling method and device, equipment and storage medium

Abstract

The invention belongs to the field of energy scheduling, and discloses an energy system scheduling method, device and equipment and a storage medium, and the method comprises the steps: constructing a mathematical model of a regional integrated energy system, and building an operation constraint model and supply and demand balance constraint of each component of the regional integrated energy system based on collected energy flow data of electric and cold energy flows; establishing a dynamic electricity price model based on the difference between the real-time net power grid demand and the average demand; a master-slave game model with a power grid as a leader and an energy storage operator and a user as followers is established, the energy storage operator and the user are in a cooperative game relationship, and the maximization of the overall income of the park is taken as a common target; and solving the master-slave game model by adopting a depth deterministic strategy gradient optimization algorithm, converging each main strategy to a game equilibrium point through multiple iterative learning, and outputting an optimal dynamic electricity price, an energy storage scheduling plan and a user load curve.

Description

Technical Field

[0001] This invention belongs to the field of energy dispatching, and particularly relates to an energy system dispatching method, apparatus, equipment and storage medium. Background Technology

[0002] With the continuous growth of energy demand and the large-scale integration of renewable energy, the dispatching methods of traditional energy systems face numerous challenges. Regional integrated energy systems can improve energy utilization efficiency by integrating multiple energy forms such as electricity, cooling, and heating, but issues such as multi-energy coupling and conflicts of interest among multiple stakeholders complicate system optimization and dispatching.

[0003] In related technologies, scheduling methods for integrated energy systems mainly fall into two categories: centralized optimization and distributed optimization. While centralized optimization methods can achieve global optimum, they require complete information from all stakeholders, leading to issues such as privacy protection and communication burden in practical applications. Distributed optimization methods, such as game theory, can protect the privacy of each stakeholder. However, existing research often employs traditional game models, such as Stackelberg games or Nash games, which struggle to effectively handle complex interactions among multiple stakeholders. This is especially true when some stakeholders have cooperative relationships, as traditional models cannot accurately describe such hybrid game structures.

[0004] Furthermore, existing methods for solving game equilibrium often employ traditional optimization algorithms such as KKT conditions and dual decomposition. These methods have limited ability to solve nonlinear and nonconvex problems and struggle with high-dimensional state spaces. Although some studies have attempted to introduce reinforcement learning algorithms, they mostly employ algorithms with discrete action spaces, such as Q-learning and DQN, which are less adaptable to scheduling problems with continuous action spaces. Summary of the Invention

[0005] In view of this, the present invention discloses an energy system scheduling method, apparatus, equipment and storage medium, which can solve the shortcomings of related technologies.

[0006] To achieve the above objectives, the present invention discloses the following technical solution:

[0007] According to a first aspect of the present invention, an energy system scheduling method is proposed, comprising: A mathematical model of a regional integrated energy system is constructed, and operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system are established based on the collected energy flow data of electricity and cooling energy. The regional integrated energy system includes a power grid, a photovoltaic power generation system, energy storage devices, a water-cooled storage system, and user terminals. Establish a dynamic electricity pricing model based on the difference between real-time net grid demand and average demand; A master-slave game model is established with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient, the energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system, and the users' strategy is to adjust the electricity consumption period of the transferable load. The master-slave game model is solved using a deep deterministic policy gradient optimization algorithm. Through multiple iterations, the strategies of each agent converge to the game equilibrium point, outputting the optimal dynamic electricity price, energy storage scheduling plan, and user load curve. The Actor network of the deep deterministic policy gradient optimization algorithm is used to generate deterministic action strategies for each agent based on the system state, and the Critic network is used to evaluate the long-term payoff Q value of the state-action pair.

[0008] According to a second aspect of the present invention, an energy system dispatching device is provided, applied to the server side of an air conditioning system, the device comprising: The first construction unit is to construct a mathematical model of the regional integrated energy system, and to establish operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system based on the collected energy flow data of electricity and cooling energy; wherein, the regional integrated energy system includes a power grid, a photovoltaic power generation system, an energy storage device, a water-cooled storage system, and a user terminal; The second building block: establishing a dynamic electricity price model based on the difference between real-time net grid demand and average demand; The third building block: Establish a master-slave game model with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient. The energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system. The users' strategy is to adjust the electricity consumption period of the transferable load. The solution unit employs a deep deterministic policy gradient optimization algorithm to solve the master-slave game model. Through multiple iterations, it learns to converge the strategies of each agent to the game equilibrium point, outputting the optimal dynamic electricity price, energy storage scheduling plan, and user load curve. The Actor network of the deep deterministic policy gradient optimization algorithm is used to generate deterministic action strategies for each agent based on the system state, while the Critic network is used to evaluate the long-term payoff Q-value of the state-action pair.

[0009] According to a third aspect of the present invention, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor implements the steps of the method as described in the first aspect by running the executable instructions.

[0010] According to a fourth aspect of the invention, a computer-readable storage medium is provided having computer instructions stored thereon that, when executed by a processor, implement the steps of the method as described in the first aspect.

[0011] As can be seen from the above technical solutions, the energy system scheduling method disclosed in this invention is as follows: On the one hand, by treating the power grid as the leader and energy storage operators and users as followers, with a cooperative game relationship among the followers, this hybrid game structure can more accurately describe the complex interactions among multiple entities in the actual system. It considers both the dominant position of the power grid and the win-win cooperation between energy storage operators and users, solving the problem that traditional single-game models struggle to handle hybrid game structures. On the other hand, by establishing a dynamic electricity price model based on the difference between real-time net grid demand and average demand, it can more sensitively reflect changes in supply and demand, guide users to rationally adjust their electricity consumption behavior, achieve peak shaving and valley filling, and improve system operating efficiency. Furthermore, addressing the difficulty of traditional optimization algorithms in handling high-dimensional continuous action spaces, this invention employs a deep deterministic policy gradient algorithm. By using an Actor-Critic framework to process the continuous action space, it can effectively solve nonlinear, non-convex game equilibrium problems, improving solution efficiency and accuracy. Attached Figure Description

[0012] Figure 1 This is a flowchart of an exemplary embodiment of an energy system scheduling method; Figure 2 This is a framework diagram of a comprehensive energy system game model provided in an exemplary embodiment; Figure 3 This is a schematic diagram of a deep deterministic policy gradient network structure provided in an exemplary embodiment; Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment; Figure 5 This is a block diagram of an energy system dispatching device provided in an exemplary embodiment. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present invention as detailed in the appended claims.

[0014] It should be noted that the steps of the corresponding methods in other embodiments are not necessarily performed in the order shown and described in this invention. In some other embodiments, the methods may include more or fewer steps than those described in this invention. Furthermore, a single step described in this invention may be broken down into multiple steps in other embodiments; and multiple steps described in this invention may be combined into a single step in other embodiments.

[0015] With the continuous growth of energy demand and the large-scale integration of renewable energy, the dispatching methods of traditional energy systems face numerous challenges. Regional integrated energy systems can improve energy utilization efficiency by integrating multiple energy forms such as electricity, cooling, and heating, but issues such as multi-energy coupling and conflicts of interest among multiple stakeholders complicate system optimization and dispatching.

[0016] In related technologies, scheduling methods for integrated energy systems mainly fall into two categories: centralized optimization and distributed optimization. While centralized optimization methods can achieve global optimum, they require complete information from all stakeholders, leading to issues such as privacy protection and communication burden in practical applications. Distributed optimization methods, such as game theory, can protect the privacy of each stakeholder. However, existing research often employs traditional game models, such as Stackelberg games or Nash games, which struggle to effectively handle complex interactions among multiple stakeholders. This is especially true when some stakeholders have cooperative relationships, as traditional models cannot accurately describe such hybrid game structures.

[0017] Furthermore, existing methods for solving game equilibrium often employ traditional optimization algorithms such as KKT conditions and dual decomposition. These methods have limited ability to solve nonlinear and nonconvex problems and struggle with high-dimensional state spaces. Although some studies have attempted to introduce reinforcement learning algorithms, they mostly employ algorithms with discrete action spaces, such as Q-learning and DQN, which are less adaptable to scheduling problems with continuous action spaces.

[0018] To address the shortcomings of related technologies, this invention proposes an energy system scheduling method, apparatus, equipment, and storage medium.

[0019] Figure 1 This is a flowchart illustrating an exemplary embodiment of an energy system scheduling method. For example... Figure 1 As shown, the method may include the following steps: Step 101: Construct a mathematical model of the regional integrated energy system, and establish operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system based on the collected energy flow data of electricity and cooling energy flow; wherein, the regional integrated energy system includes a power grid, a photovoltaic power generation system, an energy storage device, a water-cooled storage system, and a user terminal.

[0020] Specifically, the operational constraints of the power grid include power purchase constraints, power sales constraints, and mutual exclusion constraints between power purchase and sales states; the operational constraints of the water-based cold storage system model include dynamic balance constraints of cold storage capacity, cooling / release rate constraints, and cold storage tank capacity constraints; the output power of the photovoltaic power generation system model is a function of ambient temperature and solar radiation intensity; the operational constraints of the energy storage device model include dynamic update constraints of state of charge (SOC), upper and lower limits of SOC, charging and discharging power constraints, and equal SOC at the beginning and end of the cycle; and the user-side model includes constraints on the proportion of transferable load and total load balance constraints.

[0021] As the primary energy supplier in a regional integrated energy system, the power grid provides a stable power supply to the entire system. Assuming the power grid's supply capacity is unlimited, but is constrained by supply and demand balance, its power output can be expressed as:

[0022]

[0023]

[0024]

[0025] in, This represents the net interaction with the power grid. , , These represent the power purchased from the grid at time t and its maximum and minimum values, respectively. , , Let be the internet power at time t and its maximum and minimum values, respectively; , These represent the status of purchasing and selling electricity to the grid at time t, with values ​​of 0 or 1.

[0026] Water-based cooling systems are used for peak-valley electricity price arbitrage, providing cooling during periods of low electricity prices and releasing cooling during periods of high electricity prices. Their operation modes include direct supply mode and cooling release mode.

[0027] in, This indicates the remaining cooling capacity of the ice storage device (kWh or RT·h). Indicates refrigeration efficiency (dimensionless). Indicates the chilled water cooling / ice filling rate (kW); Indicates the cooling efficiency (dimensionless). This indicates the rate of cooling release (kW).

[0028] Cooling and release rate constraints: ; in, For maximum cooling rate, This represents the maximum rate of cooling release.

[0029] Water storage tank capacity constraints: ; in, For minimum cold storage capacity, This is the upper limit of the cold storage tank capacity (kWh).

[0030] Based on the nonlinear relationship between irradiance and temperature, the output power of photovoltaic (PV) power generation equipment is not only related to the energy conversion efficiency of the PV panels but also affected by ambient temperature and solar radiation intensity. Therefore, it exhibits fluctuations under different seasons and weather conditions. The functional relationship between output power and ambient temperature and solar radiation intensity is shown below: ; ; ; in, Let be the power output of the photovoltaic power generation equipment at time t; Let t be the actual temperature of the photovoltaic panel at time t during operation, in °C; Let t be the actual ambient temperature at time t, in °C; The operating temperature of the photovoltaic panel under standard test conditions, in °C; The ambient temperature under standard test conditions, in °C; Let be the photovoltaic power generation efficiency at time t; Photovoltaic power generation efficiency under standard test conditions; For inverter efficiency; Solar radiation intensity under standard test conditions, W / m2; The actual light intensity at time t, in W / m²; This is the temperature power coefficient, typically taken as 0.0047 / ℃; This indicates the area of ​​the solar panel, expressed in m².

[0031] Energy storage devices: Although there are various energy storage methods, they can be mainly divided into three categories according to energy type: chemical energy storage, electromagnetic energy storage, and physical energy storage. Different energy storage methods have their own advantages and disadvantages, but their basic working processes are not significantly different, including two parts: charging and releasing energy. The general model for energy storage can be represented as: ; ; ; ; ; in, and , representing the state of charge (SOC) of the energy storage unit at time t and time t+1, respectively. and These represent the charging power and releasing power at time 1 and 2, respectively. This indicates the maximum energy that the energy storage unit can store. This indicates the rated charge / discharge power. It is a 0-1 variable, when storing energy. =1, when energy is released =0, the introduction of this variable ensures that energy storage and energy release do not occur simultaneously. To ensure the sustainability of the energy storage unit's operation, the above formula stipulates that the SOC at the end of the optimization is equal to its initial value, ensuring a smooth start-up in the next working cycle, and T represents the total number of optimization periods.

[0032] Throughout a complete optimized operating cycle, the total charge and discharge amounts of the ESS should remain consistent: ; The energy storage capacity of the i-th IES at time t is the sum of the capacity at time t-1 and the charging / discharging power at time t: ; Charging power upper and lower limit constraints: ; Slope rate constraint: ; On the user side: Transferable loads can be used intermittently and are not subject to continuity constraints, shifting from one time period to any other. The shifted load must maintain the same total load as the unshifted load to meet the user's electricity needs. ; ; in This indicates the electrical load of the system after the translation. This indicates the system electrical load before translation. The percentage of loads that can be moved is taken as 0.10.

[0033] Electricity load supply and demand balance constraints: ; Cooling load supply and demand balance constraints: .

[0034] Step 102: Establish a dynamic electricity price model based on the difference between real-time net grid demand and average demand.

[0035] The dynamic electricity price in step S2 takes into account both the fluctuation characteristics of user electricity demand and the fluctuation characteristics of renewable energy output. It is defined as a function of the difference between net grid demand and average net grid demand in each period. The expression for the dynamic electricity price model is as follows: ; ; The dynamic electricity price for period t; Time-of-use pricing; For the user's electricity consumption demand in period t. This represents the average daily electricity consumption of a user. It is the variance of electricity demand; It is the price adjustment coefficient, which is always greater than 0.

[0036] The power grid influences market demand and supply by adjusting prices, but must meet the following constraints: ; , These are the minimum and maximum values ​​for dynamic electricity prices, used to limit the pricing behavior of power grid companies. The lower limit is used to protect the profits of the power grid, while the upper limit is set based on the user's acceptable level.

[0037] Step 103: Establish a master-slave game model with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient. The energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system. The users' strategy is to adjust the electricity consumption period of the transferable load.

[0038] The multi-agent game model is a leader-follower game model with the power grid as the leader and energy companies and users as followers, where the energy companies and users are in a cooperative game. The integrated energy system game model established in this invention, which includes the power grid, is as follows: Figure 2 As shown.

[0039] The participants include the following entities: (1) Regional power grid: As the dominant player in the game, it is responsible for formulating dynamic electricity pricing strategies, guiding market supply and demand balance by adjusting electricity price signals, and maximizing its own benefits and overall social welfare. (2) Energy storage operators and users: As followers of the power grid, they optimize their own strategies to maximize overall revenue based on the dynamic electricity price published by the power grid. Among them, energy storage operators take advantage of peak-valley arbitrage by selecting charging and discharging time and power. Photovoltaic power generation is given priority to users, and energy storage optimizes charging and discharging plans according to user needs to reduce users' electricity costs; users adjust their electricity consumption behavior according to the dynamic electricity price, with the goal of maximizing the user's welfare function, and provide information on the transferable load periods to energy storage operators.

[0040] The power grid's strategy is The formulation of price adjustment coefficients, which influence market demand and supply by adjusting prices, must meet the following constraints: .

[0041] The strategy for energy storage devices is charging and discharging power. It is necessary to meet the energy storage capacity and power constraints: ; The user's strategy is to adjust power consumption. It is necessary to meet the constraints of load shifting and total balance: ; ; The objective function of the power grid is to maximize profits: ; ; ; in Total load demand; For the power generation cost function, , and is a coefficient.

[0042] Since there is a cooperative relationship between users and energy storage, the overall revenue function of the park can be expressed as the sum of the revenues of energy storage operators and users:

[0043]

[0044]

[0045]

[0046]

[0047] in, To generate profits from batteries, arbitrage is achieved through low storage and high generation. Cost of energy storage charging and discharging; Benefits from water-based cooling.

[0048] Users maximize the difference between utility and cost by adjusting their electricity consumption: subtracting the cost from the utility function yields the user's welfare function, and the objective function is to maximize the user's welfare function. For the price offered by the supplier, each user adjusts their consumption strategy to maximize their own welfare.

[0049] max ; ; in The utility function representing a user's utility over a specific period of time is characterized by a quadratic function. It represents the consumption level of a certain commodity. It is a parameter that represents the user's level of preference.

[0050] The constraints satisfying the principles of increasing utility function and decreasing marginal utility are as follows:

[0051] .

[0052] Step 104: Solve the master-slave game model using a deep deterministic strategy gradient optimization algorithm, and through multiple iterations of learning, make the strategies of each subject converge to the game equilibrium point, and output the optimal dynamic electricity price, energy storage scheduling plan and user load curve; wherein, the Actor network of the deep deterministic strategy gradient optimization algorithm is used to generate deterministic action strategies of each subject according to the system state, and the Critic network is used to evaluate the long-term payoff Q value of the state-action pair.

[0053] The game equilibrium solution driven by the DDPG algorithm defines the state space in the environment modeling. (State), Action Space (Action) Reward (Reward), etc.

[0054] The DDPG algorithm's network structure comprises an Actor policy network and a Critic value evaluation network. The Actor network consists of a three-layer fully connected neural network with 256 hidden nodes, using the ReLU activation function. The output layer normalizes action values ​​using the Tanh function, taking the current state as input and outputting a deterministic action. The Critic network uses the same structure, taking the joint vector of state and action as input and outputting the Q-value that evaluates the long-term reward of the state-action pair. The specific structure is as follows Figure 3 As shown.

[0055] The interactive learning and strategy optimization process is as follows: Initialize the Actor and Critic network parameters and create an experience replay pool to store interaction data for state, action, reward, and next state.

[0056] The game begins to iterate, with the power grid, as the leader, initializing dynamic electricity price signals and releasing initial electricity prices based on historical load data, renewable energy output forecasts, and market supply and demand.

[0057] Energy storage operators and users, acting as followers, optimize their charging and discharging strategies, cooling and cold storage strategies, and electricity consumption adjustment strategies based on the dynamic electricity prices released by the grid. The revenue of each entity is calculated, and the system status is updated as environmental feedback.

[0058] The power grid updates dynamic electricity prices based on follower response results and fluctuations in renewable energy output. Through multiple iterations, the power grid adjusts electricity prices, and energy storage and user adjustment strategies are combined to eventually reach a stable equilibrium point.

[0059] Randomly sample small batches of data from the replay pool. Update the Critic network by minimizing the temporal difference error (TD-error) during empirical replay and training. ; in, The Q-value of the next state-action pair for the target network; Gamma is the discount factor used to balance the importance of current and future rewards. Actor network update: policy gradient ascent, maximizing the Q-value. ; in, For the policy function of the Actor network, The gradient of the Q-value with respect to the action guides the direction of policy updates.

[0060] Update the target network. Every 100 iterations, synchronize the parameters of the main network to the target network.

[0061] When the marginal gains of each agent approach zero, the payoffs converge, that is, when the game reaches equilibrium. At this point, no matter how the agents change their strategies, they will not achieve greater payoffs. This approach considers both maximizing social welfare and ensuring the economic interests of individual agents, thus optimizing the overall efficiency and stability of the system.

[0062] On one hand, this invention treats the power grid as the leader and energy storage operators and users as followers, with a cooperative game relationship among the followers. This hybrid game structure more accurately describes the complex interactions between multiple stakeholders in a real-world system, considering both the grid's dominant position and the mutually beneficial cooperation between energy storage operators and users, thus solving the problem that traditional single-game models struggle to handle hybrid game structures. On the other hand, by establishing a dynamic electricity price model based on the difference between real-time net grid demand and average demand, it can more sensitively reflect changes in supply and demand, guiding users to rationally adjust their electricity consumption behavior, achieving peak shaving and valley filling, and improving system operating efficiency. Furthermore, addressing the difficulty of traditional optimization algorithms in handling high-dimensional continuous action spaces, this invention employs a deep deterministic policy gradient algorithm. By using an Actor-Critic framework to process the continuous action space, it can effectively solve nonlinear, non-convex game equilibrium problems, improving solution efficiency and accuracy.

[0063] Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment. Please refer to... Figure 4 At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, memory 408, and non-volatile memory 410, and may also include other hardware required for its functions. One or more embodiments of the present invention can be implemented in software, for example, the processor 402 reads the corresponding computer program from the non-volatile memory 410 into memory 408 and then runs it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0064] Please refer to Figure 5 An energy system dispatching device can be applied to, for example Figure 5 The device shown, in order to implement the technical solution of the present invention, includes: The first construction unit 501 is used to construct a mathematical model of a regional integrated energy system, and to establish operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system based on the collected energy flow data of electrical and cooling energy flows; wherein, the regional integrated energy system includes a power grid, a photovoltaic power generation system, an energy storage device, a water-cooled storage system, and a user terminal; The second building unit 502 is used to establish a dynamic electricity price model based on the difference between real-time net grid demand and average demand. The third building unit 503 is used to establish a master-slave game model with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient. The energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system. The users' strategy is to adjust the electricity consumption period of the transferable load. The solution unit 504 is used to solve the master-slave game model using a deep deterministic policy gradient optimization algorithm, and through multiple iterations of learning, to make the strategies of each subject converge to the game equilibrium point, and output the optimal dynamic electricity price, energy storage scheduling plan and user load curve; wherein, the Actor network of the deep deterministic policy gradient optimization algorithm is used to generate the deterministic action strategy of each subject according to the system state, and the Critic network is used to evaluate the long-term payoff Q value of the state-action pair.

[0065] Optionally, the dynamic electricity price model is expressed as:

[0066] in, The dynamic electricity price for period t. For time-of-use electricity pricing, The net electricity demand of the power grid during period t. This represents the average daily electricity consumption of a user. It is the variance of electricity demand; It is the price adjustment coefficient.

[0067] Optionally, the operating constraints of the power grid include power purchase constraints, power sales constraints, and mutual exclusion constraints between power purchase and sales states; the operating constraints of the water-based cold storage system model include dynamic balance constraints of cold storage capacity, cooling / release rate constraints, and cold storage tank capacity constraints; the output power of the photovoltaic power generation system model is a function of ambient temperature and solar radiation intensity; the operating constraints of the energy storage device model include dynamic update constraints of state of charge (SOC), upper and lower limits of SOC, charging and discharging power constraints, and equal SOC at the beginning and end of the cycle; the user-side model includes constraints on the proportion of transferable load and total load balance constraints.

[0068] Optionally, the dynamic electricity price satisfies preset upper and lower price limits, wherein the lower price limit is used to guarantee the grid profit, and the upper price limit is set based on the user's acceptable level.

[0069] Optionally, the objective function of the power grid is to maximize the difference between electricity sales revenue and generation cost, expressed as:

[0070]

[0071]

[0072] in, Total load demand; For the power generation cost function, , and For coefficients; The overall revenue of the park is the sum of the revenue of the energy storage operator and the total welfare of users. The user welfare function is the difference between the quadratic utility function and the electricity cost.

[0073] Optionally, the solving unit 504 is specifically used for: Define a state space, action space, and reward function; the state space includes time-of-use electricity price, electricity / cooling load demand, photovoltaic predicted output, energy storage SOC, and water storage cooling capacity; the action space includes dynamic electricity price adjustment, energy storage charging and discharging power, water storage cooling / releasing rate, and user load shifting; the reward function corresponds to the objective function value of each game subject. Initialize the Actor network and Critic network, and create an experience replay pool; In each iteration of the game, the power grid, as the leader, takes the lead in releasing dynamic electricity prices; energy storage operators and users, as followers, optimize their own strategies and execute them based on the electricity prices. The system calculates rewards and updates status based on the actions of each subject, and stores interactive data in the experience replay pool; Data is sampled from the experience replay pool, the Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by the policy gradient ascent method. Repeat the iterations until the marginal returns of each subject approach zero, at which point the system reaches game equilibrium.

[0074] Furthermore, both the Actor network and the Critic network are neural networks containing three fully connected layers with 256 hidden layer nodes, and the ReLU activation function is used; the output layer of the Actor network uses the Tanh function to normalize the action values.

[0075] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0076] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0078] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0079] For any other form of computer-readable medium (or computer-readable storage medium) as described above, computer instructions may be stored thereon, which, when executed by a processor, implement one or more of the above embodiments, thereby realizing the technical solution of the present invention.

[0080] The present invention also proposes a computer program that, when executed by a processor, implements one or more of the embodiments described above, thereby realizing the technical solution of the present invention. This computer program may be specifically recorded on the above-described or other computer-readable media, and the present invention does not impose any limitations on this.

[0081] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0082] The foregoing has described specific embodiments of the invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] The terminology used in one or more embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in one or more embodiments of the invention and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0084] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of the present invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0085] The above description is merely a preferred embodiment of one or more embodiments of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present invention should be included within the protection scope of one or more embodiments of the present invention.

Claims

1. An energy system dispatching method, characterized in that, include: A mathematical model of a regional integrated energy system is constructed, and operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system are established based on the collected energy flow data of electricity and cooling energy. The regional integrated energy system includes a power grid, a photovoltaic power generation system, energy storage devices, a water-cooled storage system, and user terminals. Establish a dynamic electricity pricing model based on the difference between real-time net grid demand and average demand; A master-slave game model is established with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient, the energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system, and the users' strategy is to adjust the electricity consumption period of the transferable load. The master-slave game model is solved using a deep deterministic policy gradient optimization algorithm. Through multiple iterations, the strategies of each agent converge to the game equilibrium point, outputting the optimal dynamic electricity price, energy storage scheduling plan, and user load curve. The Actor network of the deep deterministic policy gradient optimization algorithm is used to generate deterministic action strategies for each agent based on the system state, and the Critic network is used to evaluate the long-term payoff Q value of the state-action pair.

2. The method according to claim 1, characterized in that, The dynamic electricity price model is expressed as follows: in, The dynamic electricity price for period t. For time-of-use electricity pricing, The net electricity demand of the power grid during period t. This represents the average daily electricity consumption of a user. It is the variance of electricity demand; It is the price adjustment coefficient.

3. The method according to claim 1, characterized in that, The operational constraints of the power grid include power purchase constraints, power sales constraints, and mutual exclusion constraints between power purchase and sales states; the operational constraints of the water-based cold storage system model include dynamic balance constraints of cold storage capacity, cooling / releasing rate constraints, and cold storage tank capacity constraints; the output power of the photovoltaic power generation system model is a function of ambient temperature and solar radiation intensity; the operational constraints of the energy storage device model include dynamic update constraints of state of charge (SOC), upper and lower limits of SOC, charging and discharging power constraints, and equal SOC at the beginning and end of the cycle; the user-side model includes constraints on the proportion of transferable load and total load balance constraints.

4. The method according to claim 1, characterized in that, The dynamic electricity price satisfies preset upper and lower price limits, wherein the lower price limit is used to guarantee the grid profit, and the upper price limit is set based on the user's acceptable level.

5. The method according to claim 1, characterized in that, The objective function of the power grid is to maximize the difference between electricity sales revenue and generation cost, expressed as: in, Total load demand; For the power generation cost function, , and For coefficients; The overall revenue of the park is the sum of the revenue of the energy storage operator and the total welfare of users. The user welfare function is the difference between the quadratic utility function and the electricity cost.

6. The method according to claim 1, characterized in that, The process involves multiple iterative learning sessions to converge the strategies of each agent to a game equilibrium point, outputting the optimal dynamic electricity price, energy storage dispatch plan, and user load curve, including: Define a state space, action space, and reward function; the state space includes time-of-use electricity price, electricity / cooling load demand, photovoltaic predicted output, energy storage SOC, and water storage cooling capacity; the action space includes dynamic electricity price adjustment, energy storage charging and discharging power, water storage cooling / releasing rate, and user load shifting; the reward function corresponds to the objective function value of each game subject. Initialize the Actor network and Critic network, and create an experience replay pool; In each iteration of the game, the power grid, as the leader, takes the lead in releasing dynamic electricity prices; energy storage operators and users, as followers, optimize their own strategies and execute them based on the electricity prices. The system calculates rewards and updates status based on the actions of each subject, and stores interactive data in the experience replay pool; Data is sampled from the experience replay pool, the Critic network is updated by minimizing the temporal difference error, and the Actor network is updated by the policy gradient ascent method. Repeat the iterations until the marginal returns of each subject approach zero, at which point the system reaches game equilibrium.

7. The method according to claim 6, characterized in that, Both the Actor network and the Critic network are neural networks containing three fully connected layers with 256 hidden layer nodes, using the ReLU activation function; the output layer of the Actor network uses the Tanh function to normalize the action values.

8. An energy system dispatching device, characterized in that, The device includes: The first construction unit is to construct a mathematical model of the regional integrated energy system, and to establish operational constraint models and supply-demand balance constraints for each component of the regional integrated energy system based on the collected energy flow data of electricity and cooling energy; wherein, the regional integrated energy system includes a power grid, a photovoltaic power generation system, an energy storage device, a water-cooled storage system, and a user terminal; The second building block: establishing a dynamic electricity price model based on the difference between real-time net grid demand and average demand; The third building block: Establish a master-slave game model with the power grid as the leader and energy storage operators and users as followers. The energy storage operators and users are in a cooperative game relationship with the common goal of maximizing the overall revenue of the park. The power grid's strategy is to optimize the price adjustment coefficient. The energy storage operators' strategy is to optimize the charging and discharging power of the energy storage device and the operation strategy of the water storage cooling system. The users' strategy is to adjust the electricity consumption period of the transferable load. The solution unit employs a deep deterministic policy gradient optimization algorithm to solve the master-slave game model. Through multiple iterations, it learns to converge the strategies of each agent to the game equilibrium point, outputting the optimal dynamic electricity price, energy storage scheduling plan, and user load curve. The Actor network of the deep deterministic policy gradient optimization algorithm is used to generate deterministic action strategies for each agent based on the system state, while the Critic network is used to evaluate the long-term payoff Q-value of the state-action pair.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor implements the steps of the method as described in any one of claims 1-7 by running the executable instructions.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-7.

Images