Energy storage power station system operation control model generation method, device, equipment and medium

Abstract

The invention discloses an energy storage power station system operation control model generation method and device, electronic equipment and a storage medium. The method comprises the following steps: performing digital mirroring on components of the energy storage power station system to obtain a digital agent of the energy storage power station system; determining a state space and an action space of the digital intelligent agent according to the state and the parameters of the component, and constraining a storage battery pack action and a power grid action in the action space according to a preset boundary condition; wherein the state space is used for describing the operation state of the digital agent, and the action space is used for describing the executable action of the digital agent; and constructing a simulation environment according to the digital intelligent agent, the state space and the action space after the action of the storage battery pack and the action of the power grid are constrained, and learning an operation control strategy of the energy storage power station system through a deep reinforcement learning algorithm in the simulation environment to obtain an operation control model. High-precision simulation and intelligent regulation and control of the operation process of the energy storage power station system can be achieved, and the energy utilization efficiency and economical efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power station simulation, and in particular to a method, device, electronic equipment and storage medium for generating an operation and control model of an energy storage power station system. Background Art

[0002] With the development of smart grid and virtualization technologies, the operation and control of power systems are gradually evolving towards greater intelligence and efficiency. Traditional power system analysis relies on physical models and historical data. Faced with increasingly complex power networks and the growing demand for renewable energy access, its limitations are gradually becoming apparent. To address these issues, researchers have begun exploring solutions based on the concept of digital twins. However, in existing virtual power plant systems, control strategies are primarily used to coordinate energy flows within the control system to best meet load demands. These strategies include, but are not limited to, rule-based methods, planning-based methods, and heuristic algorithms. Although these methods perform well in their respective application scenarios, they are generally targeted at direct control or optimization of actual systems and are less effective in fully digital environments. Summary of the Invention

[0003] The present invention provides a method, device, electronic equipment and storage medium for generating an operation and control model of an energy storage power station system, which can realize high-precision simulation of the operation process of the energy storage power station system and intelligent regulation of the energy storage power station system, thereby improving energy utilization efficiency and economy.

[0004] According to one aspect of the present invention, a method for generating an operation and control model of an energy storage power station system is provided, the method comprising:

[0005] Digitally mirroring the components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; wherein the components include solar panels, an energy storage power station, a power grid, and electrical loads; and the energy storage power station is a battery bank including multiple types of batteries;

[0006] Determine the state space and action space of the digital agent based on the state and parameters of the components, and constrain the battery pack actions and grid actions in the action space based on preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent;

[0007] A simulation environment is constructed based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the power grid action. In the simulation environment, an operation and control model is obtained by learning the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm.

[0008] According to another aspect of the present invention, a device for generating an operation and control model of an energy storage power station system is provided, the device comprising:

[0009] A digital mirroring module, configured to digitally mirror components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; the components include solar panels, an energy storage power station, a power grid, and electrical loads; the energy storage power station is a battery pack including multiple types of batteries;

[0010] an action constraint module, configured to determine the state space and action space of the digital agent based on the states and parameters of the components, and to constrain the battery pack actions and grid actions in the action space based on preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent;

[0011] A model generation module is used to construct a simulation environment based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the grid action, and to obtain an operation and control model by learning the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm in the simulation environment.

[0012] According to another aspect of the present invention, an electronic device is provided, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating an energy storage power station system operation and control model according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating an energy storage power station system operation and control model according to any embodiment of the present invention when executed.

[0017] The technical solution of an embodiment of the present invention is to digitally mirror the components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; wherein, the components include solar panels, energy storage power stations, power grids and electric loads, and the energy storage power station is a battery pack including multiple types of batteries; the state space and action space of the digital intelligent entity are determined according to the states and parameters of the components, and the battery pack actions and power grid actions in the action space are constrained according to preset boundary conditions; wherein, the state space is used to describe the operating state of the digital intelligent entity, and the action space is used to describe the executable actions of the digital intelligent entity; a simulation environment is constructed based on the digital intelligent entity, the state space and the action space after constraining the battery pack actions and the power grid actions, and in the simulation environment, the operation and control strategy of the energy storage power station system is learned by a deep reinforcement learning algorithm to obtain an operation and control model. The technical solution of the embodiment of the present invention can achieve high-precision simulation of the operation process of the energy storage power station system by determining the digital intelligent agent of the energy storage power station system and constructing a simulation environment based on the digital intelligent agent, state space and restricted action space. Then, in the simulation environment, the operation and control strategy is learned through a deep reinforcement learning algorithm to obtain an operation and control model, which can realize intelligent regulation of the energy storage power station system and improve energy utilization efficiency and economy.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flow chart of a method for generating an operation and control model of an energy storage power station system provided in accordance with the first embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the topology of an energy storage power station system provided according to the first embodiment of the present invention;

[0022] Figure 3 This is a flow chart of a method for generating an energy storage power station system operation and control model according to a second embodiment of the present invention;

[0023] Figure 4 This is a structural diagram of a device for generating an operation and control model of an energy storage power station system according to a third embodiment of the present invention;

[0024] Figure 5 It is a structural diagram of an electronic device for implementing the method for generating an operation and control model of an energy storage power station system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Example 1

[0028] Figure 1 A flowchart of a method for generating an operation and control model of an energy storage power station system is provided for the first embodiment of the present invention. This embodiment is applicable to the case where an energy storage power station system is controlled by an operation and control model. The method can be executed by an energy storage power station system operation and control model generating device. The energy storage power station system operation and control model generating device can be implemented in the form of hardware and / or software. The energy storage power station system operation and control model generating device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0029] S110. Digitally mirror the components of the energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; wherein the components include solar panels, energy storage power stations, power grids, and electrical loads, and the energy storage power station is a battery pack including multiple types of batteries.

[0030] The energy storage power station system is a system that integrates multiple energy conversion and storage technologies with multiple types of battery energy storage power stations as the core. Its components include solar panels, energy storage power stations, power grids, and electrical loads. Through coordinated operation, it can achieve stable, efficient, and sustainable electricity production and utilization. For example, Figure 2 A schematic diagram of the topology of an energy storage power station system is shown in FIG. Figure 2 As shown in the figure, solar panels convert solar radiation into electrical energy, providing power input for the system; energy storage power stations store electrical energy in the form of chemical energy and release the stored electrical energy when needed to provide a continuous power supply; the power grid meets energy needs at different times and scenarios through a combination of electricity purchase and sales operations; the electrical load is the power consumption in the system.

[0031] Among them, multi-type batteries refer to the integration of two or more battery systems with different technical routes in energy storage power stations, which realize efficient operation of energy storage and release through collaborative management and optimized control. These battery types have significant differences in chemical properties, energy density, power density, cycle life, charge and discharge efficiency, cost and applicable scenarios. Through complementary advantages, they meet the diverse needs of energy storage power stations. Generally speaking, multi-type batteries include lithium batteries, vanadium titanium batteries, lead-acid batteries, flow batteries, etc.

[0032] In an embodiment of the present invention, the components of the energy storage power station system can be digitally mirrored to obtain a digital intelligent entity of the energy storage power station system. Specifically, the relevant data of each component of the energy storage power station system can be obtained in real time through sensors and monitoring equipment, and each component can be digitally mirrored based on the relevant data of each component, that is, a digital model of each component is constructed, and the digital models of each component are combined to obtain a digital intelligent entity of the energy storage power station system. It should be noted that the digital intelligent entity should be able to accurately reflect the operating status and change rules of the energy storage power station system. Optionally, before digitally mirroring each component based on the relevant data, the relevant data can be pre-processed such as cleaning and format conversion to ensure the accuracy and availability of the data. By digitally mirroring the components of the energy storage power station system to obtain a digital intelligent entity of the energy storage power station system, accurate monitoring and management of the energy storage power station system can be achieved, and the operating efficiency and stability of the system can be improved.

[0033] S120. Determine the state space and action space of the digital agent based on the status and parameters of the components, and constrain the battery pack actions and grid actions in the action space based on preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent.

[0034] The state space describes the operational state of the digital agent and encompasses all observable information about the digital agent at a specific point in time. The state space should fully describe the current state of the energy storage power station system to facilitate decision-making by the reinforcement learning model. The action space describes the digital agent's executable actions. Based on the current state, the digital agent can select an action from the action space and execute it, thereby affecting the state at the next moment.

[0035] In an embodiment of the present invention, the state space and action space of the digital agent can be determined based on the state and parameters of the components, providing a clear input and output space for the deep reinforcement learning algorithm. At the same time, because the state space of the digital agent is based on the digital agent, whether the action of the digital agent is directly derived from the deterministic strategy network or sampled from the random strategy, it only has upper and lower bounds on each dimension of the action, and does not constrain the rationality between the actions. In other words, in the actions given by the digital agent, the actions between different dimensions conflict. This erroneous action will hinder the training of the entire algorithm and is meaningless in a real physical scenario. Therefore, it is also necessary to constrain the battery pack action and the power grid action in the action space according to the preset boundary conditions so that the actions have real physical meanings, ensuring that the operation and control strategy learned by the algorithm meets the actual needs and boundary conditions of the system, thereby improving the convergence speed and accuracy of the deep reinforcement learning algorithm. Among them, the preset boundary conditions can be set by technical personnel according to actual conditions.

[0036] S130. Construct a simulation environment based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the grid action. In the simulation environment, learn the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm to obtain an operation and control model.

[0037] Among them, deep reinforcement learning algorithms are a technology that combines the perception ability of deep learning with the decision-making ability of reinforcement learning. They can be divided into model-based deep reinforcement learning algorithms and model-free deep reinforcement learning algorithms. In the embodiments of the present invention, model-free deep reinforcement learning algorithms are mainly used, especially algorithms based on policy gradients (PG).

[0038] In an embodiment of the present invention, a simulation environment can be first constructed based on the digital agent, the state space, and the action space after constraining the battery group action and the grid action, so as to train the deep reinforcement learning algorithm in the simulation environment so that it can learn the operation and control strategy of the energy storage power station system while taking into account the actual operation constraints and requirements of the energy storage power station system. Then, a suitable deep reinforcement learning algorithm is selected, a suitable reward function is designed, the performance of the digital agent at each time step is evaluated, and the deep reinforcement learning algorithm is guided to learn the appropriate operation and control strategy to obtain the operation and control model. By constructing a simulation environment based on the digital agent, the state space, and the action space after constraining the battery group action and the grid action, and learning the operation and control strategy of the energy storage power station system through the deep reinforcement learning algorithm in the simulation environment to obtain the operation and control model, it is possible to realize intelligent regulation of the energy storage power station system, improve energy utilization efficiency and economy, and promote the sustainable development of the new energy system.

[0039] The technical solution of an embodiment of the present invention digitally mirrors the components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system. The components include solar panels, energy storage power stations, power grids, and electrical loads, and the energy storage power station is a battery pack including multiple types of batteries. The state space and action space of the digital intelligent entity are determined based on the states and parameters of the components, and the battery pack actions and power grid actions in the action space are constrained according to preset boundary conditions. The state space is used to describe the operating state of the digital intelligent entity, and the action space is used to describe the executable actions of the digital intelligent entity. A simulation environment is constructed based on the digital intelligent entity, the state space, and the action space after constraining the battery pack actions and power grid actions. In the simulation environment, the operation and control strategy of the energy storage power station system is learned using a deep reinforcement learning algorithm to obtain an operation and control model. The technical solution of an embodiment of the present invention, by determining the digital intelligent entity of the energy storage power station system and constructing a simulation environment based on the digital intelligent entity, the state space, and the constrained action space, can achieve high-precision simulation of the operation process of the energy storage power station system. Then, in the simulation environment, the operation and control strategy is learned using a deep reinforcement learning algorithm to obtain an operation and control model, which can realize intelligent control of the energy storage power station system and improve energy utilization efficiency and economic efficiency.

[0040] Example 2

[0041] Figure 3 This is a flow chart of a method for generating an energy storage power station system operation and control model provided by the second embodiment of the present invention. The embodiment of the present invention is optimized based on the above embodiment. For solutions not fully described in the embodiment of the present invention, please refer to the above embodiment. Figure 3 As shown, the method includes:

[0042] S210: Determine the output power of the solar panel, the charge state of the battery pack, the power sold and bought by the power grid, and the input power of the electrical load at the current moment.

[0043] In an embodiment of the present invention, it is necessary to first determine the relevant data of each component of the energy storage power station system at the current moment, namely the output power of the solar panel, the storage status of the battery pack, the power sold and purchased by the power grid, and the input power of the electrical load at the current moment, so as to construct a digital intelligent entity of the energy storage power station system based on the relevant data of each component.

[0044] Optionally, determining the output power of the solar panel at the current moment, the storage state of the battery pack, the power sold and bought by the power grid, and the input power of the electric load at the current moment includes: determining the output power of the solar panel at the current moment based on the rated power, derating factor, standard incident radiation, and incident radiation at the current moment of the solar panel; determining the storage state of the battery pack at the current moment based on the storage state of the battery pack at the previous moment, the charge and discharge power at the current moment, and the charge and discharge efficiency; determining the power sold and bought by the power grid at the current moment, and determining the input power of the electric load at the current moment based on the power load data of the power user.

[0045] In the embodiment of the present invention, it is understood that the output power of the solar panel depends on many variables, including the type of material, temperature, and solar radiation incident on the surface of the component. The embodiment of the present invention does not consider the impact of temperature on the output power of the solar panel, and the output power of the solar panel at the current moment can be determined based on the rated power, derating factor, standard incident radiation, and incident radiation of the solar panel at the current moment. Optionally, the output power of the solar panel at the current moment is determined based on the rated power, derating factor, standard incident radiation, and incident radiation of the solar panel at the current moment, and the calculation formula is:

[0046]

[0047] Among them, P PV is the output power of the solar panel, Y PV is the rated power of the solar panel, f PV is the derating factor, is the incident radiation at the current moment, is the standard incident radiation.

[0048] If there are higher requirements for the accuracy of the solar panel's output power, the effect of temperature on the solar panel's output power can be considered on the basis of the above calculation formula. The output power of the solar panel at the current moment can be determined according to the solar panel's rated power, derating factor, standard incident radiation, current incident radiation, power temperature coefficient, current temperature and standard temperature. The calculation formula is:

[0049]

[0050] Among them, αP is the power temperature coefficient, T c is the current temperature, T c，STC is the standard temperature.

[0051] In an embodiment of the present invention, the current state of charge of the battery pack, i.e., the amount of charge of the battery pack at the current moment, can be determined based on the previous state of charge of the battery pack, the current charge and discharge power, and the current charge and discharge efficiency. Alternatively, the current state of charge of the battery pack can be determined based on the previous state of charge of the battery pack, the current charge and discharge power, and the current charge and discharge efficiency using the following calculation formula:

[0052]

[0053] in, is the current state of charge of the battery pack, is the battery pack's state of charge at the last moment, is the charging power at the current moment, is the discharge power at the current moment, η batt It is the charge and discharge efficiency. It should be noted that the battery pack's charge state should be within the normal power range, the battery pack's charge and discharge power should not exceed the maximum charge and discharge power, and the battery can only be in the charge or discharge state at any time.

[0054] In an embodiment of the present invention, the electricity power sold and purchased by the power grid at the current moment can be obtained from the corresponding database, and the input power of the electricity load at the current moment can be determined based on the electricity load data of the electricity users. The electricity load data comes from real electricity users and the time span is more than one year.

[0055] S220 , representing each component in the energy storage power station system according to the output power, the power storage state, the purchased and sold power, and the input power, to obtain a digital intelligent entity of the energy storage power station system.

[0056] In an embodiment of the present invention, after determining the output power of the solar panel, the storage state of the battery pack, the power sold and purchased by the power grid, and the input power of the electric load at the current moment, each component in the energy storage power station system can be represented according to the output power, storage state, power sold and purchased, and input power. Specifically, the solar panel can be represented according to the output power of the solar panel, the energy storage station can be represented according to the storage state of the battery pack, the power grid can be represented according to the power sold and purchased by the power grid, and the electric load can be represented according to the input power of the electric load. The digital representations of the components are then combined to obtain a digital intelligent entity of the energy storage power station system.

[0057] S230: Determine the state space and action space of the digital agent according to the state and parameters of the components, and constrain the battery pack action and grid action in the action space according to preset boundary conditions.

[0058] Optionally, determining the state space and action space of the digital agent based on the status and parameters of the components includes: representing the operating state of the digital agent at the current moment according to the output power of the solar panel and the storage state of the battery pack at the current moment, to obtain the state space of the digital agent; representing the executable actions of the digital agent at the current moment according to the selling and buying power of the power grid and the charging and discharging power of the battery pack at the current moment, to obtain the action space of the digital agent.

[0059] In this embodiment of the present invention, within the energy storage power station system environment, the state space of the digital agent is a continuous state space, represented by multiple continuous variables. This solution represents the current operating state of the digital agent based on the output power of the solar panels and the battery pack's charge state, where the battery pack's charge state is represented by the charge states of the multiple batteries in the battery pack. Alternatively, the state space of the digital agent can be represented as:

[0060]

[0061] Among them, S is the state space of the digital agent, P PV is the output power of the solar panel at the current moment, is the state of charge of the first battery in the battery pack, the state of charge of the second battery in the battery pack, is the state of charge of the nth battery in the battery pack, and n is the total number of batteries in the battery pack.

[0062] In this embodiment of the present invention, within the energy storage power station system environment, the action space of the digital agent is a continuous action space, represented by multiple continuous variables. This solution represents the executable actions of the digital agent at the current moment based on the current power sold by the power grid and the charge and discharge power of the battery pack. The charge and discharge power of the battery pack is represented by the charge and discharge power of the multiple batteries in the battery pack. Optionally, the action space of the digital agent can be represented as:

[0063]

[0064] Among them, A is the action space of the digital agent, P net is the power sold and purchased by the power grid at the current moment, is the charge and discharge power of the first battery in the battery pack, The charge and discharge power of the second battery in the battery pack, is the charge and discharge power of the nth battery in the battery pack, and n is the total number of batteries in the battery pack.

[0065] Optionally, the constraints on the battery group actions and grid actions in the action space according to preset boundary conditions include: if the output power of the solar panel at the current moment is greater than the rated purchase and sale power of the grid, the rated purchase and sale power of the grid is used as the purchase power threshold of the grid at the current moment; the rated purchase and sale power of the grid is used as the sales power threshold of the grid at the current moment; the minimum value of the difference between the output power of the solar panel and the purchase and sale power of the grid at the current moment, the maximum charging power of the battery group and the remaining capacity of the battery group is used as the charging power threshold of the battery group at the current moment; the minimum value of the maximum discharge power of the battery group and the current capacity of the battery group is used as the discharge power threshold of the battery group at the current moment; if the solar panel at the current moment is greater than the rated purchase and sale power of the grid, the rated purchase and sale power of the grid is used as the sales power threshold of the grid at the current moment; The output power of the solar panel is less than or equal to the rated buying and selling power of the power grid, the minimum value of the maximum discharge power of the battery pack and the remaining capacity of the battery pack is added to the output power of the solar panel, and the addition result and the minimum value of the rated buying and selling power of the power grid are used as the buying power threshold of the power grid at the current moment; the rated buying and selling power of the power grid is used as the selling power threshold of the power grid at the current moment; the difference between the output power of the solar panel and the buying and selling power of the power grid at the current moment, the maximum charging power of the battery pack and the remaining capacity of the battery pack is used as the charging power threshold of the battery pack at the current moment; the minimum value of the maximum discharge power of the battery pack and the current capacity of the battery pack is used as the discharge power threshold of the battery pack at the current moment.

[0066] In an embodiment of the present invention, it is necessary to constrain the battery pack actions and grid actions in the action space according to preset boundary conditions so that the actions have real physical meanings, ensure that the operation and control strategies learned by the algorithm meet the actual needs and boundary conditions of the system, and thereby improve the convergence speed and accuracy of the deep reinforcement learning algorithm. Specifically, if the output power of the solar panel at the current moment is greater than the power supply of the grid, the rated power supply of the grid is used as the power supply threshold of the grid at the current moment, the rated power supply of the grid is used as the power supply threshold of the grid at the current moment, the difference between the output power of the solar panel and the power supply of the grid at the current moment, the maximum charging power of the battery pack and the minimum value of the remaining capacity of the battery pack are used as the charging power threshold of the battery pack at the current moment, and the maximum discharge power of the battery pack and the minimum value of the current capacity of the battery pack are used as the discharge power threshold of the battery pack at the current moment. If the output power of the solar panel at the current moment is less than or equal to the power purchase and sale power of the power grid, the minimum value of the maximum discharge power of the battery pack and the remaining capacity of the battery pack is added to the output power of the solar panel, and the addition result and the minimum value of the rated power purchase and sale power of the power grid are used as the power purchase power threshold of the power grid at the current moment, and the rated power purchase and sale power of the power grid is used as the power sale power threshold of the power grid at the current moment. The difference between the output power of the solar panel and the power purchase and sale power of the power grid at the current moment, the maximum charging power of the battery pack and the minimum value of the remaining capacity of the battery pack are used as the charging power threshold of the battery pack at the current moment, and the maximum discharge power of the battery pack and the minimum value of the current capacity of the battery pack are used as the discharge power threshold of the battery pack at the current moment.

[0067] S240. Construct a simulation environment based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the grid action. In the simulation environment, learn the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm to obtain an operation and control model.

[0068] S250: Verify and evaluate the performance of the operation control model, and adjust the operation control strategy based on the verification and performance evaluation results.

[0069] In an embodiment of the present invention, after obtaining the operation and control model, it is necessary to verify and evaluate the performance of the operation and control model, including testing in a simulation environment and an actual environment. Based on the verification and performance evaluation results, the operation and control strategy of the energy storage power station system is optimized and adjusted.

[0070] Optionally, the operation and control model is verified and performance evaluated, and the operation and control strategy is adjusted according to the verification results and performance evaluation results, including: for the digital intelligent body at the current moment, determining the target operation and control strategy through the operation and control model to control the digital intelligent body, and determining the input power of the electric load at the next moment; calculating the difference between the input power of the electric load at the next moment and the target input power of the electric load at the next moment, and adjusting the operation and control strategy until the difference is less than a preset threshold; wherein, the target input power is predicted based on the electricity load data of the power user and a pre-trained prediction model.

[0071] In an embodiment of the present invention, the target operation and control strategy can be determined for the digital intelligent body at the current moment through the operation and control model, and the digital intelligent body at the current moment can be controlled. After the control action is completed, the input power of the electric load at the next moment is determined, and the input power of the electric load at the next moment is compared with the target input power of the electric load at the next moment to determine whether the execution effect of the operation and control strategy meets expectations. Specifically, the difference between the input power of the electric load at the next moment and the target input power of the electric load at the next moment can be calculated, and when the difference is less than the preset threshold, it is considered that the execution effect of the operation and control strategy meets expectations. If the execution effect of the operation and control strategy does not meet expectations, it is necessary to continue to adjust the operation and control strategy until the difference is less than the preset threshold. Among them, the target input power is predicted based on the electricity load data of the power user and the pre-trained prediction model, and is the expected value of the input power of the electric load. The preset threshold can be set by technical personnel according to actual conditions.

[0072] S260: Update the operation and control model according to the adjusted operation and control strategy, and control the energy storage power station system through the updated operation and control model.

[0073] In an embodiment of the present invention, after the operation and control strategy is adjusted until the execution effect of the operation and control strategy meets expectations, the operation and control model can be updated according to the adjusted operation and control strategy, that is, the deep reinforcement learning algorithm is retrained with the adjusted operation and control strategy to obtain an updated operation and control model, and the energy storage power station system is controlled by the updated operation and control model, and the optimized operation and control strategy is applied to the actual energy storage power station system to further improve energy utilization efficiency and economy.

[0074] The technical solution of the embodiment of the present invention determines the output power of the solar panel, the storage state of the battery pack, the power sold and purchased by the power grid, and the input power of the electric load at the current moment; represents each component in the energy storage power station system according to the output power, storage state, power sold and purchased, and input power, to obtain a digital intelligent body of the energy storage power station system; determines the state space and action space of the digital intelligent body according to the state and parameters of the components, and constrains the battery pack action and the power grid action in the action space according to preset boundary conditions; constructs a simulation environment according to the digital intelligent body, the state space, and the action space after constraining the battery pack action and the power grid action, and learns the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm in the simulation environment to obtain an operation and control model; verifies and evaluates the performance of the operation and control model, and adjusts the operation and control strategy according to the verification results and performance evaluation results; updates the operation and control model according to the adjusted operation and control strategy, and controls the energy storage power station system through the updated operation and control model. The technical solution of the embodiment of the present invention can achieve high-precision simulation of the operation process of the energy storage power station system by determining the digital intelligent agent of the energy storage power station system and constructing a simulation environment based on the digital intelligent agent, state space, and restricted action space. Then, in the simulation environment, the operation and control strategy is learned through a deep reinforcement learning algorithm to obtain an operation and control model, which can realize intelligent regulation of the energy storage power station system and improve energy utilization efficiency and economy. At the same time, by adjusting the operation and control strategy based on the verification results and performance evaluation results, and controlling the energy storage power station system based on the operation and control model updated according to the adjusted operation and control strategy, the execution results of the operation and control strategy can be optimized, further improving energy utilization efficiency and economy.

[0075] Example 3

[0076] Figure 4 This is a schematic diagram of the structure of a device for generating an operation and control model of an energy storage power station system provided by the third embodiment of the present invention. Figure 4 As shown, the device includes:

[0077] A digital mirroring module 310 is configured to digitally mirror the components of the energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; the components include solar panels, an energy storage power station, a power grid, and electrical loads; the energy storage power station is a battery bank including multiple types of batteries;

[0078] An action constraint module 320 is configured to determine the state space and action space of the digital agent based on the states and parameters of the components, and to constrain the battery pack actions and grid actions in the action space according to preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent;

[0079] The model generation module 330 is used to construct a simulation environment based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the grid action, and to obtain an operation and control model by learning the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm in the simulation environment.

[0080] Optionally, the digital mirror module 310 includes:

[0081] a data determination unit, configured to determine the output power of the solar panel, the state of charge of the battery pack, the power sold by the power grid, and the input power of the electrical load at a current moment;

[0082] The digital mirror unit is used to represent each component in the energy storage power station system according to the output power, the power storage state, the purchased power and the input power, so as to obtain a digital intelligent entity of the energy storage power station system.

[0083] Optionally, the data determination unit is specifically configured to:

[0084] Determining the output power of the solar panel at a current moment according to the rated power, derating factor, standard incident radiation of the solar panel and the incident radiation at a current moment;

[0085] Determining the current state of charge of the battery pack according to the previous state of charge of the battery pack, the current charge and discharge power, and the current charge and discharge efficiency;

[0086] The power sold and purchased by the power grid at the current moment is determined, and the input power of the electric load at the current moment is determined based on the power load data of the power user.

[0087] Optionally, the action constraint module 320 includes:

[0088] a state space determining unit, configured to represent the current operating state of the digital agent according to the output power of the solar panel and the power storage state of the battery pack, thereby obtaining a state space of the digital agent;

[0089] The action space determination unit is used to represent the executable actions of the digital agent at the current moment according to the power sold and purchased by the power grid and the charge and discharge power of the battery pack at the current moment, so as to obtain the action space of the digital agent.

[0090] Optionally, the action constraint module 320 includes:

[0091] a first action constraint unit configured to, if the output power of the solar panel at a current moment is greater than the rated purchase and sale power of the power grid, use the rated purchase and sale power of the power grid as the purchase power threshold of the power grid at a current moment; use the rated purchase and sale power of the power grid as the sales power threshold of the power grid at a current moment; use the minimum value of the difference between the output power of the solar panel and the sales and sale power of the power grid at a current moment, the maximum charging power of the battery pack, and the remaining capacity of the battery pack as the charging power threshold of the battery pack at a current moment; and use the minimum value of the maximum discharge power of the battery pack and the current capacity of the battery pack as the discharge power threshold of the battery pack at a current moment;

[0092] The second action constraint unit is used to add the minimum value of the maximum discharge power of the battery pack and the remaining capacity of the battery pack to the output power of the solar panel if the output power of the solar panel at the current moment is less than or equal to the rated purchase and sale power of the power grid, and use the addition result and the minimum value of the rated purchase and sale power of the power grid as the purchase power threshold of the power grid at the current moment; use the rated purchase and sale power of the power grid as the purchase power threshold of the power grid at the current moment; use the difference between the output power of the solar panel and the purchase and sale power of the power grid at the current moment, the maximum charging power of the battery pack and the remaining capacity of the battery pack as the charging power threshold of the battery pack at the current moment; and use the minimum value of the maximum discharge power of the battery pack and the current capacity of the battery pack as the discharge power threshold of the battery pack at the current moment.

[0093] Optionally, the device further includes:

[0094] An operation control strategy adjustment module is used to verify and evaluate the performance of the operation control model, and adjust the operation control strategy based on the verification results and performance evaluation results;

[0095] The operation and control model updating module is used to update the operation and control model according to the adjusted operation and control strategy, and control the energy storage power station system through the updated operation and control model.

[0096] Optionally, the operation control strategy adjustment module is specifically configured to:

[0097] For the digital agent at the current moment, a target operation and control strategy is determined by the operation and control model to control the digital agent, and the input power of the electric load at the next moment is determined;

[0098] Calculate the difference between the input power of the electric load at the next moment and the target input power of the electric load at the next moment, and adjust the operation and control strategy until the difference is less than a preset threshold; wherein the target input power is predicted based on the power load data of the power user and a pre-trained prediction model.

[0099] The energy storage power station system operation and control model generation device provided in the embodiment of the present invention can execute the energy storage power station system operation and control model generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] Example 4

[0101] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0102] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0103] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0104] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for generating an energy storage power station system operation and control model.

[0105] In some embodiments, the method for generating an operation and control model of an energy storage power station system may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for generating an operation and control model of an energy storage power station system described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for generating an operation and control model of an energy storage power station system in any other appropriate manner (e.g., by means of firmware).

[0106] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0108] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0111] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0112] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0113] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for generating an operation and control model of an energy storage power station system, characterized in that: The method comprises: Digitally mirroring the components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; wherein the components include solar panels, an energy storage power station, a power grid, and electrical loads; and the energy storage power station is a battery bank including multiple types of batteries; Determine the state space and action space of the digital agent based on the state and parameters of the components, and constrain the battery pack actions and grid actions in the action space based on preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent; A simulation environment is constructed based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the power grid action. In the simulation environment, an operation and control model is obtained by learning the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm.

2. The method according to claim 1, characterized in that The digital mirroring of the components of the energy storage power station system to obtain a digital intelligent entity of the energy storage power station system includes: Determining the output power of the solar panel, the state of charge of the battery pack, the power sold and bought by the power grid, and the input power of the electrical load at the current moment; Each component in the energy storage power station system is represented according to the output power, the power storage state, the purchased power, and the input power to obtain a digital intelligent entity of the energy storage power station system.

3. The method according to claim 2, characterized in that The determining of the output power of the solar panel, the state of charge of the battery pack, the power sold by the power grid, and the input power of the electrical load at the current moment includes: Determining the output power of the solar panel at a current moment according to the rated power, derating factor, standard incident radiation of the solar panel and the incident radiation at a current moment; Determining the current state of charge of the battery pack according to the previous state of charge of the battery pack, the current charge and discharge power, and the current charge and discharge efficiency; The power sold and purchased by the power grid at the current moment is determined, and the input power of the electric load at the current moment is determined based on the power load data of the power user.

4. The method according to claim 1, wherein Determining the state space and action space of the digital agent according to the state and parameters of the component includes: Representing the current operating state of the digital agent according to the output power of the solar panel and the power storage state of the battery pack, thereby obtaining a state space of the digital agent; The executable actions of the digital agent at the current moment are represented according to the electricity sales power of the power grid and the charge and discharge power of the battery pack at the current moment, thereby obtaining the action space of the digital agent.

5. The method according to claim 1, wherein The constraining the battery pack action and the grid action in the action space according to the preset boundary conditions includes: If the output power of the solar panel at the current moment is greater than the rated purchase and sale power of the power grid, the rated purchase and sale power of the power grid is used as the purchase power threshold of the power grid at the current moment; the rated purchase and sale power of the power grid is used as the sales power threshold of the power grid at the current moment; the minimum value of the difference between the output power of the solar panel and the sales power of the power grid at the current moment, the maximum charging power of the battery pack, and the remaining capacity of the battery pack is used as the charging power threshold of the battery pack at the current moment; the minimum value of the maximum discharge power of the battery pack and the current capacity of the battery pack is used as the discharge power threshold of the battery pack at the current moment; If the output power of the solar panel at the current moment is less than or equal to the rated buying and selling power of the power grid, the minimum value of the maximum discharge power of the battery pack and the remaining capacity of the battery pack is added to the output power of the solar panel, and the addition result and the minimum value of the rated buying and selling power of the power grid are used as the buying power threshold of the power grid at the current moment; the rated buying and selling power of the power grid is used as the selling power threshold of the power grid at the current moment; the difference between the output power of the solar panel and the buying and selling power of the power grid at the current moment, the maximum charging power of the battery pack and the remaining capacity of the battery pack are used as the charging power threshold of the battery pack at the current moment; the maximum discharge power of the battery pack and the minimum value of the current capacity of the battery pack are used as the discharge power threshold of the battery pack at the current moment.

6. The method according to claim 1, characterized in that After learning the operation and control strategy of the energy storage power station system by a deep reinforcement learning algorithm in the simulation environment to obtain an operation and control model, the method further includes: Verifying and evaluating the performance of the operation control model, and adjusting the operation control strategy based on the verification and performance evaluation results; The operation and control model is updated according to the adjusted operation and control strategy, and the energy storage power station system is controlled by the updated operation and control model.

7. The method according to claim 6, characterized in that The verification and performance evaluation of the operation control model, and adjustment of the operation control strategy based on the verification results and performance evaluation results, include: For the digital agent at the current moment, a target operation and control strategy is determined by the operation and control model to control the digital agent, and the input power of the electric load at the next moment is determined; Calculate the difference between the input power of the electric load at the next moment and the target input power of the electric load at the next moment, and adjust the operation and control strategy until the difference is less than a preset threshold; wherein the target input power is predicted based on the power load data of the power user and a pre-trained prediction model.

8. A device for generating an operation and control model of an energy storage power station system, characterized in that: The device comprises: A digital mirroring module, configured to digitally mirror components of an energy storage power station system to obtain a digital intelligent entity of the energy storage power station system; the components include solar panels, an energy storage power station, a power grid, and electrical loads; the energy storage power station is a battery pack including multiple types of batteries; an action constraint module, configured to determine the state space and action space of the digital agent based on the states and parameters of the components, and to constrain the battery pack actions and grid actions in the action space based on preset boundary conditions; wherein the state space is used to describe the operating state of the digital agent, and the action space is used to describe the executable actions of the digital agent; A model generation module is used to construct a simulation environment based on the digital intelligent agent, the state space, and the action space after constraining the battery pack action and the grid action, and to obtain an operation and control model by learning the operation and control strategy of the energy storage power station system through a deep reinforcement learning algorithm in the simulation environment.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating an energy storage power station system operation and control model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the energy storage power station system operation and control model generation method according to any one of claims 1 to 7 when executed.

Images