Multi-time scale network following / network construction energy storage cooperative control method and device

Abstract

The invention relates to the technical field of electric power systems, and discloses a multi-time scale network following / network construction energy storage cooperative control method and device, and the method comprises the steps: enabling a model prediction control algorithm to accurately solve an energy storage network following and network construction control proportion, optimizing power adjustment, and adapting to the change of a power grid. The reinforcement learning algorithm is combined with multi-dimensional parameters, the converter control mode is dynamically and intelligently adjusted according to the control proportion, comprehensive evaluation is achieved, and real-time response is achieved. Double-layer optimization cooperative control with the upper layer mainly based on model prediction and the lower layer mainly based on reinforcement learning is adopted, the problem of network following / network construction cooperative control in different power grid characteristics and multiple time scales can be solved, and the active supporting effect of energy storage on a novel power system is improved.

Description

Technical Field

[0001] The present application relates to the technical field of power systems, and in particular to a collaborative control method and device for multi-time-scale grid-following / grid-building energy storage. Background Art

[0002] Currently, converters for new energy stations and energy storage systems mostly use grid-following control. However, in areas with high renewable energy penetration, grid characteristics are often weak. When grid-following converters are connected to weak grids, they are prone to oscillation, posing a risk of system instability. To address this issue, the introduction of strong supporting grid-following control has become an important direction. However, converting all energy storage converters to grid-following control is not a good idea. On the one hand, it would significantly increase construction costs. On the other hand, when the grid characteristics are strong, grid-following control is also prone to causing safety accidents.

[0003] Furthermore, with the large-scale integration of new energy sources and the application of various grid support technologies, the strength and weakness of the power grid are in a state of dynamic change. The requirements of new power systems for grid characteristics are not fixed and vary significantly across different timescales. In the short term, when the grid experiences small disturbances, it requires strong characteristics to quickly withstand them and maintain stable operation. In the medium to long term, when large power sources or loads are connected or disconnected from the grid, the flexible control resources in the power system must be able to respond quickly, effectively reducing grid inertia and ensuring system balance and stability.

[0004] In summary, existing technologies have significant shortcomings in addressing the diverse characteristics of power grids and the coordinated control of grid-following and grid-forming systems across multiple timescales. Therefore, there is an urgent need to find effective technical solutions to address the challenges of coordinated control across diverse power grid characteristics and multiple timescales, thereby enhancing the proactive support provided by energy storage to new power systems. Summary of the Invention

[0005] In view of this, the present application provides a collaborative control method and device for multi-time scale grid-following / grid-building energy storage to solve the problem of obvious deficiencies in grid-following / grid-building collaborative control in response to different grid characteristics and multiple time scales.

[0006] On the first aspect, the present application provides a collaborative control method for multi-time-scale grid-following / grid-building energy storage, the method comprising: based on the strength and weakness characteristics of the power grid and the support requirements of systems at different time scales, a model predictive control algorithm is used to solve the control ratio of energy storage grid-following and grid-building; based on the control ratio of energy storage grid-following and grid-building, combined with the multi-dimensional parameters of energy storage, a reinforcement learning algorithm is used to realize dynamic and intelligent adjustment of the control mode of each energy storage converter in the station.

[0007] In this application, the model predictive control algorithm can accurately solve the control ratio of energy storage to grid and network formation, optimize power regulation, and adapt to grid changes. The reinforcement learning algorithm combines multi-dimensional parameters to dynamically and intelligently adjust the converter control mode according to the control ratio, comprehensively evaluate, and respond in real time. The use of a two-layer optimized collaborative control with model prediction as the upper layer and reinforcement learning as the lower layer can solve the problem of grid-following / network formation collaborative control with different grid characteristics and multiple time scales, and improve the active support effect of energy storage on new power systems.

[0008] In an optional embodiment, the process of solving the control ratio of energy storage to grid-following and grid-forming includes: solving the system voltage support strength; superimposing the system voltage support strength with the strength of the energy storage system as the input of the prediction model; and combining multiple response indicators and using the prediction model to obtain the control ratio of energy storage to grid-following and grid-forming.

[0009] In an optional embodiment, the process of solving the system voltage support strength includes: obtaining the Thevenin equivalent circuit of the entire power grid based on the Thevenin equivalent principle; solving the access port voltage when the network access device is connected to the power grid and the no-load voltage modulus of the access port when the network access device is not connected to the power grid based on the Thevenin equivalent circuit; and taking the ratio of the access port voltage when the network access device is connected to the power grid to the no-load voltage modulus of the access port when the network access device is not connected to the power grid as the system voltage support strength.

[0010] In an optional implementation, the response index includes: an inertia response index, a primary frequency modulation response index, and a secondary frequency modulation response index.

[0011] In an optional embodiment, the process of dynamically and intelligently adjusting the control mode of each energy storage converter in the station includes: obtaining the health status, charge state, voltage, temperature, and energy storage grid-following and grid-forming control ratio of the energy storage system at the current moment; obtaining the working mode of each energy storage converter at the current moment; and using a reinforcement learning algorithm to obtain the working mode of each converter at the next moment based on the health status, charge state, voltage, temperature, energy storage grid-following and grid-forming control ratio, and working mode of each energy storage converter of the energy storage system at the current moment.

[0012] In an optional embodiment, the reinforcement learning algorithm includes: using the health status, state of charge, voltage, temperature, and the ratio of energy storage grid-following and grid-forming control of the energy storage system as state quantities of the evaluation value of the decision-making process in the reinforcement learning algorithm; using the control mode switching of each energy storage converter according to the decision of the intelligent agent as the action quantity of the evaluation value; constructing a behavior value function based on the evaluation value, and iteratively learning the behavior value function until the evaluation value converges.

[0013] In the second aspect, the present application provides a collaborative control device for multi-time-scale grid-following / grid-building energy storage, which includes: an upper-level control module for using a model predictive control algorithm to solve the energy storage grid-following and grid-building control ratio; a lower-level control module for using a reinforcement learning algorithm based on the energy storage grid-following and grid-building control ratio, combined with the multi-dimensional parameters of energy storage, to achieve dynamic and intelligent adjustment of the control mode of each energy storage converter in the station.

[0014] In a third aspect, the present application provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the multi-time-scale collaborative control method of grid / grid-forming energy storage of the above-mentioned first aspect or any corresponding embodiment thereof.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the multi-time-scale collaborative control method for grid-following / grid-building energy storage according to the first aspect or any corresponding embodiment thereof.

[0016] In a fifth aspect, the present application provides a computer program product comprising computer instructions, which are used to enable a computer to execute the multi-time-scale grid-following / grid-building energy storage collaborative control method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0017] Compared with related technologies, the embodiment of the present application provides a multi-time-scale collaborative control method for grid-following / grid-building energy storage, which addresses the technical difficulties of grid-following converters being prone to resonance and instability when connected to weak grids under the "double high" background of new power systems, the poor economy and multi-machine coupling of grid-building converters under strong grids, and the dynamic changes in the strength and weakness characteristics of the grid with the penetration rate of new energy and the lack of multi-time-scale collaborative control strategies. A multi-timescale grid-following / grid-forming energy storage two-layer collaborative control scheme is proposed. The upper layer uses a model predictive control algorithm to solve the grid-following / grid-forming control ratio based on voltage support strength indicators such as the grid short-circuit ratio and multi-timescale requirements such as short-term inertia support, medium-time primary frequency modulation, and long-time secondary frequency modulation. The lower layer combines real-time parameters such as energy storage health status, charge status, voltage, and temperature, and dynamically adjusts the control mode of individual converters through a reinforcement learning algorithm. This achieves a dynamic balance between the inertia support capability of the grid-forming converter in weak grid scenarios and the economic and efficient operation of the grid-following converter in strong grid scenarios, improves the response speed of the energy storage system to grid disturbances, reduces the operating costs and safety risks of all scenarios, and provides a flexible and efficient energy storage support solution for new power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 1 is a flow chart of a method for collaborative control of multi-time-scale grid / grid-connected energy storage according to an embodiment of the present application; Figure 2 is a model predictive control chart according to an embodiment of the present application; Figure 3 1 is a flow chart of solving the control ratio of energy storage grid connection and grid construction according to an embodiment of the present application; Figure 4 is a Thevenin equivalent circuit diagram according to an embodiment of the present application; Figure 5 This is a diagram of an upper-level control concept based on model predictive control according to an embodiment of the present application; Figure 6 This is a schematic diagram of a process for dynamically and intelligently adjusting the control mode of each energy storage converter in a station according to an embodiment of the present application; Figure 7 is a schematic diagram of a lower-level control strategy based on a reinforcement learning algorithm according to an embodiment of the present application; Figure 8 1 is a flow chart of another method for collaborative control of multi-time-scale grid / grid-connected energy storage according to an embodiment of the present application; Figure 9 This is a structural block diagram of a multi-time-scale coordinated control device for grid-following / grid-building energy storage according to an embodiment of the present application; Figure 10 It is a structural diagram of a computer device provided in an optional embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0021] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0022] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0023] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0024] According to an embodiment of the present application, an embodiment of a collaborative control method for multi-time-scale grid / grid-forming energy storage is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] In this embodiment, a multi-time scale grid / grid-building energy storage collaborative control method is provided, which can be used for the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 1 This is a flow chart of a method for collaborative control of multi-time scale grid / grid-based energy storage according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps: Step S1: Based on the strong and weak characteristics of the power grid and the support requirements of the system at different time scales, the model predictive control algorithm is used to solve the control ratio of energy storage following the grid and forming the grid.

[0026] In modern power systems, energy storage systems must both adjust to grid frequency and voltage fluctuations (grid-following control) and proactively maintain stable voltage and frequency support during grid failures or weak grid conditions (grid-building control). Properly allocating these two control modes is key to improving energy storage system performance. Model Predictive Control (MPC) algorithms establish a dynamic mathematical model of the energy storage system, combining the grid's real-time operating status with future forecasts to determine the optimal control strategy using a rolling optimization approach.

[0027] Specifically, model predictive control, also known as moving level control or retreating level control, is a widely used control strategy whose core principle is to use a mathematical model of the system to predict future behavior and then use this knowledge to generate control actions to maximize certain performance objectives. Model predictive control charts such as Figure 2 shown. Figure 2Reference trajectory generation involves generating a reference trajectory based on the system's desired output or setpoint. The reference trajectory is typically a time-varying curve that represents the desired state of the system over a period of time. Predictive models utilize a mathematical or empirical model of the system to predict the system's output over a period of time based on the current system state and inputs. The predictive model is the core component of model predictive control, and its accuracy directly impacts control effectiveness. Rolling optimization involves using an optimization algorithm to determine the optimal control input sequence at each sampling moment based on the predicted future output and the reference trajectory. The goal of the optimization algorithm is to minimize a performance metric, such as the error between the system output and the reference trajectory, or the change in the control input. Feedback correction involves comparing the actual system output with the output of the predictive model to generate an error signal. The predictive model is then corrected based on the error signal to improve prediction accuracy. Feedback correction can employ various feedback control methods, such as proportional-integral-derivative (PID) control. Figure 2 middle, is the reference trajectory, is the predicted trajectory, u is the trajectory output after rolling optimization, and y is the actual driving trajectory of the controlled object, that is, the current state of the controlled object. is the model prediction output.

[0028] Step S2: Based on the control ratio of energy storage following the grid and building the grid, combined with the multi-dimensional parameters of energy storage, a reinforcement learning algorithm is used to achieve dynamic and intelligent adjustment of the control mode of each energy storage converter in the station.

[0029] After determining the proportion of energy storage grid-following and grid-building control, the key challenge in optimizing energy storage system operation is how to precisely control each PCS within the station to ensure efficient collaboration within a complex and changing operating environment. Reinforcement learning (RL) algorithms enable the intelligent agent (i.e., the PCS control system) to continuously learn through trial and error through interaction with the environment (grid and energy storage system operating conditions), autonomously exploring the optimal control strategy with the goal of maximizing long-term cumulative rewards.

[0030] First, the system comprehensively considers the proportion of energy storage grid-following and grid-building control, as well as the multi-dimensional parameters of energy storage devices, such as the charge and discharge efficiency of energy storage batteries, the capacity limitations of power converters, and the response speed differences between different devices. It then constructs a reinforcement learning model consisting of a state space, an action space, and a behavior value function. The state variable describes the current operating state of the energy storage system, the action variable defines the control modes that the energy storage converter can adopt (such as constant power control or constant voltage control), and the behavior value, designed based on the control objective, evaluates the impact of different control actions on system performance. During operation, the energy storage converter selects a control mode based on its current state. After executing an action, it receives reward signals from the environment and adjusts its control strategy accordingly. Through continuous iterative training, the reinforcement learning algorithm enables the energy storage converter to adapt to different operating conditions, dynamically adjust the control mode, and achieve optimal coordination among the energy storage devices within the station, effectively improving the overall economic efficiency and reliability of the energy storage system and providing higher-quality ancillary services to the power grid.

[0031] In some optional embodiments, such as Figure 3 As shown in Figure 2, the process of solving the control ratio of energy storage to grid and grid construction includes: Step S11: Calculate the system voltage support strength.

[0032] Specifically, system voltage support strength is a key indicator for measuring the power system's ability to maintain voltage stability in the face of external disturbances. Its quantitative assessment is crucial for ensuring the safe operation of the power grid. The system short-circuit ratio (SCR) and critical short-circuit ratio (CSCR) are typically used as core parameters to characterize system voltage support strength. The SCR is calculated by calculating the ratio of the system's short-circuit capacity to the power capacity connected to the system, reflecting the system's ability to withstand short-circuit faults. A larger SCR value indicates a stronger system and a greater ability to maintain voltage stability under disturbances. The critical short-circuit ratio acts as a threshold; when the actual SCR falls below this value, the system faces the risk of voltage instability. Therefore, system voltage support strength can be used to predict whether a power grid is weak or strong.

[0033] Optionally, the actual solution process requires comprehensive consideration of factors such as grid topology, line parameters, and load distribution. Using power system analysis software, the grid short-circuit ratio can be accurately calculated through methods such as power flow and short-circuit calculations. This can then be compared and analyzed with the critical short-circuit ratio to comprehensively assess the system's voltage support strength. Furthermore, auxiliary indicators such as voltage stability limit and reactive power reserve can be incorporated to further refine and improve the assessment of system voltage support strength, ensuring the accuracy and reliability of the results.

[0034] Optionally, the process of solving the system voltage support strength includes: obtaining the Thevenin equivalent circuit of the entire power grid based on the Thevenin equivalent principle; solving the access port voltage when the network-access device is connected to the grid and the no-load voltage modulus of the access port when the network-access device is not connected to the grid based on the Thevenin equivalent circuit; and taking the ratio of the access port voltage when the network-access device is connected to the grid to the no-load voltage modulus of the access port when the network-access device is not connected to the grid as the system voltage support strength.

[0035] In an exemplary embodiment, based on the grid topology and operating status data, and according to the Thevenin equivalence principle, obtaining the Thevenin equivalent circuit of the entire grid includes the following steps: S1, with the new energy grid-connected node as the equivalent port, the grid area outside the equivalent port is converted into an active two-terminal network; S2, based on the grid topology and operating status data, solves the open-circuit voltage of the port when it is unloaded through power flow calculation to obtain the Thevenin equivalent potential; S3, set each independent power source in the fundamental frequency positive sequence network to zero, solve the port input impedance by circuit simplification, and obtain the Thevenin equivalent impedance; S4, connect the Thevenin equivalent potential and the Thevenin equivalent impedance in series to obtain the Thevenin equivalent circuit.

[0036] Specifically, according to the Thevenin equivalence principle, when looking at the grid from any point SYS in the grid, the entire grid can be represented by a Thevenin equivalent circuit, as follows: Figure 4 Thevenin equivalent circuit is composed of thevenin equivalent potential Thevenin equivalent impedance Thevenin equivalent potential Equal to the no-load voltage on SYS when the network device is not connected to the grid , the Thevenin equivalent impedance Z is equal to the equivalent impedance of the network viewed from point SYS when each independent power source in the fundamental frequency positive sequence network is set to zero. Figure 4 In (a), SYS is any point in the power grid. It is the no-load voltage on SYS when the network-connected device is not connected to the grid; Figure 4 In (b), is the Thevenin equivalent impedance, SYS is any point in the grid, is the Thevenin equivalent potential, It is the no-load voltage on SYS when the network device is not connected to the grid. It is the voltage at the port of the networked device, that is, the voltage at point SYS , is the impedance phasor of the network equipment, It is the current phasor of the network equipment.

[0037] Assuming that the no-load voltage modulus on SYS when the network device is not connected to the grid Equal to rated voltage .

[0038] in, The meaning is to express the impedance of the network equipment in phasor form; It means calculating the short-circuit capacity of the system at the access point (SYS), which is used to reflect the system's ability to provide short-circuit current; It means calculating the rated capacity of the networked equipment, reflecting the power carrying capacity of the equipment itself; The meaning is to simplify the short-circuit ratio into the ratio of the impedance of the network equipment to the Thevenin equivalent impedance of the system, which more intuitively reflects the impedance matching relationship between the system and the equipment.

[0039] The following formula is the voltage on the port of the network device after the network device is connected to the grid, that is, the voltage on point SYS .

[0040] According to the above formula, the network access device After connecting to the grid, the voltage modulus on its port is the short circuit ratio λ SCR Impedance angle and And changes.

[0041] Based on the concept of infinite power supply, the voltage support strength of any point in the power grid is defined as the ability to maintain the voltage modulus of the access point close to the no-load voltage of the access point, and is expressed as U sys / U sys0 It is characterized by the system voltage support strength. The system voltage support strength calculation formula is as follows:

[0042] From the above formula, we can know that the system voltage support strength K vtg The value range of is [0,1]. Z th When it is equal to zero, the system voltage support strength K vtg Equal to 1; whenZ th When it is equal to infinity, the system voltage support strength K vtg Equal to zero.

[0043] Step S12: superimpose the system voltage support strength and the energy storage system strength and use them as input to the prediction model.

[0044] Specifically, energy storage systems possess unique strength characteristics. Factors such as their charge and discharge capabilities, response speed, and capacity collectively determine their supporting role within the power system. Superimposing the system voltage support strength with the strength of the energy storage system aims to integrate the comprehensive characteristics of the power grid and energy storage system, providing a more comprehensive and accurate data foundation for subsequent control strategy development.

[0045] Step S13: Combining multiple response indicators, using a prediction model to obtain the energy storage grid-following and grid-building control ratios.

[0046] Optionally, the response index includes: an inertia response index, a primary frequency modulation response index, and a secondary frequency modulation response index.

[0047] After solving the voltage support strength of the system, it is superimposed with the strength of the energy storage system as the input of the model prediction. Then, combined with the inertia response and frequency modulation response results, the system's grid-following / grid-forming control ratio is output. The control idea is as follows: Figure 5 shown. Figure 5 middle, M is the system moment of inertia; D is the load damping coefficient; 1 / ( MS+D ) is the machine-network interface model; k is the integral controller coefficient; λ G is the inertia response index; λ f1 It is the primary frequency modulation response index; λ f2 is the secondary frequency modulation response index; λ k For the network / network control ratio, S represents the parameter for converting the time domain function to the frequency domain function.

[0048] In some optional embodiments, such as Figure 6 As shown in FIG, the process of dynamically and intelligently adjusting the control mode of each energy storage converter in the station includes: Step S21: Obtain the current health status, state of charge, voltage, temperature, and energy storage grid-following and grid-forming control ratio of the energy storage system.

[0049] Specifically, the stable operation of energy storage systems relies on real-time monitoring of their core parameters. To determine the current health status of the energy storage system, sensor networks and fault diagnosis technologies are required. For example, by analyzing data such as the internal resistance of the battery pack and the number of charge and discharge cycles, combined with fault prediction models, it is possible to determine whether the battery has potential risks such as aging or thermal runaway. The state of charge (SOC) can be determined using algorithms such as the ampere-hour integration method and the Kalman filter, which comprehensively consider the battery's charge and discharge current, voltage, and other parameters to accurately calculate the remaining battery capacity. Voltage parameters are directly measured using voltage sensors, providing a basis for evaluating the system's power quality and operational stability. Temperature monitoring uses temperature sensors deployed in key locations such as battery modules and converters to monitor the operating temperature of the equipment in real time and prevent safety accidents caused by overheating.

[0050] Step S22: Obtain the current working mode of each energy storage converter.

[0051] Specifically, the energy storage converter, a key device connecting energy storage equipment to the grid, has a direct impact on the efficiency of the energy storage system. The current operating mode of each energy storage converter can be read in real time through the converter's communication interface.

[0052] Step S23: Based on the current state of the energy storage system, state of charge, voltage, temperature, energy storage grid-following and grid-forming control ratio, and the operating mode of each energy storage converter, a reinforcement learning algorithm is used to determine the operating mode of each converter at the next moment.

[0053] In some optional embodiments, the reinforcement learning algorithm includes: using the health status, state of charge, voltage, temperature, and the ratio of energy storage grid-following and grid-forming control of the energy storage system as state quantities of the evaluation value of the decision-making process in the reinforcement learning algorithm; using the control mode switching of each energy storage converter according to the decision of the intelligent agent as the action quantity of the evaluation value; constructing a behavior value function based on the evaluation value, and iteratively learning the behavior value function until the evaluation value converges.

[0054] Specifically, the lower-level control method is based on the network-following / network-building control ratio λ solved by the upper-level k , combined with the energy storage health status (SOH), state of charge (SOC), voltage (U) and temperature (T), a reinforcement learning algorithm is used to dynamically adjust the control mode of each energy storage converter in the station.

[0055] Specifically, the reinforcement learning algorithm learns from its own continuous exploration experience and does not need to know the environment model, so it does not need to know the state transition function. Just select the maximum value from the table, which greatly simplifies the decision-making process. The values in the table are the result of step-by-step iterative learning. The agent needs to constantly interact with the environment to enrich table to cover all possible situations. If the values in the table no longer change significantly, it means that the results have converged.

[0056] The iterative calculation formula for the behavior value function of the decision-making process in the reinforcement learning algorithm is:

[0057] in, is the state-behavior pair of the decision-making process at time t; s t+1 is the state at time t+1; γ is the discount factor, which reflects the importance of the reward value of the next action to the Q value of this action; α is the learning factor, which determines the extent to which new information covers old information; Meaning: in State of the moment Next, for all possible actions , calculate the corresponding Value, and then take the largest value, which is equivalent to being in state Under this state, determine which action can obtain the greatest expected value; where the formula refers to the same physical quantity (i.e., the action at time t+1). is the function parameter, subscript Is the traversal identifier.

[0058] Specifically, Figure 7 This is a schematic diagram of the lower-level control strategy based on the reinforcement learning algorithm. The reinforcement learning algorithm is used to control the operating mode of each energy storage converter in the station, so that it can meet the dual goals of strong support for the power grid and economic operation at different time scales. The grid-following / grid-forming control ratio, energy storage health status (SOH), state of charge (SOC), voltage (U) and temperature (T) of the station in this application are all state variables of the reinforcement learning algorithm. s t Each energy storage converter switches its control mode according to the decision of the intelligent agent, which is an action of the reinforcement learning algorithm. , What is learned is the evaluation value of each state-action pair, that is, the state-action evaluation value when the power grid system strength changes and meets the active support requirements of the energy storage station. After a period of iterative learning, The values in the table will stabilize, indicating that the learning results have converged. At this time, the goal of active and efficient operation of the energy storage station under different grid system strengths can be met.

[0059] Specifically, Figure 8 This is a flow chart of the collaborative control method for multi-time scale grid / grid-connected energy storage. Figure 8 In the process, the voltage and equivalent impedance of the power grid port in the area where the energy storage station is located at time t are first read. Based on the read data, the strength of the power grid system in the area where the energy storage station is located at time t is calculated. Secondly, the calculated system strength and the input system support requirements at different time scales at time t are combined, and the model predictive control algorithm is used to obtain the grid-following / grid-forming controller ratio λ in the station. k After that, we judge λ k Is there a change? If λ k If there is no change, read the working mode of each converter at time t. k If there is a change, then the health status (SOH), state of charge (SOC), voltage (U) and temperature (T) of each energy storage system at time t are input. k Regardless of whether there is a change, the relevant information (such as the converter working mode or the energy storage system status) is input into the reinforcement learning algorithm. Through the reinforcement learning algorithm, the working mode of each converter at time t+1 is output, and then the process is looped back to the data reading step to continue the next round.

[0060] In this embodiment, a multi-time scale coordinated control device for grid / grid energy storage is also provided. The device is used to implement the above-mentioned embodiments and preferred implementation methods, and the details that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0061] This embodiment provides a multi-time scale grid / grid-building energy storage collaborative control device. Figure 9 This is a structural block diagram of a multi-time scale coordinated control device for grid / grid energy storage according to an embodiment of the present application, such as Figure 9 As shown, the device includes: The upper control module 90 is configured to employ a model predictive control algorithm to determine the energy storage grid-following and grid-forming control ratios, including: determining the system voltage support strength; superimposing the system voltage support strength with the energy storage system strength as input to a prediction model; and utilizing the prediction model to determine the energy storage grid-following and grid-forming control ratios based on multiple response indicators. The lower control module 91 is used to realize dynamic intelligent adjustment of the control mode of each energy storage converter in the station based on the control ratio of energy storage following the grid and building the grid, combined with the multi-dimensional parameters of energy storage, and using the reinforcement learning algorithm.

[0062] It can be understood that this device is used as a multi-timescale grid-following / grid-forming energy storage collaborative control device for new power systems, and achieves precise adaptation of the energy storage system to the dynamic characteristics of the power grid through two-layer control logic. The core logic of this device is based on the "model predictive control (MPC) + reinforcement learning" two-layer optimization proposed in the document: through two-layer collaborative control, this device breaks through the limitations of traditional single control modes in adaptability to strong and weak power grids and multi-timescale adjustment capabilities. It not only solves the resonance risk of grid-following converters in weak power grids and the cost and stability issues of grid-forming converters in strong power grids, but also improves the active support effect of the energy storage system on the new power system by dynamically matching the real-time needs of the power grid, providing a technical solution for grid stability and economic operation in scenarios with a high proportion of new energy grid connection.

[0063] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0064] In this embodiment, the multi-time-scale coordinated control device for grid-following / grid-building energy storage is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0065] An embodiment of the present application also provides a computer device having the above-mentioned multi-time scale network tracking / network-building energy storage collaborative control device.

[0066] See also Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present application. Figure 10 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 10 A processor 10 is taken as an example.

[0067] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0068] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0069] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0070] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0071] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 10 The bus connection is taken as an example.

[0072] The input device 30 can receive input digital or character information and generate key signal input related to user settings and function control of the computer device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 40 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). Such display devices include, but are not limited to, liquid crystal displays, light emitting diodes, monitors, and plasma displays. In some optional embodiments, the display device may be a touch screen.

[0073] The embodiments of the present application also provide a computer-readable storage medium. The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0074] Part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0075] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A multi-time scale coordinated control method for grid / grid-connected energy storage, characterized in that: The method comprises: Based on the strong and weak characteristics of the power grid and the requirements for system support at different time scales, a model predictive control algorithm is used to solve the control ratio of energy storage to grid and grid construction, including: Calculate the system voltage support strength; The system voltage support strength and the energy storage system strength are superimposed and used as input to the prediction model; Combining multiple response indicators, the prediction model is used to obtain the energy storage grid-following and grid-building control ratios; Based on the control ratio of energy storage following the grid and building the grid, combined with the multi-dimensional parameters of energy storage, a reinforcement learning algorithm is used to achieve dynamic intelligent adjustment of the control mode of each energy storage converter in the station.

2. The multi-time-scale coordinated control method for grid-connected / grid-structured energy storage according to claim 1 is characterized in that: The process of solving the system voltage support strength includes: Based on the grid topology and operating status data, the Thevenin equivalent circuit of the entire grid is obtained according to the Thevenin equivalent principle. Based on the Thevenin equivalent circuit, the voltage at the access port of the network access device when the network access device is connected to the grid and the no-load voltage modulus of the access port when the network access device is not connected to the grid are calculated; The ratio of the access port voltage when the network access device is connected to the grid to the no-load voltage modulus of the access port when the network access device is not connected to the grid is used as the system voltage support strength.

3. The method according to claim 2, characterized in that Based on the grid topology and operating status data, and according to the Thevenin equivalent principle, the Thevenin equivalent circuit of the entire grid is obtained, which includes: Taking the new energy grid-connected node as the equivalent port, the grid area outside the equivalent port is converted into an active two-terminal network; Based on the grid topology and the operating status data, solving the open circuit voltage of the port when it is unloaded through power flow calculation to obtain the Thevenin equivalent potential; Set each independent power source in the fundamental frequency positive sequence network to zero, solve the port input impedance by circuit simplification, and obtain the Thevenin equivalent impedance; The Thevenin equivalent potential and the Thevenin equivalent impedance are connected in series to obtain the Thevenin equivalent circuit of the entire power grid.

4. The multi-time-scale coordinated control method for grid-connected / grid-structured energy storage according to claim 2 is characterized in that: The response index includes: an inertia response index, a primary frequency modulation response index and a secondary frequency modulation response index.

5. The multi-time-scale coordinated control method for grid-following / grid-building energy storage according to claim 1 is characterized in that: The process of dynamic intelligent adjustment of the control mode of each energy storage converter in the station includes: Obtain the current health status, state of charge, voltage, temperature, and energy storage grid-following and grid-forming control ratio of the energy storage system; Obtain the working mode of each energy storage converter at the current moment; Based on the current health status, state of charge, voltage, temperature, energy storage grid-following and grid-forming control ratio, and the operating mode of each energy storage converter, the reinforcement learning algorithm is used to obtain the operating mode of each converter at the next moment.

6. The multi-time-scale coordinated control method for grid-connected / grid-structured energy storage according to claim 5 is characterized in that: Reinforcement learning algorithms include: The health status, state of charge, voltage, temperature, and the ratio of energy storage to grid control are used as state variables for evaluating the decision-making process in the reinforcement learning algorithm. Switching the control mode of each energy storage converter according to the decision of the intelligent agent is used as the action amount of the evaluation value; A behavior value function is constructed based on the evaluation value, and the behavior value function is iteratively learned until the evaluation value converges.

7. A multi-time scale coordinated control device for grid / grid energy storage, characterized in that: The device comprises: The upper-level control module is used to use a model predictive control algorithm to solve the energy storage grid-following and grid-forming control ratio, including: solving the system voltage support strength; superimposing the system voltage support strength with the strength of the energy storage system as the input of the prediction model; combining multiple response indicators and using the prediction model to obtain the energy storage grid-following and grid-forming control ratio; The lower-level control module is used to realize dynamic intelligent adjustment of the control mode of each energy storage converter in the station based on the energy storage grid following and grid construction control ratio, combined with the multi-dimensional parameters of the energy storage, and using a reinforcement learning algorithm.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the collaborative control method for multi-time-scale grid-following / grid-forming energy storage according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the multi-time-scale grid-following / grid-building energy storage collaborative control method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the collaborative control method of multi-time-scale grid-following / grid-building energy storage according to any one of claims 1 to 6.

Images