A collaborative control method and device for multi-timescale grid / grid-based energy storage

Abstract

This application relates to the field of power system technology and discloses a collaborative control method and device for grid-connected / network-connected energy storage across multiple time scales. The method includes: a model predictive control algorithm that accurately solves the grid-connected and network-connected control ratios of energy storage, optimizes power regulation, and adapts to grid changes; and a reinforcement learning algorithm that combines multi-dimensional parameters to dynamically and intelligently adjust the converter control mode based on the control ratios, providing comprehensive evaluation and real-time response. This two-layer optimized collaborative control approach, with the upper layer primarily based on model prediction and the lower layer primarily based on reinforcement learning, can solve the grid-connected / network-connected collaborative control problem across different grid characteristics and multiple time scales, and improve the active support effect of energy storage for new power systems.

Description

Technical Field

[0001] This application relates to the field of power system technology, and in particular to a collaborative control method and device for multi-timescale grid-connected / grid-connected energy storage. Background Technology

[0002] Currently, most converters in new energy power plants and energy storage systems adopt grid-connected control. However, in areas with high new energy penetration, the grid characteristics are often weak. When grid-connected converters are connected to weak grids, they are prone to oscillations, posing a risk of system instability. To address this issue, introducing grid-based control with strong support capabilities has become an important direction. However, converting all energy storage converters to grid-based control is not a good solution. On the one hand, it would significantly increase construction costs; on the other hand, in areas with strong grid characteristics, grid-based control is also prone to causing safety accidents.

[0003] Furthermore, with the large-scale grid connection of new energy sources and the application of various grid support technologies, the strength and weakness characteristics of the power grid are constantly changing. The requirements of the new power system for grid characteristics are not fixed and vary significantly across different time scales. In the short term, when the power grid is subjected to small disturbances, it needs to possess strong characteristics to quickly resist interference and maintain stable operation. In the medium to long term, when large power sources or loads are disconnected from the grid, the flexible control resources in the power system are required to respond quickly, effectively reduce grid inertia, and ensure the balance and stability of the system.

[0004] In summary, existing technologies have significant shortcomings in addressing grid-connected / network-integrated coordinated control under different grid characteristics and multiple time scales. Therefore, there is an urgent need to find an effective technical means to solve the problem of grid-connected / network-integrated coordinated control under different grid characteristics and multiple time scales, thereby improving the active support effect of energy storage for new power systems. Summary of the Invention

[0005] In view of this, this application provides a method and device for coordinated control of grid-connected / network-connected energy storage across multiple time scales, in order to address the significant shortcomings in coordinated control of grid-connected / network-connected energy storage under different grid characteristics and multiple time scales.

[0006] Firstly, this application provides a collaborative control method for grid-connected / grid-connected energy storage across multiple time scales. The method includes: based on the characteristics of the power grid and the system support requirements at different time scales, using a model predictive control algorithm to solve the control ratio between grid-connected and grid-connected energy storage; and based on the control ratio between grid-connected and grid-connected energy storage, combined with the multi-dimensional parameters of energy storage, using a reinforcement learning algorithm to achieve dynamic and intelligent adjustment of the control mode of each energy storage converter in the power station.

[0007] In this application, the model predictive control algorithm can accurately solve the control ratio between energy storage and grid integration, optimize power regulation, and adapt to grid changes. The reinforcement learning algorithm, combined with multi-dimensional parameters, dynamically and intelligently adjusts the converter control mode based on the control ratio, providing comprehensive evaluation and real-time response. This two-layer optimized collaborative control approach, with the upper layer primarily based on model prediction and the lower layer primarily based on reinforcement learning, can solve the problem of grid integration / network integration collaborative control across different grid characteristics and multiple time scales, and improve the active support effect of energy storage for new power systems.

[0008] In one optional implementation, the process of determining the energy storage-grid integration and grid construction control ratio includes: determining the system voltage support strength; superimposing the system voltage support strength and the strength of the energy storage system as the input to the prediction model; and combining multiple response indicators to obtain the energy storage-grid integration and grid construction control ratio using the prediction model.

[0009] In one optional implementation, the process of solving the system voltage support strength includes: obtaining the Thevenin equivalent circuit of the entire power grid according to the Thevenin equivalent principle; based on the Thevenin equivalent circuit, solving the access port voltage when the connected equipment is connected to the power grid and the no-load voltage magnitude of the access port when the connected equipment is not connected to the power grid; and taking the ratio of the access port voltage when the connected equipment is connected to the power grid to the no-load voltage magnitude of the access port when the connected equipment is not connected to the power grid as the system voltage support strength.

[0010] In one optional implementation, the response metrics include: inertia response metrics, primary frequency modulation response metrics, and secondary frequency modulation response metrics.

[0011] In one optional implementation, the process of dynamically and intelligently adjusting the control modes of each energy storage converter within the power station includes: acquiring the current health status, state of charge, voltage, temperature, and grid connection and grid construction control ratio of the energy storage system; acquiring the current operating mode of each energy storage converter; and, based on the current health status, state of charge, voltage, temperature, grid connection and grid construction control ratio of the energy storage system, and the operating mode of each energy storage converter, using a reinforcement learning algorithm to obtain the operating mode of each converter for the next moment.

[0012] In one optional implementation, the reinforcement learning algorithm includes: using the health status, state of charge, voltage, temperature, and the ratio of energy storage to grid control in the energy storage system as state variables for the evaluation value of the decision-making process in the reinforcement learning algorithm; using the switching of control modes of each energy storage converter according to the agent's decision as the action variable for the evaluation value; constructing a behavior value function based on the evaluation value, and iteratively learning the behavior value function until the evaluation value converges.

[0013] Secondly, this application provides a multi-timescale grid-connected / grid-connected energy storage collaborative control device, the device comprising: an upper-level control module, used to solve the grid-connected and grid-connected control ratio of energy storage using a model predictive control algorithm; and a lower-level control module, used to dynamically and intelligently adjust the control mode of each energy storage converter in the site based on the grid-connected and grid-connected control ratio of energy storage, combined with the multi-dimensional parameters of energy storage, using a reinforcement learning algorithm.

[0014] Thirdly, this application provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the multi-timescale grid-connected / grid-based energy storage collaborative control method described in the first aspect or any of its corresponding embodiments.

[0015] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the multi-timescale grid / grid-based energy storage collaborative control method described in the first aspect or any of its corresponding embodiments.

[0016] Fifthly, this application provides a computer program product, including computer instructions, which are used to cause a computer to execute the multi-timescale grid / grid-based energy storage collaborative control method described in the first aspect or any of its corresponding embodiments.

[0017] Compared to related technologies, the present application provides a multi-timescale collaborative control method for grid-connected / grid-connected energy storage, which addresses the technical challenges of grid-connected converters being prone to resonance instability when connected to weak grids under the "dual high" background of new power systems, poor economic efficiency and multi-machine coupling of grid-connected converters under strong grids, as well as the dynamic changes in grid strength and weakness characteristics with the penetration rate of new energy sources and the lack of multi-timescale collaborative control strategies. A multi-timescale grid-connected / grid-connected energy storage dual-layer collaborative control scheme is proposed. The upper layer uses a model predictive control algorithm to solve the grid-connected / grid-connected control ratio based on the grid short-circuit ratio and other voltage support strength indicators, as well as the requirements of short-term inertia support, medium-term primary frequency regulation, and long-term secondary frequency regulation. The lower layer combines real-time parameters such as energy storage health status, state of charge, voltage, and temperature, and dynamically adjusts the control mode of individual converters through reinforcement learning algorithms. This achieves a dynamic balance between the inertia support capability of grid-connected converters in weak grid scenarios and the economical and efficient operation of grid-connected converters in strong grid scenarios. This improves the response speed of the energy storage system to grid disturbances, reduces the overall operating cost and safety risks, and provides a flexible and efficient energy storage support solution for new power systems. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the multi-timescale coordinated control method for grid / networked energy storage according to an embodiment of this application.

[0020] Figure 2 It is a model predictive control chart according to an embodiment of this application;

[0021] Figure 3 This is a schematic flowchart illustrating the process of solving the energy storage-grid and grid-building control ratio according to an embodiment of this application;

[0022] Figure 4 This is a Thevenin equivalent circuit diagram according to an embodiment of this application;

[0023] Figure 5 This is a diagram illustrating the upper-level control logic of model predictive control according to an embodiment of this application.

[0024] Figure 6 This is a flowchart illustrating the dynamic intelligent adjustment of the control mode of each energy storage converter in the power station according to an embodiment of this application.

[0025] Figure 7 This is a schematic diagram of a lower-level control strategy based on a reinforcement learning algorithm according to an embodiment of this application;

[0026] Figure 8 This is a flowchart illustrating another multi-timescale coordinated control method for grid / networked energy storage according to an embodiment of this application;

[0027] Figure 9 This is a structural block diagram of a multi-timescale coordinated control device for grid / networked energy storage according to an embodiment of this application;

[0028] Figure 10 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0030] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0031] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0032] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0033] According to an embodiment of this application, a collaborative control method for multi-timescale grid / grid-based energy storage is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] This embodiment provides a multi-timescale coordinated control method for grid-connected / grid-based energy storage, which can be used in the aforementioned mobile terminals, such as mobile phones and tablets. Figure 1 This is a flowchart of a multi-timescale coordinated control method for grid / grid-based energy storage according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0035] Step S1: Based on the characteristics of the power grid and the system support requirements at different time scales, a model predictive control algorithm is used to solve the control ratio of energy storage to grid and grid construction.

[0036] In modern power systems, energy storage systems need to both adjust to changes in grid frequency and voltage (grid-following control) and proactively build stable voltage and frequency support in the event of grid faults or weak grid conditions (grid-building control). The key to improving the performance of energy storage systems lies in how to rationally allocate the ratio between these two control modes. Model Predictive Control (MPC) algorithms establish a dynamic mathematical model of the energy storage system and combine it with real-time grid operating conditions and future predictions to solve for the optimal control strategy through rolling optimization.

[0037] Specifically, model predictive control, also known as moving-level control or retreating-level control, is a widely used control strategy. Its core principle is to use a mathematical model of the system to predict future behavior, and then use this knowledge to generate control actions to maximize certain performance objectives. A model predictive control diagram is shown below. Figure 2 As shown. Figure 2 Reference trajectory generation involves generating a reference trajectory based on the system's desired output or setpoint. The reference trajectory is typically a time-varying curve representing the system's expected state over a future period. Predictive modeling involves using a mathematical or empirical model of the system to predict its output over a future period, based on the current system state and inputs. The predictive model is the core of model predictive control, and its accuracy directly impacts control performance. Rolling optimization involves solving for the optimal control input sequence at each sampling time, based on the predicted future output and reference trajectory, using an optimization algorithm. The optimization algorithm aims to minimize a performance metric, such as the error between the system output and the reference trajectory, or the change in control input. Feedback correction involves comparing the actual system output with the predictive model output to obtain an error signal. The predictive model is then corrected based on this error signal to improve prediction accuracy. Feedback correction can employ various feedback control methods, such as proportional-integral-derivative (PID) control. Figure 2 middle, It is a reference trajectory. This is the predicted trajectory, where u is the trajectory output after scrolling optimization, and y is the actual trajectory of the controlled object, i.e., the current state of the controlled object. It is the output of the model's prediction.

[0038] Step S2: Based on the control ratio of energy storage to grid and grid construction, and combined with the multi-dimensional parameters of energy storage, a reinforcement learning algorithm is used to realize the dynamic and intelligent adjustment of the control mode of each energy storage converter in the power station.

[0039] After determining the control ratio between energy storage and the grid, the core issue for optimizing the operation of energy storage systems is how to precisely regulate each energy storage converter (PCS) within the power station to ensure efficient and coordinated operation in a complex and ever-changing environment. Reinforcement learning (RL) algorithms enable the agent (i.e., the energy storage converter control system) to learn through trial and error in its interaction with the environment (grid and energy storage system operating conditions), aiming to maximize long-term cumulative rewards and autonomously explore the optimal control strategy.

[0040] First, the system comprehensively considers the ratio of energy storage integration with the grid to grid-connected control, as well as multi-dimensional parameters of energy storage devices, such as the charging and discharging efficiency of energy storage batteries, the capacity limitations of converters, and the differences in response speed among different devices, to construct a reinforcement learning model that includes a state space, action space, and behavioral value functions. State variables describe the current operating state of the energy storage system, action variables define the control modes that the energy storage converter can adopt (such as constant power control, constant voltage control, etc.), and behavioral values ​​are designed according to the control objectives to evaluate the impact of different control actions on system performance. During operation, the energy storage converter selects a control mode based on the current state, receives reward signals from the environment after executing actions, and adjusts its control strategy accordingly. Through continuous iterative training, the reinforcement learning algorithm enables the energy storage converter to adapt to different operating conditions, dynamically adjust the control mode, achieve optimal coordination among various energy storage devices within the site, effectively improve the overall economy and reliability of the energy storage system, and provide higher-quality ancillary services to the power grid.

[0041] In some alternative implementations, such as Figure 3 As shown, the process of solving the energy storage-grid and grid-connection control ratio includes:

[0042] Step S11: Solve for the system voltage support strength.

[0043] Specifically, system voltage support strength is a key indicator measuring the power system's ability to maintain voltage stability in the face of external disturbances, and its quantitative assessment is crucial for ensuring the safe operation of the power grid. The grid short-circuit ratio (SCR) and critical short-circuit ratio (CSCR) are typically used as core parameters to characterize system voltage support strength. The grid short-circuit ratio is obtained by calculating the ratio of the system's short-circuit capacity to the capacity of the power sources connected to the system, reflecting the system's tolerance to short-circuit faults. A higher short-circuit ratio indicates a stronger system and a greater ability to maintain voltage stability under disturbances. The critical short-circuit ratio serves as a threshold; when the actual grid short-circuit ratio falls below this value, the system faces the risk of voltage instability. Therefore, system voltage support strength can be used to predict whether the power grid is weak or strong.

[0044] Optionally, in the actual solution process, it is necessary to comprehensively consider factors such as the power grid topology, line parameters, and load distribution. Using power system analysis software, through methods such as power flow calculation and short-circuit calculation, the short-circuit ratio of the power grid is accurately calculated and compared with the critical short-circuit ratio to comprehensively evaluate the system voltage support strength. At the same time, auxiliary indicators such as voltage stability limits and reactive power reserves are also combined to further refine and improve the evaluation of the system voltage support strength, ensuring the accuracy and reliability of the evaluation results.

[0045] Optionally, the process of solving for the system voltage support strength includes: obtaining the Thevenin equivalent circuit of the entire power grid according to the Thevenin equivalent principle; based on the Thevenin equivalent circuit, solving for the access port voltage when the connected equipment is connected to the power grid and the no-load voltage magnitude of the access port when the connected equipment is not connected to the power grid; and taking the ratio of the access port voltage when the connected equipment is connected to the power grid to the no-load voltage magnitude of the access port when the connected equipment is not connected to the power grid as the system voltage support strength.

[0046] In an exemplary embodiment, obtaining the Thevenin equivalent circuit of the entire power grid based on the power grid topology and operating status data, according to the Thevenin equivalent principle, includes the following steps:

[0047] S1 uses the new energy grid-connected node as the equivalent port, and the power grid area outside the equivalent port is converted into an active two-terminal network;

[0048] S2, based on the power grid topology and operating status data, calculates the open-circuit voltage of the port under no-load conditions through power flow calculation to obtain the Thevenin equivalent potential;

[0049] S3, set each independent power source in the fundamental frequency positive sequence network to zero, and solve the port input impedance by simplifying the circuit to obtain the Thevenin equivalent impedance;

[0050] S4 connects the Thevenin equivalent potential and the Thevenin equivalent impedance in series to obtain the Thevenin equivalent circuit.

[0051] Specifically, according to the Thevenin equivalence principle, when looking at the power grid from any point SYS in the power grid, the entire power grid can be represented by a Thevenin equivalent circuit, such as... Figure 4 As shown. The Thevenin equivalent circuit is derived from the Thevenin equivalent potential. Thevenin equivalent impedance Composed of phases connected in series. Thevenin equivalent potential. Equal to the no-load voltage on SYS at the point when the connected equipment is not connected to the power grid. The Thevenin equivalent impedance Z is equal to the equivalent impedance of the network viewed from point SYS when all independent power sources in the fundamental frequency positive sequence network are set to zero. Figure 4 In (a) of the diagram, SYS is any point in the power grid. It is the no-load voltage on SYS at the point when the network-connected equipment is not connected to the power grid; Figure 4 In (b) of the middle, SYS is the Thevenin equivalent impedance, and SYS is any point in the power grid. It is the Thevenin equivalent potential. It is the no-load voltage on SYS at the point when the connected equipment is not connected to the power grid. It is the voltage at the port of the network-connected device, i.e., the voltage at point SYS. , It is the impedance phasor of the network-connected equipment. It is the current phasor of the network-connected equipment.

[0052] Assuming the no-load voltage magnitude on SYS at a point when the connected equipment is not connected to the power grid equal to rated voltage And set

[0053]

[0054]

[0055]

[0056]

[0057] in, This means representing the impedance of the network-connected device in phasor form.

[0058] It means the short-circuit capacity of the system at the connection point (SYS), which reflects the system's ability to provide short-circuit current;

[0059] It means to calculate the rated capacity of the network-connected equipment, reflecting the power carrying capacity of the equipment itself;

[0060] The short-circuit ratio is simplified to the ratio of the impedance of the connected equipment to the Thevenin equivalent impedance of the system, which more intuitively reflects the impedance matching relationship between the system and the equipment.

[0061] The following formula represents the voltage at the port of the connected device after it is connected to the power grid, i.e., the voltage at point SYS. .

[0062] According to the above formula, the network access device After being connected to the power grid, the voltage magnitude at its port It depends on the short-circuit ratio λ SCR Impedance angle and And it changes.

[0063] Based on the concept of an infinite power source, the voltage support strength at any point in the power grid is defined as the ability to maintain the voltage magnitude at the connection point close to the no-load voltage at the connection point, and is used as... U sys / U sys0 This is characterized by what is called the system voltage support strength. The formula for calculating the system voltage support strength is as follows:

[0064]

[0065] From the above formula, we can see that the system voltage support strength K vtg The value range is [0, 1]. When Z th When equal to zero, the system voltage support strength K vtg Equal to 1; when Z th When the voltage is equal to infinity, the system voltage support strength K vtg It equals zero.

[0066] Step S12: The system voltage support strength and the strength of the energy storage system are superimposed and used as the input of the prediction model.

[0067] Specifically, energy storage systems possess unique strength characteristics. Their charging and discharging capabilities, response speed, and capacity collectively determine the supporting role they can play in the power system. Superimposing the system voltage support strength with the strength of the energy storage system aims to integrate the combined characteristics of the power grid and the energy storage system, providing a more comprehensive and accurate data foundation for subsequent control strategy formulation.

[0068] Step S13: Combine multiple response indicators and use a prediction model to obtain the energy storage grid connection and grid construction control ratio.

[0069] Optionally, the response metrics include: inertia response metrics, primary frequency modulation response metrics, and secondary frequency modulation response metrics.

[0070] After solving for the voltage support strength of the system, it is superimposed with the strength of the energy storage system and used as the input for model prediction. Combined with the inertial response and frequency modulation response results, the grid-following / grid-connecting control ratio of the system is output. The control logic diagram is as follows: Figure 5 As shown. Figure 5 middle, M Let the system's rotational inertia be denoted by . D For load damping coefficient; 1 / ( MS+D This is the machine-network interface model; k These are the integral controller coefficients; λG Inertial response index; λ f1 This refers to the primary frequency regulation response index; λ f2 This refers to the secondary frequency modulation response index; λ k For the network / network control ratio, S represents the parameters for transforming the time-domain function to the frequency-domain function.

[0071] In some alternative implementations, such as Figure 6 As shown, the process of dynamically and intelligently adjusting the control mode of each energy storage converter in the power station includes:

[0072] Step S21: Obtain the current health status, state of charge, voltage, temperature, and grid connection and control ratio of the energy storage system at the current moment.

[0073] Specifically, the stable operation of an energy storage system depends on the real-time monitoring of its core parameters. To obtain the current health status of the energy storage system, sensor networks and fault diagnosis technologies are required. For example, by analyzing data such as the internal resistance and charge-discharge cycle count of the battery pack, combined with fault prediction models, it is possible to determine whether the battery has potential risks such as aging or thermal runaway. The state of charge (SOC) can be obtained using algorithms such as the ampere-hour integral method and Kalman filtering, comprehensively considering parameters such as the battery's charging and discharging current and voltage to accurately calculate the remaining battery capacity. Voltage parameters are directly measured by voltage sensors, providing a basis for assessing the system's power quality and operational stability. Temperature monitoring involves deploying temperature sensors in key components such as battery modules and inverters to monitor the equipment's operating temperature in real time, preventing safety accidents caused by overheating.

[0074] Step S22: Obtain the current operating mode of each energy storage converter.

[0075] Specifically, as a key device connecting energy storage equipment to the power grid, the operating mode of the energy storage converter directly affects the operating efficiency of the energy storage system. The current operating mode of each energy storage converter can be read in real time through the converter's communication interface.

[0076] Step S23: Based on the current health status, state of charge, voltage, temperature, energy storage grid connection and grid construction control ratio, and operating mode of each energy storage converter, use reinforcement learning algorithm to obtain the operating mode of each converter at the next moment.

[0077] In some optional implementations, the reinforcement learning algorithm includes: using the health status, state of charge, voltage, temperature, and the ratio of energy storage to grid control as state variables for the evaluation value of the decision-making process in the reinforcement learning algorithm; using the switching of control modes of each energy storage converter according to the agent's decision as the action variable for the evaluation value; constructing a behavior value function based on the evaluation value, and iteratively learning the behavior value function until the evaluation value converges.

[0078] Specifically, the lower-level control method is based on the following / networking control ratio λ solved in the upper-level solution. k By combining energy storage health status (SOH), state of charge (SOC), voltage (U), and temperature (T), a reinforcement learning algorithm is used to dynamically adjust the control mode of each energy storage converter in the site.

[0079] Specifically, reinforcement learning algorithms learn from their own accumulated experience and do not need to know the environment model, nor do they need to know the state transition function. When making decisions, they only need to learn from... Simply select the maximum value from the table, which greatly simplifies the decision-making process. The values ​​in the table are the result of iterative learning step by step. The agent needs to continuously interact with the environment to enrich itself. A table was created to cover all possible scenarios. After iterating for a period of time... If the values ​​in the table no longer change significantly, it indicates that the results have converged.

[0080] The iterative calculation formula for the behavior-value function in the decision-making process of reinforcement learning algorithms is as follows:

[0081]

[0082] in, For the state-behavior pair of the decision-making process at time t; s t+1 The state at time t+1; γ is the discount factor, reflecting the importance of the reward value of the next action to the Q value of this action; α The learning factor determines the degree to which new information covers old information; among them, Meaning: In state of time Next, for all possible actions Calculate the corresponding The values ​​are then taken, and the largest value is selected, which is equivalent to the state. In this state, determine which action will yield the greatest expected value; where, in the formula... They refer to the same physical quantity (i.e., the action at time t+1), the part in parentheses It is a function parameter, subscript It is a traversal identifier.

[0083] Specifically, Figure 7 This diagram illustrates the underlying control strategy based on a reinforcement learning algorithm. The algorithm controls the operation mode of each energy storage converter within the power station, ensuring it meets the dual objectives of strong grid support and economical operation at different time scales. In this application, the grid-connected / grid-connected control ratio, energy storage health status (SOH), state of charge (SOC), voltage (U), and temperature (T) of the power station are all state variables of the reinforcement learning algorithm. s t The switching of control modes by each energy storage converter based on the decisions of the intelligent agent is an action of the reinforcement learning algorithm. , The learning process focuses on the evaluation value of each state-action pair, specifically the state-action evaluation value under varying grid system strength conditions while meeting the active support requirements of energy storage facilities. After a period of iterative learning... Once the values ​​in the table stabilize, it indicates that the learning results have converged, and at this point, the goal of active and efficient operation of energy storage stations under different power grid system intensities can be achieved.

[0084] Specifically, Figure 8 A flowchart illustrating a multi-timescale coordinated control method for grid / grid-based energy storage. Figure 8 First, data such as the grid port voltage and equivalent impedance of the area where the energy storage station is located are read at time t. Based on the read data, the grid system strength of the area where the energy storage station is located at time t is calculated. Second, combining the calculated system strength and the system support requirements at different time scales at time t, a model predictive control algorithm is applied to obtain the grid-connected / grid-building controller ratio λ within the station. k Then, determine λ. k Has there been any change, if λ k If there is no change, then read the operating mode of each converter at time t. If λ k If there are changes, input the state of health (SOH), state of charge (SOC), voltage (U), and temperature (T) of each energy storage system at time t. Wherein, regardless of λ... k Regardless of whether there are changes, relevant information (such as converter operating mode or energy storage system status) is input into the reinforcement learning algorithm. Through the reinforcement learning algorithm, the operating mode of each converter at time t+1 is output, and then the process loops back to the data reading step to continue the next round of the process.

[0085] This embodiment also provides a multi-timescale grid-connected / grid-based energy storage collaborative control device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0086] This embodiment provides a multi-timescale coordinated control device for grid / grid-based energy storage. Figure 9 This is a structural block diagram of a multi-timescale grid / grid-based energy storage collaborative control device according to an embodiment of this application, such as... Figure 9 As shown, the device includes:

[0087] The upper-level control module 90 is used to solve the energy storage grid connection and grid construction control ratio using a model predictive control algorithm, including: solving the system voltage support strength; superimposing the system voltage support strength and the strength of the energy storage system as the input of the prediction model; and combining multiple response indicators to obtain the energy storage grid connection and grid construction control ratio using the prediction model.

[0088] The lower-level control module 91 is used to dynamically and intelligently adjust the control mode of each energy storage converter in the station based on the control ratio of energy storage to grid and network construction, combined with the multi-dimensional parameters of energy storage, and using reinforcement learning algorithms.

[0089] This device is understood to be applied to multi-timescale grid-connected / grid-connected energy storage collaborative control devices in new power systems. It achieves precise adaptation of the energy storage system to the dynamic characteristics of the power grid through a two-layer control logic. The core logic of this device is based on the "Model Predictive Control (MPC) + Reinforcement Learning" two-layer optimization proposed in the document. Through two-layer collaborative control, this device overcomes the limitations of traditional single control modes in adaptability to strong and weak power grids and multi-timescale regulation capabilities. It solves the resonance risk of grid-connected converters in weak power grids and the cost and stability issues of grid-connected converters in strong power grids. Furthermore, by dynamically matching the real-time demands of the power grid, it enhances the active support effect of the energy storage system for new power systems, providing a technical solution for grid stability and economic operation in scenarios with a high proportion of new energy grid integration.

[0090] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0091] In this embodiment, the multi-timescale grid / grid-based energy storage collaborative control device is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0092] This application also provides a computer device having the above-mentioned multi-timescale grid / networked energy storage collaborative control device.

[0093] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application, such as... Figure 10 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 10 Take a processor 10 as an example.

[0094] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0095] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0096] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0098] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0099] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0100] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0101] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0102] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A collaborative control method for multi-timescale grid / grid-based energy storage, characterized in that, The method includes: Based on the characteristics of the power grid and the system support requirements at different time scales, a model predictive control algorithm is used to solve the control ratio between energy storage and grid connection, including: Solving for the system voltage support strength specifically includes: Based on the power grid topology and operating status data, the Thevenin equivalent circuit of the entire power grid is obtained according to the Thevenin equivalence principle. Based on the Thevenin equivalent circuit, the access port voltage when the grid-connected device is connected to the grid and the no-load voltage magnitude of the access port when the grid-connected device is not connected to the grid are calculated. The ratio of the voltage at the access port of the connected device when it is connected to the power grid to the magnitude of the no-load voltage at the access port of the connected device when it is not connected to the power grid is used as the system voltage support strength. The voltage support strength of the system and the strength of the energy storage system are superimposed and used as the input to the prediction model; By combining multiple response indicators, the prediction model is used to obtain the energy storage grid connection and grid construction control ratio, wherein the response indicators include: inertia response indicator, primary frequency regulation response indicator and secondary frequency regulation response indicator. Based on the aforementioned energy storage grid connection and grid construction control ratio, and combined with the multi-dimensional parameters of energy storage, a reinforcement learning algorithm is adopted to realize the dynamic intelligent adjustment of the control mode of each energy storage converter in the power station. The multi-dimensional parameters include: the charging and discharging efficiency of the energy storage battery, the capacity limit of the converter, the response speed difference of different devices, the energy storage health status, the state of charge, voltage, and temperature.

2. The method according to claim 1, characterized in that, Based on the power grid topology and operating status data, and according to the Thevenin equivalence principle, the Thevenin equivalent circuit of the entire power grid is obtained as follows: Using new energy grid-connected nodes as equivalent ports, the power grid area outside the equivalent ports is converted into an active two-terminal network; Based on the power grid topology and the operating status data, the open-circuit voltage when the port is unloaded is calculated by power flow calculation to obtain the Thevenin equivalent potential. By setting each independent power source in the fundamental frequency positive sequence network to zero, and solving for the port input impedance through circuit simplification, the Thevenin equivalent impedance is obtained. By connecting the Thevenin equivalent potential and the Thevenin equivalent impedance in series, the Thevenin equivalent circuit of the entire power grid is obtained.

3. The multi-timescale grid / grid-based energy storage collaborative control method according to claim 1, characterized in that, The process of dynamically and intelligently adjusting the control modes of each energy storage converter within the power station includes: Obtain the current health status, state of charge, voltage, temperature, and grid connection and control ratio of the energy storage system at any given time; Obtain the current operating mode of each energy storage converter; Based on the current health status, state of charge, voltage, temperature, grid connection and grid construction control ratio of the energy storage system, and the operating mode of each energy storage converter, the operating mode of each converter at the next moment is obtained using reinforcement learning algorithms.

4. The multi-timescale grid / grid-based energy storage collaborative control method according to claim 3, characterized in that, Reinforcement learning algorithms include: The health status, state of charge, voltage, temperature, and the ratio of energy storage to grid control are used as state variables for evaluating the decision-making process in the reinforcement learning algorithm. The switching of control modes of each energy storage converter based on the decision of the intelligent agent is used as the action quantity of the evaluation value. A behavior value function is constructed based on the evaluation value, and the behavior value function is iteratively learned until the evaluation value converges.

5. A multi-timescale grid / grid-based energy storage collaborative control device, characterized in that, The device includes: The upper-level control module is used to solve the energy storage-grid connection and grid-connection control ratio using model predictive control algorithms, including: solving the system voltage support strength; specifically including: Based on the power grid topology and operating status data, the Thevenin equivalent circuit of the entire power grid is obtained according to the Thevenin equivalence principle. Based on the Thevenin equivalent circuit, the access port voltage when the grid-connected device is connected to the grid and the no-load voltage magnitude of the access port when the grid-connected device is not connected to the grid are calculated. The ratio of the voltage at the access port of the connected device when it is connected to the power grid to the magnitude of the no-load voltage at the access port of the connected device when it is not connected to the power grid is used as the system voltage support strength. The voltage support strength of the system and the strength of the energy storage system are superimposed and used as the input of the prediction model. Combining multiple response indicators, the prediction model is used to obtain the control ratio of energy storage following the grid and grid construction. The response indicators include: inertia response indicator, primary frequency regulation response indicator and secondary frequency regulation response indicator. The lower-level control module is used to dynamically and intelligently adjust the control mode of each energy storage converter in the power station based on the energy storage grid connection and grid construction control ratio, combined with the multi-dimensional parameters of energy storage, and using reinforcement learning algorithms. The multi-dimensional parameters include: the charging and discharging efficiency of the energy storage battery, the capacity limit of the converter, the response speed difference of different devices, the energy storage health status, the state of charge, voltage, and temperature.

6. A computer device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the multi-timescale grid-connected / grid-based energy storage collaborative control method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the multi-timescale grid-connected / grid-based energy storage collaborative control method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the multi-timescale grid-connected / grid-based energy storage collaborative control method as described in any one of claims 1 to 4.

Images