A control method for grid-connected energy storage converters considering battery factors

By constructing state vectors and projecting security constraints, the safety issue of grid-connected frequency regulation of energy storage power stations was solved, achieving a balance between grid frequency stability and battery health, and improving the safety of grid-connected frequency regulation of energy storage power stations.

CN123121868APending Publication Date: 2026-11-17HUADIAN ELECTRIC POWER SCI INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611539653.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-30
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing technologies, it is difficult to effectively improve the safety of grid-connected frequency regulation of energy storage power stations. Traditional virtual inertia and damping schemes are prone to parameter overruns and system instability when real-time operating conditions change.

Method used

By collecting real-time data from the power grid and energy storage batteries, a state vector is constructed. The original virtual inertia and damping are output using a reinforcement learning strategy network. Then, a safety constraint projection based on KKT conditions is used to map them into the feasible stability domain, thereby controlling the output active power of the grid-type energy storage converter and taking into account both grid frequency stability and battery health.

Benefits of technology

This technology effectively reduces battery risks and avoids system instability caused by parameter overruns during the dynamic adjustment of grid inertia damping, thereby improving the safety of grid-connected frequency regulation of energy storage power stations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123121868A_ABST
    Figure CN123121868A_ABST
Patent Text Reader

Abstract

This application relates to a control method, system, device, and medium for a grid-connected energy storage converter that considers battery factors. The method includes: real-time acquisition of grid data signals and battery data information from the energy storage battery; constructing a state vector S based on the grid data information and battery data information; inputting the state vector S into a reinforcement learning policy network πθ, and outputting the original virtual inertia J. raw With the original damping D raw Using a safety constraint projection based on KKT conditions, J raw and D raw The virtual inertia J and damping D are obtained by mapping to the feasible stability region Ω. Based on the virtual inertia J and damping D, the output active power of the grid-connected energy storage converter is controlled. This application realizes the consideration of the healthy life of the grid-connected battery during the dynamic adjustment of grid inertia damping, effectively reducing battery risk. Furthermore, the inertia damping obtained by reinforcement learning is subjected to safety constraint projection processing to avoid system instability that may be caused by parameter out-of-bounds errors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of power grid control technology, and in particular to a control method, system, device and medium for a grid-type energy storage converter that takes into account battery factors. Background Technology

[0002] Traditional power systems rely on the rotating mechanical inertia of synchronous thermal power generators to resist frequency fluctuations. However, with the continuous expansion of the power system and the increase in the proportion of new energy power generation, the inertia level of the power grid continues to decline, and frequency stability faces severe challenges. The rotating mechanical inertia provided by thermal power units alone is no longer sufficient. The virtual inertia provided by grid-connected energy storage power stations has gradually become a key variable for resisting frequency fluctuations. However, traditional solutions mostly use fixed or offline optimized virtual inertia J and damping D, which have obvious limitations and are difficult to adapt to real-time operating condition changes.

[0003] With the continuous advancement of intelligent and information technology, adaptive dynamic adjustment of inertia damping schemes have emerged in existing technologies. However, these existing virtual inertia schemes usually only consider the response sensitivity and stability of the power grid in response to frequent fluctuations. Directly using the output virtual inertia J and damping D for power grid regulation and control can easily lead to parameter over-limit, causing system instability and safety risks.

[0004] Currently, no effective solution has been proposed for improving the safety of grid-connected frequency regulation of energy storage power stations in related technologies. Summary of the Invention

[0005] This application provides a control method, system, device, and medium for a grid-connected energy storage converter that takes battery factors into account, in order to at least solve the problem of how to improve the safety of grid-connected frequency regulation of energy storage power stations in related technologies.

[0006] In a first aspect, embodiments of this application provide a control method for a grid-connected energy storage converter that considers battery factors, the method comprising: Real-time acquisition of power grid data signals and energy storage battery data information; Based on the power grid data and the battery data, a state vector S is constructed; The state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw ; The original virtual inertia J is projected using a safety constraint projection based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; Based on the virtual inertia J and damping D, the active power output of the grid-type energy storage converter is controlled.

[0007] In some embodiments, the method includes: The reinforcement learning policy network πθ is optimized and updated online based on the reward function R, wherein the reward function R includes a frequency deviation squared term, a voltage deviation squared term, and a battery fatigue damage deviation squared term. The reward function R is used to ensure that the control quantity output by the reinforcement learning policy network πθ takes into account both grid frequency stability and energy storage battery health.

[0008] In some embodiments, the state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw include: The reinforcement learning policy network πθ is the policy network πθ of the reinforcement learning algorithm TD3, and the policy network πθ includes an input layer, a first hidden layer, a second hidden layer and an output layer; The input layer has 6 neurons to receive a 6-dimensional state vector S; the first and second hidden layers each have 256 neurons, and the ReLU activation function is used; the output layer has 2 neurons, and the tanh activation function is used. The state vector S is input into the policy network πθ, and the nonlinear features of the operating conditions are extracted through two hidden layers. The output layer outputs the original virtual inertia J. raw With the original damping D raw .

[0009] In some embodiments, a safety constraint projection based on KKT conditions is used to represent the original virtual inertia J. raw and the original damping D raw Mapping to the feasible stability region Ω, the virtual inertia J and damping D are obtained as follows: The original virtual inertia J is projected using a safety constraint projection based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω yields the virtual inertia J and damping D actually used for control, wherein the constraints of the feasible stability region Ω are:

[0010] in, The meaning is... The meaning is...

[0011] In some embodiments, real-time acquisition of grid data signals from the power grid and battery data information from the energy storage battery includes: Real-time acquisition of power grid data signals, wherein the power grid data signals include power grid frequency f, frequency change rate df / dt, voltage amplitude V, and voltage change rate dV / dt; Real-time acquisition of battery data information from energy storage batteries, including state of charge (SOC) and battery temperature (T).

[0012] In some embodiments, constructing the state vector S based on the power grid data information and the battery data information includes: The grid frequency f, frequency change rate df / dt, voltage amplitude V, voltage change rate dV / dt, state of charge (SOC), and battery temperature T are time-synchronized and aligned, and normalized respectively to map them to a unified numerical range of the reinforcement learning network input. The normalized power grid data signal and battery data information are concatenated in a preset order to obtain the state vector. .

[0013] In some embodiments, controlling the output active power of the grid-type energy storage converter based on the virtual inertia J and damping D includes: Based on the virtual inertia J and damping D, the active power command is calculated using the virtual synchronous machine active power control formula:

[0014] To control the output active power of the grid-type energy storage converter, wherein, ω = ω - ω0, ω = 2πf is the real-time angular frequency of the power grid, f is the real-time power grid frequency, ω0 = 2πf0 is the rated angular frequency of the power grid, f0 is the rated power grid frequency, J is the virtual inertia, and D is the damping.

[0015] In a second aspect, embodiments of this application provide a control system for a grid-type energy storage converter that considers battery factors. The system is used to execute the method described in the first aspect above. The system includes a data acquisition module, a reinforcement learning module, a safety constraint module, and a power calculation module. The data acquisition module is used to acquire grid data signals from the power grid and battery data information from the energy storage battery in real time. The reinforcement learning module is used to construct a state vector S based on the power grid data information and the battery data information; The state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw ; The safety constraint module is used to project the original virtual inertia J using a safety constraint based on KKT conditions. raw and the original damping D rawMapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; The power calculation module is used to control the output active power of the grid-type energy storage converter based on the virtual inertia J and damping D.

[0016] Thirdly, embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0018] Compared to related technologies, this application provides a control method, system, device, and medium for a grid-connected energy storage converter that considers battery factors. The method involves real-time acquisition of grid data signals and battery data information from the energy storage battery; constructing a state vector S based on the grid and battery data information; and inputting the state vector S into a reinforcement learning policy network πθ to output the original virtual inertia J. raw With the original damping D raw Using a safety-constrained projection based on KKT conditions, the original virtual inertia J is transformed. raw and the original damping D raw By mapping to the feasible stability region Ω, virtual inertia J and damping D are obtained. Based on virtual inertia J and damping D, the output active power of the grid-connected energy storage converter is controlled. This realizes the consideration of the healthy life of the grid-connected battery during the dynamic adjustment of grid inertia damping, effectively reducing battery risk. Furthermore, the inertia damping obtained by reinforcement learning is subjected to safety constraint projection processing to avoid system instability that may be caused by parameter out-of-bounds. This solves the problem of how to improve the safety of grid-connected frequency regulation of energy storage power stations. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the steps of a grid-type energy storage converter control method considering battery factors according to an embodiment of this application; Figure 2 This is a schematic diagram of the hardware architecture implementation according to an embodiment of this application; Figure 3 This is a schematic diagram of the online tuning process for reinforcement learning according to an embodiment of this application; Figure 4This is a schematic diagram of a planar security domain according to an embodiment of this application; Figure 5 This is a schematic diagram illustrating the frequency drop suppression effect according to an embodiment of this application; Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0021] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0022] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0023] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0024] This application provides a control method for a grid-type energy storage converter that takes battery factors into account. Figure 1 This is a flowchart illustrating the steps of a grid-connected energy storage converter control method considering battery factors according to an embodiment of this application, as follows: Figure 1 As shown, the method includes the following steps: Step S102: Real-time acquisition of power grid data signals and energy storage battery data information; Specifically, step S102 involves real-time acquisition of grid data signals from the power grid, including grid frequency f, frequency change rate df / dt, voltage amplitude V, and voltage change rate dV / dt; and real-time acquisition of battery data information from the energy storage battery, including state of charge (SOC) and battery temperature T.

[0025] It should be noted that the execution of the method in the embodiments of this application is based on a specific hardware architecture. Figure 2 This is a schematic diagram of the hardware architecture implementation according to an embodiment of this application, such as... Figure 2 As shown, the power grid (such as a medium-voltage AC actual power grid / simulated power grid) is both the power interaction object and the object being measured. On the one hand, it completes bidirectional AC power flow with the grid-type energy storage converter; on the other hand, it outputs analog voltage and current signals to the PMU synchronous phasor measurement device.

[0026] The PMU (Phase Quantity Measurement Unit) is an independent hardware measurement unit that receives analog voltage and current signals from the power grid, performs synchronization phasor calculations, and outputs digital quantities: power grid frequency f, frequency change rate df / dt, voltage amplitude V, and voltage change rate dV / dt. This electrical measurement data is then sent to the DSP-FPGA controller. The PMU only processes power grid electrical quantities and does not receive data from the battery management system (BMS). Battery status signals are directly fed into the DSP-FPGA controller from the battery.

[0027] The DSP-FPGA controller (the hardware carrier for algorithm execution) receives grid measurement data output by the PMU and battery status data output by the BMS; it completes state vector construction, policy network inference, KKT security constraint projection, reward function calculation, TD3 policy network parameter iterative update, and outputs active power commands. The grid-type energy storage converter (control execution unit) receives the active power command P issued by the DSP-FPGA. cmd On the one hand, it interacts with the energy storage battery via DC power, and on the other hand, it outputs / absorbs active power to the grid to achieve virtual inertia and damping support.

[0028] The BMS sensors inside the energy storage battery (energy storage power unit) collect the battery's state of charge (SOC) and temperature (T), and upload the SOC and T data to the DSP-FPGA controller. This enables bidirectional DC power exchange with the grid-type energy storage converter.

[0029] Step S104: Construct a state vector S based on grid data and battery data. Specifically, step S104 involves time synchronization and alignment of the grid frequency f, frequency change rate df / dt, voltage amplitude V, voltage change rate dV / dt, state of charge (SOC), and battery temperature T, followed by normalization processing to map them to a unified numerical range for the reinforcement learning network input. The normalized grid data signal and battery data information are then concatenated in a preset order to obtain the state vector. .

[0030] It should be noted that, as Figure 2 As shown, steps S102 and S104 are in Figure 2 In terms of hardware implementation: The PMU synchronous phasor measurement device acquires the voltage and current signals of the power grid, calculates and outputs the power grid frequency f, frequency change rate df / dt, voltage amplitude V, and voltage change rate dV / dt, and sends them to the DSP-FPGA controller; the energy storage battery's built-in BMS acquires the SOC and battery temperature T and uploads them to the DSP-FPGA controller. The DSP-FPGA controller uses the PMU sampling clock as the time reference to timestamp the power grid electrical quantities and battery state quantities, ensuring that all variables in the same set of state vectors are taken from the same operating conditions at the same time.

[0031] Step S106: Input the state vector S into the reinforcement learning policy network πθ, and output the original virtual inertia J. raw With the original damping D raw ; Specifically, in step S106, the reinforcement learning policy network πθ is the policy network πθ of the reinforcement learning algorithm TD3. The policy network πθ includes an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer has 6 neurons to receive the 6-dimensional state vector S. The first and second hidden layers each have 256 neurons, and the ReLU function is used for activation. The output layer has 2 neurons, and the tanh function is used for activation. The state vector S is input into the policy network πθ, and the nonlinear features of the working condition are extracted through the two hidden layers. The output layer outputs the original virtual inertia J. raw With the original damping D raw .

[0032] Furthermore, the reinforcement learning policy network πθ is optimized and updated online based on the reward function R (the update period can be 80ms). The reward function R includes squared frequency deviation terms, squared voltage deviation terms, and squared battery fatigue damage deviation terms. The reward function R is used to ensure that the control output of the reinforcement learning policy network πθ balances grid frequency stability and battery health. Specifically, the process of updating the policy network πθ is as follows: 1. Set the current state S t Action (J) t D t ), Reward R t The next state S t+1 Stored as an experience sample in the experience replay buffer; 2. Randomly draw a batch of samples from the experience replay buffer and utilize the reward R. t Calculate the Bellman objective value; 3. Update the two evaluation Critic networks of TD3 to minimize the Q-value prediction loss; 4. Following the delayed update strategy, after a specified number of steps, update the network parameters θ of the strategy network πθ based on the gradient information of the Critic network to maximize the cumulative expected reward. 5. Perform soft updates on the target policy network and target evaluation network. After completing a single update, enter the next control cycle and repeat the closed-loop process of data collection, inference, execution, and update.

[0033] It should be noted that, Figure 3 This is a schematic diagram of the online tuning process for reinforcement learning according to an embodiment of this application, as shown below. Figure 3As shown, the reinforcement learning policy network of this application embodiment can update network weights online during operation; the network only outputs the original actions, which cannot be directly used for control and must be processed by safety constraint projection; the input includes both grid electrical quantities and battery status, taking into account grid stability and battery health conditions.

[0034] Step S108: Using safety constraint projection based on KKT conditions, the original virtual inertia J is... raw and the original damping D raw Mapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; Specifically, step S108 involves using a safety constraint projection based on KKT conditions to transform the original virtual inertia J. raw and the original damping D raw Mapping to the feasible stability region Ω yields the virtual inertia J and damping D actually used for control, where the constraints of the feasible stability region Ω are:

[0035] in, The meaning is... The meaning is...

[0036] It should be noted that, Figure 4 This is a schematic diagram of a planar security domain according to an embodiment of this application, such as... Figure 4 As shown, applying safety constraint projection to the inertia damping obtained from reinforcement learning can effectively prevent system instability that may be caused by parameter out-of-bounds errors.

[0037] Step S110: Based on the virtual inertia J and damping D, control the output active power of the grid-type energy storage converter.

[0038] Specifically, in step S110, based on the virtual inertia J and damping D, the active power command is calculated using the active power control formula of the virtual synchronous machine.

[0039] To control the output active power of the grid-type energy storage converter, wherein, ω = ω - ω0, ω = 2πf is the real-time angular frequency of the power grid, f is the real-time power grid frequency, ω0 = 2πf0 is the rated angular frequency of the power grid, f0 is the rated power grid frequency, J is the virtual inertia, and D is the damping.

[0040] It should be noted that, Figure 5 This is a schematic diagram illustrating the frequency drop suppression effect according to an embodiment of this application, such as... Figure 5As shown, this application's embodiment utilizes a reinforcement learning-based online inertia-damping parameter tuning method. Compared to offline supervised learning artificial neural networks, the policy network in this application belongs to an online reinforcement learning training architecture. It does not require a working condition-parameter label dataset and relies on online iteration using reward functions fed back from the power grid and battery environment. Furthermore, the network's original output is not directly used for control; a KKT safety constraint projection is added to restrict the parameters to the stability domain. The state input incorporates SOC and battery temperature, balancing power grid stability and battery life, overcoming the shortcomings of traditional neural networks such as offline training, poor generalization, and lack of safety protection. This achieves millisecond-level adaptation and ensures system stability.

[0041] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0042] This application provides a control system for a grid-type energy storage converter that considers battery factors. The system is used to execute the method provided in the above embodiments. The system includes a data acquisition module, a reinforcement learning module, a safety constraint module, and a power calculation module. The data acquisition module is used to collect grid data signals from the power grid and battery data information from the energy storage battery in real time. The reinforcement learning module is used to construct a state vector S based on grid data and battery data. Input the state vector S into the reinforcement learning policy network πθ, and output the original virtual inertia J. raw With the original damping D raw ; The safety constraint module is used to project the original virtual inertia J using safety constraint conditions based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; The power calculation module is used to control the output active power of the grid-type energy storage converter based on the virtual inertia J and damping D.

[0043] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0044] This embodiment provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0045] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0046] Optionally, the electronic device may further include a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a control method for a grid-type energy storage converter that takes battery factors into account. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device casing, or an external keyboard, touchpad, or mouse.

[0047] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0048] Furthermore, in conjunction with the control method for a grid-type energy storage converter considering battery factors in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the control methods for a grid-type energy storage converter considering battery factors in the above embodiments.

[0049] In one embodiment, Figure 6 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 6 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 6As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network, the internal memory provides an environment for the operating system and computer programs to run, the computer programs are executed by the processor to implement a control method for a grid-connected energy storage converter that considers battery factors, and the database stores data.

[0050] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0051] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0052] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A control method for a grid-connected energy storage converter considering battery factors, characterized in that, The method includes: Real-time acquisition of power grid data signals and energy storage battery data information; Based on the power grid data and the battery data, a state vector S is constructed; The state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw ; The original virtual inertia J is projected using a safety constraint projection based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; Based on the virtual inertia J and damping D, the active power output of the grid-type energy storage converter is controlled.

2. The method according to claim 1, characterized in that, The method includes: The reinforcement learning policy network πθ is optimized and updated online based on the reward function R, wherein the reward function R includes a frequency deviation squared term, a voltage deviation squared term, and a battery fatigue damage deviation squared term. The reward function R is used to ensure that the control quantity output by the reinforcement learning policy network πθ takes into account both grid frequency stability and energy storage battery health.

3. The method according to claim 2, characterized in that, The state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw include: The reinforcement learning policy network πθ is the policy network πθ of the reinforcement learning algorithm TD3, and the policy network πθ includes an input layer, a first hidden layer, a second hidden layer and an output layer; The input layer has 6 neurons to receive a 6-dimensional state vector S; the first and second hidden layers each have 256 neurons, and the ReLU activation function is used; the output layer has 2 neurons, and the tanh activation function is used. The state vector S is input into the policy network πθ, and the nonlinear features of the operating conditions are extracted through two hidden layers. The output layer outputs the original virtual inertia J. raw With the original damping D raw .

4. The method according to claim 1, characterized in that, The original virtual inertia J is projected using a safety constraint projection based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω, the virtual inertia J and damping D are obtained as follows: The original virtual inertia J is projected using a safety constraint projection based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω yields the virtual inertia J and damping D actually used for control, wherein the constraints of the feasible stability region Ω are: in, The meaning is... The meaning is...

5. The method according to claim 1, characterized in that, Real-time acquisition of power grid data signals and energy storage battery data includes: Real-time acquisition of power grid data signals, wherein the power grid data signals include power grid frequency f, frequency change rate df / dt, voltage amplitude V, and voltage change rate dV / dt; Real-time acquisition of battery data information from energy storage batteries, including state of charge (SOC) and battery temperature (T).

6. The method according to claim 5, characterized in that, Based on the power grid data and the battery data, the state vector S is constructed as follows: The grid frequency f, frequency change rate df / dt, voltage amplitude V, voltage change rate dV / dt, state of charge (SOC), and battery temperature T are time-synchronized and aligned, and normalized respectively to map them to a unified numerical range of the reinforcement learning network input. The normalized power grid data signal and battery data information are concatenated in a preset order to obtain the state vector. .

7. The method according to claim 1, characterized in that, Based on the virtual inertia J and damping D, controlling the output active power of the grid-type energy storage converter includes: Based on the virtual inertia J and damping D, the active power command is calculated using the virtual synchronous machine active power control formula: To control the output active power of the grid-type energy storage converter, wherein, ω = ω - ω0, ω = 2πf is the real-time angular frequency of the power grid, f is the real-time power grid frequency, ω0 = 2πf0 is the rated angular frequency of the power grid, f0 is the rated power grid frequency, J is the virtual inertia, and D is the damping.

8. A control system for a grid-connected energy storage converter that considers battery factors, characterized in that, The system is used to perform the method according to any one of claims 1 to 7, and the system includes a data acquisition module, a reinforcement learning module, a security constraint module, and a power calculation module; The data acquisition module is used to acquire grid data signals from the power grid and battery data information from the energy storage battery in real time. The reinforcement learning module is used to construct a state vector S based on the power grid data information and the battery data information; The state vector S is input into the reinforcement learning policy network πθ, and the original virtual inertia J is output. raw With the original damping D raw ; The safety constraint module is used to project the original virtual inertia J using a safety constraint based on KKT conditions. raw and the original damping D raw Mapping to the feasible stability region Ω, we obtain the virtual inertia J and damping D; The power calculation module is used to control the output active power of the grid-type energy storage converter based on the virtual inertia J and damping D.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.