Multi-region power grid energy storage adaptive virtual inertia control method based on data driving and related device

Through the data-driven adaptive virtual inertial control method for energy storage in multi-region grids, the Dueling DQN agent adjusts the inertial constant of the virtual inertial control unit, the problem of slow response speed of traditional virtual inertial control is solved, and the stability and inertia of the power grid are improved. The power compensation capability achieves reasonable compensation of inertial power and coordinated energy storage in different scenarios.

CN119209597AActive Publication Date: 2024-12-27STATE GRID HUBEI ELECTRIC POWER CO LTD +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411122815.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-27
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

In traditional virtual inertial control methods, fixed inertial constants are difficult to adapt to complex grid scenarios, resulting in slow response speed and inability to effectively compensate for grid inertial power, affecting the stability of grid frequency.

Method used

Adopting the adaptive virtual inertia control method of energy storage in multi-region grid based on data-driven energy, the inertia constant of the virtual inertia control unit is adjusted through the Dueling DQN agent, and the deep Q network algorithm and the L1 parameter norm penalty term are used to adapt according to the frequency changes of the power grid. Adjust inertial power compensation.

Benefits of technology

The stability and inertial power compensation capabilities of multi-region power grids are improved, reasonable inertial power compensation and energy storage coordinated cooperation in different scenarios, and the frequency stability of the power grid is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119209597A_ABST
    Figure CN119209597A_ABST
Patent Text Reader

Abstract

The invention provides a multi-regional power grid energy storage adaptive virtual inertia control method based on data driving and a related device. The method comprises the following steps: establishing at least two regional power grid models; adjusting inertia constants of the virtual inertia control units of the two regions by adopting a deep Q network improvement algorithm under a competition framework, establishing an action space, a state space and a reward function required by the Dueling DQN intelligent agent, establishing a target loss function, iterating neural network parameters of the Dueling DQN intelligent agent, and obtaining a target loss function of the Dueling DQN intelligent agent; and an L1 parameter norm penalty term is introduced to improve a target loss function, so that the generalization ability of the Dueling DQN intelligent agent is improved. According to the method, the inertia constants in the virtual inertia control units in different regions can be adaptively adjusted, reasonable compensation of inertia power in different scenes and cooperation between multi-region energy storage are realized, and the stability of a multi-region power grid is jointly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of energy storage virtual inertia control, and in particular to a data-driven multi-regional power grid energy storage adaptive virtual inertia control method and related devices. Background Art

[0002] With the proposal of the national dual carbon target, renewable energy has been more widely used in the power grid. Renewable energy such as photovoltaic and wind power is generally connected to the power grid through power electronic inverters. Unlike traditional synchronous generators, inverters have no rotor inertia. With the increase in the penetration rate of new energy such as photovoltaic and wind power, the inertia of the power system has also decreased, which has led to a decrease in the stability of the power grid and weakened rigid interference capabilities. At the same time, the uncertainty and volatility of wind and solar power output have had an adverse impact on the stability of the power grid frequency.

[0003] Energy storage, as an energy source, can use virtual inertia control to compensate for additional inertia power of the power grid, thereby reducing or eliminating the impact of large-scale access to wind and solar energy on the power grid. However, traditional virtual inertia control uses a fixed inertia constant, which makes it difficult for the control method to compensate for the required inertia power and has the problem of slow response speed. Fixed inertia constants are also difficult to adapt to the complex and diverse scenarios of the power grid. Inappropriate inertia power compensation may even aggravate the fluctuation of the power grid frequency and cause more severe effects. Therefore, improving the inertia compensation capability and adaptability of the virtual inertia control strategy is a key issue. With the rapid development of artificial intelligence technology, the use of data-driven adjustment of the inertia constant is a new approach. Summary of the invention

[0004] In order to improve the compensation capability of the inertial power of multi-regional power grid energy storage, the present invention provides a data-driven multi-regional power grid energy storage adaptive virtual inertia control method and related devices.

[0005] The present invention is implemented by the following technical solutions:

[0006] A data-driven multi-regional power grid energy storage adaptive virtual inertia control method comprises the following steps:

[0007] Step 1: Establish at least two regional power grid models. Each regional power grid includes a wind power system, a thermal power plant, and an energy storage device. The wind power system is connected to the power grid in the form of constant power control as a disturbance source. The thermal power plant participates in the primary and secondary frequency regulation of the regional power grid. The load frequency controller adopts an integral controller, and the energy storage device adopts virtual inertia control to compensate for inertia power.

[0008] Step 2: Use the improved deep Q network algorithm under the competition framework to adjust the inertia constants of the virtual inertial control units in the two regions, and establish the action space, state space and reward function required by the Dueling DQN agent;

[0009] Step 3: Based on the action space, state space, and reward function established in step 2, establish the target loss function, iterate the neural network parameters of the Dueling DQN agent, introduce the L1 parameter norm penalty term to improve the target loss function, and improve the generalization ability of the Dueling DQN agent.

[0010] Furthermore, the regional power grid model is expressed as:

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018] Where i = 1, 2; Δf i is the frequency deviation of region i, H i is the inertia time constant of region i, D i is the damping constant of region i, ΔP W,i is the power increment of the wind power system output, ΔP V,i is the inertia compensation power increment of the energy storage output, ΔP L,i is the load change of the power grid, ΔP M,i is the output power increment of the thermal power plant, ΔP G,i is the power increment of the turbine system output, T t,i is the time constant of the turbine system, T g,i is the time constant of the generator, ΔP C,i is the control signal of the thermal power plant; R i K is the adjustment coefficient; S,i is the integral coefficient, β i is the coefficient of deviation, ΔP TIE is the interconnection line power change, T 12 is the coupling coefficient ΔP wind,i is the initial wind power change, T W,i is the time constant of the wind turbine generator set, K V,i is the inertia time constant, T E,i is the energy storage time constant.

[0019] Furthermore, the Dueling DQN agent obtains frequency information of each area from the external power grid as the state space, and the frequency information includes the frequency deviation of the regional power grid and the integral of the frequency deviation; the action space is the inertia constant of the virtual inertial control unit of each area; the reward function is established according to the frequency information obtained by the Dueling DQN agent to realize the feedback of the external environment to the agent after the agent outputs an action.

[0020] Furthermore, in step 3, the L1 parameter norm penalty term is introduced to improve the target loss function, specifically including:

[0021] The L1 parameter norm penalty term is used to improve the traditional mean square loss function C0, add the structural risk term C1, and add regularization operations:

[0022] C=C0+C1

[0023]

[0024]

[0025]

[0026] Among them, N is the number of samples, γ is the discount coefficient; χ is the penalty coefficient, and ω represents the weight in the neural network parameters.

[0027] Furthermore, the DuelingDQN agent network is composed of two small networks, which are equivalent to three parameters, where θ is the parameter of the convolutional neural network part shared by the two small networks; α and β are the network parameters of the small network itself; the Q function Q(s, a; θ, α, β) of the Dueling DQN agent is shown as follows:

[0028] Q(s,a;θ,α,β)=V(s;θ,β)+A(s,a;θ,α)

[0029] To avoid the problem that the state value and advantage function change while the sum of the two, Q function, remains unchanged, resulting in the problem that the action value cannot be identified, the Q function is modified to:

[0030]

[0031] Among them, maxA(s,a max ; θ, α) is the maximum advantage function value, a max The action to maximize the advantage function;

[0032] The Dueling DQN agent gradually updates the neural network parameters by minimizing the loss function, thereby gradually achieving an accurate estimate of the Q value:

[0033] Loss=(yQ(s t ,a t ;θ,α,β)) 2

[0034]

[0035] Furthermore, the Dueling DQN agent continuously interacts with the external power grid to obtain sample data, extracts sample data to train the Q network until convergence. When the neural network can accurately estimate the Q value, it selects the action that maximizes the Q value, and finally realizes data-driven multi-regional power grid energy storage adaptive virtual inertia control:

[0036]

[0037] Furthermore, the specific steps of extracting sample data to train the Q network are as follows:

[0038] Initialize the neural network parameters θ, α, β, and set the number of training episodes.

[0039] (1) In each Episode, initialize the first state s t ;

[0040] (2) Select action a t Act on the environment to get the next state s t+1 , environmental feedback reward r t ;

[0041] (3) t ,a t ,r t ,s t+1 ) This four-tuple is stored in the experience playback set;

[0042] (4) Sampling samples from the experience replay set and updating the network through the objective function containing the structural risk term;

[0043] (5)s t =s t+1 , determine whether the maximum number of training steps has been reached, if yes, go to step (6), otherwise go to step (2);

[0044] (6) Determine whether the maximum number of training episodes has been reached. If so, the training ends; otherwise, go to step (1).

[0045] A data-driven multi-regional power grid energy storage adaptive virtual inertial control system, comprising: a computer-readable storage medium and a processor;

[0046] The computer-readable storage medium is used to store executable instructions;

[0047] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the data-driven multi-regional power grid energy storage adaptive virtual inertia control method.

[0048] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the data-driven multi-regional power grid energy storage adaptive virtual inertia control method.

[0049] Compared with the existing methods, the present invention has the following advantages:

[0050] The method proposed in the present invention is based on data-driven. According to the changes in the power grid scenarios, the data-driven Dueling DQN intelligent agent utilizes the frequency changes of the multi-regional power grid to adaptively adjust the inertia constants in the virtual inertial control units of different regions, thereby achieving reasonable compensation of inertial power in different scenarios and coordinated cooperation between multi-regional energy storage, and jointly improving the stability of the multi-regional power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to an embodiment of the present invention.

[0052] Figure 2 This is a multi-region power grid frequency control model according to an embodiment of the present invention.

[0053] Figure 3 This is the Dueling DQN agent Q function structure of the embodiment of the present invention.

[0054] Figure 4 It is a flow chart of Dueling DQN agent training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The embodiment of the present invention provides a data-driven multi-regional power grid energy storage adaptive virtual inertia control method, such as Figure 1 As shown, the method includes:

[0057] Step 1: First, establish at least two regional power grid models, each of which includes a wind power system, a thermal power plant, and an energy storage device. The wind power system is connected to the power grid in the form of constant power control as a disturbance source to maximize the use of clean energy. The thermal power plant participates in the primary and secondary frequency regulation of the regional power grid. The load frequency controller adopts an integral control controller. The energy storage device participates in the primary frequency regulation of the regional power grid due to its fast charging and discharging response speed. The control method is to use virtual inertia control to provide additional inertia power for the power grid and improve the stability of the frequency. This embodiment is illustrated by taking two regions as an example, and the regional power grid model is expressed as:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] Where i = 1, 2; Δf i is the frequency deviation of region i, H i is the inertia time constant of region i, D i is the damping constant of region i, ΔP W,i is the power increment of the wind power system output, ΔP V,i is the inertia compensation power increment of the energy storage output, ΔP L,i is the load change of the power grid, ΔP M,i is the output power increment of the thermal power plant, ΔP G,i is the power increment of the turbine system output, T t,i is the time constant of the turbine system, T g,i is the time constant of the generator, ΔP C,i is the control signal of the thermal power plant; R i K is the adjustment coefficient; S,i is the integral coefficient, β i is the coefficient of deviation, ΔP TIE is the power change of the interconnection line, T 12 is the coupling coefficient ΔP wind,i is the initial wind power change, T W,i is the time constant of the wind turbine generator set, K V,i is the inertia time constant, T E,i is the energy storage time constant.

[0066] The microgrids in the two regions are coupled to each other through interconnection lines. Each regional grid is equipped with a virtual inertial control unit that uses energy storage as energy. The structural model of the unit and the multi-regional grid frequency control model are as follows: Figure 2 The frequency deviation and other information of the two regional power grids are input into the Dueling DQN agent through the communication line, and then the Dueling DQN agent outputs the adjustment instructions of the virtual inertia constants of the two regions.

[0067] Step 2: Use the improved deep Q network algorithm (Dueling DQN) under the competition framework to adjust the inertia constants of the virtual inertial control units of the two regional power grids, and establish the action space, state space and reward function required by the Dueling DQN agent. The Dueling DQN agent is arranged at the connection line between the two regions to promote the coordination of the virtual inertial control units of the energy storage in the two regions.

[0068] In order to achieve the coordination of the virtual inertial control units in the two regions, the Dueling DQN agent needs to obtain the frequency information of region 1 and region 2 from the external power grid, specifically the frequency deviation and the integral of the frequency deviation of the two regional power grids. The state space is:

[0069] s=(Δf1,∫Δf1,Δf2,∫Δf2)

[0070] The action space of the agent is the inertia constant of the virtual inertial control unit in the two regions:

[0071] a=(K V1 ,K V2 )

[0072] Among them, K V1 ,K V2 are the virtual inertia constants of region 1 and region 2, respectively, V1 The size range is [K V1,min ,K V1,max ], K V2 The size range is [K V2,min ,K V2,max ] Since the action space of Dueling DQN agent is a discrete space, it is necessary to discretize the action variables and divide K V1 ,K V2 The interval is divided into 8 discrete actions, and the changes are Δk V1 ,Δk V2 , then the action set of Dueling DQN agent is:

[0073]

[0074]

[0075] In different scenarios and different time periods, the virtual inertia constants of the virtual inertia control units in the two areas will be different. The inertia constant will be adaptively adjusted according to the changes in the external power grid frequency information. The inertia control unit selects the appropriate virtual inertia constant at different stages.

[0076] The reward function is established based on the frequency information obtained by the Dueling DQN agent to realize the feedback of the external environment to the agent after the agent outputs the action:

[0077]

[0078]

[0079] r=r1+r2

[0080] Among them, λ1,λ2,λ3,λ4 are the weights of the reward function. The larger the frequency deviation, the larger the weight, and the smaller the reward obtained by the Dueling DQN agent. Through the reward function, the Dueling DQN agent can judge the quality of its own actions and seek long-term rewards.

[0081] Step 3: Based on the action space, state space and reward function established in step 2, establish the target loss function and iterate the neural network parameters of the Dueling DQN agent. Dueling DQN is an improved algorithm for deep Q networks. Different from the value function approximation method of DQN, Dueling DQN is based on a competition framework. The Q function Q(s,a;θ,α,β) is composed of the state value V(s;θ,β) and the advantage function A(s,a;θ,α). The L1 parameter norm penalty term is introduced to improve the loss function and enhance the adaptability of the DuelingDQN agent.

[0082] The DuelingDQN agent network consists of two small networks, which can be equivalent to three parameters. θ is the parameter of the convolutional neural network shared by the two small networks; α and β are the network parameters of the small network itself. The Q function Q(s, a; θ, α, β) of the DuelingDQN agent is as follows: Figure 3 shown.

[0083] Q(s,a;θ,α,β)=V(s;θ,β)+A(s,a;θ,α)

[0084] In order to avoid the problem that the state value and advantage function change while the sum of the two, the Q function, remains unchanged, resulting in the problem that the action value cannot be identified, the Q function is modified to:

[0085]

[0086] Among them, maxA(s,a max ; θ, α) is the maximum advantage function value, a max The action that maximizes the advantage function.

[0087] The Dueling DQN agent gradually updates the neural network parameters by minimizing the loss function, thereby gradually achieving an accurate estimation of the Q value.

[0088] Loss=(yQ(s t ,a t ;θ,α,β)) 2

[0089]

[0090] DuelingDQN agents have an experience replay pool, and the training method is offline training. The agent randomly extracts a certain number of samples from the experience replay pool and updates the neural network according to the loss function. Therefore, DuelingDQN agents have the ability to replay experience. When extracting samples, the loss function is:

[0091]

[0092]

[0093] Where N is the number of samples and γ is the discount factor.

[0094] The L1 parameter norm penalty term is used to improve the traditional mean square loss function C0, add the structural risk term C1, and add regularization operations:

[0095]

[0096] C=C0+C1

[0097] χ is the penalty coefficient, and ω represents the weight in the neural network parameters.

[0098] By improving the mean square loss function, the generalization ability of Dueling DQN agent is improved. DuelingDQN agent obtains sample data by continuously interacting with the external power grid, extracts sample data to train the Q network until convergence. When the neural network can accurately calculate the Q value, the action that maximizes the Q value is selected, and finally realizes data-driven multi-regional power grid energy storage adaptive virtual inertia control.

[0099]

[0100] The training process is as follows: Figure 4:

[0101] Initialize the neural network parameters θ, α, β, and set the number of training episodes.

[0102] (1) In each Episode, initialize the first state s t ;

[0103] (2) Select action a t Act on the environment to get the next state s t+1 , environmental feedback reward r t ;

[0104] (3) t ,a t ,r t ,s t+1 ) This four-tuple is stored in the experience playback set;

[0105] (4) Sampling samples from the experience replay set and updating the network through the objective function containing the structural risk term;

[0106] (5)s t =s t+1 , determine whether the maximum number of training steps has been reached, if yes, go to step (6), otherwise go to step (2);

[0107] (6) Determine whether the maximum number of training episodes has been reached. If so, the training ends; otherwise, go to step (1).

[0108] After the training, the data-driven Dueling DQN agent adjusts the inertia constants of the virtual inertial control units of the two regions according to the frequency changes of the power grids in the two regions, improves the inertia power compensation capability of the energy storage systems in the two regions, achieves coordinated cooperation between the two regions, and jointly improves the stability of the power grids in the two regions.

[0109] Another aspect of the present invention provides a data-driven multi-regional power grid energy storage adaptive virtual inertial control system, comprising: a computer-readable storage medium and a processor;

[0110] The computer-readable storage medium is used to store executable instructions;

[0111] The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the data-driven multi-regional power grid energy storage adaptive virtual inertia control method described in the first aspect.

[0112] On the other hand, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data-driven multi-regional power grid energy storage adaptive virtual inertia control method described in the first aspect.

[0113] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0114] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A data-driven multi-regional power grid energy storage adaptive virtual inertia control method, characterized in that: The steps include: Step 1: Establish at least two regional power grid models. Each regional power grid includes a wind power system, a thermal power plant, and an energy storage device. The wind power system is connected to the power grid in the form of constant power control as a disturbance source. The thermal power plant participates in the primary and secondary frequency regulation of the regional power grid. The load frequency controller adopts an integral controller, and the energy storage device adopts virtual inertia control to compensate for inertia power. Step 2: Use the improved deep Q network algorithm under the competition framework to adjust the inertia constants of the virtual inertial control units in the two regions, and establish the action space, state space and reward function required by the Dueling DQN agent; Step 3: Based on the action space, state space, and reward function established in step 2, establish the target loss function, iterate the neural network parameters of the Dueling DQN agent, introduce the L1 parameter norm penalty term to improve the target loss function, and improve the generalization ability of the Dueling DQN agent.

2. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 1, characterized in that: The regional power grid model is expressed as: Where i = 1, 2; Δf i is the frequency deviation of region i, H i is the inertia time constant of region i, D i is the damping constant of region i, ΔP W,i is the power increment of the wind power system output, ΔP V,i is the inertia compensation power increment of the energy storage output, ΔP L,i is the load change of the power grid, ΔP M,i is the output power increment of the thermal power plant, ΔP G,i is the power increment of the turbine system output, T t,i is the time constant of the turbine system, T g,i is the time constant of the generator, ΔP C,i is the control signal of the thermal power plant; R i K is the adjustment coefficient; S,i is the integral coefficient, β i is the coefficient of deviation, ΔP TIE is the interconnection line power change, T 12 is the coupling coefficient ΔP wind,i is the initial wind power change, T W,i is the time constant of the wind turbine generator set, K V,i is the inertia time constant, T E,i is the energy storage time constant.

3. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 1, characterized in that: The Dueling DQN agent obtains the frequency information of each area from the external power grid as the state space, and the frequency information includes the frequency deviation of the regional power grid and the integral of the frequency deviation; the action space is the inertia constant of the virtual inertial control unit of each area; the reward function is established according to the frequency information obtained by the Dueling DQN agent to realize the feedback of the external environment to the agent after the agent outputs the action.

4. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 1, characterized in that: In step 3, the L1 parameter norm penalty term is introduced to improve the target loss function, including: The L1 parameter norm penalty term is used to improve the traditional mean square loss function C0, add the structural risk term C1, and add regularization operations: C=C0+C1 Among them, N is the number of samples, γ is the discount coefficient; χ is the penalty coefficient, and ω represents the weight in the neural network parameters.

5. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 1, characterized in that: The DuelingDQN agent network consists of two small networks, which are equivalent to three parameters, where θ is the parameter of the convolutional neural network shared by the two small networks; α and β are the network parameters of the small network itself; the Q function Q(s, a; θ, α, β) of the Dueling DQN agent is shown as follows: Q(s,a;θ,α,β)=V(s;θ,β)+A(s,a;θ,α) To avoid the problem that the state value and advantage function change while the sum of the two, Q function, remains unchanged, resulting in the problem that the action value cannot be identified, the Q function is modified to: Among them, maxA(s,a max ; θ, α) is the maximum advantage function value, a max The action to maximize the advantage function; The Dueling DQN agent gradually updates the neural network parameters by minimizing the loss function, thereby gradually achieving an accurate estimate of the Q value: Loss=(yQ(s t ,a t ;θ,α,β)) 2 6. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 5, characterized in that: The Dueling DQN agent continuously interacts with the external power grid to obtain sample data, extracts sample data to train the Q network until convergence. When the neural network can accurately estimate the Q value, it selects the action that maximizes the Q value, and finally realizes data-driven multi-regional power grid energy storage adaptive virtual inertia control:

7. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 6, characterized in that: The specific steps of extracting sample data to train the Q network are as follows: Initialize the neural network parameters θ, α, β, and set the number of training episodes. (1) In each Episode, initialize the first state s t ; (2) Select action a t Act on the environment to get the next state s t+1 , environmental feedback reward r t ; (3) t ,a t ,r t ,s t+1 ) This four-tuple is stored in the experience playback set; (4) Sampling samples from the experience replay set and updating the network through the objective function containing the structural risk term; (5)s t =s t+1 , determine whether the maximum number of training steps has been reached, if yes, go to step (6), otherwise go to step (2); (6) Determine whether the maximum number of training episodes has been reached. If so, the training ends; otherwise, go to step (1).

8. The data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to claim 7, characterized in that: After the training, the data-driven Dueling DQN agent adjusts the inertia constant of the regional virtual inertial control unit according to the frequency changes of the regional power grid, improves the inertia power compensation capability of the energy storage systems in the two regions, and realizes the coordinated cooperation between the two regions.

9. A data-driven multi-regional power grid energy storage adaptive virtual inertial control system, comprising: A computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium to execute the data-driven multi-regional power grid energy storage adaptive virtual inertia control method described in any one of claims 1-8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the data-driven multi-regional power grid energy storage adaptive virtual inertia control method according to any one of claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • New energy station energy storage primary frequency modulation adaptive control method and system

    CN117578506A

  • Inertia damping adaptive control method and system based on deep Q network

    CN117913791A

  • Photovoltaic power generation virtual inertia compensation system and method based on super capacitor energy storage

    US20220029581A1