Multi-energy storage distribution network voltage regulation method and apparatus, and device

Abstract

The present invention relates to the technical field of power distribution networks. Disclosed are a multi-energy storage power distribution network voltage regulation method and apparatus, and a device. The method comprises: establishing a voltage optimization model for a power distribution network, converting an energy storage constraint in the voltage optimization model of the power distribution network on the basis of physical information of an energy storage system, establishing a security model of the energy storage system, forming a training guide model of the energy storage system on the basis of a Markov decision model, training each energy storage system by using a TD3 algorithm, sharing network parameters obtained through training between different energy storage systems, and adjusting the voltage of the power distribution network. By means of optimizing an operation strategy of an energy storage system, the present invention realizes efficient and sustainable operation of the energy storage system. A charging and discharging strategy of the energy storage system is optimized by means of a DRL algorithm having shared parameters, improving voltage stability of the power distribution network, reducing energy loss, improving the overall efficiency of the system, enhancing the collaboration performance of multiple energy storage systems, and optimizing control of the energy storage system in an active power distribution network.

Description

A voltage regulation method, device and equipment for a multi-energy storage power distribution network TECHNICAL FIELD

[0001] The present application relates to the technical field of power distribution networks, in particular to a voltage regulation method, device and equipment for a multi-energy storage power distribution network. BACKGROUND

[0002] In recent years, the progress of science and technology and the enhancement of environmental awareness have increased the proportion of renewable energy in energy utilization, which is of great significance for environmental protection and energy depletion mitigation. However, the large-scale grid connection of renewable energy with strong uncertainty and randomness and flexible load inevitably brings great challenges to the safe and stable operation of the distribution network (ADN). For example, too high a penetration rate of renewable energy can cause the voltage of some nodes to exceed the safety threshold; too high a demand for load can cause the voltage of some nodes to be lower than the safety threshold. Energy storage systems play a crucial role in distribution networks with large-scale grid connection of renewable energy and flexible load. Energy storage systems can provide the necessary regulation capacity when the voltage exceeds or is lower than the safety threshold. For example, when the power generated by renewable energy is excessive, energy storage devices can store excess energy; conversely, when power demand increases or renewable energy output decreases, energy storage devices can release energy to maintain the stable operation of the power grid. Due to the strong randomness and uncertainty of renewable energy sources such as wind and solar energy, energy storage systems can be used to smooth the output fluctuations of these energy sources and reduce the impact on the power grid. Energy storage systems can also quickly respond to changes in grid demand, improving the adaptability and resistance of the power grid to emergencies. Rapid energy release during peak demand or renewable energy supply shortage helps to maintain supply and demand balance. In summary, energy storage systems are one of the key technologies for achieving high penetration of renewable energy grid connection and stable operation of the power grid. With the development of technology and the reduction of cost, the application of energy storage systems is expected to be more widespread, playing a positive role in the development of future smart grids.

[0003] However, so far, the traditional energy storage control model usually needs to write the constraints on energy storage as penalty terms into the reward function of the agent when performing voltage control on the distribution network in the context of DRL. When there are too many penalty terms in the reward function, it will cause training fluctuations and affect the convergence speed, and the stability cannot be guaranteed. The current energy storage control method is difficult to adapt to the voltage control of the distribution network using deep reinforcement learning algorithm, and a more flexible and intelligent solution is needed to cope with the development of artificial intelligence. SUMMARY

[0004] The present application provides a voltage regulation method, device and equipment for a multi-energy storage power distribution network, which can reduce the difficulty of training the energy storage system through parameter sharing, thereby improving the learning efficiency, promoting knowledge transfer, improving the universality and transferability, and enhancing the collaboration performance of the multi-energy storage system.

[0005] To solve the above technical problems, the application provides a voltage regulation method for a multi-energy storage power distribution network, comprising:

[0006] establishing a power distribution network voltage optimization model, the constraints of the power distribution network voltage optimization model including active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and energy storage constraints;

[0007] According to the physical information of the energy storage system, the energy storage constraints in the power distribution network voltage optimization model are converted to establish an energy storage system safety model;

[0008] Based on the Markov decision model, the interaction process of the energy storage system and the power distribution network voltage optimization model is defined to form a training guide model of the energy storage system;

[0009] The Actor-Critic framework is improved by using a double evaluation network, a strategy delay update and a target strategy smoothing method to form a TD3 algorithm;

[0010] The TD3 algorithm is used to train a plurality of energy storage systems in the power distribution network in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system;

[0011] When each energy storage system is trained, the neural network parameters of each energy storage system are shared in the power distribution network.

[0012] Further, the power distribution network voltage optimization model is established, and the constraints of the power distribution network voltage optimization model include active power constraints, reactive power constraints, voltage constraints and energy storage constraints, specifically:

[0013] The sum of the voltage deviation of each node of the power distribution network is minimized as the objective function of the power distribution network voltage optimization model, specifically:

[0014] In the formula, V i,t is the voltage value of node i at time t; V0 is the reference voltage of the power distribution network; T is the time range; N is the number of nodes of the power distribution network;

[0015] The active power constraints and the reactive power constraints of the power distribution network voltage optimization model are respectively:

[0016] In the formula, P i,t is the actual active power injection amount of node i at time t; Q i,t is the actual reactive power injection amount of node i at time t; V i,t is the voltage value of node i at time t; V j,tV ij,t is the conductance between node i and node j; B ij,t is the susceptance between node i and node j; θ ij,t is the phase difference between node i and node j;

[0017] The power flow injection constraint of the distribution network voltage optimization model is:

[0018] In the formula, P i,t is the actual active power injection; is the active power absorbed or released by the energy storage system; is the active power output of the photovoltaic inverter; is the active power output of the wind turbine; is the active power demand of the load; Q i,t is the actual reactive power injection; is the reactive power injection of the photovoltaic inverter; is the reactive power injection of the static reactive power compensator; is the reactive power demand of the load;

[0019] The voltage constraint of the distribution network voltage optimization model is: V min ≤V i,t ≤V max , i∈N

[0020] In the formula, V i,t is the voltage value of node i at time t; V min is the minimum voltage value of each node under normal operation of the distribution network; V max is the maximum voltage value of each node under normal operation of the distribution network;

[0021] The energy storage constraint of the distribution network voltage optimization model is:

[0022] In the formula, is the energy content of the energy storage system at node i at time t; E i min is the minimum energy storage level of the energy storage system; E i max is the maximum energy storage level of the energy storage system; is the charging power of the energy storage system at node i at time t; is the maximum charging power of the energy storage system; is the discharging power of the energy storage system at node i at time t; is the maximum discharging power of the energy storage system; is the energy storage level of the energy storage system at node i at time t; is the energy storage level of the energy storage system at node i at time t-1; is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging; and Δt is a time step.

[0023] Further, the energy storage constraint in the power distribution network voltage optimization model is converted according to the physical information of the energy storage system to establish an energy storage system safety model, specifically:

[0024] In the DRL environment, the charging active power and discharging active power of the energy storage system are defined according to the physical information of the energy storage system;

[0025] The charging active power of the energy storage system is specifically:

[0026] The charging reactive power of the energy storage system is specifically:

[0027] In the formula, [·] + / - = max / min{·, 0}; is the charging power of the energy storage system at node i at time t; is the discharging power of the energy storage system at node i at time t; is the absorption action and discharge action of the active power of the energy storage system at node i at time t; is the maximum charging power of the energy storage system; is the maximum discharging power of the energy storage system; E i max is the maximum energy storage level of the energy storage system; E i min is the minimum energy storage level of the energy storage system; is the energy content of the energy storage system at node i at time t; η c is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging; and Δt is a time step.

[0028] The energy storage constraint in the power distribution network voltage optimization model is converted according to the charging active power and discharging active power of the energy storage system to establish an energy storage system safety model, specifically:

[0029] In the formula, is the energy storage level of the energy storage system at node i at time t+1; is the energy storage level of the energy storage system at node i at time t; is the device efficiency when the energy storage system is charging; the device efficiency when the energy storage system is discharged; the charging power of the energy storage system at node i at time t; the discharging power of the energy storage system at node i at time t; and Δt is a time step.

[0030] Further, the Markov decision model is used to define the interaction process of the energy storage system and the power distribution network voltage optimization model, and a training guide model of the energy storage system is formed, specifically as follows:

[0031] The load active power, load reactive power, voltage value of each node, state of charge of the energy storage system, and natural power output of the renewable energy of the power distribution network are obtained to form an observation space, specifically as follows:

[0032] In the formula, O l is the observation space of the energy storage system l; is the load active power; is the load reactive power; V i,t is the voltage value; is the state of charge of the energy storage system; is the natural power output of the photovoltaic power station; is the natural power output of the wind turbine;

[0033] The absorption and release actions of the energy storage system active power are set as an action set to form an action space, specifically as follows:

[0034] In the formula, A l is the action space of the energy storage system l; is the absorption and release action of the energy storage system active power at node i at time t; wherein, represents charging of the energy storage system, represents discharging of the energy storage system;

[0035] The reward function is set in combination with the objective function of the power distribution network voltage optimization model, specifically as follows: R=C α r1+C β r2

[0036] In the formula, R is the reward function; C α is the optimization weight coefficient of the power distribution network voltage optimization model; C β is the out-of-limit weight coefficient of the power distribution network voltage optimization model; V i,t is the voltage value of node i at time t; V0 is the reference voltage of the power distribution network; b1 is the control target of the power distribution network; V min is the minimum voltage value of each node under normal operation of the power distribution network; Vmax The maximum voltage value of each node under normal operation of the power distribution network.

[0037] Further, the operation process of the training guidance model of the energy storage system is specifically:

[0038] The energy storage system controls the energy storage system to read the current time state value of the power distribution network in which the energy storage system is located through the current observation value in the observation space;

[0039] The energy storage system controls the energy storage system to form a current action value according to the current control strategy;

[0040] According to the current action value and the preset state transition relationship, the next time state value and the reward value of the power distribution network in which the energy storage system is located are obtained;

[0041] According to the current observation value, the current action value and the reward value, control experience data is formed, and the control experience data is stored in the verification pool;

[0042] Wherein, the preset state transition relationship is specifically:

[0043] In the formula, s t+1 is the next time state value; s t is the current time state value; o 1:L,t is the current observation value of the energy storage system l; a 1:L,t is the current action value; ω t is an uncertain parameter; is a state transition function.

[0044] Further, the method of using double evaluation network, policy delay update and target policy smoothing improves the Actor-Critic framework to form the TD3 algorithm, specifically:

[0045] The basic network of the TD3 algorithm is set to 1 Actor network, 1 target Actor network, 2 Critic networks and 2 target Critic networks;

[0046] The Actor network is set to a delay update policy;

[0047] The target value calculation formula of the Actor-Critic framework is improved by adding Gaussian noise, specifically:

[0048] In the formula, y t is the target Q value at time t; r(o t ,a t ) is the observation value o t In action a treward value of the state s t i＝1,2 is a discount factor; is an observation o t action target value function of the state s t ; ε is a Gaussian noise;

[0049] updating the network parameters of the Actor-Critic framework using the preset soft update coefficient.

[0050] Further, the combination of the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system is used to train a plurality of energy storage systems in the power distribution network, specifically:

[0051] In combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system, the parameters to be trained in the TD3 algorithm are trained to obtain the trained parameters;

[0052] The trained parameters are determined as the neural network parameters of the energy storage system.

[0053] Further, after the training of each energy storage system is completed, the control of each energy storage system to share the trained neural network parameters in the power distribution network further includes:

[0054] Each energy storage system acquires the shared neural network parameters of other energy storage systems in the power distribution network, and adjusts the voltage of the power distribution network in combination with its own neural network parameters and the acquired neural network parameters.

[0055] The present application provides a voltage regulation method for a multi-energy storage power distribution network. First, an active power distribution network with a large amount of renewable energy access is modeled. A safety model suitable for deep reinforcement learning method is constructed in combination with the actual physical model of the energy storage system. A partially observable Markov model is built. The TD3 algorithm is used to train each energy storage system. The trained network parameters are shared among different energy storage systems. Finally, the voltage of the power distribution network is adjusted in combination with its own network parameters and the network parameters obtained through sharing. The present application optimizes the operation strategy of the energy storage system, realizes the efficient and sustainable operation of the energy storage system, and provides more reliable, stable and efficient energy services for the power system. The DRL algorithm optimizes the charging and discharging strategy of the energy storage system through parameter sharing to support the voltage stability of the power distribution network, reduce energy loss, improve the overall efficiency of the system, and optimize the control of the energy storage system in the active power distribution network.

[0056] Correspondingly, the present application provides a voltage regulation device for a multi-energy storage power distribution network, comprising: an optimization model establishing module, a safety model establishing module, a guide model establishing module, an algorithm forming module, a training module and a sharing module.

[0057] The optimization model establishing module is configured to establish a power distribution network voltage optimization model, and constraints of the power distribution network voltage optimization model include active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and energy storage constraints.

[0058] The safety model establishing module is configured to convert the energy storage constraints in the power distribution network voltage optimization model according to physical information of the energy storage system, and establish an energy storage system safety model.

[0059] The guidance model establishing module is configured to define an interaction process of the energy storage system and the power distribution network voltage optimization model based on a Markov decision model, and form a training guidance model of the energy storage system.

[0060] The algorithm forming module is configured to improve an Actor-Critic framework by using a method of double evaluation network, policy delay update and target policy smoothing, and form a TD3 algorithm.

[0061] The training module is configured to combine the power distribution network voltage optimization model, the energy storage system safety model and the training guidance model of the energy storage system, and train a plurality of energy storage systems in the power distribution network by using the TD3 algorithm.

[0062] The sharing module is configured to control each energy storage system to share the trained neural network parameters in the power distribution network when the training of each energy storage system is completed.

[0063] The present application provides a voltage regulation device for a multi-energy storage power distribution network, which is based on the organic combination of modules. Firstly, an active power distribution network with a large number of renewable energy sources is modeled. Then, a safety model suitable for deep reinforcement learning method is constructed by combining the actual physical model of the energy storage system. A partially observable Markov model is built. The TD3 algorithm is used to train each energy storage system. The trained network parameters are shared among different energy storage systems. Finally, the voltage of the power distribution network is regulated by combining the network parameters of the energy storage system and the network parameters obtained through sharing. The present application optimizes the operation strategy of the energy storage system, realizes the efficient and sustainable operation of the energy storage system, and provides more reliable, stable and efficient energy services for the power system. The DRL algorithm optimizes the charging and discharging strategy of the energy storage system through parameter sharing, supports the voltage stability of the power distribution network, reduces energy loss, improves the overall efficiency of the system, and optimizes the control of the energy storage system in the active power distribution network.

[0064] The present application also provides a voltage regulation device for a multi-energy storage power distribution network, which comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the voltage regulation method for the multi-energy storage power distribution network is realized. BRIEF DESCRIPTION OF DRAWINGS

[0065] FIG. 1 is a flowchart of an embodiment of a voltage regulation method for a multi-energy storage power distribution network according to the present application;

[0066] FIG. 2 is a flowchart of an embodiment of a method for sharing energy storage system parameters according to the present application;

[0067] FIG. 3 is a structural diagram of an embodiment of a voltage regulation device for a multi-energy storage power distribution network according to the present application;

[0068] FIG. 4 is a structural diagram of an embodiment of a voltage regulation device for a multi-energy storage power distribution network according to the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0070] The flowcharts shown in the drawings are only illustrative, and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be further decomposed, combined or partially merged, so that the actual execution order can be changed according to the actual situation.

[0071] Some embodiments of the present application will be described in detail below with reference to the drawings. In the case of no conflict, the embodiments described below and the features in the embodiments can be combined with each other.

[0072] Embodiment 1

[0073] Referring to FIG. 1, which is a flowchart of an embodiment of a voltage regulation method for a multi-energy storage power distribution network according to the present application, the method includes steps 101 to 106, and each step is as follows:

[0074] Step 101: Establish a power distribution network voltage optimization model, and the constraints of the power distribution network voltage optimization model include active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and energy storage constraints.

[0075] Further, in the first embodiment of the present application, in order to improve the voltage quality, an active power distribution network with a large amount of renewable energy is modeled, and the sum of voltage deviation amounts of each node of the power distribution network within a time range T is minimized as an objective function, specifically:

[0076] In the formula, V i,tVit is the voltage value of node i at time t; V0 is the network reference voltage of distribution network; T is the time range; N is the number of nodes of distribution network.

[0077] The dynamic characteristics of active distribution network can be described as active power constraints and reactive power constraints, and the active power constraints and reactive power constraints of the voltage optimization model of the distribution network are respectively:

[0078] In the formula, P i,t is the actual active power injection of node i at time t; Q i,t is the actual reactive power injection of node i at time t; V i,t is the voltage value of node i at time t; V j,t is the voltage value of node j at time t; G ij,t is the real part of the admittance element between node i and node j, that is, the conductance; B ij,t is the imaginary part of the admittance element between node i and node j, that is, the susceptance; θ ij,t is the phase difference between node i and node j.

[0079] The photovoltaic inverter, wind turbine and energy storage system are connected to the power generation side, and the static reactive power compensator is connected to the demand side. The actual active power injection P i,t is obtained by the active power output of the photovoltaic inverter connected to node i at time t The active power output of the wind turbine The active power absorbed or released by the energy storage system And the active power demand of the load The actual reactive power injection Q i,t is obtained by the reactive power injection of the static reactive power compensator connected to node i at time t The reactive power injection of the photovoltaic inverter And the reactive power demand of the load Therefore, for the nodes connected to the control device, the power flow injection constraint of the voltage optimization model of the distribution network is represented as:

[0080] In the formula, P i,t is the actual active power injection; is the active power absorbed or released by the energy storage system; is the active power output of the photovoltaic inverter; is the active power output of the wind turbine; is the active power demand of the load; Q i,t is the actual reactive power injection; is the reactive power injection of the photovoltaic inverter; This refers to the reactive power injection amount of the static reactive power compensator. This represents the reactive power demand of the load.

[0081] The voltage constraint of the distribution network voltage optimization model is: V min ≤V i,t ≤V max , i∈N

[0082] In the formula, V i,t V represents the voltage value of node i at time t. min V represents the minimum voltage value at each node under normal operating conditions of the distribution network. max This represents the maximum voltage value at each node under normal operating conditions of the distribution network.

[0083] The energy storage constraints for the distribution network voltage optimization model are:

[0084] In the formula, E represents the energy content of the energy storage system at node i at time t. i min E represents the minimum energy storage level of the energy storage system. i max This represents the maximum energy storage level of the energy storage system. The charging power of the energy storage system at node i at time t; This is the maximum charging power of the energy storage system; Let be the discharge power of the energy storage system at node i at time t; This represents the maximum discharge power of the energy storage system. Let be the energy storage level of the energy storage system at node i at time t; Let be the energy storage level of the energy storage system at node i at time t-1; Equipment efficiency when charging an energy storage system; Δt represents the equipment efficiency of the energy storage system during discharge; Δt is the time step. The energy storage constraints of the distribution network voltage optimization model include coupling constraints on energy storage between two consecutive time steps.

[0085] Step 102: Based on the physical information of the energy storage system, transform the energy storage constraints in the power distribution network voltage optimization model to establish a safety model for the energy storage system.

[0086] Furthermore, in the first embodiment of the present invention, based on the physical information of the energy storage system, the energy storage constraints in the distribution network voltage optimization model are transformed to establish an energy storage system safety model, specifically as follows:

[0087] In the DRL environment, the charging active power and discharging active power of the energy storage system are defined based on the physical information of the energy storage system.

[0088] wherein the charging active power of the energy storage system is specifically:

[0089] the charging reactive power of the energy storage system is specifically:

[0090] wherein [·] + / - = max / min{·, 0}; is the charging power of the energy storage system at node i at time t; is the discharging power of the energy storage system at node i at time t; is the absorption and release action of the active power of the energy storage system at node i at time t; is the maximum charging power of the energy storage system; is the maximum discharging power of the energy storage system;E i max is the maximum energy storage level of the energy storage system;E i min is the minimum energy storage level of the energy storage system; is the energy content of the energy storage system at node i at time t;η c is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging;Δt is the time step;

[0091] According to the charging active power and the discharging active power of the energy storage system, the energy storage constraint in the power distribution network voltage optimization model is converted, and an energy storage system safety model is established, specifically:

[0092] wherein, is the energy storage level of the energy storage system at node i at time t+1; is the energy storage level of the energy storage system at node i at time t; is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging; is the charging power of the energy storage system at node i at time t; is the discharging power of the energy storage system at node i at time t;Δt is the time step.

[0093] In the first embodiment of the present application, the present application defines two mutually exclusive quantities of the charging active power and the discharging active power of the energy storage system, and based on the definition formula of the two quantities, the state transition equation of the energy storage constraint in the distribution network voltage optimization model can be converted to realize the hard constraint of the system without using the penalty function energy storage system. The present application can improve the utilization efficiency of the energy storage system in the DRL method by constructing a safety model and controlling the energy storage system in the DRL, so that the energy storage system is well combined with the deep reinforcement learning algorithm and plays a good role in the voltage control of the distribution network.

[0094] Step 103: defining the interaction process of the energy storage system and the distribution network voltage optimization model based on the Markov decision model to form a training guide model of the energy storage system.

[0095] In the first embodiment of the present application, the role of the partially observable Markov decision model in DRL is to provide a structured mathematical model for describing the interaction process of the agent and the environment, thereby guiding the learning and decision optimization of the agent. The environment of the present application is the distribution network voltage optimization model, and the agent is the energy storage system. The Markov decision model of the present application describes the probability of the system from one state to the next state, and the process has no memory, that is, the next state only depends on the current state and does not depend on the previous state.

[0096] In the first embodiment of the present application, the partially observable Markov decision model adopts Markov elements are constructed, wherein L represents the number of agents; S={s1,s2,L,s l ,L,s L} is the state of the environment of each agent; O={o1,o2,L,o l ,L,o L} is the local observation space of each agent, and the elements of the observation space are derived from the state space; A={a1,a2,L,a l ,L,a L} is the action space of each agent; R={r1,r2,L,r l ,L,r L} is the reward function of each agent; is the state transition function, and its expression is wherein ω is an uncertain parameter for representing the uncertainty and randomness of the environment (DN).

[0097] Further, in the first embodiment of the present application, the interaction process of the energy storage system and the distribution network voltage optimization model is defined based on the Markov decision model to form a training guide model of the energy storage system, specifically:

[0098] The agent can perceive the current state of the system by observing the space O = {o1, o2, …, o l ,…,o L}, and then give the control action. The local observation space of the system can be formed by obtaining the load active power, load reactive power, voltage value of each node, state of charge of the energy storage system and natural power output of the renewable energy of the power distribution network, and specifically:

[0099] In the formula, O l is the observation space of the energy storage system l; is the load active power; is the load reactive power;V i,t is the voltage value; is the state of charge of the energy storage system; is the natural power output of the photovoltaic power station; is the natural power output of the wind turbine. In the voltage control of the power distribution network, the natural active power output of the photovoltaic power station is related to the solar radiation intensity, shading, temperature, etc. The natural active power output of the wind turbine is affected by the wind speed, air density, etc. The load active power demand and the load reactive power demand of the load are affected by the user's power consumption behavior.

[0100] The control optimization of the present application does not involve OLTC, capacitor, SVC and wind-solar power generation, so the action of the power distribution network is mainly the absorption and release of the active power of the energy storage system. Therefore, the absorption and release actions of the active power of the energy storage system are set as the action set to form the action space, and specifically:

[0101] In the formula, A l is the action space of the energy storage system l; is the absorption and release action of the active power of the energy storage system at node i at time t; wherein, indicates that the energy storage system is charging, indicates that the energy storage system is discharging.

[0102] In order to make all agents work cooperatively, all agents are controlled to share the same reward function, the reward function is set in combination with the objective function of the power distribution network voltage optimization model, and the voltage constraint of the power distribution network voltage optimization model is added as a penalty term to the reward function, and specifically: R = C α r1+C β r2

[0103] In the formula, R is the reward function;C αis an optimization weight coefficient of the voltage optimization model of the power distribution network; C β is an out-of-limit weight coefficient of the voltage optimization model of the power distribution network; V i,t is a voltage value of the node i at the t time; V0 is a reference voltage of the power distribution network; b1 is a control target of the power distribution network; V min is a minimum voltage value of each node under normal operation of the power distribution network; V max is a maximum voltage value of each node under normal operation of the power distribution network.

[0104] Further, in the first embodiment of the present application, the running process of the training guide model of the energy storage system is specifically as follows:

[0105] The energy storage system reads the current time state value of the power distribution network in which the energy storage system is located through the current observation value in the observation space.

[0106] The energy storage system forms a current action value according to the current control strategy.

[0107] According to the current action value and a preset state transition relationship, a next time state value and a reward value of the power distribution network in which the energy storage system is located are obtained.

[0108] Control experience data are formed according to the current observation value, the current action value and the reward value, and the control experience data are stored in a verification pool.

[0109] The preset state transition relationship is specifically as follows:

[0110] In the formula, s t+1 is a next time state value; s t is a current time state value; o 1:L,t is a current observation value of the energy storage system l; a 1:L,t is a current action value; ω t is an uncertain parameter. is a state transition function.

[0111] In the first embodiment of the present application, in the Markov decision model, at the t time, the environment has a state s t ; the intelligent agent reads the state of the environment through the observation value o t , and then gives an action a l,t according to the control strategy , which has an influence on the environment so that the environment state is obtained according to the state transition relationship to obtain the environment state s t+1 at the next time; in this process, the intelligent agent obtains a reward value r l,t . According to the data in the process, an array (o l,t , al,t ,r l,t ,o l,t+1 ) as control experience is stored in the test pool for subsequent use.

[0112] Step 104: improve the Actor-Critic framework by using the method of double evaluation network, policy delay update and target policy smoothing to form the TD3 algorithm.

[0113] Further, in the first embodiment of the present application, the Actor-Critic framework is improved by using the method of double evaluation network, policy delay update and target policy smoothing to form the TD3 algorithm, specifically:

[0114] The basic network of the TD3 algorithm is set to 1 Actor network, 1 target Actor network, 2 Critic networks and 2 target Critic networks;

[0115] The delay update policy is set for the Actor network;

[0116] The target value calculation formula of the Actor-Critic framework is improved by adding Gaussian noise, specifically:

[0117] In the formula, y t is the target Q value at time t; r(o t ,a t ) is the reward value of the observation o t under action a t ; γmin i＝1,2 is the discount factor; is the target value function of the observation o t under action ; ε is the Gaussian noise;

[0118] The network parameters of the Actor-Critic framework are updated using the preset soft update coefficient.

[0119] In the first embodiment of the present application, the Actor-Critic (AC) framework can form a reinforcement learning algorithm TD3 applied to continuous action space. Among them, the input of the Critic evaluation network is the observation o t and the action amount a t , and the output is the scalar Q value function estimate Q θ (o t ,a t ), the Q value is used to evaluate the pros and cons of the current control policy, and the to-be-trained parameter of the Critic evaluation network is θ. The Actor action network takes the observation o t as input, and outputs continuous action The to-be-trained parameters of the actor network are Responsible for policy improvement according to the Q value estimated by the critic network.

[0120] In the first embodiment of the present application, the TD3 algorithm of the present application uses 2 critic networks and 2 target critic networks, and the smaller value is taken when updating the target value, avoiding the situation that the current network and the target network are too similar and cannot be independently estimated due to the slow change of the policy. The actor and the critic are coupled with each other in the AC framework, and when the critic estimation value is inaccurate, it will affect the update direction of the actor, produce wrong actions, and may cause the learning behavior to diverge. Therefore, the present application delays the update frequency of the actor, so that the actor can be updated when the critic estimation error is low, which can maximize the avoidance of action errors caused by inaccurate critic estimation. The present application averages the target value by adding Gaussian noise to the next action, reduces the variance of the target value, and realizes the smoothing of the target policy. Through the methods of double evaluation network, policy delay update and target policy smoothing, the Actor-Critic framework is improved, the overestimation problem of Q value existing in the traditional AC framework is suppressed, and the training efficiency and stability are improved. The formula for calculating the target value is updated after the improvement of the Actor-Critic framework:

[0121] Wherein, ε is the added Gaussian noise, ε ~ clip[N(0, σ)-c, c], c is the upper and lower limit of the noise.

[0122] In the first embodiment of the present application, while improving the Actor-Critic framework, the TD3 algorithm provides a parameter update method using a soft update coefficient τ, wherein the parameter update of the actor network is: The time difference error calculation of the critic at each step is: t =y t -Q θ (o t ,a t );The parameter update calculation method in the critic network is: The parameter update of the target critic network is: j′ ←τθ j′ +(1-τ)θ′ j .

[0123] Step 105: combine the power distribution network voltage optimization model, the energy storage system safety model and the training guidance model of the energy storage system, and train a plurality of energy storage systems in the power distribution network using the TD3 algorithm.

[0124] Further, in the first embodiment of the present application, in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system, the TD3 algorithm is used to train a plurality of energy storage systems in the power distribution network, specifically:

[0125] In combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system, the parameters to be trained in the TD3 algorithm are trained to obtain the trained parameters.

[0126] The trained parameters are determined as the neural network parameters of the energy storage system.

[0127] In the first embodiment of the present application, after the multi-energy storage system is modeled as a multi-agent system, one energy storage system is one agent, in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system, the TD3 algorithm can be used to train the neural network parameters of the energy storage system. The trained parameters are

[0128] Step 106: After the training of each energy storage system is completed, control each energy storage system to share the trained neural network parameters of itself in the power distribution network.

[0129] Further, in the first embodiment of the present application, after the training of each energy storage system is completed, control each energy storage system to share the trained neural network parameters of itself in the power distribution network, further comprising:

[0130] Each energy storage system acquires the neural network parameters shared by other energy storage systems in the power distribution network, and adjusts the voltage of the power distribution network in combination with its own neural network parameters and the acquired neural network parameters.

[0131] In the first embodiment of the present application, after the training of each energy storage system is completed, each energy storage system will obtain a trained neural network parameter. Each energy storage system shares the trained neural network parameter obtained by itself among different agents, so that the energy storage system can still obtain the control strategy of other energy storage systems without obtaining the observation information in the control area of other energy storage systems, avoiding the propagation of global information and improving the learning rate of the agent and the cooperativeness between different agents.

[0132] As an example of the first embodiment of the application, referring to FIG. 2, it is a flowchart of an embodiment of the energy storage system parameter sharing method provided by the application. The power distribution network is controlled by N energy storage systems. In the training process, each energy storage system is trained according to the observation values (i.e., local observation information) of its control area. After training, the trained parameters are shared with other energy storage systems in the power distribution network. In addition, the experience replay buffer shared by multiple energy storage systems can improve sampling efficiency while reducing memory usage, which helps to improve convergence speed and stability.

[0133] In summary, the first embodiment of the application provides a voltage regulation method for a multi-energy storage power distribution network. First, an active power distribution network with a large amount of renewable energy access is modeled. A safety model suitable for deep reinforcement learning methods is constructed by combining the actual physical model of the energy storage system. A partially observable Markov model is built. The TD3 algorithm is used to train each energy storage system. The network parameters obtained by training are shared among different energy storage systems. Finally, the voltage of the power distribution network is regulated by combining its own network parameters and the network parameters obtained through sharing. The application optimizes the operation strategy of the energy storage system, realizes the efficient and sustainable operation of the energy storage system, and provides more reliable, stable and efficient energy services for the power system. The DRL algorithm optimizes the charging and discharging strategy of the energy storage system to support the voltage stability of the power distribution network, reduce energy loss, improve the overall efficiency of the system, and optimize the control of the energy storage system in the active power distribution network.

[0134] Embodiment 2

[0135] Referring to FIG. 3, it is a structural schematic diagram of an embodiment of the voltage regulation device for a multi-energy storage power distribution network provided by the application. The device includes an optimization model establishment module 201, a safety model establishment module 202, a guidance model establishment module 203, an algorithm formation module 204, a training module 205 and a sharing module 206.

[0136] The optimization model establishment module 201 is used to establish a power distribution network voltage optimization model. The constraints of the power distribution network voltage optimization model include active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and energy storage constraints.

[0137] The safety model establishment module 202 is used to convert the energy storage constraints in the power distribution network voltage optimization model according to the physical information of the energy storage system, and establish a safety model of the energy storage system.

[0138] The guidance model establishment module 203 is used to define the interaction process of the energy storage system and the power distribution network voltage optimization model based on the Markov decision model, and form a training guidance model of the energy storage system.

[0139] The algorithm forming module 204 is configured to improve the Actor-Critic framework by adopting a method of double evaluation network, policy delay update and target policy smoothing to form a TD3 algorithm;

[0140] The training module 205 is configured to train a plurality of energy storage systems in the power distribution network by using the TD3 algorithm in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system.

[0141] The sharing module 206 is configured to share the trained neural network parameters of each energy storage system in the power distribution network after the training of each energy storage system is completed.

[0142] Further, in the second embodiment of the present application, a power distribution network voltage optimization model is established, and the constraints of the power distribution network voltage optimization model include active power constraints, reactive power constraints, voltage constraints and energy storage constraints, specifically:

[0143] The sum of voltage deviation amounts of each node in the power distribution network is minimized as the objective function of the power distribution network voltage optimization model, specifically:

[0144] In the formula, V i,t is the voltage value of node i at time t; V0 is the reference voltage of the power distribution network; T is the time range; N is the number of nodes in the power distribution network;

[0145] The active power constraints and the reactive power constraints of the power distribution network voltage optimization model are respectively:

[0146] In the formula, P i,t is the actual active power injection amount of node i at time t; Q i,t is the actual reactive power injection amount of node i at time t; V i,t is the voltage value of node i at time t; V j,t is the voltage value of node j at time t; G ij,t is the conductance between node i and node j; B ij,t is the susceptance between node i and node j; θ ij,t is the phase difference between node i and node j;

[0147] The power flow injection constraint of the power distribution network voltage optimization model is:

[0148] In the formula, P i,t is the actual active power injection amount; is the active power amount absorbed or discharged by the energy storage system; is the active power output amount of the photovoltaic inverter; active power output of the wind turbine; active power demand of the load; i,t actual reactive power injection; reactive power injection of the photovoltaic inverter; reactive power injection of the static reactive power compensator; reactive power demand of the load;

[0149] The voltage constraint of the distribution network voltage optimization model is: V min ≤V i,t ≤V max , i∈N

[0150] In the formula, V i,t is the voltage value of node i at time t; V min is the minimum voltage value of each node under normal operation of the distribution network; V max is the maximum voltage value of each node under normal operation of the distribution network;

[0151] The energy storage constraint of the distribution network voltage optimization model is:

[0152] In the formula, is the energy content of the energy storage system at node i at time t; E i min is the minimum energy storage level of the energy storage system; E i max is the maximum energy storage level of the energy storage system; is the charging power of the energy storage system at node i at time t; is the maximum charging power of the energy storage system; is the discharging power of the energy storage system at node i at time t; is the maximum discharging power of the energy storage system; is the energy storage level of the energy storage system at node i at time t; is the energy storage level of the energy storage system at node i at time t-1; is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging; and Δt is the time step.

[0153] Further, in the second embodiment of the present application, the energy storage constraint in the distribution network voltage optimization model is converted according to the physical information of the energy storage system, and an energy storage system safety model is established, specifically:

[0154] In the DRL environment, the charging active power and the discharging active power of the energy storage system are defined according to the physical information of the energy storage system;

[0155] wherein the charging active power of the energy storage system is specifically:

[0156] the charging reactive power of the energy storage system is specifically:

[0157] in the formula, [·]+ / -=max / min{·,0}; is the charging power of the energy storage system at node i at time t; is the discharging power of the energy storage system at node i at time t; is the absorption action and the release action of the active power of the energy storage system at node i at time t; is the maximum charging power of the energy storage system; is the maximum discharging power of the energy storage system; i max is the maximum energy storage level of the energy storage system; i min is the minimum energy storage level of the energy storage system; is the energy content of the energy storage system at node i at time t; c is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging;Δt is a time step.

[0158] According to the charging active power and the discharging active power of the energy storage system, the energy storage constraint in the power distribution network voltage optimization model is converted, and an energy storage system safety model is established, specifically:

[0159] in the formula, is the energy storage level of the energy storage system at node i at time t+1; is the energy storage level of the energy storage system at node i at time t; is the device efficiency when the energy storage system is charging; is the device efficiency when the energy storage system is discharging; is the charging power of the energy storage system at node i at time t; is the discharging power of the energy storage system at node i at time t;Δt is a time step.

[0160] Further, in the second embodiment of the present application, the interaction process of the energy storage system and the power distribution network voltage optimization model is defined based on a Markov decision model, and a training guidance model of the energy storage system is formed, specifically:

[0161] The active power of the load, the reactive power of the load, the voltage value of each node, the state of charge of the energy storage system and the natural power output of the renewable energy of the power distribution network are acquired to form an observation space, specifically:

[0162] In the formula, O l is the observation space of the energy storage system l; is the load active power; is the load reactive power; V i,t is the voltage value; is the state of charge of the energy storage system; is the natural power output of the photovoltaic power station; is the natural power output of the wind turbine;

[0163] The absorption action and the release action of the energy storage system active power are set as an action set to form an action space, and specifically:

[0164] In the formula, A l is the action space of the energy storage system l; is the absorption action and the release action of the energy storage system active power at node i at t; wherein, indicates that the energy storage system is charged, indicates that the energy storage system is discharged;

[0165] In combination with the objective function of the power distribution network voltage optimization model, a reward function is set, and specifically: R=C α r1+C β r2

[0166] In the formula, R is the reward function; C α is the optimization weight coefficient of the power distribution network voltage optimization model; C β is the out-of-limit weight coefficient of the power distribution network voltage optimization model; V i,t is the voltage value of node i at t; V0 is the reference voltage of the power distribution network; b1 is the control target of the power distribution network; V min is the minimum voltage value of each node under normal operation of the power distribution network; V max is the maximum voltage value of each node under normal operation of the power distribution network.

[0167] Further, in the second embodiment of the present application, the running process of the training guide model of the energy storage system is specifically:

[0168] The current time state value of the power distribution network in which the energy storage system is located is read by the energy storage system through the current observation value in the observation space;

[0169] The current action value is formed according to the current control strategy;

[0170] According to the current action value and the preset state transition relationship, the next time state value and the reward value of the power distribution network in which the energy storage system is located are obtained.

[0171] forming control experience data according to the current observation value, the current action value and the reward value, and storing the control experience data into a verification pool;

[0172] The preset state transition relationship is specifically:

[0173] In the formula, s t+1 is a next time state value; s t is a current time state value; o 1:L,t is a current observation value of the energy storage system l; a 1:L,t is a current action value; ω t is an uncertain parameter; is a state transition function.

[0174] Further, in the second embodiment of the present application, the Actor-Critic framework is improved by using a double evaluation network, a policy delay update and a target policy smoothing method to form a TD3 algorithm, and the TD3 algorithm is specifically:

[0175] The basic network of the TD3 algorithm is set to 1 Actor network, 1 target Actor network, 2 Critic networks and 2 target Critic networks;

[0176] A delay update policy is set for the Actor network;

[0177] The target value calculation formula of the Actor-Critic framework is improved by adding Gaussian noise, and the target value calculation formula is specifically:

[0178] In the formula, y t is a target Q value at time t; r(o t ,a t ) is a reward value of the observation value o t under the action a t ; γmin i＝1,2 is a discount factor; is a target value function of the observation value o t under the action ; and ε is Gaussian noise.

[0179] The network parameters of the Actor-Critic framework are updated by using a preset soft update coefficient.

[0180] Further, in the second embodiment of the present application, the TD3 algorithm is used to train a plurality of energy storage systems in the power distribution network in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guidance model of the energy storage system, and the TD3 algorithm is specifically:

[0181] The training parameters in the TD3 algorithm are trained in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system, to obtain the trained parameters;

[0182] The trained parameters are determined as the neural network parameters of the energy storage system.

[0183] Further, in the second embodiment of the present application, after each energy storage system is trained, the energy storage system shares the trained neural network parameters in the power distribution network, and further comprises:

[0184] Each energy storage system obtains the shared neural network parameters of other energy storage systems in the power distribution network, and adjusts the voltage of the power distribution network in combination with the neural network parameters of itself and the obtained neural network parameters.

[0185] In summary, the second embodiment of the present application provides a voltage adjustment device for a multi-energy storage power distribution network, which is based on the organic combination of modules. First, an active power distribution network with a large number of renewable energy sources is modeled. A safety model suitable for deep reinforcement learning method is constructed in combination with the actual physical model of the energy storage system. A partially observable Markov model is built. The TD3 algorithm is used to train each energy storage system. The trained network parameters are shared among different energy storage systems. Finally, the voltage of the power distribution network is adjusted in combination with the network parameters of itself and the network parameters obtained through sharing. Through the optimization of the operation strategy of the energy storage system, the present application realizes the efficient and sustainable operation of the energy storage system, and provides more reliable, stable and efficient energy services for the power system. The DRL algorithm optimizes the charging and discharging strategy of the energy storage system through parameter sharing to support the voltage stability of the power distribution network, reduce energy loss, improve the overall efficiency of the system, and optimize the control of the energy storage system in the active power distribution network.

[0186] Referring to FIG. 4, it is a structural schematic diagram of an embodiment of the voltage adjustment device for a multi-energy storage power distribution network provided by the present application. The device can be a server, which includes a processor, a memory and a network interface connected through a system bus. The memory can include a non-volatile storage medium and an internal memory.

[0187] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0188] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions that, when executed, can cause the processor to perform any kind of P2P-based broadcast system data transmission method.

[0189] The internal memory provides an environment for the running of a computer program in a non-volatile storage medium, which, when executed by the processor, can cause the processor to perform any kind of P2P-based broadcast system data transmission method.

[0190] The network interface is used for network communication, such as sending assigned tasks.

[0191] Those skilled in the art can understand that the structure shown in FIG. 4 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0192] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0193] The above specific embodiments further illustrate the purpose, technical scheme and beneficial effects of the present application. It should be understood that the above description is only for specific embodiments of the present application and is not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A voltage regulation method for a multi-energy storage power distribution network, characterized in that, The application relates to a method for training a storage system in a power distribution network. The method comprises the following steps: establishing a power distribution network voltage optimization model, wherein the constraints of the power distribution network voltage optimization model comprise active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and storage constraints; transforming the storage constraints in the power distribution network voltage optimization model according to physical information of the storage system to establish a storage system safety model; defining an interaction process of the storage system and the power distribution network voltage optimization model based on a Markov decision model to form a training guide model of the storage system; improving an Actor-Critic framework by using a double evaluation network, a policy delay update and a target policy smoothing method to form a TD3 algorithm; training a plurality of storage systems in the power distribution network by using the TD3 algorithm in combination with the power distribution network voltage optimization model, the storage system safety model and the training guide model of the storage system; 2. The method of voltage regulation for a multi-energy distribution grid of claim 1, wherein, controlling the storage systems to share the neural network parameters trained by the storage systems in the power distribution network after the training of the storage systems is completed. The sum of voltage deviation amounts of each node of the power distribution network is minimized as the objective function of the power distribution network voltage optimization model, specifically: In the formula, V i,t is the voltage value of node i at time t; V0 is the network reference voltage of the power distribution network; T is the time range; N is the number of nodes of the power distribution network; The active power constraint and the reactive power constraint of the power distribution network voltage optimization model are respectively: In the formula, P i,t is the actual active power injection of node i at time t; Q i,t is the actual reactive power injection of node i at time t; V i,t is the voltage value of node i at time t. V j,t Vj(t) is the voltage value of the node j at time t; G ij,t Gij is the conductance between node i and node j; B ij,t Bij is the susceptance between node i and node j; θ ij,t is the phase difference between node i and node j; The power flow injection constraint of the distribution network voltage optimization model is: In the formula, P i,t is the actual active power injection; an amount of active power absorbed or delivered by the energy storage system; active power output of a photovoltaic inverter; active power output of a wind turbine; Q is the amount of reactive power demand; i,t Q is the amount of actual reactive power injection; For the amount of reactive power injection of a photovoltaic inverter; For static var compensator reactive power injection amount; The power distribution network voltage optimization model is established, and the constraints of the power distribution network voltage optimization model comprise active power constraints, reactive power constraints, voltage constraints and storage constraints. The voltage constraint of the power distribution network voltage optimization model is: In the formula, V i,t is the voltage value of node i at time t; V min is the minimum voltage value of each node under normal operation of the power distribution network; V max is the maximum voltage value of each node under normal operation of the power distribution network; The energy storage constraint of the power distribution network voltage optimization model is: In the formulae, Ei(t) is the energy content of the energy storage system at node i at time t; i min Emin is the minimum energy storage level of the energy storage system; i max Emax is the maximum energy storage level of the energy storage system; Pcharge(i, t) is the charging power of the energy storage system at node i at time t; maximal charging power for the energy storage system; Pdischarge(i, t) is the discharging power of the energy storage system at node i at time t; for the maximum discharge power of the energy storage system; a storage level of the energy storage system at node i at time t; a storage level of the energy storage system at node i at time t-1; Device efficiency when charging an energy storage system; The load reactive power demand is calculated.

3. The method of voltage regulation for a multi-energy distribution grid of claim 2, wherein, The device efficiency of the storage system during discharging is calculated. The physical information of the storage system is defined to calculate the charging active power and the discharging active power of the storage system in a DRL environment. The charging active power of the energy storage system is specifically: The charging reactive power of the energy storage system is specifically: In the formula, [·] + / - = max / min{·, 0}; Pcharge(i, t) is the charging power of the energy storage system at node i at time t; Pdischarge(i, t) is the discharging power of the energy storage system at node i at time t; absorbing and discharging actions of the energy storage system at node i at time t; to the maximum charging power of the energy storage system; E i max E i min E Qi(t) is the energy content of the energy storage system at node i at time t; η c η is the device efficiency when charging the energy storage system; The device efficiency of the storage system during discharging is calculated. According to the charging active power and the discharging active power of the energy storage system, the energy storage constraint in the power distribution network voltage optimization model is converted, an energy storage system safety model is established, and specifically: In the formulae, a storage level of the energy storage system at the node i at time t+1; a storage level of the energy storage system at node i at time t; Device efficiency when charging an energy storage system; Device efficiency for discharging of energy storage system; Pcharge(i, t) is the charging power of the energy storage system at node i at time t; The discharging power of the storage system at the node i at the time t is calculated.

4. The method of voltage regulation for a multi-energy storage power distribution grid of claim 2, wherein, The natural power output of the fan is calculated. Obtain the load active power, load reactive power, voltage value of each node, state of charge of energy storage system and natural power output of renewable energy of the power distribution network to form an observation space, specifically: In the formula, O l is the observation space of the energy storage system l; to load real power; for loading the reactive power; V i,t for the voltage value; state of charge for an energy storage system; for the natural power output of a photovoltaic power plant; The discharging of the storage system is represented. The absorbing action and the discharging action of the active power of the energy storage system are set as an action set, an action space is formed, and specifically: In the formula, A l is the action space of the energy storage system l; is the absorption action and the discharge action of the energy storage system at the node i at the time t; wherein, representing charging of the energy storage system, The reward function is set in combination with the objective function of the power distribution network voltage optimization model. The operation process of the training guide model of the storage system is as follows: R = C α r1 + C β r2 In the formula, R is a reward function; C α is an optimization weight coefficient of the distribution network voltage optimization model; C β is an out-of-limit weight coefficient of the distribution network voltage optimization model; V i,t is a voltage value of node i at time t; V0 is a reference voltage of the distribution network; b1 is a control target of the distribution network; V min is a minimum voltage value of each node under normal operation of the distribution network; V max is a maximum voltage value of each node under normal operation of the distribution network.

5. The method of voltage regulation for a multi-energy distribution grid of claim 4, wherein, The current state value of the power distribution network in which the storage system is located is read by the storage system through observation of the current observation value in the observation space. The current action value is formed according to the current control strategy. The next time state value and the reward value of the power distribution network in which the storage system is located are obtained according to the current action value and a preset state transition relationship. The control experience data is formed according to the current observation value, the current action value and the reward value, and the control experience data is stored in a test pool. The state transition function is calculated. The preset state transition relationship is specifically as follows: where s t+1 is the next time state value; s t is the current time state value; o 1:L,t is the current observation value of the energy storage system l; a 1:L,t is the current action value; ω t is the uncertain parameter; The TD3 algorithm is formed by improving the Actor-Critic framework by using the double evaluation network, the policy delay update and the target policy smoothing method.

6. The method of voltage regulation for a multi-energy distribution grid of claim 1, wherein, The basic network of the TD3 algorithm is set as one Actor network, one target Actor network, two Critic networks and two target Critic networks. The delay update strategy is set for the Actor network. ​ The target value calculation formula of the Actor-Critic framework is improved by increasing Gaussian noise, specifically: where y t is the target Q value at time t; r(o t ,a t ) is the observed value o t the reward value at action a t ; and γmin i＝1,2 is the discount factor; For the observed value o t In the action a target value function; and ε is a Gaussian noise; The network parameters of the Actor-Critic framework are updated by using the preset soft update coefficient.

7. The method of voltage regulation for a multi-energy distribution grid of claim 1, wherein, The TD3 algorithm is used to train the plurality of energy storage systems in the power distribution network in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system. The training guide model of the energy storage system is combined with the power distribution network voltage optimization model and the energy storage system safety model, and the TD3 algorithm is used to train the to-be-trained parameters in the TD3 algorithm. The trained parameters are determined as the neural network parameters of the energy storage system.

8. The method of voltage regulation for a multi-energy distribution grid of claim 1, wherein, After the training of each energy storage system is completed, the trained neural network parameters of each energy storage system are shared in the power distribution network. Each energy storage system acquires the shared neural network parameters of other energy storage systems in the power distribution network, and adjusts the voltage of the power distribution network in combination with the neural network parameters of the energy storage system and the acquired neural network parameters.

9. A voltage regulating device for a multi-energy power distribution network, characterized by, The method comprises the following steps: The method comprises the following steps: The optimization model establishment module is configured to establish a power distribution network voltage optimization model, and the constraints of the power distribution network voltage optimization model comprise active power constraints, reactive power constraints, power flow injection constraints, voltage constraints and energy storage constraints. The safety model establishment module is configured to convert the energy storage constraints in the power distribution network voltage optimization model according to the physical information of the energy storage system, and establish an energy storage system safety model. The guide model establishment module is configured to define the interaction process of the energy storage system and the power distribution network voltage optimization model based on a Markov decision model, and form a training guide model of the energy storage system. The algorithm formation module is configured to improve the Actor-Critic framework by using a double evaluation network, a policy delay update and a target policy smoothing method, and form a TD3 algorithm. The training module is configured to train a plurality of energy storage systems in the power distribution network by using the TD3 algorithm in combination with the power distribution network voltage optimization model, the energy storage system safety model and the training guide model of the energy storage system. The sharing module is configured to control each energy storage system to share the trained neural network parameters in the power distribution network after the training of each energy storage system is completed.

10. A voltage regulation device for a multi-energy storage distribution network, characterized in that, The voltage regulation method for the multi-energy storage power distribution network comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the voltage regulation method for the multi-energy storage power distribution network according to any one of claims 1 to 8 when executing the computer program.

Images