Method and system for regulating and controlling long feeder voltage after loop closing of power distribution network based on deep reinforcement learning
Through deep reinforcement learning and multi-agent collaborative optimization strategies, the problem of insufficient voltage control accuracy of long feeders after the distribution network is closed is solved, and precise voltage control and improved stability of the power system are achieved.
Patent Information
- Application Number
- CN202510811078.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-30
AI Technical Summary
In the existing technology, the voltage control accuracy of long feeders after the distribution network is closed is insufficient and the adaptability is poor, resulting in voltage fluctuations and instability, affecting the stability and safety of the power system.
A multi-agent collaborative optimization strategy based on deep reinforcement learning is adopted, combined with the AMAPPO algorithm. Through data processing and model building before and after loop closing, precise control of long feeder voltage is achieved, including data preprocessing, dynamic model construction and agent voltage regulation method training.
It achieves precise voltage regulation in a dynamic and uncertain environment, improves power quality and regulation adaptability, and ensures stable and efficient operation of the power system.
Smart Images
Figure CN120728618A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of power system control technology, and specifically relates to a method and system for controlling the voltage of long feeders after the distribution network is closed based on deep reinforcement learning. Background Art
[0002] In my country, urban 10kV distribution networks generally adopt a ring network design and open-loop operation. During distribution network maintenance, load transfer can be achieved without interrupting power to users by closing the ring and switching power. However, due to the potential for excessive closing currents or surge currents, closing the ring can lead to voltage fluctuations and instability on long feeders on the load side after closing the ring, or even tripping protection circuits, causing wider power outages. With technological advancements and accumulated experience, the safety of closing the ring is gradually improving. In recent years, emerging technologies such as artificial intelligence and the Internet of Things have rapidly developed. Deep reinforcement learning, due to its powerful representation and decision-making capabilities under uncertainty, has attracted widespread attention and begun to be applied to voltage regulation in power distribution networks. Numerous methods based on deep reinforcement learning have been developed for active distribution network voltage regulation. However, as distribution networks grow in size, the requirements for communication equipment become higher. Furthermore, the dimensionality of the state and action spaces of intelligent agents also increases, which affects the speed and performance of agent training. When multiple nodes or devices in a distribution network need to collaborate, the use of multi-agent deep reinforcement learning algorithms for coordinated control is a current research hotspot in deep reinforcement learning for distribution network voltage regulation. Summary of the Invention
[0003] The main purpose of this invention is to overcome the defects of insufficient accuracy and poor adaptability of long feeder voltage control after the distribution network is closed in the existing technology. By introducing a deep reinforcement learning algorithm and combining it with a multi-agent collaborative optimization strategy, it can achieve precise control of long feeder voltage in a dynamic and uncertain environment, comprehensively improve the power quality of the distribution network and the intelligence level of voltage dispatching control, and ensure stable and efficient operation of the power system.
[0004] To achieve the above objectives, the present invention provides a method for controlling the voltage of a long feeder after the distribution network is closed based on deep reinforcement learning. The method comprises: Analysis of distribution network operation characteristics before closing the loop and judgment of closing conditions; Active distribution network system model and problem modeling in uncertain environment after loop closing; Training and simulation results analysis of voltage regulation method based on AMAPPO.
[0005] Preferably, before the analysis of the operating characteristics of the distribution network before closing the loop and the judgment of the closing conditions, the data input and cleaning work is also included, specifically including that the acquisition system obtains the voltage amplitude, phase angle, and current electrical parameters of the closing point and related lines, and pre-processes the electrical parameters, including removing outliers, filling missing data, and verifying data consistency to ensure the accuracy and reliability of the data.
[0006] Preferably, the analysis of the operating characteristics of the distribution network before closing the loop and the judgment of the closing conditions specifically include judging the voltage amplitude, phase angle, phase sequence and current parameters at the closing point calculated according to the closing criterion to determine whether the closing conditions are met; if the conditions are met, proceed to the next step; if not, make corresponding adjustments to the distribution network until the closing conditions are met.
[0007] Preferably, the closing condition specifically includes that, in the closing operation of the distribution network, the voltage phase difference on both sides of the closing point is controlled within a small range to ensure the stability and safety of the system; the voltage phase difference on both sides of the closing point is controlled within a small range of 5°, that is, ; Where, is the voltage phase at the closing point A side, is the voltage phase at the closing point B side; Define coefficients , 220kV and below shall not exceed 20% of the rated voltage, Take 0.2; meet the following conditions: ; Where, is the voltage amplitude at the closing point A side, is the voltage amplitude at the closing point B side; Constraints on the closing loop current: ; ; Where, is the closed-loop steady-state current, It is the setting value of the second stage of overcurrent protection. is the loop-closing impulse current, It is a set value for overcurrent protection.
[0008] Preferably, the active distribution network system model and problem modeling in the uncertain environment after the loop is closed specifically include, taking into account the output fluctuation and load change factors of the distributed power source, building a dynamic model that reflects the uncertainty of the system; based on the dynamic model, modeling the long feeder voltage control problem after the distribution network loop is closed; Clarify the objective function and constraints of the problem. The objective function includes minimizing voltage deviation and voltage fluctuation. The constraints include the equipment operating range and voltage upper and lower limits. Select the average node voltage deviation as the objective function: ; Where, is the voltage of each node under uncertain environment, is the rated voltage of each node; For the system, power balance should be achieved, and the constraints are: ; in, are the active power of generator, wind turbine, photovoltaic, and load respectively. They are the reactive power of generator, wind turbine, photovoltaic, and load respectively; and Node and The voltage amplitude; For nodes and The phase angle difference between and are the real and imaginary parts of the node admittance matrix elements; At the same time, considering the power over-limit problem after the ring-closing power transfer, it is necessary to ensure that the load of any feeder does not exceed its maximum transmission power, that is: ; Where, is the feeder load, Maximum transmission power of the feeder.
[0009] Preferably, the AMAPPO-based voltage regulation method training and simulation result analysis uses the active distribution network system model and problem modeling results in the uncertain environment after loop closing as the training environment, and uses a deep reinforcement learning algorithm to enable the intelligent agent to learn the AMAPPO-based voltage regulation method.
[0010] Preferably, the voltage regulation method based on AMAPPO includes an actor network, a critic network and an attention network; The actor network is an intelligent agent The actor network in interacts with the environment based on local observations obtained from the environment , the actor network takes action ; For each PV inverter agent , its action can be expressed as , which represents the reactive power injected into the photovoltaic inverter to regulate the voltage; the correction equation for the photovoltaic injected reactive power is: ; The equation states that when the node voltage Less than the lower limit of allowable voltage is a positive value and less than , when the node voltage Greater than the upper limit of the allowable voltage Should be a negative value and greater than ,in, ; The critic network and attention network: for the agent The purpose of the critic network in the given state is to , calculate the value function; each agent joins the attention network, takes the state encoding of all agents as input, and generates contributions from other agents; the other agents, including wind turbines, the attention network will generate the respective contributions of photovoltaics and wind turbines.
[0011] Preferably, the algorithm flow of the AMAPPO-based voltage regulation method training is: Initialize the environment, each agent Get the initial observation value; Each agent Taking action simultaneously ; Each agent Correction is performed according to the defined correction equation; Each agent Execute the corrected action ; Each agent Get new observations ; Determine whether the convergence criterion is met. If converged, end the iteration; if not, continue the iteration.
[0012] Preferably, the simulation result analysis is to use iterative strategy extraction to verify the VIPER method, and extract an explainability strategy from the trained AMAPPO-based voltage regulation method; The extraction and verification VIPER method comprises the following steps: Initialize the trajectory experience pool and decision tree strategy; Calculate state distribution; Store the trajectory into the trajectory experience pool; The trajectories in the trajectory experience pool are weighted and resampled and then stored in the trajectory buffer; The decision tree is trained using the data in the trajectory buffer.
[0013] On the other hand, the present invention also provides a long feeder voltage control system after distribution network loop closing based on deep reinforcement learning, the system comprising: Analysis and judgment module, used for analyzing the operating characteristics of the distribution network before closing the loop and judging the closing conditions; Model module, used for active distribution network system model and problem modeling in uncertain environment after loop closing; Training and analysis module, used for training the voltage regulation method based on AMAPPO and analyzing simulation results.
[0014] The technical effects achieved by the present invention are: 1. Improve power quality: The present invention uses a voltage regulation strategy based on deep reinforcement learning to effectively cope with the complex situation of the active distribution network in an uncertain environment after the loop is closed, achieve precise control of the long feeder voltage, and significantly improve the power quality of the long feeder after the weak distribution network is closed.
[0015] 2. Enhanced control adaptability: Compared with traditional voltage control methods, the deep reinforcement learning method adopted in this invention has stronger adaptability and improves the flexibility and reliability of control. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 Flowchart of the long feeder voltage control method after distribution network loop closure based on deep reinforcement learning; Figure 2 Block diagram of voltage regulation algorithm based on AMAPPO; Figure 3 A process for extracting explainable voltage control strategies for active distribution networks. DETAILED DESCRIPTION
[0017] In order to more clearly understand the technical means of the present invention, the following Figure 1-3 The present invention is further described in detail with reference to specific embodiments.
[0018] like Figure 1 As shown, a method for voltage control of long feeders after distribution network loop closing based on deep reinforcement learning includes the following steps: Step 1: Analysis of distribution network operating characteristics and determination of closing conditions before closing the loop Based on the operational characteristics of the distribution network before loop closure, an equivalent analysis model of each distribution network topology is established. This model can accurately reflect the electrical connection relationship and parameter characteristics of the distribution network before loop closure.
[0019] Before conducting distribution network loop analysis, data input and cleaning are required.
[0020] Traditional methods for determining closing conditions in distribution networks face numerous challenges. First, the calculation process is complex and time-consuming, relying heavily on technician experience and simplified models. This results in poor judgment accuracy and makes it difficult to fully consider complex operating conditions. Second, insufficient real-time monitoring and processing capabilities make it difficult to cope with rapid changes in operating conditions and accurately handle the coupled relationships between multiple factors, such as voltage and phase. In contrast, deep learning-based methods offer significant advantages. They offer not only rapid and accurate calculations but also automated learning, real-time monitoring, and simultaneous processing of multiple variables, effectively preventing misjudgments.
[0021] Using grid operation data from a 10kV distribution station in a first-tier city as the data source, a data collection system was used to obtain key electrical parameters such as voltage amplitude, phase angle, and current at the closing point and related lines. Before entering the simulation software, this data was preprocessed, including removing outliers, filling in missing data, and verifying data consistency to ensure accuracy and reliability. Based on the closing criteria, the calculated parameters such as voltage amplitude, phase angle, phase sequence, and current at the closing point were evaluated to determine whether the closing conditions were met. If so, the system proceeded to the next step; if not, the distribution network was adjusted accordingly until the closing conditions were met.
[0022] During distribution network closing operations, the voltage phase difference on both sides of the closing point should generally be kept within a small range to ensure system stability and safety. Specifically, the phase difference should be as close to 0° as possible, but a certain range of phase difference is permitted in actual operation. In this method, the phase difference on both sides of the closing point should be controlled within 5° to ensure good application results.
[0023] ; Where, is the voltage phase at the closing point A side, It is the voltage phase at the closing point B side.
[0024] The voltage amplitude difference on both sides of the loop closing point should be controlled within a certain range. , generally 220kV and below does not exceed 20% of the rated voltage, Take 0.2. If the voltage difference is too large, a large closing current will be generated when closing the loop, which may cause protection to operate. Taking a 10KV medium-voltage distribution network as an example, the following conditions should be met: ; Where, is the voltage amplitude at the closing point A side, is the voltage amplitude on the B side of the closing point.
[0025] A large impact current will be generated at the moment of loop closing operation. If the loop closing current is too large, it may have an impact on the stability of the system, causing system voltage fluctuations, frequency changes, etc., and corresponding constraints must be placed on the loop closing current.
[0026] ; ; Where, is the closed-loop steady-state current, It is the setting value of the second stage of overcurrent protection. is the loop-closing impulse current, It is a set value for overcurrent protection.
[0027] Step 2: Active distribution network system model and problem modeling in uncertain environment after loop closure Model the active distribution network system under uncertain post-loop conditions. Consider factors such as output fluctuations of distributed generation (DGs) and load variations, and construct a dynamic model that reflects system uncertainty. Based on this dynamic model, model the voltage regulation problem for long feeders after the distribution network loop is closed. Define the objective function, such as minimizing voltage deviation and voltage fluctuation, as well as the constraints, such as the equipment operating range and upper and lower voltage limits.
[0028] This method uses the average value of node voltage deviation as the objective function: ; Where, is the voltage of each node under uncertain environment, is the rated voltage of each node.
[0029] For the system, power balance should be achieved, and the constraints are: ; in, are the active power of generator, wind turbine, photovoltaic, and load respectively. They are the reactive power of generator, wind turbine, photovoltaic, and load respectively. and Node and The voltage amplitude; For nodes and The phase angle difference between and are the real and imaginary parts of the node admittance matrix elements.
[0030] At the same time, considering the power over-limit problem after the ring-closing power transfer, it is necessary to ensure that the load of any feeder does not exceed its maximum transmission power: ; Where, is the feeder load, Maximum transmission power of the feeder.
[0031] Step 3: AMAPPO-based voltage regulation strategy training This paper introduces the Attentional Multi-Agent Proximal Optimization (AMAPPO) method. Using the active distribution network system model and problem modeling results in a closed-loop uncertain environment as a training environment, the agent learns an AMAPPO-based voltage regulation strategy through a deep reinforcement learning algorithm.
[0032] like Figure 2 As shown, the voltage regulation algorithm based on AMAPPO includes an actor network, a critic network, and an attention network.
[0033] Actor Networks: Agents The actor network in interacts with the environment based on local observations obtained from the environment , the actor network takes action Taking photovoltaic as an example, for each photovoltaic inverter agent , its action can be expressed as , which represents the reactive power injected into the PV inverter to regulate the voltage. The correction equation for the PV injected reactive power is: ; This equation states that when the node voltage Less than the lower limit of allowable voltage Should be a positive value and less than , when the node voltage Greater than the upper limit of the allowable voltage Should be a negative value and greater than .in, .
[0034] Critic and Attention Networks: Agents The purpose of the critic network in the given state is to , calculate the value function. The reward of also depends on the actions of other agents, so when calculating the agent When calculating the value function for a given agent, it is necessary to consider the contributions of other agents. To achieve this, each agent is joined to an attention network, which takes the state encodings of all agents as input and generates contributions from other agents. For example, in this method, other agents such as wind turbines can also be added, and the attention network will generate the contributions of both photovoltaic and wind turbines.
[0035] The algorithm flow is: (1) Initialize the environment, each agent Get the initial observation value.
[0036] (2) Each agent Taking action simultaneously .
[0037] (3) Each agent Correction is performed according to the defined correction equation.
[0038] (4) Each agent Execute the corrected action .
[0039] (5) Each agent Get new observations .
[0040] (6) Determine whether the convergence criterion is met. If convergence is achieved, the iteration ends; if not, continue the iteration.
[0041] The Validation and Iterative Policy Extraction (VIPER) method is used to extract an interpretable policy from the trained AMAPPO-based voltage regulation policy, such as Figure 3 As shown, The extraction steps are as follows: (1) Initialize the trajectory experience pool and decision tree strategy.
[0042] (2) Calculate the state distribution.
[0043] (3) Store the trajectory into the trajectory experience pool.
[0044] (4) The trajectories in the trajectory experience pool are weighted and resampled and then stored in the trajectory buffer.
[0045] (5) Use the data in the trajectory buffer to train the decision tree.
[0046] This explainable strategy clearly demonstrates the decision-making process and basis for voltage regulation, making it easier for engineers to understand and apply. The extracted explainable strategy was analyzed through simulation results. Simulations under different operating conditions verified its effectiveness in improving power quality on long feeders after closing a weak distribution network loop, demonstrating improvements in voltage compliance and reduced voltage fluctuations.
[0047] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.
Claims
1. A method for voltage control of long feeders after distribution network loop closing based on deep reinforcement learning, characterized in that: The method comprises: Analysis of distribution network operation characteristics before closing the loop and judgment of closing conditions; Active distribution network system model and problem modeling in uncertain environment after loop closing; Training and simulation results analysis of voltage regulation method based on AMAPPO.
2. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 1, characterized in that: Before the analysis of the distribution network operating characteristics before closing the loop and the judgment of the closing conditions, the data input and cleaning work is also included. Specifically, the acquisition system obtains the voltage amplitude, phase angle, and current electrical parameters of the closing point and related lines, and pre-processes the electrical parameters, including removing outliers, filling in missing data, and verifying data consistency to ensure the accuracy and reliability of the data.
3. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 2, characterized in that: The analysis of distribution network operation characteristics and judgment of closing conditions before closing the loop specifically includes judging the voltage amplitude, phase angle, phase sequence and current parameters at the closing point calculated according to the closing criterion to determine whether the closing conditions are met; If the conditions are met, proceed to the next step; if not, make corresponding adjustments to the distribution network until the loop closing conditions are met.
4. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 3 is characterized in that: The closing condition specifically includes that, during the closing operation of the distribution network, the voltage phase difference on both sides of the closing point is controlled within a small range to ensure the stability and safety of the system; the voltage phase difference on both sides of the closing point is controlled within a small range of 5°, that is, ; Where, is the voltage phase at the closing point A side, is the voltage phase at the closing point B side; Define coefficients , 220kV and below shall not exceed 20% of the rated voltage, Take 0.2; The following conditions are met: ; Where, is the voltage amplitude at the closing point A side, is the voltage amplitude at the closing point B side; Constraints on the closing loop current: ; ; Where, is the closed-loop steady-state current, It is the setting value of the second stage of overcurrent protection. is the loop-closing impulse current, It is a set value for overcurrent protection.
5. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 1, characterized in that: The active distribution network system model and problem modeling in the post-loop uncertain environment specifically include building a dynamic model that reflects system uncertainty by taking into account the output fluctuations and load changes of distributed power sources; Based on the dynamic model, the voltage regulation problem of long feeders after the distribution network is closed is modeled; Clarify the objective function and constraints of the problem. The objective function includes minimizing voltage deviation and voltage fluctuation. The constraints include the equipment operating range and voltage upper and lower limits. Select the average node voltage deviation as the objective function: ; Where, is the voltage of each node under uncertain environment, is the rated voltage of each node; For the system, power balance should be achieved, and the constraints are: ; in, are the active power of generator, wind turbine, photovoltaic, and load respectively. They are the reactive power of generator, wind turbine, photovoltaic, and load respectively; and Node and The voltage amplitude; For nodes and The phase angle difference between and are the real and imaginary parts of the node admittance matrix elements; At the same time, considering the power over-limit problem after the ring-closing power transfer, it is necessary to ensure that the load of any feeder does not exceed its maximum transmission power, that is: ; Where, is the feeder load, Maximum transmission power of the feeder.
6. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 1, characterized in that: The training and simulation results analysis of the AMAPPO-based voltage regulation method uses the active distribution network system model and problem modeling results in the uncertain environment after loop closing as the training environment, and uses the deep reinforcement learning algorithm to enable the intelligent agent to learn the AMAPPO-based voltage regulation method.
7. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 6, characterized in that: The voltage regulation method based on AMAPPO includes an actor network, a critic network and an attention network; The actor network is an intelligent agent The actor network in interacts with the environment based on local observations obtained from the environment , the actor network takes action ; For each PV inverter agent , its action can be expressed as , which represents the reactive power injected into the photovoltaic inverter to regulate the voltage; the correction equation for the photovoltaic injected reactive power is: ； The equation states that when the node voltage Less than the lower limit of allowable voltage is a positive value and less than , when the node voltage Greater than the upper limit of the allowable voltage Should be a negative value and greater than ,in, ; The critic network and attention network: for the agent The purpose of the critic network in the given state is to , calculate the value function; each agent joins the attention network, takes the state encoding of all agents as input, and generates contributions from other agents; the other agents, including wind turbines, the attention network will generate the respective contributions of photovoltaics and wind turbines.
8. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning according to claim 7, characterized in that: The algorithm flow of the AMAPPO-based voltage regulation method training is as follows: Initialize the environment, each agent Get the initial observation value; Each agent Taking action simultaneously ; Each agent Correction is performed according to the defined correction equation; Each agent Execute the corrected action ; Each agent Get new observations ; Determine whether the convergence criterion is met. If converged, end the iteration; if not, continue the iteration.
9. The method for controlling voltage of long feeders after closing the distribution network loop based on deep reinforcement learning as claimed in claim 8, characterized in that: The simulation results analysis is to use iterative policy extraction to verify the VIPER method and extract the explainability policy from the trained AMAPPO-based voltage regulation method; The extraction and verification VIPER method comprises the following steps: Initialize the trajectory experience pool and decision tree strategy; Calculate state distribution; Store the trajectory into the trajectory experience pool; The trajectories in the trajectory experience pool are weighted and resampled and then stored in the trajectory buffer; The decision tree is trained using the data in the trajectory buffer.
10. A long feeder voltage control system after distribution network loop closing based on deep reinforcement learning, characterized in that: The system comprises: Analysis and judgment module, used for analyzing the operating characteristics of the distribution network before closing the loop and judging the closing conditions; Model module, used for active distribution network system model and problem modeling in uncertain environment after loop closing; Training and analysis module, used for training the voltage regulation method based on AMAPPO and analyzing simulation results.