A load restoration method for distribution networks based on multi-agent coordination mechanism in virtual power plants

Abstract

This invention relates to the field of distribution network dispatching and virtual power plant collaborative control technology, specifically to a distribution network load restoration method based on a multi-agent coordination mechanism under a virtual power plant. This method first models the adjustable resources of the virtual power plant to quantify its power regulation potential, then abstracts the load restoration process into a Markov decision process. Subsequently, it configures three types of agents: switch control agents, virtual power plant collaborative agents, and global coordination agents. Finally, it achieves multi-agent training and execution. This invention can fully release the power support potential of the virtual power plant, solve problems such as slow convergence of single agents, susceptibility to local optima, and spatial adaptation defects in mixed actions of multiple agents, ensure the safe operation of the power grid, and improve the flexibility, efficiency, and safety of load restoration after distribution network faults.

Description

Technical Field

[0001] This invention relates to the field of distribution network dispatching and virtual power plant collaborative control technology, specifically to a distribution network load restoration method based on a multi-agent coordination mechanism under a virtual power plant. Background Technology

[0002] As the core link connecting the main grid and end users in the power system, the operational stability of the distribution network has a decisive impact on the guarantee of electricity supply for people's livelihood and the continuity of social production. However, the distribution network has a complex topology and densely distributed nodes, making it susceptible to faults caused by factors such as equipment aging and failure, line short-circuit faults, and extreme weather disasters, leading to frequent local or even regional power outages. Therefore, rapid load restoration after a fault is crucial. Traditional load restoration methods achieve network topology reconstruction by controlling the closing of sectionalizing switches and tie switches, thereby completing the load restoration process. With the gradual maturation of Virtual Power Plant (VPP) technology, distributed resources such as distributed photovoltaics, energy storage devices, and controllable loads are aggregated and integrated through VPPs and incorporated into the unified dispatch system of the distribution network, providing flexible power regulation guarantees for load restoration after a fault. However, existing technical solutions still face three major technical bottlenecks in practical applications:

[0003] 1. Traditional load restoration solutions struggle to fully utilize the adjustable resources of a VPP;

[0004] Existing schemes for VPPs to participate in distribution network load restoration do not systematically incorporate various types of adjustable resources of VPPs, such as photovoltaic curtailment, energy storage charging and discharging, and controllable load reduction, into the load restoration decision-making chain. This results in the inability to deliver surplus photovoltaic power to power-deficient areas, and the disconnect between the charging and discharging rhythm of energy storage and changes in load gaps. As a result, the power regulation potential of VPPs is largely idle, making it difficult for VPPs to play their core value in load restoration.

[0005] 2. Single-agent reinforcement learning struggles to adapt to high-dimensional state and action spaces;

[0006] As distribution networks evolve towards medium and large scale, their state and action spaces expand exponentially: the state dimension needs to cover the opening and closing states of numerous switches, as well as multi-source parameters such as photovoltaic output, energy storage SOC, and controllable load boundaries on the VPP side; the action dimension includes discrete operations of switching, closing, and opening, and continuous adjustment of VPP power. This excessively large state and action space leads to slow training convergence and a tendency to get trapped in local optima in existing single-agent deep reinforcement learning-based schemes.

[0007] 3. Multi-agent reinforcement learning suffers from a defect in adapting to mixed action spaces;

[0008] To overcome the limitations of single-agent scaling, the industry has attempted to introduce a multi-agent framework. Load restoration involving virtual power plants involves two types of actions: discrete actions such as switching on / off states, and continuous actions such as VPP energy storage charging / discharging power and photovoltaic curtailment adjustment. Current multi-agent solutions often use a single algorithm to handle all actions: if an algorithm adapted for discrete actions is chosen, the continuous VPP actions must be forcibly discretized, leading to a loss of adjustment accuracy; if an algorithm adapted for continuous actions is used, it is difficult to efficiently handle discrete switching operations. This adaptation deficiency results in low multi-agent collaboration efficiency, preventing deep coordination between switching and VPP actions. Summary of the Invention

[0009] This invention addresses the technical problems existing in the prior art by providing a distribution network load restoration method based on a multi-agent coordination mechanism under a virtual power plant.

[0010] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a method for distribution network load restoration based on multi-agent coordination mechanism under virtual power plant, the method comprising the following steps;

[0011] S1. Modeling the adjustable resources of the virtual power plant: Collect three types of core parameters: distributed photovoltaic, energy storage system and controllable load, and construct the total adjustable power model of the virtual power plant to quantify its power regulation potential.

[0012] S2. Construct a Markov decision process: Abstract the load recovery process into a Markov decision quintuple, transforming the complex load recovery problem into an ordered decision process;

[0013] S3, Configure three types of intelligent agents, including:

[0014] S301, Switch Control Agent: Employs the QMIX algorithm, which outputs local and global Q values ​​of switch actions through a local Q network and a centralized hybrid network, thereby unifying local decision-making and global topology optimization;

[0015] S302, Virtual Power Plant Collaborative Intelligent Agent: Adopting the MADDPG algorithm, relying on the Actor-Critic dual network to output the virtual power plant's continuous adjustment actions that meet the constraints and evaluate the comprehensive value of the actions;

[0016] S303, Global Coordination Agent: Eliminates the difference in value scale between two types of agents through value fusion, adapts action weights according to the recovery phase, verifies power grid constraints to prevent illegal actions, and solves the coordination problem;

[0017] S4. Implement multi-agent training and execution: Initialize parameters and experience pool, randomly generate fault scenario collection data, and synchronously update the parameters of each agent after the sample size reaches the target; when a fault occurs, load the pre-trained model, collect the status in real time, and iteratively execute action decisions and constraint verification until the load is restored or the maximum number of actions is reached.

[0018] The beneficial effects of this invention are:

[0019] This invention systematically integrates photovoltaic, energy storage, and controllable load resources within a virtual power plant to construct a total adjustable power model. It is then combined with the MADDPG algorithm to adapt to continuous adjustment actions, avoiding resource idleness and adjustment accuracy loss, fully releasing the power support potential of the VPP, and improving load recovery flexibility.

[0020] This invention employs a hybrid architecture of switch-controlled intelligent agent (QMIX algorithm) and virtual power plant collaborative intelligent agent (MADDPG algorithm), decomposing the high-dimensional state and action space to solve the problems of slow convergence and easy trapping in local optima by single intelligent agents. At the same time, through a two-layer network and comprehensive value evaluation, it achieves the unification of local decision-making and global optimization.

[0021] This invention's global coordination agent eliminates scale differences through value standardization, dynamically adjusts weights based on the recovery phase, balances the priorities of topology reconstruction and resource optimization, and conducts multi-dimensional joint action verification to prevent risks such as voltage overruns and topology loops. It also dynamically corrects strategies to ensure operational safety and improves the response speed and recovery effect of distribution network line faults. Attached Figure Description

[0022] Figure 1 The architecture diagram of the distribution network load restoration method based on multi-agent coordination mechanism under the virtual power plant provided by the present invention is shown below.

[0023] Figure 2 The flowchart illustrates the distribution network load restoration method based on a multi-agent coordination mechanism in a virtual power plant, as provided by this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] See Figures 1 to 2 A method for distribution network load restoration based on multi-agent coordination mechanism under virtual power plant, the method includes the following steps;

[0026] S1. Modeling the adjustable resources of the virtual power plant: Collect three types of core parameters: distributed photovoltaic, energy storage system and controllable load, and construct the total adjustable power model of the virtual power plant to quantify its power regulation potential.

[0027] The specific parameters for collecting the three types of core resources within the virtual power plant are as follows:

[0028] Distributed photovoltaics: Real-time irradiance, ambient temperature, and output prediction deviation are collected. These parameters directly affect the actual output of photovoltaics and are the key basis for quantifying the adjustable potential of photovoltaics.

[0029] Energy storage system: Collects the current state of charge (SOC) and charge / discharge power boundary. SOC determines the total amount of electrical energy that can be released or stored at present, while the charge / discharge power boundary limits the rate of charge and discharge of energy storage. Together, they determine the power regulation capability of energy storage.

[0030] Controllable load: Collect the adjustment range of commercial central air conditioning and charging piles, and clarify the power range that can be reduced or transferred during the load recovery process;

[0031] Based on the collected data, a total adjustable power model of the VPP is constructed to quantify the power regulation potential of the VPP. The calculation formula for the total adjustable power model of the VPP is as follows:

[0032] ;

[0033] in VPP total adjustable power, in kW; Adjustable power for photovoltaic curtailment; This refers to the energy storage charging and discharging power (discharging is positive, charging is negative). Controllable load power;

[0034] This model provides a clear understanding of the adjustable power that the VPP can provide at different times, offering data support for subsequent load restoration decisions.

[0035] S2. Constructing a Markov Decision Process: The load recovery process is abstracted into a Markov decision quintuple, wherein the Markov decision quintuple is... The specific definition is as follows:

[0036] state space ;

[0037] Where U is the set of three-phase voltages at the node, satisfying , Let be the voltage at node i after the k-th operation. , They are nodes The minimum and maximum allowable voltage values ​​are important indicators for measuring the operating status of the power grid. Exceeding voltage limits will affect the safe and stable operation of the power grid. N is the set of sectionalizing switch states (0 = open, 1 = closed). Sectionalizing switches are used to divide the distribution network into different areas, and their states directly determine the topology of the distribution network and the load supply range. L is the set of tie line switch states (0 = open, 1 = closed). Tie line switches are used to connect different distribution network areas. After a fault occurs, the load can be transferred by closing the tie line switch. This is a set of VPP resource status, including photovoltaic output, energy storage SOC, and controllable load adjustment range, reflecting the current resource status and adjustment capability of the VPP.

[0038] Action Space A ;

[0039] in , For discrete operation of segmented switches, Discrete operation of tie line switches enables distribution network topology reconfiguration by operating the switches, thereby changing the power supply path of the loads;

[0040] The continuous adjustment amount for photovoltaic curtailment can be adjusted according to load demand to increase or decrease the amount of photovoltaic power supplied to the distribution network. To provide continuous power for energy storage charging and discharging, peak shaving and valley filling and load balancing are achieved by controlling the charging and discharging of energy storage. To ensure power supply to critical loads by continuously reducing the power demand of controllable loads when there is a large load gap;

[0041] State transition probability P: Determined by power flow calculation of distribution network, involving node power balance, line impedance characteristics and VPP resource response characteristics (such as energy storage charging and discharging delay, photovoltaic output fluctuation). Given the current state and actions, the power flow calculation can determine the state of the grid at the next moment, thereby realizing the transition between states.

[0042] Reward settings ;

[0043] in , As a reward for the switch agent, This is the load recovery weight, used to incentivize the contribution of switching actions to load recovery; The higher the load recovery rate, the greater the reward. This is the penalty weight for invalid switches, used to penalize invalid switch operations. The penalty is for the number of invalid switches, such as operations that cause a topology loop or operations on faulty lines. The more invalid operations, the greater the penalty.

[0044] , Rewards for VPP collaborative agents. Weighting for network loss penalties; The network loss ratio is used to incentivize VPP actions to reduce network loss. The greater the network loss, the greater the penalty. As a penalty weight for voltage exceeding the limit, The degree of voltage exceeding the limit is determined; the more severe the voltage exceedance, the greater the penalty, in order to ensure the stability of the power grid voltage.

[0045] Discount factor Used to balance short-term load recovery effectiveness with long-term power grid operational stability. The value ranges from 0 to 1. The closer it is to 1, the more the system prioritizes long-term stable operation of the power grid; The closer it is to 0, the more the system focuses on the short-term load recovery effect;

[0046] By constructing an MDP, the complex load recovery problem is transformed into an ordered decision-making process, providing a clear framework for the training and decision-making of agents.

[0047] S3, Configure three types of intelligent agents, including:

[0048] S301. Switch Control Agent: To address the discreteness of switch operations in distribution networks, the QMIX algorithm is used to configure the switch control agent. A two-layer structure of local Q network and centralized hybrid network is designed to achieve consistency between local action value and global topology optimization objective.

[0049] The inputs to the local Q-network include the switching state of the fault-associated region, with values ​​{0, 1} representing open and closed; and the node voltage amplitude vector. Unit: pu; Line current RMS vector The unit is kA; these input information can reflect the local power grid operation status and switch status in the fault area, providing a basis for local decision-making on switch actions;

[0050] The output is a vector of local Q values ​​representing the discrete switch action. s represents the state. To perform an action, each element represents the expected cumulative reward for performing the corresponding switch action in the current state. The local Q value can be used to preliminarily determine the merits of different switch actions in the local region.

[0051] The network structure adopts a multi-layer fully connected neural network, with ReLU activation function in the hidden layers and linear activation function in the output layer;

[0052] The loss function formula is: ;

[0053] Wherein: represents the loss function of the local Q network; is the mathematical expectation operator, representing the expectation of calculating the loss for the sampled batch data in the experience pool; is the target Q value calculated based on the immediate reward and the output of the target network. By minimizing this loss function, the global Q value output by the centralized hybrid network can be made consistent with the optimization goal determined by the global coordination agent, achieving the unity of local decision-making and global optimization; the specific calculation formula is:

[0054] ;

[0055] Where is the discount factor, is the executed action and the next state after that, is the action in the next state. By minimizing this loss function, the local Q value output by the local Q network can continuously approach the target Q value, improving the accuracy of local decision-making;

[0056] The input of the centralized hybrid network includes the local Q value vector output by the local Q network ; the whole network switch state vector . The whole network switch state vector can reflect the topological structure of the entire distribution network. Combining with the local Q value vector can evaluate the global impact of switch actions;

[0057] The output is the global Q value of the switch action , representing the value of this switch action to the global load restoration under the consideration of the whole network topology and power distribution. The global Q value can avoid the limitations of local decision-making and achieve global topology optimization;

[0058] The network structure adopts a multi-layer fully connected neural network. The hidden layer uses the ReLU activation function, and the output layer uses a linear activation, which is similar to the local Q network structure, ensuring that the network has good fitting ability and computational efficiency;

[0059] The loss function ;

[0060] Where is calculated by the global coordination agent; [[ID=5​​​​S302.VPP Cooperative Agent: To address the continuity and multiple constraints of VPP resource regulation, the MADDPG algorithm is used to configure the VPP cooperative agent, and an Actor-Critic dual network structure is designed to achieve optimal power regulation in the continuous action space.

[0062] The Actor network input includes the VPP resource state vector. The switching status of the fault-related area, with values ​​{0, 1} representing open and closed, and the node voltage deviation vector. The unit is pu;

[0063] Including photovoltaic power output Unit kW, energy storage Units: % and current power of controllable load. The unit is kW; the above parameters reflect the resource status and regulation potential of the VPP; the switching status of the fault-related area affects the power transmission path of the VPP; the node voltage deviation vector reflects the degree of deviation of the grid voltage and provides a reference for VPP regulation actions;

[0064] The output is a continuous VPP adjustment action vector. ,satisfy:

[0065] The amount of photovoltaic curtailment adjustment cannot be negative and cannot exceed the maximum output of photovoltaic power, ensuring that photovoltaic regulation is within a reasonable range.

[0066] The charging and discharging power of energy storage devices must be between their minimum and maximum charging and discharging power to avoid damage caused by overcharging and discharging.

[0067] The controllable load reduction amount cannot be positive (i.e., the load cannot be increased), and the reduction amount cannot exceed the minimum allowable reduction amount, so as to ensure the user's basic electricity needs.

[0068] Represents the adjustment amount of photovoltaic power output. Represents the current output of distributed photovoltaic power. The charging and discharging power representing energy storage. , Represents the maximum and minimum charge / discharge power of energy storage. This represents the amount of controllable load reduction. This is the minimum allowable reduction in controllable load.

[0069] The network structure adopts a multi-layer fully connected neural network. The hidden layer uses the ReLU activation function, and the output layer uses the Tanh activation function. The output value is scaled to the legal action range through linear mapping. The Tanh activation function can map the output value to the interval [-1,1]. Then, through linear mapping, it can be adjusted to meet the VPP adjustment action constraint range to ensure the legality of the output action.

[0070] The policy gradient loss function is: ;

[0071] in, As evaluated by the Critic network, by maximizing this loss function, the Actor network can learn the optimal policy and output the VPP adjustment action that maximizes the action value. This represents the policy gradient loss function of the Actor network; Let be the mathematical expectation operator, representing the expectation of state s and action s. Calculate the expected value based on the current strategy and dynamic distribution of the environment;

[0072] The input to the Critic network includes the VPP action vector. VPP coverage area load restoration Unit kW, network loss rate Unit %, voltage amplitude vector The VPP action vector, measured in units of pu, is the object of Critic network evaluation; load restoration reflects the contribution of VPP actions to load restoration; and the network loss rate and voltage amplitude vector reflect the impact of VPP actions on grid operating efficiency and stability. Combining these inputs allows for a comprehensive evaluation of the value of VPP actions.

[0073] The output is the value of the VPP action. This is used to evaluate the comprehensive contribution of the action to load recovery, network loss control and voltage stability, and to provide a basis for strategy optimization of the Actor network;

[0074] The network structure adopts a multi-layer fully connected neural network, with ReLU activation function in the hidden layer and linear activation function in the output layer, to ensure that the network can accurately fit the complex relationship between VPP action value and input parameters.

[0075] The loss function is: ;

[0076] in: The loss function of the Critic network is represented by this. For mathematical expectation operators; To enable the Critic network to perform VPP actions in the current state s The predictive value; From instant rewards The loss function is calculated from the target network output. By minimizing this loss function, the Critic network can accurately evaluate the value of VPP actions, providing reliable value feedback to the Actor network. The specific calculation method is as follows:

[0077] ;

[0078] in As a discount factor, To perform the action The next state after that, For the next state The following actions are to be performed;

[0079] S303. Global Coordination Agent: A global coordination agent is set up to address the issues of differing output value metrics, asynchronous actions, and potential violations of power grid security constraints by the switch control agent and the VPP cooperative agent. Specifically, it implements the following functions:

[0080] Value standardization processing: Receiving the global Q value of the switch-controlled agent. and the VPP local value output by the VPP collaborative agent. Since the numerical ranges of the two types of values ​​may differ significantly, direct fusion could lead to one type of value dominating the decision-making process and affecting its rationality. Therefore, value standardization is necessary, as shown in the following formula:

[0081] ;

[0082] ;

[0083] in This represents the standardized global Q-value of the switch; This represents the raw global Q value output by the switch-controlled agent; , These represent the minimum and maximum values ​​of the global Q-value during the training process; This represents the local value of the standardized VPP; This represents the original local value output by the VPP collaborative agent; , These are the minimum and maximum values ​​of the local value of VPP during training, respectively. These extreme value parameters are dynamically updated according to a preset training period to adapt to changes in the value range during training. Through standardization, the two types of values ​​are uniformly mapped to the [-1,1] interval, eliminating value scale differences and laying the foundation for subsequent value fusion.

[0084] Dynamic weight adjustment: Based on the current stage of distribution network load recovery, dynamically adjust the weight ratio of switch actions and VPP actions, ensuring the weights meet the normalization condition. The global value fusion formula is:

[0085] ;

[0086] in This represents the overall value after fusion; Represents the dynamic weight of the switching action; Represents the dynamic weights of VPP actions; and These are the standardized switching Q value and VPP value, respectively;

[0087] When the load restoration ratio is small, the system is in a critical period of topology reconfiguration. At this time, the main problem for the distribution network is to restore power supply to the loads by adjusting switch states to construct a reasonable power supply topology. Therefore, setting... For larger values, To minimize this value, priority is given to ensuring the contribution of switching actions to restoring network topology connectivity. When the load restoration ratio is relatively large, the system enters the resource optimization-dominated period. At this time, the distribution network topology is basically reasonable, and the main task is to further improve the load restoration ratio, reduce network losses, and ensure voltage stability by optimizing the utilization efficiency of VPP resources. Therefore, the following settings are implemented: For larger values, To achieve a smaller value, prioritize optimizing the utilization efficiency of VPP resources to improve recovery performance;

[0088] Joint action legality verification: Verify the legality of the joint execution result of the switching action and the VPP action. The verification criteria include, but are not limited to:

[0089] Node voltage constraints: ;

[0090] in: The node after the k-th operation The voltage; , They are nodes The minimum and maximum allowable values ​​of voltage are important indicators for the safe operation of the power grid. Exceeding voltage limits may lead to problems such as damage to electrical equipment and instability of the power grid.

[0091] Line current constraints: ;

[0092] in: For the line The current; , These are the minimum and maximum allowable values ​​for the line current, respectively. Exceeding the maximum value may cause faults such as line overheating and tripping, affecting the reliability of power grid supply.

[0093] Topological constraints: The distribution network maintains radial connectivity. Distribution networks typically adopt a radial topology. Ring network operation may lead to problems such as relay protection malfunction and increased short-circuit current, while islanding may cause some areas to lose power.

[0094] If a joint action is detected to violate any of the above constraints, or if a switch is not closed, resulting in the VPP power not being effectively delivered to the target area, the global coordinating agent will temporarily reduce the dynamic weight of the corresponding agent to guide the system to select a safer and more legal combination of actions, thereby ensuring the safe and stable operation of the power grid.

[0095] S4. Implement multi-agent training and execution:

[0096] S401. Initialization Settings: For the switch-controlled agent, initialize the QMIX local Q network, centralized hybrid network, and corresponding target network. Set up an experience storage pool to store sample data during training. The size of the experience storage pool is determined according to training requirements. Set the learning rate, which affects the update speed of network parameters and needs to be adjusted according to the training convergence. Set the batch size, which determines the number of samples used for each parameter update. A reasonable batch size can improve training efficiency and stability.

[0097] For VPP collaborative agents, the Actor network, Critic network, and corresponding target network of MADDPG are initialized, and the experience storage pool is shared with the switch agent to realize the shared use of sample data and improve sample utilization. The learning rate is set, and different learning rates can be set for the Actor network and the Critic network to suit their respective training needs.

[0098] For a globally coordinating agent, initialize the value standardization parameters. , , , The initial values ​​can be determined based on experience or pre-training data; the initial values ​​of the dynamic weights can be set to... = =0.5, which is dynamically adjusted according to the recovery phase during training;

[0099] S402, Fault Scenario Generation and Experience Acquisition: Randomly simulate distribution network fault scenarios, such as disconnecting a line, to simulate the types of faults that may occur in the actual distribution network, providing diverse fault environments for agent training. Reset the distribution network operating state and VPP resource state so that each fault scenario simulation starts from the initial state, ensuring the fairness and consistency of training;

[0100] Switching intelligent agents are based on Greedy strategy Select discrete switch action , - Greedy strategies are set with larger parameters in the early stages of training. The value is increased to encourage the agent to explore more and try different switching actions; as training progresses, the value is gradually decreased. This value enables the agent to utilize the learned optimal strategy to select actions and obtain rewards after execution. The current state s and the action ,award Next state Store in the experience storage pool;

[0101] VPP collaborative agents output continuous actions based on the Actor network. Receive a reward after execution The current state s and the action ,award Next state The data is stored in the experience storage pool, and by continuously collecting experience samples, data support is provided for updating the agent's parameters.

[0102] S403, Parameter Update: When the number of valid samples in the experience storage pool exceeds the batch size, the network parameters of each agent are updated synchronously to ensure that the parameter updates of each agent are based on sufficient sample data and improve the reliability of parameter updates.

[0103] For the switch-controlling agent, switch experience samples are sampled from the experience storage pool, and then processed according to the formula. Calculate the target Q value; where This indicates the current state of the switch-controlled agent. Next action The instant rewards received; This is a discount factor used to balance short-term recovery effects with long-term grid stability; The next state after the action is performed; The action to be performed in the next state; The next state Q value output by the target network is determined; the QMIX network parameters are updated by minimizing the mean square error between the predicted Q value and the target Q value, and the target network is updated by soft update. Soft update allows the parameters of the target network to slowly follow the changes of the main network parameters, improving the stability of training.

[0104] For VPP collaborative agents, VPP experience samples are sampled from the experience storage pool; the Critic network is updated by minimizing the value prediction error; the Actor network optimizes the policy parameters through backpropagation of the gradient of the Critic network, enabling the Actor network to learn better action policies.

[0105] For the global coordination agent, after each round of parameter updates, the value standardization parameters and dynamic weights are fine-tuned based on the overall network load recovery rate and network loss data. By fine-tuning the parameters, the global coordination agent can better adapt to changes in the power grid state and improve the accuracy of global decision-making.

[0106] S404, Fault Response: After a fault occurs, the system triggers the load recovery process, loading the model parameters of the pre-trained switch control agent (local Q network, centralized hybrid network and its target network), VPP collaborative agent (Actor network, Critic network and its target network), and global coordination agent to ensure that the agent can make decisions based on the pre-trained policy.

[0107] The dynamic parameters of the initialization execution phase, such as the recovery phase identifier, are used to determine the current recovery phase of the distribution network; the current action count is used to record the number of actions that have been executed, so as to avoid excessive action counts leading to resource waste or grid instability.

[0108] Real-time acquisition of initial operating data of the distribution network after a fault occurs (node ​​voltage, switch status, load power, etc.) and initial state of VPP resources (photovoltaic output, energy storage SOC, controllable load boundary, etc.) constitutes the initial state, providing input data for the agent's first decision;

[0109] S405, Action Decision: Real-time collection of current distribution network operation data (node ​​voltage, switch status, load power, line power flow, etc.) and VPP resource real-time status (photovoltaic output, energy storage SOC, controllable load current power, etc.) to form the current status, ensuring that the agent's decision is based on the latest grid status;

[0110] The switch control agent will use relevant information from the current state s (such as the switch state of the fault-associated area and the node voltage magnitude vector) to control the switch. Line current effective value vector Input the local Q-network and calculate the action of each discrete switch. The corresponding local Q-value vector , local Q-value vector and the entire network switch state vector Input a centralized hybrid network to calculate each action. global Q value ;

[0111] The VPP collaborative agent will store relevant information in the current state s, including, for example, the VPP resource state vector. The Actor network takes inputs such as the fault-related area switch status as inputs and outputs a deterministic continuous VPP adjustment action vector. ;

[0112] The global coordinating agent first receives the global Q value from the switching control agent. VPP actions from the VPP collaborative agent The value standardization process is obtained by performing the standardization process. and ;

[0113] The current load recovery phase of the distribution network is determined by indicators such as the current load recovery ratio and network connectivity, and the switching action weights are dynamically adjusted accordingly. and VPP action weights ;

[0114] Then, the fused global value is calculated, and the switching action is... and VPP actions The joint execution results are preliminarily verified, and adjustments may be made temporarily based on the verification results. or Based on the value of integration Based on the pre-verification results, the final switch action to be executed is determined or confirmed. and VPP actions and the switch action command The VPP regulation action command is sent to the distribution network switch execution unit for execution. Send to the VPP resource control unit for execution;

[0115] S406. Constraint Verification and Iteration: After the action is executed, the overall network state is updated through distribution network power flow calculation to obtain new data such as node voltage, line current, and load recovery status of each node, which constitutes the next state. ;

[0116] The global coordinating agent for the next state A comprehensive power grid safety constraint verification is performed, including node voltage constraints, line current constraints, topology constraints, and VPP equipment constraints. If any safety constraint is found to be violated during the verification, it is considered an illegal action. The system will handle the situation according to a preset strategy, such as reverting to the state before the action was executed, triggering emergency measures (such as disconnecting some non-critical loads), adjusting the agent weights, and then returning to S405 for the next round of decision-making.

[0117] If all safety constraints are met, check whether the restored load ratio has reached the preset threshold and whether the preset maximum number of actions has been reached. If all constraints are met and the restoration target is achieved, stop the process and output the results, including the restored load ratio, network loss rate, voltage stability, etc. If all constraints are met but the restoration target has not been achieved and the maximum number of actions has not been exceeded, update the current status. The action count is incremented, and the process returns to S405 for iteration. If the maximum number of actions is reached but the recovery goal is not achieved, the process stops and the current result is output. The reasons for not achieving the recovery goal are analyzed to provide a reference for subsequent optimization.

[0118] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the above embodiments are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant, characterized in that, This method Includes the following steps; S1. Modeling the adjustable resources of the virtual power plant: Collect three types of core parameters: distributed photovoltaic, energy storage system and controllable load, and construct the total adjustable power model of the virtual power plant to quantify its power regulation potential. S2. Construct a Markov decision process: Abstract the load recovery process into a Markov decision quintuple, transforming the complex load recovery problem into an ordered decision process; S3, Configure three types of intelligent agents, including: S301, Switch Control Agent: Employs the QMIX algorithm, which outputs local and global Q values ​​of switch actions through a local Q network and a centralized hybrid network, thereby unifying local decision-making and global topology optimization; In S301, the inputs of the local Q network include the switching state of the fault-associated region, with values ​​{0, 1} representing open and closed; and the node voltage amplitude vector. Line current RMS vector The output is a local Q-value vector of discrete switch actions. s represents the state. To execute actions, the network structure employs a multi-layer fully connected neural network, with ReLU activation function in the hidden layers and linear activation function in the output layer. loss function ; in Represents the loss function of the local Q network; The mathematical expectation operator represents the expectation of the loss calculated on the sampled batch data in the experience pool; To be based on instant rewards The target Q value, calculated from the output of the target network, is specifically calculated using the following formula: ; in As a discount factor, To perform the action The next state after that, The action to be performed in the next state; S302, Virtual Power Plant Collaborative Intelligent Agent: Adopting the MADDPG algorithm, relying on the Actor-Critic dual network to output the virtual power plant's continuous adjustment actions that meet the constraints and evaluate the comprehensive value of the actions; S303, Global Coordination Agent: Eliminates the value scale difference between the switch control agent and the virtual power plant collaborative agent through value fusion, adapts the action weight according to the recovery phase, and verifies the power grid constraints to prevent illegal actions; S4. Implement multi-agent training and execution: Initialize parameters and experience pool, randomly generate fault scenario collection data, and synchronously update the parameters of each agent after the sample size reaches the target; when a fault occurs, load the pre-trained model, collect the status in real time, and iteratively execute action decisions and constraint verification until the load is restored or the maximum number of actions is reached.

2. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 1, characterized in that, In S1, the calculation formula for the total adjustable power model of the virtual power plant is: ； in This represents the total adjustable power of VPP. Adjustable power for photovoltaic curtailment; For energy storage charging and discharging power; This refers to the controllable load power.

3. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 1, characterized in that, In S2, the Markov decision quintuple is: The specific definition is as follows: state space Where U is the set of three-phase voltages at a node, satisfying , Let be the voltage at node i after the k-th operation. , They are nodes Minimum and maximum allowable voltage values; N is the set of sectionalizing switch states; L is the set of tie-line switch states; A set of VPP resource states; Action Space A ;in , For discrete operation of segmented switches, For discrete operation of the tie line switch; This is a continuous adjustment amount for photovoltaic curtailment. For continuous power charging and discharging of energy storage, This refers to the continuous reduction of controllable load. State transition probability P; Reward settings ;in , As a reward for the switching agent, For load recovery weight, To restore the load ratio, Penalty weight for invalid switches, This represents the number of invalid switches. , Rewards for VPP collaborative agents. As a weight for network loss penalties, This represents the network loss ratio. As a penalty weight for voltage exceeding the limit, The degree of voltage exceeding the limit; Discount factor .

4. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 1, characterized in that, In S301, the input to the centralized hybrid network includes the local Q-value vector output by the local Q-network. ; Network-wide switch state vector ; The output is the global Q value of the switching action. The network structure adopts a multi-layer fully connected neural network, with ReLU activation function in the hidden layers and linear activation function in the output layer. loss function ; in Calculated by a globally coordinating agent; The loss function representing a centralized hybrid network; is the mathematical expectation operator, representing the expectation of the loss calculated on the sampled batch data in the experience pool.

5. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 1, characterized in that, In S302, the Actor-Critic dual network includes an Actor network and a Critic network. The input to the Actor network includes a virtual power plant resource state vector. Switching status of fault-related regions, node voltage deviation vector ; Including photovoltaic power output Energy storage and the current power of controllable load ; The output is a continuous virtual power plant regulation action vector. ,satisfy: 、 、 ; in Represents the continuous adjustment amount of photovoltaic curtailment. Represents the current output of distributed photovoltaic power. Represents the continuous power of energy storage charging and discharging. , Represents the maximum and minimum charge / discharge power of energy storage. This represents the continuous reduction in controllable load. This is the minimum allowable reduction in controllable load. The network structure adopts a multi-layer fully connected neural network, with the hidden layer using the ReLU activation function and the output layer using the Tanh activation function. The output value is scaled to the range of legal actions through a linear mapping. Policy gradient loss function ; in This represents the policy gradient loss function of the Actor network; Let be the mathematical expectation operator, representing the vector of continuous virtual power plant regulation actions for state s. Calculate the expected value based on the current strategy and dynamic distribution of the environment. This was obtained through evaluation by the Critic network.

6. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 5, characterized in that, The input to the Critic network includes a continuous virtual power plant regulation action vector. Virtual power plant coverage area load restoration Network loss ratio Voltage amplitude vector The output is the value of the virtual power plant's actions. The network structure adopts a multi-layer fully connected neural network, with ReLU activation function in the hidden layers and linear activation function in the output layer. The loss function is: ; in The loss function of the Critic network is represented by this. For mathematical expectation operators; To enable the Critic network to perform VPP actions in the current state s The predictive value; From instant rewards The calculation is obtained from the output of the target network, and the specific calculation method is as follows: ; in, As a discount factor, To perform the action The next state after that, For the next state The actions to be performed.

7. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 1, characterized in that, In S303, the global value standardization formula for the dynamically adjusted weights of the global coordinating agent is as follows: ; ; in This represents the standardized global Q-value of the switch; This represents the raw global Q value output by the switch-controlled agent; , These represent the minimum and maximum values ​​of the global Q-value during the training process; This represents the local value of a standardized virtual power plant. This represents the original local value output by the collaborative intelligent agent of the virtual power plant. , These are the minimum and maximum local values ​​of the virtual power plant during the training process, respectively. , The parameters are dynamically updated according to the preset training period.

8. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 7, characterized in that, In S303, the global value formula for the dynamically adjusted weights of the global coordinating agent is: ； The weights satisfy the normalization condition ; in This represents the overall value after fusion; Represents the dynamic weight of the switching action; Represents the dynamic weights of VPP actions; and These are the standardized global Q-value of the switch and the local value of the virtual power plant, respectively.

9. The method for distribution network load restoration based on a multi-agent coordination mechanism under a virtual power plant according to claim 8, characterized in that, In S303, the basis for the global coordinating agent to verify power grid constraints includes at least the following: Node voltage constraints: ;in The node after the k-th operation The voltage; , They are nodes Minimum and maximum permissible values ​​for voltage; Line current constraints: ;in For the line The current; , These are the minimum and maximum allowable values ​​for the current in line l, respectively. And topological constraints.

Images