Power supply recovery decision-making method for power distribution network

Abstract

The invention discloses a power distribution network power supply recovery decision-making method, which comprises the following steps of: firstly, acquiring data required by a power distribution network power supply recovery simulation environment, and establishing the power distribution network power supply recovery simulation environment based on the acquired data; secondly, based on the constructed simulation environment, defining a Markov decision process of power supply recovery of the power distribution network, including a state space, an action space and a reward function; and based on the decision process, constructing a power supply recovery reinforcement learning agent based on a DDQN algorithm, enabling the agent to interact with a simulation environment, obtaining a training sample, and carrying out training. And after the intelligent agent training is completed, the importance of each input feature on an intelligent agent decision result is calculated by utilizing a Shapley value analysis method, a visual feature importance degree sequence is provided for a dispatcher, an interpretable power supply recovery strategy is provided for the dispatcher, and the interpretability and credibility of a decision are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power supply restoration of a power distribution network of an electric power system, and in particular to a distribution network power supply restoration decision-making method based on Shapley value analysis and reinforcement learning. Background Art

[0002] In recent years, large-scale power outages caused by extreme events have become frequent, and ensuring the security of new power systems has become a critical and urgent task. However, the increased randomness and uncertainty of new distribution networks poses severe challenges to system analysis and decision-making, making it even more difficult to maintain safe and stable operation of distribution networks. Under extreme events, rapid and reliable distribution network power restoration is crucial to ensuring power system security. Studying intelligent power restoration strategies under extreme events within the context of new distribution networks to reduce the economic losses and social impacts of power outages is of great research value and engineering significance.

[0003] At present, the power restoration problem in distribution networks is mainly solved using a method based on mixed integer programming. This method has strict optimality guarantees and strong modeling flexibility, but it still has the following shortcomings: 1) The computational complexity is high and it is difficult to meet the needs of real-time decision-making. With the expansion of the scale of distribution networks, the complexity of topological structures, and the diversification of controllable resources, the complexity of solving power restoration problems has increased significantly. The solution time of methods based on mixed integer programming has increased significantly in large-scale systems, making it difficult to meet real-time scheduling requirements. 2) It relies on precise mathematical modeling and has limited adaptability. This method is highly dependent on the precise modeling of system parameters. When there are errors in system parameters or incomplete information, it may be difficult to provide a stable and reliable power restoration solution, which affects its applicability.

[0004] In recent years, deep reinforcement learning has been widely used to solve power restoration problems in distribution networks, becoming a method with significant development potential because it does not rely on precise mathematical models, can autonomously learn optimal decision-making strategies in complex environments, and has high decision-making efficiency. However, existing power restoration methods based on reinforcement learning still have the following problems:

[0005] 1) The opaque decision-making process makes it difficult for human dispatchers to understand the agent's power restoration strategy. Traditional reinforcement learning methods rely on deep neural networks for decision-making. The network's internal computational processes are complex, making it difficult to explain the agent's behavior from a physical perspective. In distribution network power restoration tasks, dispatchers struggle to understand the specific decision-making basis of the agent, such as why a particular switch action was selected or why a particular load was prioritized for restoration. This impacts the agent's credibility in practical applications.

[0006] 2) Unknown feature contributions make it difficult to optimize input variable selection. Reinforcement learning models typically require a large number of input features (such as topology information, load distribution, voltage status, and power flow information), but different features have varying degrees of influence on the agent's decision-making. Existing methods are unable to quantify the relative importance of each input feature, resulting in redundant features potentially impacting model performance. The influence of key features is difficult to assess, hindering model interpretability.

[0007] 3) Lack of traceability makes anomaly analysis and fault diagnosis difficult. Because reinforcement learning decisions are trained based on large-scale historical interaction data, once the intelligent agent makes an abnormal decision (such as abnormal restoration priority or violation of power flow safety constraints), it is difficult to trace the key factors that led to the decision. This makes it difficult for dispatchers to debug and adjust the reinforcement learning model in practice, affecting the safety and reliability of power restoration strategies. Summary of the Invention

[0008] The purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a distribution network power restoration decision-making method based on Shapley value analysis and reinforcement learning, which can provide dispatchers with an explainable power restoration strategy and improve the explainability and credibility of the decision.

[0009] To achieve the above objectives, the present invention provides a technical solution: a distribution network power restoration decision-making method based on Shapley value analysis and reinforcement learning, comprising the following steps:

[0010] S1: Acquire the data required for the distribution network power restoration simulation environment, including distribution network topology, switch position and status, branch impedance, load distribution, and distributed generation (DG) parameters. Based on the acquired data, establish the distribution network power restoration simulation environment.

[0011] S2: Based on the established distribution network power restoration simulation environment, define the Markov decision process for distribution network power restoration, including the state space, action space, and reward function;

[0012] S3: Based on the Markov decision process, a power restoration reinforcement learning agent based on the DDQN algorithm is constructed and enabled to interact with the distribution network power restoration simulation environment to obtain training samples and perform training;

[0013] S4: After the power restoration reinforcement learning agent is trained, the Shapley value analysis method is used to calculate the importance of each input feature to the decision-making result of the power restoration reinforcement learning agent, and an intuitive feature importance ranking is provided to the dispatcher to enhance the interpretability and transparency of the decision-making process.

[0014] Further, in step 2, the state space is expressed as:

[0015]

[0016] Where: s t is the state of the distribution network at time step t; is the set of power restoration time steps; and ε are the collections of nodes and branches of the distribution network respectively; z t,ij is the on-off state of branch ij at time step t, z t,ij =1 means branch ij is in closed state, z t,ij =0 means branch ij is in disconnected state; P t,ij and Q t,ij are the active power and reactive power flowing through branch ij at time step t; y t,i is the power supply status of node i at time step t, y t,i =1 means node i is powered on, y t,i =0 means node i is in power-off state; and are the active load and reactive load of node i at time step t, respectively; and are the active and reactive outputs of the DG at node i at time step t; V t,i is the voltage amplitude of node i at time step t;

[0017] The reward function is expressed as:

[0018] r t =r1+r2+r3(2)

[0019]

[0020] Where: r t is the reward at time step t; r1 is the load recovery reward of the node; r2 is the power flow limit penalty of the branch; r3 is the voltage deviation penalty of the node; α i is the load recovery reward coefficient of node i, which is used to adjust the contribution of load recovery in the reward; β ij is the power flow over-limit penalty coefficient of branch ij, which is used to punish the situation where the branch power exceeds the rated power to prevent overload; γ i is the voltage deviation penalty coefficient of node i, which is used to control the impact of node voltage deviation from the rated value and avoid voltage instability; S ij is the maximum power allowed to flow through branch ij; V i max and V i min are the maximum and minimum voltages allowed at node i, respectively.

[0021] Furthermore, in step 4, the core idea of ​​the Shapley value analysis method is to quantify the importance of each input feature by calculating its marginal contribution to the decision-making of the power restoration reinforcement learning agent; wherein, the marginal contribution of an input feature is defined as: the difference between the output result when the feature participates in the calculation and the output result when it does not participate; by calculating all relevant feature combinations, the relative contribution of each feature to the power restoration decision is determined, thereby improving the interpretability of the decision; for a single sample feature, the larger the absolute value of the Shapley value, the more significant the impact of the feature on the agent's decision; the positive or negative value of the Shapley value reflects whether the feature will increase or decrease the output result of the agent; for the i-th input feature, the expected value of its Shapley value φ i The calculation formula is:

[0022]

[0023] Where: X is the set of all input features; S is the set X without feature x i Then all relevant feature subsets; does not contain feature x i The input subset when f(x i ) includes the feature x i Output of the reinforcement learning agent when power is restored; f S (x i ) does not include feature x i Output of the reinforcement learning agent for power restoration.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] 1. Improve the explainability of power restoration decisions.

[0026] This method combines Shapley value analysis with reinforcement learning algorithms to achieve transparency in power restoration strategies by quantifying the impact of different input features on the decision-making of a power restoration reinforcement learning agent. Compared to the "black box" nature of traditional deep reinforcement learning (DRL) methods, this method provides feature importance ranking, enabling dispatchers to clearly understand the agent's decision-making basis, improving the interpretability and credibility of power restoration plans.

[0027] 2. Optimize feature selection to improve agent training efficiency.

[0028] Through Shapley value analysis, the marginal contribution of each input feature to the agent's decision-making is quantified. This allows for the removal of redundant features, reduces the dimensionality of training data, and improves the training efficiency of the power restoration reinforcement learning agent. Furthermore, the impact of irrelevant or low-contribution features on model convergence is avoided, making the power restoration strategy more stable and reliable.

[0029] 3. Enhance the traceability of intelligent entities and improve abnormal analysis capabilities.

[0030] Because power restoration tasks involve complex topological changes and power flow constraints, this invention uses Shapley values ​​to calculate the contribution of each input variable to the final decision, making the agent's decision-making process traceable. When an abnormal restoration strategy is encountered, Shapley value analysis can identify the key variables influencing the decision, allowing dispatchers to quickly locate the problem and improve fault diagnosis and anomaly analysis capabilities.

[0031] 4. Improve the security of recovery plans.

[0032] This paper constructs a reward function for distribution network power restoration. By introducing a power flow over-limit penalty term, it reduces the risk of power flow over-limit during distribution network power restoration. By optimizing the reward function, the intelligent agent is able to favor restoration plans that meet power flow safety constraints during training, improving the safety and engineering applicability of power restoration plans.

[0033] 5. Enhance the decision-making support capabilities of dispatchers and improve the acceptability of plans.

[0034] The feature importance ranking provided by the present invention can be used as an auxiliary decision-making tool for dispatchers, enabling them to understand the decision logic of the power supply restoration reinforcement learning agent and make manual adjustments, thereby enhancing the acceptability of the power supply restoration plan and improving the practicality of the intelligent dispatching system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 4 is an implementation flow chart of the method of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0037] like Figure 1 As shown, this embodiment discloses a distribution network power supply restoration decision-making method based on Shapley value analysis and reinforcement learning, the details of which are as follows:

[0038] S1: Acquire the data required for the distribution network power supply restoration simulation environment, including distribution network topology, switch position and status, branch impedance, load distribution, and distributed generation (DG) parameters. Based on the acquired data, establish the distribution network power supply restoration simulation environment.

[0039] S2: Based on the established distribution network power restoration simulation environment, define the Markov decision process for distribution network power restoration, including the state space, action space, and reward function;

[0040] The state space is represented as:

[0041]

[0042] Where: s t is the state of the distribution network at time step t; is the set of power restoration time steps; and ε are the collections of nodes and branches of the distribution network respectively; z t,ij is the on-off state of branch ij at time step t, z t,ij =1 means branch ij is in closed state, z t,ij =0 means branch ij is in disconnected state; P t,ij and Q t,ij are the active power and reactive power flowing through branch ij at time step t; y t,i is the power supply status of node i at time step t, y t,i =1 means node i is powered on, y t,i =0 means node i is in power-off state; and are the active load and reactive load of node i at time step t, respectively; and are the active and reactive outputs of the DG at node i at time step t; V t,i is the voltage amplitude at node i at time step t.

[0043] The reward function is expressed as:

[0044] r t =r1+r2+r3 (2)

[0045]

[0046] Where: r t is the reward at time step t; r1 is the load recovery reward of the node; r2 is the power flow limit penalty of the branch; r3 is the voltage deviation penalty of the node; α i is the load recovery reward coefficient of node i, which is used to adjust the contribution of load recovery in the reward; β ij is the power flow over-limit penalty coefficient of branch ij, which is used to punish the situation where the branch power exceeds the rated power to prevent overload; γ i is the voltage deviation penalty coefficient of node i, which is used to control the impact of node voltage deviation from the rated value and avoid voltage instability; S ij is the maximum power allowed to flow through branch ij; V i max and V i min are the maximum and minimum voltages allowed at node i, respectively.

[0047] S3: Based on the Markov decision process, a power supply restoration reinforcement learning agent based on the DDQN algorithm is constructed, and it is enabled to interact with the distribution network power supply restoration simulation environment to obtain training samples and perform training.

[0048] S4: After the power restoration reinforcement learning agent is trained, the Shapley value analysis method is used to calculate the importance of each input feature to the decision-making result of the power restoration reinforcement learning agent, and an intuitive feature importance ranking is provided to the dispatcher to enhance the interpretability and transparency of the decision-making process.

[0049] The core idea of ​​the Shapley value analysis method is to quantify the importance of each input feature by calculating its marginal contribution to the decision-making of the power restoration reinforcement learning agent. The marginal contribution of an input feature is defined as the difference between the output result when the feature participates in the calculation and the output result when it does not participate. By calculating all relevant feature combinations, the relative contribution of each feature to the power restoration decision is determined, thereby improving the interpretability of the decision. For a single sample feature, the larger the absolute value of the Shapley value, the more significant the impact of the feature on the agent's decision. The positive or negative sign of the Shapley value reflects whether the feature will increase or decrease the output result of the agent. For the i-th input feature, the expected value of its Shapley value φ i The calculation formula is:

[0050]

[0051] Where: X is the set of all input features; S is the set X without feature x i Then all relevant feature subsets; does not contain feature x i The input subset when f(x i ) includes the feature x i Output of the reinforcement learning agent when power is restored; f S (x i ) does not include feature x i Output of the reinforcement learning agent for power restoration.

[0052] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A distribution network power restoration decision-making method based on Shapley value analysis and reinforcement learning, characterized in that: The following steps are involved: S1: Acquire the data required for the distribution network power restoration simulation environment, including distribution network topology, switch position and status, branch impedance, load distribution, and distributed generation (DG) parameters. Based on the acquired data, establish the distribution network power restoration simulation environment. S2: Based on the established distribution network power restoration simulation environment, define the Markov decision process for distribution network power restoration, including the state space, action space, and reward function; S3: Based on the Markov decision process, a power restoration reinforcement learning agent based on the DDQN algorithm is constructed and enabled to interact with the distribution network power restoration simulation environment to obtain training samples and perform training; S4: After the power restoration reinforcement learning agent is trained, the Shapley value analysis method is used to calculate the importance of each input feature to the decision-making result of the power restoration reinforcement learning agent, and an intuitive feature importance ranking is provided to the dispatcher to enhance the interpretability and transparency of the decision-making process.

2. A distribution network power restoration decision-making method based on Shapley value analysis and reinforcement learning according to claim 1, characterized in that: In step 2, the state space is represented as: Where: s t is the state of the distribution network at time step t; is the set of power restoration time steps; and ε are the collections of nodes and branches of the distribution network respectively; z t,ij is the on-off state of branch ij at time step t, z t,ij =1 means branch ij is in closed state, z t,ij =0 means branch ij is in disconnected state; P t,ij and Q t,ij are the active power and reactive power flowing through branch ij at time step t; y t,i is the power supply status of node i at time step t, y t,i =1 means node i is powered on, y t,i =0 means node i is in power-off state; and are the active load and reactive load of node i at time step t, respectively; and are the active and reactive outputs of the DG at node i at time step t; V t,i is the voltage amplitude of node i at time step t; The reward function is expressed as: r t ＝r1+r2+r3 (2) Where: r t is the reward at time step t; r1 is the load recovery reward of the node; r2 is the power flow limit penalty of the branch; r3 is the voltage deviation penalty of the node; α i is the load recovery reward coefficient of node i, which is used to adjust the contribution of load recovery amount in the reward; β ij is the power flow over-limit penalty coefficient of branch ij, which is used to punish the situation where the branch power exceeds the rated power to prevent overload; γ i is the voltage deviation penalty coefficient of node i, which is used to control the impact of node voltage deviation from the rated value and avoid voltage instability; S ij is the maximum power allowed to flow through branch ij; V i max and V i min are the maximum and minimum voltages allowed at node i, respectively.

3. The distribution network power restoration decision-making method based on Shapley value analysis and reinforcement learning according to claim 1 is characterized by: In step 4, the core idea of ​​the Shapley value analysis method is to quantify the importance of each input feature by calculating its marginal contribution to the decision-making of the power restoration reinforcement learning agent. The marginal contribution of an input feature is defined as the difference between the output result when the feature participates in the calculation and the output result when the feature does not participate. By calculating all relevant feature combinations, the relative contribution of each feature to the power restoration decision is determined, thereby improving the interpretability of the decision. For a single sample feature, the larger the absolute value of the Shapley value, the more significant the impact of the feature on the agent's decision. The positive or negative value of the Shapley value reflects whether the feature will increase or decrease the output result of the agent. For the i-th input feature, the expected value of its Shapley value is φ i The calculation formula is: Where: X is the set of all input features; S is the set X without feature x i Then all relevant feature subsets; does not contain feature x i The input subset when f(x i ) includes the feature x i Output of the reinforcement learning agent when power is restored; f S (x i ) does not include feature x i Output of the reinforcement learning agent for power restoration.

Images