Power distribution network fault recovery method based on graph reinforcement learning and related device

Abstract

The invention belongs to the technical field of power distribution network fault recovery, and provides a power distribution network fault recovery method based on graph reinforcement learning and a related device, and the method comprises the steps: constructing a power distribution network fault recovery problem graph model, and carrying out the embedding of a power distribution network model graph based on a graph attention neural network; based on the power distribution network fault recovery problem graph model, an agent set of multi-agent reinforcement learning is constructed, and the power distribution network fault recovery problem is modeled as a Markov decision process; and solving the Markov decision process, carrying out multi-agent training, and generating a fault recovery strategy by the trained agents according to the current state of the power distribution network. According to the method, the technologies of graph theory, deep learning and reinforcement learning are deeply fused, and efficient and accurate power distribution network fault recovery strategy generation is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of distribution network fault recovery, and in particular relates to a distribution network fault recovery method based on graph reinforcement learning and related devices. Background Art

[0002] Distribution network fault recovery refers to the process of adjusting the system network topology to restore the power supply to the downstream power-off loads by regulating the feeder and tie line switches after the distribution network line fails and is cut off. The current distribution network fault recovery methods mainly use mathematical optimization methods, artificial intelligence algorithms and heuristic search algorithms.

[0003] The DQN algorithm is trained and updated based on the Markov Decision Process, which includes state space, action space, reward function, and state transition probability. In this process, the agent interacts with the environment, observes the current state, takes actions, and obtains rewards.

[0004] The advantage of this method is that it avoids over-reliance on the latest experience and makes training more stable.

[0005] In DQN, the selection of the maximum Q value and the evaluation of the Q value are performed through the same network, which can easily lead to over-estimation of the Q value, especially when the rewards of certain actions are noisy. When the Q of an action is too high, the effectiveness of the strategy execution will decrease. Double DQN reduces the risk of over-estimation by using two networks to select and evaluate actions separately. At the same time, the DQN algorithm is applied to the fault recovery scenario of the distribution network. Due to the large number of topological connection relationships in the distribution network and the lack of effective representation of the network topology, it is difficult to ensure the accuracy of the generated recovery strategy.

[0006] Distribution network fault recovery strategy based on GCN graph convolutional neural network. GCN is used to model the topological structure of the distribution network and process the graph structure data of the distribution network to capture the correlation and transmission information between nodes, and then combined with reinforcement learning to learn effective recovery strategies.

[0007] The distribution network can be modeled as a graph structure, where nodes represent devices in the power system (such as substations, distribution lines, load nodes, etc.), and edges represent the physical connections between these devices. The goal of fault recovery is to restore power supply as quickly as possible and reduce the impact of the fault by disconnecting and reclosing these edges and adjusting the node status.

[0008] GCN uses the adjacency matrix of the graph to perform convolution operations, and the edge weights in the adjacency matrix are fixed, so all neighbor nodes contribute equally to the information of the target node. In the distribution network, the degree of electrical connection between different nodes may vary greatly. In the fault recovery scenario, the node types are complex, and GCN aggregates information for all nodes in the same way. It cannot distinguish the feature differences between different types of nodes, resulting in a lack of flexibility in its information transmission mechanism. At the same time, each layer of GCN can only aggregate the information of neighboring nodes, so multiple convolutional layers are required to capture the influence of distant nodes. However, as the number of network layers increases, GCN is prone to the problem of gradient vanishing or over-smoothing, that is, the features of different nodes gradually converge as they propagate, resulting in the inability to distinguish the differences between nodes. Summary of the invention

[0009] The purpose of the present invention is to provide a distribution network fault recovery method and related devices based on graph reinforcement learning to solve the problem of difficulty in ensuring the accuracy of generated recovery strategies.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] In a first aspect, the present invention provides a distribution network fault recovery method based on graph reinforcement learning, comprising:

[0012] Construct a graph model for distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network;

[0013] Based on the graph model of distribution network fault restoration problem, a multi-agent reinforcement learning agent set is constructed, and the distribution network fault restoration problem is modeled as a Markov decision process.

[0014] The Markov decision process is solved and multi-agent training is performed. After training, the agents generate fault recovery strategies according to the current state of the distribution network.

[0015] Furthermore, the construction of the distribution network fault recovery problem graph model includes:

[0016] Modeling a distribution network as a tree diagram Where V={V0,V1,...,V N} and E={E1,E2,…,E N} represent the set of nodes and edges respectively; for node i∈V, let s i =p i +jq i is the complex power injection, is the complex voltage; the active and reactive power flow constraints of the distribution network are:

[0017]

[0018] In the formula, for node i and node, G ij and B ij Respectively represent the real and imaginary parts of the inter-node admittance; θ ij Represents the phase angle difference between nodes;

[0019] The adjacency matrix and characteristic matrix are constructed for the distribution network. The adjacency matrix contains the topological connection relationship of the power grid, and the characteristic matrix considers the electrical characteristics related to the operating state. For the adjacency matrix A, if the node connection (i, j) ∈ E, then the corresponding adjacency matrix element A ij =1, otherwise A ij =0; the feature matrix X corresponding to the graph is an N×M matrix, which stores the eigenvalues ​​corresponding to each node, and M is the dimension of the node feature. Specifically, the feature matrix expression is as follows:

[0020] X=[f C ,f line ,f gen ,f load ]

[0021] In the formula, f line ,f gen ,f load They are the characteristics of the transmission line, the characteristics of the generator and the characteristics of the load.

[0022] Furthermore, the distribution network model graph embedding based on the graph attention neural network includes:

[0023] The graph attention network is used to embed the features of the distribution network topology state graph data. Specifically, GATs obtains the feature embedding expression of the distribution network topology and distribution network state through a multi-head attention mechanism. The convolution calculation principle of the graph attention layer with a multi-head attention mechanism is defined as follows:

[0024]

[0025] In the formula, σ represents the nonlinear activation function, is the set of neighboring nodes of node i, K represents the number of attention mechanisms adopted, It represents the normalized attention factor of node j to node k calculated by the kth attention mechanism, W k is the kth weight matrix.

[0026] Furthermore, based on the distribution network fault recovery problem graph, a multi-agent reinforcement learning agent set is constructed, and the distribution network fault recovery problem is modeled as a Markov decision process, including:

[0027] Construct the state space, action space, and reward function for a Markov decision process.

[0028] Further, specific:

[0029] Construct the state space of a Markov decision process:

[0030]

[0031] in, Indicates the fault switch number, Indicates the switch number of the last action, S switch Represents the state of all switches in the distribution network. If S switch =1 means the switch is in the closed state, otherwise it is disconnected;

[0032] The intelligent agent controls the switching of the lines through switches, thereby controlling the operating state of the distribution network and constructing the action space of the Markov decision process:

[0033]

[0034] Among them, N switch Indicates the switch number. If a=(1,0,...,0,0), it means that the switch numbered 1 will be activated.

[0035] Construct the reward function for the Markov decision process:

[0036]

[0037] Among them, P load represents the reward value after full load failure recovery, and Respectively represent the penalty values ​​for voltage over-limit and operation over-limit; represents the penalty for increased load loss, P loop represents the penalty when the distribution network forms a ring network; ω represents the weight parameter of each constraint.

[0038] Furthermore, the Markov decision process is solved and multi-agent training is performed. After training, the agents generate a fault recovery strategy according to the current distribution network state, including:

[0039] The dual deep Q network DDQN algorithm is used to solve the Markov process. The proposed DDQN algorithm introduces two independent neural networks Q1(s,a;θ1) and Q2(s,a;θ2) for selecting actions and evaluating action values ​​respectively; the calculation formula of the target Q value is as follows

[0040] Q target (s,a)=R+γ·Q(s′,argmax a′ Q(s′,a′;θ1);θ2)

[0041] Among them, argmax a′Q(s′,a′;θ1) represents the action of selecting the maximum Q value;

[0042] After training, the agent generates a fault recovery strategy based on the current distribution network status, including the specific line switch operation position and operation sequence, and provides a feasible fault recovery strategy that meets various distribution network constraints.

[0043] Further, function update: DDQN uses time difference error to update the neural network parameters, and the calculation formula is as follows:

[0044] δ=|Q(s,a)-Q target(s,a) |=|Q(s,a;θ)-(R+γ·Q(s′,argmax a′ Q(s′,a′;θ1);θ2))|

[0045] The loss function aims to minimize the difference between the estimated Q value and the target Q value, and the network parameters are updated using the gradient descent method.

[0046] In a second aspect, the present invention provides a distribution network fault recovery system based on graph reinforcement learning, comprising:

[0047] The model building module is used to build a graph model of distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network;

[0048] The problem conversion module is used to construct a set of agents for multi-agent reinforcement learning based on the distribution network fault recovery problem graph, and model the distribution network fault recovery problem as a Markov decision process;

[0049] The solution output module is used to solve the Markov decision process and train multiple agents. After training, the agents generate fault recovery strategies according to the current distribution network status.

[0050] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the steps of a distribution network fault recovery method based on graph reinforcement learning.

[0051] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the distribution network fault recovery method based on graph reinforcement learning are implemented.

[0052] Compared with the prior art, the present invention has the following technical effects:

[0053] The present invention proposes an innovative distribution network fault recovery method based on graph reinforcement learning. This method deeply integrates graph theory, deep learning and reinforcement learning techniques, aiming to achieve efficient and accurate distribution network fault recovery strategy generation.

[0054] The present invention first simplifies the complex distribution network structure into an easy-to-process graph form by constructing a tree graph model of the distribution network, where nodes represent buses and edges represent connecting lines. This modeling method is not only intuitive, but also convenient for subsequent graph embedding processing.

[0055] By constructing the adjacency matrix and the feature matrix, the present invention fully captures the topological connection relationship and operating status characteristics of the distribution network. The adjacency matrix reflects the connection status between nodes in the power grid, while the feature matrix contains the electrical characteristics of the nodes, such as transmission line parameters, generator status and load conditions, providing a rich information basis for subsequent graph embedding.

[0056] The use of graph attention neural networks (GATs) for graph embedding is a highlight of this invention. GATs can deeply explore the complex relationship between the topology and state of the distribution network through a multi-head attention mechanism, accurately extract features even in the presence of measurement errors, and provide high-quality input data for the reinforcement learning algorithm. This graph embedding method significantly improves the accuracy and robustness of the fault recovery strategy.

[0057] The present invention cleverly models the distribution network fault recovery problem as a Markov decision process (MDP), which is the key to connecting graph embedding and reinforcement learning. By defining the state space, action space and reward function, the present invention constructs a complete MDP framework, providing a clear optimization target for the subsequent reinforcement learning algorithm.

[0058] The state space contains the faulty switch number, the switch number of the last action, and the status of all switches in the distribution network. This information together constitutes the complete state of the current distribution network and provides a basis for decision-making for the intelligent agent.

[0059] The action space is defined as the switching operation of the switch, and only one switch is switched at a time to avoid state space explosion while ensuring the feasibility of the strategy.

[0060] The reward function comprehensively considers multiple factors such as load recovery, voltage limit exceeding, operation limit exceeding, load loss increase and ring network formation. Through reasonable weight distribution, it guides the intelligent agent to learn a fault recovery strategy that satisfies the constraints and is efficient.

[0061] The present invention adopts the Dual Deep Q Network (DDQN) algorithm to solve the MDP, which is an advanced algorithm in the field of deep reinforcement learning. DDQN effectively avoids the overestimation problem and improves the stability and convergence of the strategy by introducing two independent neural networks for selecting actions and evaluating action values ​​respectively.

[0062] By updating the neural network parameters through time difference error, DDQN can continuously learn and optimize the fault recovery strategy. After a lot of training, the agent can quickly generate feasible fault recovery strategies including switch operation positions and sequences according to the current distribution network status. These strategies not only meet various distribution network constraints, but also have a high fault recovery rate and have significant practical application value.

[0063] In summary, the distribution network fault recovery method based on graph reinforcement learning proposed in the present invention achieves efficient and accurate solution to the distribution network fault recovery problem by constructing a graph model of the distribution network, using a graph attention neural network for graph embedding, modeling the fault recovery problem as a Markov decision process, and applying a deep reinforcement learning algorithm for solving it. This method not only improves the generation speed and quality of the fault recovery strategy, but also enhances the robustness of the strategy to measurement errors, providing a strong guarantee for the safe and stable operation of the distribution network. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flow chart of the present invention.

[0065] Figure 2 This is a logic block diagram of the present invention. DETAILED DESCRIPTION

[0066] The present invention is further described below in conjunction with the accompanying drawings:

[0067] Example 1, please refer to Figure 1 The present invention provides a distribution network fault recovery method based on graph reinforcement learning, comprising:

[0068] Construct a graph model for distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network;

[0069] Based on the graph model of distribution network fault restoration problem, a multi-agent reinforcement learning agent set is constructed, and the distribution network fault restoration problem is modeled as a Markov decision process.

[0070] The Markov decision process is solved and multi-agent training is performed. After training, the agents generate fault recovery strategies according to the current state of the distribution network.

[0071] This technical solution aims to solve the distribution network fault recovery problem by combining graph theory, graph attention neural network and multi-agent reinforcement learning.

[0072] By constructing a graphical model of the distribution network fault recovery problem, the scheme can intuitively and efficiently represent the complex structure and operation status of the distribution network. The use of the graphical model simplifies the representation of the problem and makes subsequent processing and analysis more convenient.

[0073] The distribution network model graph embedding method based on graph attention neural network can deeply mine the correlation information between the distribution network topology and status, and accurately extract features that are useful for fault recovery even in the presence of measurement errors or incomplete data.

[0074] The distribution network fault recovery problem is modeled as a Markov decision process, and based on this, a multi-agent reinforcement learning agent set is constructed, so that the scheme can learn recovery strategies that adapt to different fault scenarios. The use of multi-agents enhances the decision-making ability of the scheme and its adaptability to complex environments.

[0075] Through multi-agent training, each agent can learn the optimal strategy for a specific task or scenario, so that it can quickly and accurately generate recovery plans when a failure occurs.

[0076] Since the solution uses a reinforcement learning algorithm, it can continuously optimize the recovery strategy through trial and error and feedback, making the recovery process more efficient and accurate.

[0077] The collaborative work of multiple agents can further shorten the recovery time, reduce the impact of faults on the operation of the distribution network, and improve the stability and reliability of the overall system.

[0078] The solution is highly flexible and scalable. For distribution networks of different sizes and structures, it only needs to adjust the number and parameters of the graph model and intelligent agents to adapt to new application scenarios.

[0079] With the continuous advancement of technology and the continuous accumulation of data, the solution can also improve the performance and accuracy of recovery strategies through continuous learning and optimization.

[0080] In summary, this technical solution provides an efficient, accurate and adaptable solution to the distribution network fault recovery problem by combining graph theory, graph attention neural network and multi-agent reinforcement learning. This solution can not only improve the recovery efficiency and accuracy, but also has strong flexibility and scalability, providing a strong guarantee for the safe and stable operation of the distribution network.

[0081] Embodiment 2, the present invention provides a distribution network fault recovery method based on graph reinforcement learning, specifically comprising:

[0082] Step 1: Construct a graph model of distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network;

[0083] Step 2: Model the distribution network fault restoration problem as a Markov decision process;

[0084] Step 3: Study the fault recovery strategy based on the DDQN algorithm and train the distribution network decision-making fault recovery agent.

[0085] Preferably, the implementation of step 1 includes the following steps:

[0086] Step 1.1: Modeling the distribution network fault recovery problem

[0087] Modeling a distribution network as a tree diagram Where V={V0,V1,…,V N} and E={E1,E2,...,E N} respectively represent the set of nodes (busbars) and edges (connections). For node i∈V, let s i =p i +jq i is the complex power injection, is the complex voltage. Consider the active and reactive power flow constraints of the distribution network:

[0088]

[0089] In the formula, for node i and node, G ij and B ij Respectively represent the real and imaginary parts of the inter-node admittance; θ ij Represents the phase angle difference between nodes.

[0090] Step 1.2, construct the distribution network adjacency matrix and characteristic matrix;

[0091] The adjacency matrix and characteristic matrix are constructed for the distribution network. The adjacency matrix contains the topological connection relationship of the power grid, and the characteristic matrix considers the electrical characteristics related to the operating state. For the adjacency matrix A, if the node connection (i, j) ∈ E, then the corresponding adjacency matrix element A ij =1, otherwise A ij = 0. The feature matrix X corresponding to the graph is an N×M matrix that stores the eigenvalues ​​corresponding to each node. M is the dimension of the node feature. Specifically, the feature matrix expression is as follows:

[0092] X=[f C ,f line ,f gen ,f load ]

[0093] In the formula, f line ,f gen ,f load They are the characteristics of the transmission line, the characteristics of the generator and the characteristics of the load.

[0094] Step 1.3: Graph Embedding Method Based on Graph Attention Neural Network

[0095] In this step, the graph attention network is used to embed the distribution network topology state graph data. Specifically, GATs can obtain the feature embedding expression of the distribution network topology and distribution network state through the multi-head attention mechanism, and improve the decision accuracy of the reinforcement learning algorithm when there is measurement error in the distribution network. The convolution calculation principle of the graph attention layer with multi-head attention mechanism is defined as follows:

[0096]

[0097] In the formula, σ represents the nonlinear activation function, is the set of neighboring nodes of node i, K represents the number of attention mechanisms adopted, It represents the normalized attention factor of node j to node k calculated by the kth attention mechanism, W k is the kth weight matrix.

[0098] The implementation of step 2 includes the following steps:

[0099] Step 2.1: State space definition

[0100] Construct the state space of a Markov decision process:

[0101]

[0102] in, Indicates the fault switch number, Indicates the switch number of the last action, S switch Represents the state of all switches in the distribution network. If S switch =1 means the switch is in the closed state, otherwise it is disconnected.

[0103] Step 2.2: Action Space Definition

[0104] The intelligent agent controls the switching of the line through the switch, thereby controlling the operating state of the distribution network. Since a complete load switching operation includes the switching of a group of segment switches and circuit breakers, only one switch is switched at a time to avoid state space explosion. Construct the action space of the Markov decision process:

[0105]

[0106] Among them, N switch Indicates the switch number. If a=(1,0,...,0,0), it means that switch numbered 1 will be activated.

[0107] Step 2.3, reward function definition:

[0108] Construct the reward function for the Markov decision process:

[0109]

[0110] Among them, P load represents the reward value after full load failure recovery, and They represent the penalty values ​​for voltage exceeding the limit and operation exceeding the limit respectively. represents the penalty for increased load loss, P loop It represents the penalty when the distribution network forms a ring network. ω represents the weight parameter of each constraint.

[0111] The deep reinforcement learning algorithm proposed in step 3 is used to solve the distribution network fault recovery problem. The implementation of step 3 includes the following steps:

[0112] Step 3.1, Q function definition

[0113] The dual deep Q network (DDQN) algorithm is used to solve the Markov process established in step 2. The proposed DDQN algorithm structure is as follows: Figure 2 DDQN introduces two independent neural networks Q1(s,a;θ1) and Q2(s,a;θ2) for selecting actions and evaluating action values ​​respectively. The calculation formula of the target Q value is as follows

[0114] Q target (s,a)=R+γ·Q(s′,argmax a′ Q(s′,a′;θ1);θ2)

[0115] Among them, argmax a′ Q(s′, a′; θ1) represents the action of selecting the maximum Q value.

[0116] Step 3.2: Q function update

[0117] DDQN uses temporal difference error to update neural network parameters. The calculation formula is as follows:

[0118] δ=|Q(s,a)-Q target(s,a) |=|Q(s,a;θ)-(R+γ·Q(s′,argmax a′ Q(s′,a′;θ1);θ2))|

[0119] The loss function aims to minimize the difference between the estimated Q value and the target Q value. The network parameters are updated using the gradient descent method. The algorithm flow is shown in Table 1.

[0120] Step 3.3: After training, the agent is able to generate a fault recovery strategy based on the current distribution network status, including the specific line switch operation position and operation sequence, and provide a feasible fault recovery strategy that meets various distribution network constraints.

[0121] Table 1 Deep reinforcement learning DQN algorithm flow chart

[0122]

[0123]

[0124] The present invention proposes a distribution network fault recovery method based on graph reinforcement learning. The technical principles of the method mainly revolve around graph theory, graph attention neural networks (GATs) and dual deep Q network (DDQN) algorithms.

[0125] The distribution network is modeled as a tree graph, where nodes represent buses and edges represent connecting lines. This modeling method simplifies the complex structure of the distribution network and facilitates subsequent graph embedding and reinforcement learning processing.

[0126] By constructing the adjacency matrix and feature matrix, the topological connection relationship and operating status characteristics of the distribution network are captured, providing a rich information basis for graph embedding.

[0127] Graph Attention Neural Networks (GATs):

[0128] GATs are used to embed features in distribution network topology state graph data. Through the multi-head attention mechanism, GATs can deeply explore the complex relationship between distribution network topology and state, and accurately extract features even in the presence of measurement errors.

[0129] This graph embedding method improves the decision accuracy of reinforcement learning algorithms for distribution network fault restoration problems.

[0130] Dual Deep Q Network (DDQN) Algorithm:

[0131] The distribution network fault restoration problem is modeled as a Markov decision process (MDP), and a complete MDP framework is constructed by defining the state space, action space and reward function.

[0132] The DDQN algorithm is used to solve this MDP. By introducing two independent neural networks for selecting actions and evaluating action values, the over-estimation problem is avoided and the stability and convergence of the strategy are improved.

[0133] By updating the neural network parameters through temporal difference errors, DDQN can continuously learn and optimize the fault recovery strategy.

[0134] This method can quickly generate a feasible fault recovery strategy including switch operation positions and sequences, significantly shortening the fault recovery time.

[0135] Enhance decision-making accuracy:

[0136] Through the graph embedding method of graph attention neural network and reinforcement learning of DDQN algorithm, this method can accurately capture the topology and state characteristics of the distribution network and learn recovery strategies that adapt to different fault scenarios.

[0137] Improved robustness:

[0138] The method is robust to measurement errors and can accurately generate recovery strategies even in the presence of incomplete or noisy data.

[0139] Meeting distribution network constraints:

[0140] The generated fault recovery strategy satisfies various distribution network constraints, such as voltage over-limit, operation over-limit, load loss increase, and ring network formation, ensuring the feasibility and safety of the recovery strategy.

[0141] Flexibility and scalability:

[0142] This method has strong flexibility and scalability. For distribution networks of different scales and structures, it only needs to adjust the number and parameters of graphical models and intelligent agents to adapt to new application scenarios.

[0143] In summary, the distribution network fault recovery method based on graph reinforcement learning proposed in this invention realizes efficient and accurate solution to the distribution network fault recovery problem by combining graph theory, graph attention neural network and dual deep Q network algorithm. This method not only improves the efficiency and accuracy of fault recovery, but also enhances robustness, meets various constraints of the distribution network, and has flexibility and scalability, providing a strong guarantee for the safe and stable operation of the distribution network.

[0144] In yet another embodiment of the present invention, a distribution network fault recovery system based on graph reinforcement learning is provided, which can be used to implement the above-mentioned distribution network fault recovery method based on graph reinforcement learning. Specifically, the system includes:

[0145] The model building module is used to build a graph model of distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network;

[0146] The problem conversion module is used to construct a set of agents for multi-agent reinforcement learning based on the distribution network fault recovery problem graph, and model the distribution network fault recovery problem as a Markov decision process;

[0147] The solution output module is used to solve the Markov decision process and train multiple agents. After training, the agents generate fault recovery strategies according to the current distribution network status.

[0148] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0149] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of a distribution network fault recovery method based on graph reinforcement learning.

[0150] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the above-mentioned embodiment in a distribution network fault recovery method based on graph reinforcement learning.

[0151] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A distribution network fault recovery method based on graph reinforcement learning, characterized in that: include: Construct a graph model for distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network; Based on the graph model of distribution network fault restoration problem, a multi-agent reinforcement learning agent set is constructed, and the distribution network fault restoration problem is modeled as a Markov decision process. The Markov decision process is solved and multi-agent training is performed. After training, the agents generate fault recovery strategies according to the current state of the distribution network.

2. A distribution network fault recovery method based on graph reinforcement learning according to claim 1, characterized in that: The construction of the distribution network fault recovery problem graph model comprises: Modeling a distribution network as a tree diagram ,in and Represent the set of nodes and edges respectively; for nodes ,make is the complex power injection, is the complex voltage; the active and reactive power flow constraints of the distribution network are: In the formula, for the node and nodes, and Respectively represent the real and imaginary parts of the inter-node admittance; Represents the phase angle difference between nodes; The adjacency matrix and characteristic matrix are constructed for the distribution network. The adjacency matrix contains the topological connection relationship of the power grid, and the characteristic matrix considers the electrical characteristics related to the operating status. , if the node connection , then the corresponding adjacency matrix element ,otherwise ; The feature matrix corresponding to the graph for The matrix stores the eigenvalues ​​corresponding to each node. is the dimension of the node feature. Specifically, the feature matrix expression is as follows: In the formula, They are the characteristics of the transmission line, the characteristics of the generator and the characteristics of the load.

3. A distribution network fault recovery method based on graph reinforcement learning according to claim 2, characterized in that: The distribution network model graph embedding based on the graph attention neural network includes: The graph attention network is used to embed the features of the distribution network topology state graph data. Specifically, GATs obtains the feature embedding expression of the distribution network topology and distribution network state through a multi-head attention mechanism. The convolution calculation principle of the graph attention layer with a multi-head attention mechanism is defined as follows: In the formula, represents the nonlinear activation function, For Node The set of neighboring nodes of Represents the number of attention mechanisms adopted, It means that by Nodes calculated by the attention mechanism For Node The normalized attention factor of For the A weight matrix.

4. A distribution network fault recovery method based on graph reinforcement learning according to claim 1, characterized in that: Based on the distribution network fault recovery problem graph model, a multi-agent reinforcement learning agent set is constructed, and the distribution network fault recovery problem is modeled as a Markov decision process, including: Construct the state space, action space, and reward function for a Markov decision process.

5. A distribution network fault recovery method based on graph reinforcement learning according to claim 4, characterized in that: Specific: Construct the state space of a Markov decision process: in, Indicates the fault switch number, Indicates the switch number of the last action, S switch Represents the state of all switches in the distribution network. If S switch =1 means the switch is in the closed state, otherwise it is disconnected; The intelligent agent controls the switching of the lines through switches, thereby controlling the operating state of the distribution network and constructing the action space of the Markov decision process: Among them, N switch Indicates the switch number. If a=(1,0,...,0,0), it means that the switch numbered 1 will be activated. Construct the reward function for the Markov decision process: Among them, P load represents the reward value after full load failure recovery, and Respectively represent the penalty values ​​for voltage over-limit and operation over-limit; represents the penalty for increased load loss, P loop represents the penalty when the distribution network forms a ring network; ω represents the weight parameter of each constraint.

6. A distribution network fault recovery method based on graph reinforcement learning according to claim 1, characterized in that: The Markov decision process is solved and multi-agent training is performed. After training, the agent generates a fault recovery strategy according to the current distribution network state, including: Using double depth The proposed DDQN algorithm introduces two independent neural networks to solve the Markov process. and They are used to select actions and evaluate action values ​​respectively; the goal The value is calculated as follows in, Indicates the maximum selection Actions of values; After training, the agent generates a fault recovery strategy based on the current distribution network status, including the specific line switch operation position and operation sequence, and provides a feasible fault recovery strategy that meets various distribution network constraints.

7. A distribution network fault recovery method based on graph reinforcement learning according to claim 1, characterized in that: Function update: DDQN uses temporal difference error to update the neural network parameters. The calculation formula is as follows: The loss function aims to make the estimate Values ​​and Goals The difference between the values, the minimum network parameters are updated using the gradient descent method.

8. A distribution network fault recovery system based on graph reinforcement learning, characterized in that: include: The model building module is used to build a graph model of distribution network fault recovery problem and embed the distribution network model graph based on graph attention neural network; The problem conversion module is used to construct a set of agents for multi-agent reinforcement learning based on the distribution network fault recovery problem graph, and model the distribution network fault recovery problem as a Markov decision process; The solution output module is used to solve the Markov decision process and train multiple agents. After training, the agents generate fault recovery strategies according to the current distribution network status.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a distribution network fault recovery method based on graph reinforcement learning as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a distribution network fault recovery method based on graph reinforcement learning as described in any one of claims 1 to 7 are implemented.

Images