Power distribution network voltage reactive power optimization method fusing graph neural network and reinforcement learning

Abstract

The invention discloses a power distribution network voltage reactive power optimization method fusing a graph neural network and reinforcement learning, and relates to the crossing field of power system automation and deep reinforcement learning, and the method comprises the steps: 1, extracting the topological information of a power distribution network through the graph neural network GNN, taking the minimum line loss as an objective function, building a continuous reactive power optimization model of the power distribution network, and obtaining a continuous reactive power optimization model of the power distribution network; meanwhile, constraints of safe operation of the power distribution network are added; 2, describing a reactive power optimization problem of the power distribution network as a Markov decision process according to the model constructed in the step 1; 3, the GNN is embedded into the deep Q network, and a unique network is constructed to solve the constructed Markov decision process; and training the constructed model by using historical operation data of the power grid. The main innovation point of the method is that the dependence on an accurate physical model is eliminated, the rapid reactive power optimization of the power distribution network can be realized, and meanwhile, the trained model has relatively strong generalization capability and can be solved in real time, so that the calculation burden is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system automation and relates to a distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning. Background Art

[0002] To promote the construction of a clean and low-carbon energy system, the development of solar and wind energy will become an important task in the future. According to preliminary estimates, by 2030, my country's total installed capacity of wind and solar power generation is expected to exceed 1.6 billion kilowatts. With the rapid development of smart grids, the structure and operation of distribution systems have also changed significantly.

[0003] However, with the access of distributed generation (DG) with high penetration rate, the stable operation of distribution network faces severe challenges. The access of a large number of photovoltaic power generation systems has changed the distribution network from a single power supply structure and a unidirectional power flow mode to a multi-point power supply structure and a frequently changing bidirectional power flow mode. This change may lead to an increase in the phenomenon of voltage over-limit in the distribution network. At the same time, improving the economic efficiency of the distribution network has also become the focus of many researchers.

[0004] In distribution networks with high DG penetration, traditional reactive power optimization methods are difficult to effectively deal with their volatility and randomness. For example, mathematical optimization methods, intelligent optimization methods, random optimization methods, and model prediction methods all have problems in terms of computational complexity, local optimal traps, and high dependence on prediction data and models. In contrast, reinforcement learning (RL) can extract the best operating strategy from historical data through continuous interaction with the environment, reducing dependence on precise physical models. Therefore, the reactive power optimization method based on deep reinforcement learning (DRL) can quickly adapt to the power flow changes in the power grid, improve the system network loss, and have stronger online adjustment capabilities, becoming one of the hot topics of research today. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning, which can get rid of the dependence on precise physical models and realize rapid reactive power optimization of distribution networks.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning, comprising the following steps:

[0007] S1. Extract the topological information of the distribution network through the graph neural network GNN, build a voltage regulation equipment model, take the minimum line loss as the objective function, establish a continuous reactive power optimization model of the distribution network, and add constraints for the safe operation of the distribution network;

[0008] S2. Based on the continuous reactive power optimization model of the distribution network, the reactive power optimization problem is described as a Markov decision process, and a distribution network voltage reactive power optimization network DQN based on the deep Q network is constructed, including: taking the voltage regulating device as the decision-making agent, and defining the state space, agent action space, reward function, and state transfer process.

[0009] S3. Use the GNN-DQN combined algorithm to train the grid voltage and reactive power optimization network to obtain the distribution network voltage and reactive power optimization model; specifically: the graph neural network GNN extracts the topological information of the distribution network to generate the state of each node and aggregates these states into the state of the distribution network voltage and reactive power optimization network DQN. The voltage and reactive power optimization network DQN estimates the Q value of each action based on the state and outputs the optimal action; the distribution network voltage and reactive power optimization network DQN includes a strategy network Actor and a value network Critic. The value network Critic supervises the training of the strategy network Actor by updating the Q value.

[0010] S4. Input the real-time collected distribution network topology information into the distribution network voltage and reactive power optimization model to obtain the distribution network voltage and reactive power optimization strategy, that is, the output of the voltage regulating equipment.

[0011] Furthermore, in the aforementioned step S1, the voltage regulating equipment model includes: an on-load voltage regulating transformer tap changer OLTC model, and a capacitor bank CBS model, an on-load voltage regulating transformer tap changer OLTC ratio and a capacitor bank CBS state;

[0012] The OLTC model is:

[0013]

[0014] In the formula, represents the effective value of the current in the line between nodes i and j, r ij Represents resistance, x ij Represents reactance, represents the voltage amplitude of node j at time t, represents the tap ratio of the transformer on the line between nodes i and j, Represents the active power and reactive power on line i and line j respectively, x ij and r ij Represent the resistance and reactance values ​​on lines i and j respectively;

[0015] The CBS model is:

[0016]

[0017] In the formula, Represents the binary state of the capacitor, M cap Indicates the maximum reactive power output, represents the voltage amplitude of node i at time t, Indicates the reactive output of the capacitor bank at time t;

[0018] The OLTC ratio of the on-load tap changer and the capacitor bank CBS status are:

[0019]

[0020] Where vector k represents the turns ratio of all devices integrated over time, vector h represents the binary state of the integrated capacitor bank, and the target includes voltage regulation f v , power factor correction f PR and minimize the network loss f L ,λ V Denotes the weight coefficient of the deviation, λ PR Weighting factor for power factor correction, λ L Represents the weight coefficient of network loss, λ u represents the penalty weight for the change in OLTC ratio, λ h It represents the penalty weight for the capacitor state change, k t , k t-1 denote the transformation ratio of the tap at time t and t-1 respectively, h t ,h t-1 Represent the state quantities of the capacitor at time t and t-1 respectively.

[0021] Furthermore, the aforementioned objective function is as follows:

[0022] Min.∑P loss,i ;

[0023] Where P loss,i represents the line loss on the i-th branch;

[0024] The constraints for safe operation of the distribution network include: power flow equality constraints and voltage amplitude inequality constraints, as shown in the following formula:

[0025] The power flow equation constraint is as follows:

[0026]

[0027] Where N is the total number of nodes in the distribution network, G ij and B ij Represent the real and imaginary parts of the node admittance, V i and Vj is the node voltage amplitude, δ i and δ j Represents the phase angle difference of the node voltage;

[0028] The voltage amplitude inequality constraint is as follows:

[0029] V min ≤V i ≤V max i∈N

[0030] Where V min and V max are the upper and lower limits of the node voltage respectively.

[0031] Furthermore, in the aforementioned step S2, the state space is specifically: for the distribution network determined by the topology parameters, define the environmental state s perceived by the agent at time t t As follows:

[0032]

[0033] In the formula, is the net active power injected into node i at time t, including the sum of load and power supply; The net reactive power injected into the node; is the voltage amplitude of node i at time t; the state set S at time t is (s1, s2, …s N ), where N is the total number of nodes.

[0034] The action space is specifically: the reactive output of the on-load tap-changer OLTC and the reactive output of the capacitor bank CBS, the action a of the agent at time t t for:

[0035] a t =[Q SVC,1 ,Q SCV,2 ,…,Q SVC,N ,Q DG,1 ,Q DG,2 ,…,Q DG,N ].

[0036] The reward function is:

[0037] In the formula, the reward function consists of the main line reward and the auxiliary reward, where the main line reward is the network loss cost, C e is the cost coefficient; the auxiliary reward is the penalty for voltage crossing the limit, U t,k is the voltage amplitude at time t, η is the voltage violation penalty coefficient, U min is the minimum voltage amplitude, U max is the maximum voltage amplitude, σ is the activation function;

[0038] The state transfer process specifically includes the following steps:

[0039] S2.1, the agent based on the current environment state s t , and the reward function r t Select the best action a t ,

[0040] S2.2. The optimal action a t Input into the power system environment to perform power flow calculation and update the status s t＝ s t+1 and the reward function r t＝ r t+1 ,

[0041] S2.3. Return to execute S2.1.

[0042] Furthermore, in the aforementioned step S3, the topological information of the distribution network extracted by the graph neural network GNN specifically extracts the features of each node of the distribution network system and then updates it as follows:

[0043]

[0044] Where: H represents the feature set; σ represents the activation function, f is the aggregation function, is a collection of nodes, represents the characteristics of the current node v, Represents the characteristics of neighbor node u; A uv It is an element in the adjacency matrix, describing the connection relationship between node u and node v.

[0045] Furthermore, the objective function of the distribution network voltage reactive power optimization network DQN in the aforementioned step S3 is:

[0046]

[0047] Where Q(s, a; θ) represents the output of the distribution network voltage and reactive power optimization network DQN, r(s, a) represents the timely reward, θ represents the parameters of the distribution network voltage and reactive power optimization network DQN, and θ' represents the parameters of the target distribution network voltage and reactive power optimization network DQN.

[0048] Furthermore, in the aforementioned step S3, using the GNN-DQN combined algorithm to train the grid voltage reactive power optimization network includes the following steps:

[0049] S3.1, Initialize network parameters θ1, θ2,

[0050] S3.2, obtaining the topological information of the distribution network of the intelligent agent, including net active power, net reactive power, and voltage amplitude;

[0051] S3.3, randomly generate action a or

[0052] S3.4, by state s t Transfer to the next state s t+1 ~p(s t+1 |s t ,a t ) and reward r t , and then put it into the experience pool;

[0053] S3.5. The value network Critic calculates the target Q value as follows:

[0054]

[0055] S3.6, then update the policy network Actor according to the target Q value:

[0056]

[0057] Compared with the prior art, the beneficial technical effects of the above technical solutions adopted in the present invention are as follows: the method of the present invention gets rid of the dependence on the precise physical model, and the solution obtained is a strategy model rather than a specific solution. The model is a function fitted by a deep neural network, has strong generalization ability, and can adapt to the disturbance of the problem. In addition, the GNN-DQN algorithm used in the present invention has stronger exploration ability and generalization ability than the algorithms in other inventions (such as DQN, DDPG), and is more conducive to learning new tasks. GNN-DQN significantly improves the stability of the learning process and the accuracy of the decision-making process compared to the use of the DQN algorithm alone. In addition, the use of the GNN-DQN algorithm for controlling fast-acting discrete devices (OLTC and capacitor banks CBS) in the distribution network has not been applied so far, and is the originality of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is the interaction process between the agent and the environment in the Markov decision process.

[0059] Figure 2 This is the offline training and online execution process of the proposed model.

[0060] Figure 3 This is the overall process of GNN-DQN algorithm for reactive power voltage optimization. DETAILED DESCRIPTION

[0061] In order to better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0062] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0063] refer to Figure 3 The present invention provides a distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning, comprising the following steps:

[0064] S1. Extract the topological information of the distribution network through the graph neural network GNN, build a voltage regulation equipment model, take the minimum line loss as the objective function, establish a continuous reactive power optimization model of the distribution network, and add constraints for the safe operation of the distribution network;

[0065] S2. Based on the continuous reactive power optimization model of the distribution network, the reactive power optimization problem is described as a Markov decision process, and a distribution network voltage reactive power optimization network DQN based on the deep Q network is constructed, including: taking the voltage regulating device as the decision-making agent, and defining the state space, agent action space, reward function, and state transfer process.

[0066] S3. Use the GNN-DQN combined algorithm to train the grid voltage and reactive power optimization network to obtain the distribution network voltage and reactive power optimization model; specifically: the graph neural network GNN extracts the topological information of the distribution network to generate the state of each node and aggregates these states into the state of the distribution network voltage and reactive power optimization network DQN. The voltage and reactive power optimization network DQN estimates the Q value of each action based on the state and outputs the optimal action; the distribution network voltage and reactive power optimization network DQN includes a strategy network Actor and a value network Critic, and the value network Critic is used to supervise the training of the strategy network Actor.

[0067] S4. Input the real-time collected distribution network topology information into the distribution network voltage and reactive power optimization model to obtain the distribution network voltage and reactive power optimization strategy, that is, the output of the voltage regulating equipment.

[0068] In step S1, the constraints for safe operation of the distribution network mainly include equality constraints and inequality constraints. The constraint model is as follows: The constraints for safe operation of the distribution network include: power flow equality constraints and voltage amplitude inequality constraints, which are specifically as follows:

[0069] (1) The power flow equation constraint is as follows:

[0070]

[0071] Where N is the total number of nodes in the distribution network, G ij and B ij Represent the real and imaginary parts of the node admittance, V i and V j is the node voltage amplitude, δ i and δ j Represents the phase angle difference of the node voltage;

[0072] (2) Voltage amplitude inequality constraint, as shown below:

[0073] V min ≤V i ≤V max i∈N

[0074] Where V min and V max are the upper and lower limits of the node voltage respectively, which are also used as the penalty term of the auxiliary reward function later.

[0075] The voltage regulating equipment model of the present invention includes: an on-load voltage regulating transformer tap changer OLTC model, and a capacitor bank CBS model, an on-load voltage regulating transformer tap changer OLTC ratio and a capacitor bank CBS state;

[0076] The OLTC model is:

[0077]

[0078] In the formula, represents the effective value of the current in the line between nodes i and j, r ij Represents resistance, x ij Represents reactance, represents the voltage amplitude of node j at time t, represents the tap ratio of the transformer on the line between nodes i and j, Represents the active power and reactive power on line i and line j respectively, x ij and r ij Represent the resistance and reactance values ​​on lines i and j respectively.

[0079] The CBS model is:

[0080]

[0081] In the formula, Represents the binary state of the capacitor, M cap Indicates the maximum reactive power output, represents the voltage amplitude of node i at time t, Indicates the reactive output of the capacitor bank at time t.

[0082] In order to improve the voltage distribution and reactive power level, an optimization framework is designed. The framework sets the OLTC ratio and the state of the capacitor bank CBS at discrete time points t = 1, 2, ... The OLTC ratio of the on-load tap changer and the state of the capacitor bank CBS are:

[0083]

[0084] Where vector k represents the turns ratio of all devices integrated over time, vector h represents the binary state of the integrated capacitor bank, and the target includes voltage regulation f v , power factor correction f PR and minimize the network loss f L ,λ V Denotes the weight coefficient of the deviation, λ PR Weighting factor for power factor correction, λ L Represents the weight coefficient of network loss, λ u represents the penalty weight for the change in OLTC ratio, λ h It represents the penalty weight for the capacitor state change, k t , k t-1 denote the transformation ratio of the tap at time t and t-1 respectively, h t ,h t-1 Represent the state quantities of the capacitor at time t and t-1 respectively.

[0085] The objective function is as follows:

[0086] Min.∑P loss ;

[0087] Where P loss Indicates line loss;

[0088] Step S2, such as Figure 1 The interaction process between the agent and the environment shown can be interpreted as at each time step t, the agent is in state s t , take action a t , and then according to the state transition probability p(s t+1 |s t ,a t ) Enter the next state s t+1 , and obtain rewards r(s) from the environment t ,a t ), the goal of reinforcement learning is to find the optimal policy π* so that the expected discounted cumulative return starting from any state or state-action combination is Maximize, where γ∈(0,1) is the discount factor. The above process is called MDP, which includes the 5-tuple {s t ,a tr t ,s t+1 ,γ}. Below we transform the reactive voltage problem of the distribution network into a Markov decision process. The graph neural network (GNN-QL) algorithm is introduced, which solves the Volt / Var control problem by combining the graph neural network state with the traditional system state, which is an advancement of reinforcement learning technology. The proposed pseudocode describes the iterative process of learning the optimal strategy for decision making in a complex environment with network entities.

[0089] In step S2, the state space is specifically: for the distribution network determined by the topology parameters, define the environmental state s perceived by the agent at time t t As follows:

[0090]

[0091] In the formula, is the net active power injected into node i at time t, including the sum of load and power supply; The net reactive power injected into the node; is the voltage amplitude of node i at time t; all nodes of the distribution network are observable, and the state set S at time t is (s1, s2, …s N ), where N is the total number of nodes.

[0092] The action space is specifically: the reactive output of the on-load tap-changer OLTC and the reactive output of the capacitor bank CBS, the action a of the agent at time t t for:

[0093] a t =[Q SVC,1 ,Q SCV,2 ,…,Q SVC,N ,Q DG,1 ,Q DG,2 ,…,Q DG,N ].

[0094] The reward function is:

[0095] In the formula, the reward function consists of the main line reward and the auxiliary reward, where the main line reward is the network loss cost, C e is the cost coefficient; the auxiliary reward is the penalty for voltage crossing the limit, U t,k is the voltage amplitude at time t, η is the voltage violation penalty coefficient, U min is the minimum voltage amplitude, U max is the maximum voltage amplitude, σ is the activation function;

[0096] The state transfer process specifically includes the following steps:

[0097] S2.1, the agent based on the current environment state s t , and the reward function r t Select the best action a t ,

[0098] S2.2. The optimal action a t Input into the power system environment to perform power flow calculation and update the status s t＝ s t+1 and the reward function r t＝ r t+1 ,

[0099] S2.3. Return to execute S2.1.

[0100] It is through such interactions that the intelligent agent achieves real-time control and thus achieves the goal of improving power flow distribution.

[0101] Graph Neural Network (GNN) is a type of neural network that can process graph structured data. It learns the representation of nodes through information transmission and aggregation between nodes, and is often used to analyze the topological information of the graph structure and the characteristics of the nodes in the graph. In the distribution network scenario, the nodes of the graph can represent the bus or load in the system, and the edges represent the physical connections between the nodes (such as transmission lines).

[0102] A graph G consists of a node set V and an edge set E, denoted as G(V,E), where each node can be represented by a feature vector hv. The adjacency matrix A is used to represent the connection relationship between nodes. If there is an edge between node i and node j, then Aij = 1, otherwise Aij = 0.

[0103] refer to Figure 2 In step S3, the topological information of the distribution network extracted by the graph neural network GNN is to extract the characteristics of each node in the distribution network system and then update it as follows:

[0104]

[0105] Where: H represents the feature set; σ represents the activation function, f is the aggregation function, is a collection of nodes, represents the characteristics of the current node v, Represents the characteristics of neighbor node u; A uv It is an element in the adjacency matrix, describing the connection relationship between node u and node v.

[0106] The role of GNN in the GNN-DQN combination algorithm. GNN is used to process the graph structure information of the distribution network, extract the characteristics of each node in the system, and combine these characteristics with the traditional system state. GNN extracts the representation of each node from the connection relationship and node characteristics of the graph, and inputs the extracted graph structure information into the DQN network to estimate the Q value corresponding to each state-action.

[0107] In the DQN algorithm, the agent learns an action-value function Q(s,a)Q(s,a)Q(s,a), which represents the future cumulative reward that can be obtained by performing action aaa in a given state sss. The core of DQN is to use a deep neural network to approximate this Q-value function. By using mechanisms such as experience replay and target network, DQN can effectively stabilize the training process.

[0108] Step S3 The objective function of the distribution network voltage reactive power optimization network DQN is:

[0109]

[0110] Where Q(s, a; θ) represents the output of the distribution network voltage and reactive power optimization network DQN, r(s, a) represents the timely reward, θ represents the parameters of the distribution network voltage and reactive power optimization network DQN, and θ' represents the parameters of the target distribution network voltage and reactive power optimization network DQN.

[0111] In step S3, the grid voltage reactive power optimization network training using the GNN-DQN combined algorithm includes the following steps:

[0112] S3.1, Initialize network parameters θ1, θ2,

[0113] S3.2, obtaining the topological information of the distribution network of the intelligent agent, including net active power, net reactive power, and voltage amplitude;

[0114] S3.3, randomly generate action a or a t ~π φ (a t |s t ),

[0115] S3.4, by state s t Transfer to the next state s t+1 ~p(s t+1 |s t ,a t ) and reward r t , and then put it into the experience pool;

[0116] S3.5. The value network Critic calculates the target Q value as follows:

[0117]

[0118] S3.6, then update the policy network Actor according to the target Q value:

[0119]

[0120] In the GNN-DQN algorithm, the topological features and state information extracted by GNN are passed to DQN as input. The GNN module generates state representations for each node and aggregates these representations into the state representation of the entire system as input to DQN. Based on these state representations, DQN estimates the Q value of each possible action and selects the optimal action for control.

[0121] According to the process of the GNN-DQN algorithm and the update formula of the above parameters, the offline training process of the GNN-DQN algorithm can be summarized as shown in Table 1:

[0122] Table 1

[0123]

[0124]

[0125] According to the proposed GNN-DQN algorithm, the network of the intelligent agent is constructed, and then the intelligent agent is trained offline based on the historical operation data of the power grid, and then executed online. Figure 2 Its training and execution process is demonstrated.

[0126] For the DRL method, the reward and cost are derived from the line losses and node voltages calculated from the power flow simulation. The parameter settings of the GNN-DQN algorithm are provided in Table 2 below.

[0127]

[0128] Finally, the topological information of the real-time distribution network is input into the distribution network voltage and reactive power optimization model to obtain the distribution network voltage and reactive power optimization strategy, that is, the output of the voltage regulating equipment

[0129] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning, characterized in that: The following steps are involved: S1. Extract the topological information of the distribution network through the graph neural network GNN, build a voltage regulation equipment model, take the minimum line loss as the objective function, establish a continuous reactive power optimization model of the distribution network, and add constraints for the safe operation of the distribution network; S2. Based on the continuous reactive power optimization model of the distribution network, the reactive power optimization problem is described as a Markov decision process, and a distribution network voltage reactive power optimization network DQN based on the deep Q network is constructed, including: taking the voltage regulating device as the decision-making agent, and defining the state space, agent action space, reward function, and state transfer process. S3. Use the GNN-DQN combined algorithm to train the power grid voltage and reactive power optimization network to obtain the distribution network voltage and reactive power optimization model; specifically: the topological information of the distribution network extracted by the graph neural network GNN generates the state of each node and aggregates these states into the state of the distribution network voltage and reactive power optimization network DQN, and the voltage and reactive power optimization network DQN estimates the Q value of each action based on the state and outputs the optimal action; the distribution network voltage and reactive power optimization network DQN includes a strategy network Actor and a value network Critic, and the value network Critic supervises the training of the strategy network Actor by updating the Q value. S4. Input the real-time collected distribution network topology information into the distribution network voltage and reactive power optimization model to obtain the distribution network voltage and reactive power optimization strategy, that is, the output of the voltage regulating equipment.

2. A distribution network voltage and reactive power optimization method integrating graph neural network and reinforcement learning according to claim 1, characterized in that: In step S1, the voltage regulating equipment model includes: an on-load voltage regulating transformer tap changer OLTC model, and a capacitor bank CBS model, an on-load voltage regulating transformer tap changer OLTC ratio and a capacitor bank CBS state; The OLTC model is: In the formula, represents the effective value of the current in the line between nodes i and j, r ij Represents resistance, x ij Represents reactance, represents the voltage amplitude of node j at time t, represents the tap ratio of the transformer on the line between nodes i and j, Represents the active power and reactive power on line i and line j respectively, x ij and r ij Represent the resistance and reactance values ​​on lines i and j respectively; The CBS model is: In the formula, h t i Represents the binary state of the capacitor, M cap Indicates the maximum reactive output, V i t represents the voltage amplitude of node i at time t, Indicates the reactive output of the capacitor bank at time t; The OLTC ratio of the on-load tap changer and the capacitor bank CBS status are: Where vector k represents the turns ratio of all devices integrated over time, vector h represents the binary state of the integrated capacitor bank, and the target includes voltage regulation f v , power factor correction f PR and minimize the network loss f L ,λ V Denotes the weight coefficient of the deviation, λ PR Weighting factor for power factor correction, λ L Represents the weight coefficient of network loss, λ u represents the penalty weight for the change in OLTC ratio, λ h It represents the penalty weight for the change in capacitor state, k t , k t-1 denote the transformation ratio of the tap at time t and t-1 respectively, h t ,h t-1 Represent the state quantities of the capacitor at time t and t-1 respectively.

3. The method for optimizing voltage and reactive power of a distribution network by integrating graph neural network and reinforcement learning according to claim 1, characterized in that: The objective function is as follows: Min.∑P loss,i ; Where P loss,i represents the line loss on the i-th branch; The constraints for safe operation of the distribution network include: power flow equality constraints and voltage amplitude inequality constraints, as shown in the following formula: The power flow equation constraint is as follows: Where N is the total number of nodes in the distribution network, G ij and B ij Represent the real and imaginary parts of the node admittance, V i and V j is the node voltage amplitude, δ i and δ j Represents the phase angle difference of the node voltage; The voltage amplitude inequality constraint is as follows: In min ≤V i ≤V max i∈N Where V min and V max are the upper and lower limits of the node voltage respectively.

4. The method for optimizing voltage and reactive power of a distribution network by integrating graph neural network and reinforcement learning according to claim 3 is characterized in that: In step S2, the state space is specifically: for the distribution network determined by the topology parameters, define the environmental state s perceived by the agent at time t t As follows: In the formula, p i t is the net active power injected into node i at time t, including the sum of load and power supply; q i t The net reactive power injected into the node; v i t is the voltage amplitude of node i at time t; the state set S at time t is (s1, s2, …s N ), where N is the total number of nodes. The action space is specifically: the reactive output of the on-load tap-changer OLTC and the reactive output of the capacitor bank CBS, the action a of the agent at time t t for: a t =[Q SVC,1 ,Q SCV,2 ,…,Q SVC,N ,Q DG,1 ,Q DG,2 ,…,Q DG,N ]。 The reward function is: In the formula, the reward function consists of the main line reward and the auxiliary reward, where the main line reward is the network loss cost, C e is the cost coefficient; the auxiliary reward is the penalty for voltage crossing the limit, U t,k is the voltage amplitude at time t, η is the voltage violation penalty coefficient, U min is the minimum voltage amplitude, U max is the maximum voltage amplitude, σ is the activation function; The state transfer process specifically includes the following steps: S2.1, the agent based on the current environment state s t , and the reward function r t Select the best action a t , S2.

2. The optimal action a t Input into the power system environment to perform power flow calculation and update the status s t =s t+1 and the reward function r t =r t+1 , S2.

3. Return to execute S2.

1.

5. The method for optimizing voltage and reactive power of a distribution network by integrating graph neural network and reinforcement learning according to claim 3, characterized in that: In step S3, the topological information of the distribution network extracted by the graph neural network GNN is to extract the characteristics of each node in the distribution network system and then update it as follows: Where: H represents the feature set; σ represents the activation function, f is the aggregation function, is a collection of nodes, represents the characteristics of the current node v, Represents the characteristics of neighbor node u; A uv It is an element in the adjacency matrix, describing the connection relationship between node u and node v.

6. The method for optimizing voltage and reactive power of a distribution network by integrating graph neural network and reinforcement learning according to claim 3, characterized in that: Step S3 The objective function of the distribution network voltage reactive power optimization network DQN is: Where Q(s, a; θ) represents the output of the distribution network voltage and reactive power optimization network DQN, r(s, a) represents the timely reward, θ represents the parameters of the distribution network voltage and reactive power optimization network DQN, and θ' represents the parameters of the target distribution network voltage and reactive power optimization network DQN.

7. The method for optimizing voltage and reactive power of a distribution network by integrating graph neural network and reinforcement learning according to claim 3, characterized in that: In step S3, the grid voltage reactive power optimization network training using the GNN-DQN combined algorithm includes the following steps: S3.1, Initialize network parameters θ1, θ2, S3.2, obtaining the topological information of the distribution network of the intelligent agent, including net active power, net reactive power, and voltage amplitude; S3.3, randomly generate action a or a t ~π φ (a t |s t ), S3.4, by state s t Transfer to the next state s t+1 ~p(s t+1 |s t ,a t ) and reward r t , and then put it into the experience pool; S3.

5. The value network Critic calculates the target Q value as follows: S3.6, then update the policy network Actor according to the target Q value:

Images