Method and system for optimizing control strategy of power distribution system

Abstract

The invention discloses a power distribution system control strategy optimization method and system. The method comprises the steps of obtaining a graph structure of a power distribution network of a power distribution system; wherein the graph structure comprises nodes and edges; constructing an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network; according to the action space and the reward function, training the GNN and reinforcement learning network by adopting the graph structure to obtain the trained GNN and reinforcement learning network; and optimizing a control strategy of the power distribution system according to the trained GNN and the reinforcement learning network. According to the method, the GNN and reinforcement learning network method can be combined, more intelligent decision and strategy optimization can be obtained through the graph structure of the power distribution network, the source network load storage distributed optimal coordination and management in the power distribution system can be realized, the coordination and accuracy of the control strategy can be improved, the overall strategy optimization effect can be improved, and the method is suitable for the complex power system optimization problem.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and in particular to a method and system for optimizing a control strategy of a power distribution system. Background Art

[0002] In modern power systems, the efficiency and reliability of distribution networks have become a key issue. Traditional distribution networks have one-way power flow, highly centralized energy scheduling, and limited visualization capabilities, which make it difficult for the system to flexibly respond to changes in dynamic loads and the challenges of increasing distributed energy access. With the rapid development of renewable energy, energy storage technology, and electric vehicles, distribution networks are transforming from passive energy distribution systems to dynamic energy interaction platforms. At a time when distribution networks are undergoing profound changes, traditional static, centralized control methods can no longer adapt to the development needs of new distribution networks, and adaptive and intelligent optimization strategies have become particularly important. Summary of the invention

[0003] The present invention provides a method and system for optimizing the control strategy of a distribution system. By using the graph structure of the distribution network and the graph reinforcement learning method of GNN and DDPG, an adaptive and intelligent optimization strategy can be obtained to achieve the coordinated optimization management of the source, grid, load and storage distribution in the distribution system.

[0004] In order to achieve the above object, an embodiment of the present invention provides a method for optimizing a power distribution system control strategy, comprising:

[0005] Acquire a graph structure of a power distribution network of a power distribution system; wherein the graph structure includes nodes and edges;

[0006] constructing an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network;

[0007] According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the graph structure to obtain the trained GNN and the reinforcement learning network;

[0008] Based on the trained GNN and reinforcement learning network, the control strategy of the power distribution system is optimized.

[0009] As an improvement of the above solution, the step of obtaining the graph structure of the power distribution network of the power distribution system includes:

[0010] Acquire a topological structure of a distribution network of a power distribution system, define node attributes and edge attributes in the topological structure, and obtain a graph structure of the distribution network;

[0011] The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

[0012] As an improvement of the above scheme, the GNN and the reinforcement learning network are trained using the graph structure according to the action space and the reward function to obtain the trained GNN and reinforcement learning network, including:

[0013] Using GNN to identify the graph structure, and obtain a feature matrix of the graph structure;

[0014] According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and the reinforcement learning network.

[0015] As an improvement of the above solution, the reinforcement learning network includes a strategy network and a value network;

[0016] The policy network is used to output action selection according to the input feature matrix;

[0017] The value network is used to evaluate the state-action value under the current action selection.

[0018] As an improvement of the above solution, the action space includes the output adjustment strategy of the generator, the charging and discharging strategy of the energy storage device, and the demand response strategy of the load;

[0019] The reward function is to minimize the system operation cost while improving the stability and reliability of the system operation.

[0020] As an improvement of the above solution, the reward function includes the total cost of power generation and energy storage, voltage stability penalty and load unsatisfied penalty.

[0021] In order to achieve the above object, an embodiment of the present invention provides an optimization system for a power distribution system control strategy, including:

[0022] A network structure acquisition module, used to acquire a graph structure of a power distribution network of a power distribution system; wherein the graph structure includes nodes and edges;

[0023] An action reward construction module, used to construct an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network;

[0024] A learning network training module, used to train the GNN and the reinforcement learning network using the graph structure according to the action space and the reward function to obtain the trained GNN and the reinforcement learning network;

[0025] The control strategy optimization module is used to optimize the control strategy of the power distribution system according to the trained GNN and reinforcement learning network.

[0026] As an improvement of the above solution, the network structure acquisition module includes:

[0027] Acquire a topological structure of a distribution network of a power distribution system, define node attributes and edge attributes in the topological structure, and obtain a graph structure of the distribution network;

[0028] The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

[0029] As an improvement of the above solution, the learning network training module includes:

[0030] Using GNN to identify the graph structure, and obtain a feature matrix of the graph structure;

[0031] According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and the reinforcement learning network.

[0032] As an improvement of the above solution, the reinforcement learning network includes a strategy network and a value network;

[0033] The policy network is used to output action selection according to the input feature matrix;

[0034] The value network is used to evaluate the state-action value under the current action selection.

[0035] Compared with the prior art, the optimization method and system of the control strategy of a distribution system disclosed in the embodiment of the present invention obtains the graph structure of the distribution network of the distribution system; wherein the graph structure includes nodes and edges; according to the power characteristics of the distribution network, the action space and reward function of the distribution network are constructed; according to the action space and reward function, the GNN and reinforcement learning network are trained using the graph structure to obtain the trained GNN and reinforcement learning network; according to the trained GNN and reinforcement learning network, the control strategy of the distribution system is optimized. The distribution network can be represented as a graph structure, and the complex connection relationship and interaction influence can be captured in detail. Based on the use of GNN, the system can use the topological structure information to effectively extract and aggregate the characteristics of nodes and edges; through the propagation of information in the graph, the effective combination of global and local network states is achieved, which is helpful for more accurate state characterization; combined with GNN and reinforcement learning network methods, more intelligent decision-making and strategy optimization are obtained through the graph structure of the distribution network, and the optimal coordination and management of source, network, load and storage distribution in the distribution system is achieved, and the coordination and accuracy of the control strategy are improved, thereby improving the effect of overall strategy optimization, which is suitable for complex power system optimization problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1It is a flow chart of a method for optimizing a power distribution system control strategy provided by an embodiment of the present invention;

[0037] Figure 2 It is a structural schematic diagram of an optimization system for a power distribution system control strategy provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be noted that the terms "comprises" and "specifically" and any variations of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0040] See also Figure 1 , Figure 1 1 is a flow chart of a method for optimizing a power distribution system control strategy provided by an embodiment of the present invention. The method for optimizing a power distribution system control strategy includes:

[0041] S1, obtaining a graph structure of a power distribution network of a power distribution system; wherein the graph structure includes nodes and edges;

[0042] S2, constructing an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network;

[0043] S3, according to the action space and the reward function, using the graph structure to train the GNN and the reinforcement learning network to obtain the trained GNN and reinforcement learning network;

[0044] S4, optimizing the control strategy of the power distribution system according to the trained GNN and reinforcement learning network.

[0045] In one embodiment, MATLAB Simulink is used to build a simulation environment for a power distribution system, including modeling generators, loads, energy storage devices and their interconnected lines; defining environmental dynamics, setting state transfer functions and reward functions so that the environment can respond to actions and provide feedback; initializing parameters of GNN and DDPG networks, and initializing an experience replay buffer; at each time step, inputting the current state into the GNN to obtain a graph embedding Z; the Actor network inputs the graph embedding Z obtained from the GNN, generates corresponding actions, and applies them to the environment; recording the current state s, action a, reward r, and next state s′ into the experience replay buffer; the Critic network inputs the graph embedding Z of the current state s and the selected action a; calculating and evaluating the corresponding Q value; continuously evaluating the convergence of the strategy during the training process, adjusting training parameters and hyperparameters (such as learning rate, discount factor, etc.) to optimize the training effect, and after the training is completed, saving the trained GNN and reinforcement learning model.

[0046] Specifically, the step S1 includes:

[0047] S11, obtaining a topological structure of a distribution network of a power distribution system, defining node attributes and edge attributes in the topological structure, and obtaining a graph structure of the distribution network;

[0048] The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

[0049] For example, the distribution network is represented as an undirected graph G = (V, E), where V is a set of nodes, representing the devices in the distribution network; E is a set of edges, representing the transmission lines connecting these devices; for each node v∈V, its attribute x is defined as v :

[0050] x v =[V v ,I v ,P gen,v ,P load,v ,E v ] T ;

[0051] Among them, V v is the voltage at the node; I v is the current passing through the node; P gen,v is the power generation; P load,v is the load; E v is the energy storage level;

[0052] For each edge e = (v1, v2) ∈ E, define its attribute x e :

[0053] x e=[C e ,L e ,Z e ] T ;

[0054] Among them, C e Indicates the maximum transmission power capacity of the transmission line; L e Indicates the power loss during unit power transmission; Z e Indicates the electrical impedance characteristics of the line.

[0055] Specifically, the step S3 includes:

[0056] S31, using GNN to identify the graph structure and obtain a feature matrix of the graph structure;

[0057] S32, according to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and reinforcement learning network.

[0058] For example, GNN (Graph Neural Networks) is used to model the topological structure and characteristics of the power distribution system, and perform feature extraction and information propagation, as follows:

[0059] The GNN includes an input layer, a convolutional layer and an output layer. In the input layer of the GNN, the GNN receives the initial feature vectors of nodes and edges, including a node feature matrix and an edge feature matrix. The node feature matrix is: Edge feature matrix: Where N is the number of nodes; F is the dimension of node features; M is the number of edges; Q is the dimension of edge features;

[0060] The convolutional layer of GNN uses graph convolutional networks (GCN) to process input features, transfer and aggregate neighborhood information, and the adjacency matrix is ​​represented as follows: Normalized adjacency matrix: Where A is the adjacency matrix of the graph, which is used to describe the connectivity between nodes; D is the node degree matrix, and each diagonal element is the degree of the corresponding node;

[0061] Each GCN layer performs the following operations:

[0062]

[0063] Among them, H (l) is the node feature representation of the lth layer, and the initial state H (0) =X;W (l) is the trainable weight matrix of layer l; σ is the activation function;

[0064] In each GCN layer, nodes aggregate features from their neighbors to achieve global enhancement of local features;

[0065] The output layer of GNN generates representations of each node and the entire graph;

[0066] Z=H (L) ;

[0067] Where L is the number of layers of GCN; Z is the final node feature representation, and the representation of each node is obtained after the last layer of convolution;

[0068] By performing average pooling on each node representation, we get the feature vector of the entire graph:

[0069] z global =POOL(Z);

[0070] Among them, POOL represents a pooling operation.

[0071] Specifically, the reinforcement learning network includes a strategy network and a value network;

[0072] The policy network is used to output action selection according to the input feature matrix;

[0073] The value network is used to evaluate the state-action value under the current action selection.

[0074] Exemplarily, the reinforcement learning network DDPG (Deep Deterministic Policy Gradient) includes a policy network Actor and a value network Critic; the policy network Actor uses the graph embedding (feature matrix) generated by GNN as input, and the output includes action selection probability or strategy; the input layer of the policy network: receives the embedded representation Z of the node or graph generated by GNN; the network structure of the policy network: is a multilayer perceptron MLP (Multilayer Perceptron); the output layer of the policy network: outputs the action strategy, wherein the expression of the action strategy is:

[0075] μ(s|θ μ ) = MLP(Z);

[0076] Where s is the current state; θ μ are the parameters of the policy network;

[0077] The value network Critic estimates the state value or state-action value under the current strategy; the input layer of the value network receives the graph embedding Z of the state s and the action a; the network structure of the value network takes the state and action as joint input and processes through several hidden layers; the output layer of the value network outputs the state-action value, where the expression of the state-action value is:

[0078] Q(s,a|θ Q ) = MLP([Z,a]);

[0079] Among them, θ Q are the parameters of the value network;

[0080] Use the experience replay buffer to store interaction data and introduce the target network μ′ and Q′ to improve training stability:

[0081] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ );

[0082] Here, γ is the discount factor.

[0083] In this embodiment, the Actor and Critic networks are optimized using a joint parameter optimization method, as follows:

[0084] Optimize the policy network based on the policy gradient method, and use the gradient obtained from the Critic network to update the Actor network parameters:

[0085]

[0086] Maximizing the value function Q by optimizing the Actor network parameters θ μ ; and use the Bellman equation to optimize the Critic network, and use the target value to update the Critic network parameters, the target value is:

[0087] y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ );

[0088] Loss function:

[0089]

[0090] Minimize the loss function and optimize the estimated Q value:

[0091] Each time the experience is replayed, the Actor and Critic networks are updated alternately so that both are improved together;

[0092] To enhance training stability, use a slightly updated target network parameter approach:

[0093] θ′←τθ+(1-τ)θ′;

[0094] Where τ is the soft update coefficient.

[0095] In the embodiment of the present invention, by combining GNN and DDPG, the topological structure characteristics of the distribution network can be fully utilized to perform efficient and intelligent strategy optimization, thereby improving the operating efficiency and stability of the system. The graph embedding generated by GNN is used to improve the effectiveness of state representation. The strategy and value network in the Actor-Critic architecture are combined, and their respective optimizations improve the coordination and accuracy of the control strategy, thereby improving the effect of the overall strategy optimization, which is suitable for complex power system optimization problems.

[0096] Specifically, the action space includes the output adjustment strategy of the generator, the charging and discharging strategy of the energy storage device, and the demand response strategy of the load;

[0097] The reward function is to minimize the system operation cost while improving the stability and reliability of the system operation.

[0098] Specifically, the reward function includes the total cost of power generation and energy storage, voltage stability penalty, and load failure penalty.

[0099] In this embodiment, the reward function objective is to minimize the system operation cost and improve the stability and reliability of the system operation at the same time, as follows:

[0100] The total cost of power generation and energy storage C total :

[0101] C total =∑ i C gen,i (P gen,i )+∑ j C storage,j (a storage,j );

[0102] Voltage stability penalty S:

[0103] S=∑ v (|V v -V ref,v | 2 );

[0104] Load does not meet the penalty R:

[0105]

[0106] Combining the above into a total reward function:

[0107] L=-(α·C total +β·S+γ·R);

[0108] Among them, C gen,i (P gen,i ) is the cost function of the ith generator, C storage,j (a storage,j ) is the cost function of energy storage device j; V ref,v is the reference voltage at node v; is the load actually met; α, β, γ are weight coefficients.

[0109] The embodiment of the present invention discloses an optimization method for a distribution system control strategy, which obtains a graph structure of a distribution network of the distribution system; wherein the graph structure includes nodes and edges; constructs an action space and a reward function of the distribution network according to the power characteristics of the distribution network; trains a GNN and a reinforcement learning network according to the action space and the reward function to obtain the trained GNN and reinforcement learning network; optimizes the control strategy of the distribution system according to the trained GNN and reinforcement learning network. The distribution network can be represented as a graph structure, and the complex connection relationship and interaction influence can be captured in detail. Based on the use of GNN, the system can use the topological structure information to effectively extract and aggregate the features of nodes and edges; through the propagation of information in the graph, the effective combination of global and local network states is achieved, which helps to more accurately characterize the state; combined with GNN and reinforcement learning network methods, more intelligent decision-making and strategy optimization are obtained through the graph structure of the distribution network, and the optimal coordination and management of the source, network, load and storage distribution in the distribution system is achieved, and the coordination and accuracy of the control strategy are improved, thereby improving the effect of overall strategy optimization, which is suitable for complex power system optimization problems.

[0110] See also Figure 2 , Figure 2 1 is a schematic diagram of a structure of an optimization system 10 for a power distribution system control strategy provided by an embodiment of the present invention. The optimization system 10 for a power distribution system control strategy includes:

[0111] The network structure acquisition module 11 is used to acquire the graph structure of the power distribution network of the power distribution system; wherein the graph structure includes nodes and edges;

[0112] An action reward construction module 12, used to construct an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network;

[0113] A learning network training module 13 is used to train the GNN and the reinforcement learning network using the graph structure according to the action space and the reward function to obtain the trained GNN and the reinforcement learning network;

[0114] The control strategy optimization module 14 is used to optimize the control strategy of the power distribution system according to the trained GNN and reinforcement learning network.

[0115] Specifically, the network structure acquisition module 11 includes:

[0116] Acquire a topological structure of a distribution network of a power distribution system, define node attributes and edge attributes in the topological structure, and obtain a graph structure of the distribution network;

[0117] The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

[0118] Specifically, the learning network training module 13 includes:

[0119] Using GNN to identify the graph structure, and obtain a feature matrix of the graph structure;

[0120] According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and the reinforcement learning network.

[0121] Specifically, the reinforcement learning network includes a strategy network and a value network;

[0122] The policy network is used to output action selection according to the input feature matrix;

[0123] The value network is used to evaluate the state-action value under the current action selection.

[0124] An optimization system 10 for a power distribution system control strategy provided in an embodiment of the present invention can implement all processes of the optimization method for the power distribution system control strategy of the above-mentioned embodiment. The functions of each module in the system and the technical effects achieved are respectively the same as the functions of the optimization method for the power distribution system control strategy of the above-mentioned embodiment and the technical effects achieved, which will not be repeated here.

[0125] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for optimizing a power distribution system control strategy, characterized in that: include: Acquire a graph structure of a power distribution network of a power distribution system; wherein the graph structure includes nodes and edges; constructing an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network; According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the graph structure to obtain the trained GNN and the reinforcement learning network; Based on the trained GNN and reinforcement learning network, the control strategy of the power distribution system is optimized.

2. The method for optimizing the control strategy of a power distribution system according to claim 1, characterized in that: The obtaining of the graph structure of the power distribution network of the power distribution system comprises: Acquire a topological structure of a distribution network of a power distribution system, define node attributes and edge attributes in the topological structure, and obtain a graph structure of the distribution network; The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

3. The method for optimizing the control strategy of a power distribution system according to claim 1, characterized in that: The method of training the GNN and the reinforcement learning network using the graph structure according to the action space and the reward function to obtain the trained GNN and the reinforcement learning network includes: Using GNN to identify the graph structure, and obtain a feature matrix of the graph structure; According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and the reinforcement learning network.

4. The method for optimizing the control strategy of a power distribution system according to claim 1, characterized in that: The reinforcement learning network includes a strategy network and a value network; The policy network is used to output action selection according to the input feature matrix; The value network is used to evaluate the state-action value under the current action selection.

5. The method for optimizing the control strategy of a power distribution system according to claim 1, characterized in that: The action space includes the output adjustment strategy of the generator, the charging and discharging strategy of the energy storage device, and the demand response strategy of the load; The reward function is to minimize the system operation cost while improving the stability and reliability of the system operation.

6. The method for optimizing the control strategy of a power distribution system according to claim 5, characterized in that: The reward function includes the total cost of power generation and storage, a voltage stability penalty, and a load unsatisfied penalty.

7. An optimization system for a power distribution system control strategy, characterized in that: include: A network structure acquisition module, used to acquire a graph structure of a power distribution network of a power distribution system; wherein the graph structure includes nodes and edges; An action reward construction module, used to construct an action space and a reward function of the power distribution network according to the power characteristics of the power distribution network; A learning network training module, used to train the GNN and the reinforcement learning network using the graph structure according to the action space and the reward function to obtain the trained GNN and the reinforcement learning network; The control strategy optimization module is used to optimize the control strategy of the power distribution system according to the trained GNN and reinforcement learning network.

8. The optimization system for the power distribution system control strategy according to claim 7, characterized in that: The network structure acquisition module includes: Acquire a topological structure of a distribution network of a power distribution system, define node attributes and edge attributes in the topological structure, and obtain a graph structure of the distribution network; The nodes of the graph structure include generators, loads, and energy storage devices; and the edges of the graph structure are transmission lines.

9. The optimization system for power distribution system control strategy according to claim 7, characterized in that: The learning network training module comprises: Using GNN to identify the graph structure, and obtain a feature matrix of the graph structure; According to the action space and the reward function, the GNN and the reinforcement learning network are trained using the feature matrix to obtain the trained GNN and the reinforcement learning network.

10. The optimization system for power distribution system control strategy according to claim 7, characterized in that: The reinforcement learning network includes a strategy network and a value network; The policy network is used to output action selection according to the input feature matrix; The value network is used to evaluate the state-action value under the current action selection.

Images