Power distribution network voltage control method and system considering time sequence and topological characteristics

Abstract

The invention provides a power distribution network voltage control method and system considering time sequence and topological characteristics, and the method comprises the steps: carrying out the modeling of a power distribution network as a radial diagram, and building a voltage control mathematical model containing a photovoltaic penetration power distribution network; expressing a voltage control problem as a Markov decision problem; processing the time sequence and topological characteristics of the system by using a long short-term memory (LSTM) network and a weighted graph convolutional network (WGCN); and training an intelligent agent in the Markov decision problem by adopting a deep deterministic strategy gradient algorithm DDPG of reinforcement learning to obtain an optimal voltage control strategy. Through the method, the photovoltaic output trend is fully considered, the precision of voltage control is improved, and the training efficiency of the intelligent agent is optimized by integrating the topological structure of the power distribution network.

Description

Technical Field

[0001] This invention relates to the field of power distribution network technology, and more specifically to a power distribution network voltage control method and system that considers timing and topology characteristics. Background Technology

[0002] With the increasing severity of the global energy crisis and environmental problems, renewable energy has become a core direction for energy transformation in various countries. Among them, photovoltaic power generation has been widely used in distribution networks due to its green and low-carbon advantages. However, the volatility and uncertainty of photovoltaic power generation disrupt the unidirectional power flow characteristics of traditional distribution networks, leading to voltage limit exceedance problems. Traditional voltage control methods typically rely on accurate modeling of the distribution network, optimizing it through detailed network topology, parameters, and operating states. However, in real-world scenarios, due to the complexity and dynamic changes of distribution networks, such accurate models are often difficult to obtain. Therefore, reinforcement learning (RL), as a data-driven method, has gradually become an important tool in the field of voltage control because it does not rely on complex distribution network models and can adaptively learn optimal control strategies in dynamic environments. For example, Chinese patent application CN118693836A, "A Distribution Network Voltage Control Method and System," trains all agents based on voltage regulation maps and reinforcement learning algorithms to obtain the optimal control method. However, the aforementioned reinforcement learning methods fail to fully consider the impact of the distribution network topology on the agent's decision-making, and also neglect the agent's capture of photovoltaic output trend characteristics, thus limiting the performance and effectiveness of voltage control and resulting in low control accuracy. Summary of the Invention

[0003] The technical problem to be solved by this invention is to fully consider the photovoltaic output trend in the application scenario of photovoltaic grid connection to improve voltage control accuracy, and to optimize the training efficiency of the intelligent agent by comprehensively considering the grid topology.

[0004] The present invention solves the above-mentioned technical problems through the following technical means:

[0005] 1. A distribution network voltage control method based on timing and topology characteristics, used in photovoltaic grid connection scenarios, characterized in that it includes:

[0006] S1. Model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration.

[0007] S2. Formulate the voltage control problem as a Markov decision problem;

[0008] S3. Use Long Short-Term Memory (LSTM) network and Weighted Graph Convolutional Network (WGCN) to process the temporal and topological characteristics of the system;

[0009] S4. The agent in the above Markov decision problem is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm based on reinforcement learning to obtain the optimal voltage control strategy.

[0010] Furthermore, S1 includes the following steps:

[0011] S11. Model the distribution network as a radial diagram, where V = {0, 1, ..., N} and L = {1, 2, ..., N} represent sets with N+1 buses and N edges, respectively, and the transformer is connected to bus 0;

[0012] S12. Establish a mathematical model for photovoltaics using equation (1):

[0013]

[0014] In the formula, This represents the rated capacity of the photovoltaic system at bus i; This represents the active power generated by the photovoltaic system at bus i. This represents the reactive power generated by the photovoltaic system at bus i. This represents the maximum reactive power generated by the photovoltaic system at bus i. Let i represent the real-valued control variable for the photovoltaic system at bus i.

[0015] S13. Use equation (2) to establish the branch power flow balance equations of the distribution network:

[0016]

[0017] In the formula, This represents the active power consumed by the load at bus i; This represents the reactive power consumed by the load at bus i; v i and v i' θ represents the node voltage at bus i and bus i', respectively; ii' Indicates v i With v i' The phase angle between them; g ii' and b ii' Let i and i' represent the conductance and susceptance of the branch from bus i to bus i', respectively.

[0018] S14. The voltage control problem is expressed as equation (3):

[0019]

[0020] In the formula, v i,tvt represents the node voltage at bus i at time t; v0 represents the node voltage at bus 0, which is the reference voltage; β represents the weighting factor, used to measure the importance of line loss relative to voltage deviation. This represents the line loss on the l-th edge at time t; min indicates taking the minimum value.

[0021] Furthermore, S2 includes the following steps:

[0022] S21. Model the entire distribution network as an environment and divide it into K regions, denoted as D = {D1, D2, ... D}. j ...,D K}, D j Let j represent the j-th distribution network region, and let agent j be based on time t in D. j Information observed in the region j,t Control all photovoltaic power in this region;

[0023] S22, All local observations from agent 1 to agent I at time t {o 1,t ,o 2,t ,...,o I,t}Composition of environmental state s t , where the local observation value o of agent j at time t j,t It can be defined as equation (4):

[0024]

[0025] In the formula, and D at time t j The active and reactive power consumed by all loads within the area; and D at time t j The active and reactive power of all photovoltaic systems within the region; |v t | and θ t D at time t j The magnitude and phase angle of the voltage at all nodes within the region; D represents time t+1. j Trends in the active power output of all photovoltaic systems within the region;

[0026] S23. Action a of agent j at time t j,t It can be defined as equation (5):

[0027]

[0028] In the formula, U represents the location of agent j in D. j The total number of photovoltaic cells in the region; D represents the control of agent j.j The action of the k-th photovoltaic unit within the region, when At one time, photovoltaic power injects reactive power into the power distribution network; when At that time, photovoltaic power absorbs reactive power from the distribution network;

[0029] S24. The state transition process of the entire system from time t to t+1 is determined by the state transition function s. t+1 =T(s) t ,o 1:I,t ,a 1:I,t ,χ t The decision is determined by the environmental state s at time t. t Local observations of all agents at time t 1:I,t and action a 1:I,t and environmental uncertainty at time t χ t The impact;

[0030] S25. At time t, after agent j executes the action and completes the state transition, it will receive the corresponding reward r. j,t The reward function is designed as shown in equation (6):

[0031]

[0032] In the formula, ψ v (v i,t ) represents the voltage barrier function, where a, b, c, d, and f are all hyperparameters; |v i,t When -v0|>0.05, the deviation between the node voltage and the reference voltage in the distribution network is large, resulting in a significant penalty; |v i,t When -v0|≤0.05, the deviation between the node voltage and the reference voltage in the distribution network is small, and the penalty is calculated using a Gaussian function with mean v0 and standard deviation d.

[0033] Furthermore, S3 includes the following steps:

[0034] S31. Using equation (7), establish the forget gate f of the LSTM network at time t. t Input gate i t Memory gate c t and output gate o t :

[0035]

[0036] In the formula, x t Represents the input at time t; h t-1 This represents the hidden state at time t-1; W represents the candidate memory state at time t; f U fBoth represent the forgetting gate f at time t. t weights; b f The forget gate f represents time t. t The bias; W i U i Both represent the input gate i at time t. t weights; b i Indicates the input gate i at time t t The bias; W c U c Both represent time t. weights; b c Represents time t The bias; W o U o Both represent the output gate o at time t. t weights; b c Indicates the output gate o at time t t The bias; σ represents the sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the Hadamard product;

[0037] S32, D j Historical photovoltaic data for the first 96 time steps in the region The input is fed into the LSTM network, and the h obtained from equation (7) is processed using equation (8). t After passing through a fully connected layer, the photovoltaic active power output trend characteristics related to time t+1 are generated. And Integrate into the local observations of agent j;

[0038]

[0039] In the formula, W p Indicates the weights of the fully connected layer; b P Indicates the bias of the fully connected layer;

[0040] S33. Obtain the topology G of the connection relationships between agents in the distribution network, and simultaneously determine the shortest hop count dis of agent j and agent j' in the topology G. jj' ;

[0041] S34. Construct the input {X,A} of the WGCN network, where the feature matrix... Adjacency matrix Based on the shortest hop count dis between agent j and agent j' jj' Let the j-th row and j' column of the adjacency matrix A be defined as

[0042] S35. Set the total number of layers in the WGCN network to L, and the propagation rule is as follows: Where F (0) =X;F (L) W represents the output of the L-th layer WGCN network. (L) σ represents the weight matrix of the Lth layer WGCN network; σ represents the sigmoid function. Represents the identity matrix; express The degree matrix.

[0043] Furthermore, S4 includes the following steps:

[0044] S41. Construct an Actor network for agent j. A Critic network Among them o j Represents the local observations of agent j; a j ψ represents the output action of agent j; j and θ j These represent the parameters of the Actor network and the Critic network, respectively.

[0045] S42, Actor Network Based on Agent j and Critic Network Create each with ψ j 'Target Actor Network with parameters' With θ j 'Target Critic Network with parameters' And initialize its parameters to be the same as the original network, where o j ' represents the local observation of agent j at the next time step; a j 'Indicates the output action of agent j at the next moment;

[0046] S43. Add Gaussian noise to the output of the Actor network using equation (9).

[0047]

[0048] In the formula, ξ t Let $\mathbf{a}$ be the standard deviation of the Gaussian distribution.

[0049] S44. During the training process, each agent j acquires an experience {o} through interaction with the power distribution network environment. j,t ,a j,t ,r j,t ,o j,t+1} and store it in the shared experience buffer M;

[0050] S45. During the network update phase, a batch of samples containing M experiences is randomly sampled from the experience buffer M. The learning process involves assembling the observations of each agent j into a feature matrix. Input into WGCN, and use the aggregated observations of each agent j for updates to their respective Actor and Critic networks;

[0051] S46. Update the Actor network and Critic network using equations (10) and (11) respectively:

[0052]

[0053] In the formula, ζ ψ The learning rate of the Actor network. Indicates the relationship between ψ j Find the gradient;

[0054]

[0055] In the formula, ζ θ γ is the learning rate of the Critic network, and γ is the discount rate.

[0056] S47. The target Actor and Critic network are tracked step by step using equation (12):

[0057]

[0058] In the formula, For soft update parameters,

[0059] S48. Following the process in S43-S47, adjust the Actor network parameters ψ of agent j. j and Critic network parameters θ j Perform iterative training to obtain the optimal parameters of agent j. The corresponding optimal Actor network and optimal parameters The corresponding optimal Critic network is used to form the optimal agent j, thereby obtaining all the optimal agents and outputting the optimal voltage control strategy.

[0060] This invention also provides a distribution network voltage control system based on timing and topology characteristics, used in scenarios where photovoltaic power is connected to the grid. The system operates using the aforementioned method and includes the following modules:

[0061] The control model building module is used to model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration.

[0062] The decision model building module is used to formulate the voltage control problem as a Markov decision problem;

[0063] The feature processing module is used to process the temporal and topological features of the system using a Long Short-Term Memory (LSTM) network and a Weighted Graph Convolutional Network (WGCN).

[0064] The reinforcement learning module is used to train the agent in the above Markov decision problem using the Deep Deterministic Policy Gradient (DDPG) algorithm of reinforcement learning to obtain the optimal voltage control policy.

[0065] The advantages of this invention are:

[0066] (1) This invention utilizes a Long Short-Term Memory (LSTM) network to capture trend features in photovoltaic time series and generate key feature information for the future. These features are integrated into the observation space of the agent, enhancing its ability to perceive future changes in the system, thereby improving the accuracy and adaptability of the agent's control strategy in response to photovoltaic fluctuations.

[0067] (2) This invention introduces a weighted graph convolutional network (WGCN) to efficiently extract the topological structure and association characteristics between agents. By constructing a weighted adjacency matrix based on the number of hops in the topological connections, WGCN can more accurately capture the direct relationships and potential associations between agents. This design enables each agent to indirectly obtain the perception of the global environment even when relying only on local observations, which significantly improves the training efficiency of the decentralized framework. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the distribution network voltage control method based on timing and topology characteristics according to the present invention.

[0069] Figure 2 This is a schematic diagram illustrating the distribution network modeling environment of this invention;

[0070] Figure 3 This is a schematic diagram of the topology of intelligent agent connections in an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram illustrating the spatiotemporal three-dimensional dynamic representation of distribution network node voltage under a typical summer day scenario, according to an embodiment of the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Example 1

[0074] This embodiment provides a distribution network voltage control method based on timing and topology characteristics, applicable to scenarios where photovoltaic power is integrated into the grid, such as... Figure 1 As shown, it includes the following steps:

[0075] S1. Model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration. The specific implementation steps are as follows:

[0076] S11. Model the distribution network as a radial graph G = (V, L), where V = {0, 1, ..., N} and L = {1, 2, ..., N} represent sets with N+1 buses and N edges, respectively, and the transformer is connected to bus 0.

[0077] S12. Establish a mathematical model for photovoltaics using equation (1):

[0078]

[0079] In the formula, This represents the rated capacity of the photovoltaic system at bus i; This represents the active power generated by the photovoltaic system at bus i. This represents the reactive power generated by the photovoltaic system at bus i. This represents the maximum reactive power generated by the photovoltaic system at bus i. Let i represent the real-valued control variable for the photovoltaic system at bus i.

[0080] S13. Use equation (2) to establish the branch power flow balance equations of the distribution network:

[0081]

[0082] In the formula, This represents the active power consumed by the load at bus i; This represents the reactive power consumed by the load at bus i; v i and v i' θ represents the node voltage at bus i and bus i', respectively; ii' Indicates v i With v i' The phase angle between them; g ii' and b ii' Let i and i' represent the conductance and susceptance of the branch from bus i to bus i', respectively.

[0083] S14. The voltage control problem is expressed as equation (3):

[0084]

[0085] In the formula, v i,t vt represents the node voltage at bus i at time t; v0 represents the node voltage at bus 0, which is the reference voltage; β represents the weighting factor, used to measure the importance of line loss relative to voltage deviation. This represents the line loss on the l-th edge at time t; min indicates taking the minimum value.

[0086] S2. Formulate the voltage control problem as a Markov decision problem. The specific implementation steps are as follows:

[0087] S21. Model the entire distribution network as an environment and divide it into K regions, denoted as D = {D1, D2, ... D}. j ...,D K}, D j Let j represent the j-th distribution network region, and let agent j be based on time t in D. j Information observed in the region j,t Control all photovoltaic systems within the region; in this embodiment, such as... Figure 2 As shown, the entire power distribution network is divided into 5 zones.

[0088] S22, All local observations from agent 1 to agent I at time t {o 1,t ,o 2,t ,...,o I,t}Composition of environmental state s t , where the local observation value o of agent j at time t j,t It can be defined as equation (4):

[0089]

[0090] In the formula, and D at time t j The active and reactive power consumed by all loads within the area; and D at time t j The active and reactive power of all photovoltaic systems within the region; |v t | and θ t D at time t j The magnitude and phase angle of the voltage at all nodes within the region; D represents time t+1. j Trends in the active power output of all photovoltaic systems within the region;

[0091] S23. Action a of agent j at time t j,t It can be defined as equation (5):

[0092]

[0093] In the formula, U represents the location of agent j in D. j The total number of photovoltaic cells in the region; D represents the control of agent j. j The action of the k-th photovoltaic unit within the region, when At one time, photovoltaic power injects reactive power into the power distribution network; when At that time, photovoltaic power absorbs reactive power from the distribution network;

[0094] S24. The state transition process of the entire system from time t to t+1 is determined by the state transition function s. t+1 =T(s) t ,o 1:I,t ,a 1:I,t ,χ t The decision is determined by the environmental state s at time t. t Local observations of all agents at time t 1:I,t and action a 1:I,t and environmental uncertainty at time t χ t The impact;

[0095] S25. At time t, after agent j executes the action and completes the state transition, it will receive the corresponding reward r. j,t The reward function is designed as shown in equation (6):

[0096]

[0097] In the formula, ψ v (v i,t ) represents the voltage barrier function, where a, b, c, d, and f are all hyperparameters; |v i,t When -v0|>0.05, the deviation between the node voltage and the reference voltage in the distribution network is large, resulting in a significant penalty; |v i,t When -v0|≤0.05, the deviation between the node voltage and the reference voltage in the distribution network is small, and the penalty is calculated using a Gaussian function with mean v0 and standard deviation d.

[0098] S3. Use a Long Short-Term Memory (LSTM) network and a Weighted Graph Convolutional Network (WGCN) to process the temporal and topological characteristics of the system. The specific implementation steps are as follows:

[0099] S31. Using equation (7), establish the forget gate f of the LSTM network at time t. t Input gate i t Memory gate c t and output gate o t :

[0100]

[0101] In the formula, x t Represents the input at time t; h t-1 This represents the hidden state at time t-1; W represents the candidate memory state at time t; f U f Both represent the forgetting gate f at time t. t weights; b f The forget gate f represents time t. t The bias; W i U i Both represent the input gate i at time t. t weights; b i Indicates the input gate i at time t t The bias; W c U c Both represent time t. weights; b c Represents time t The bias; W o U o Both represent the output gate o at time t. t weights; b c Indicates the output gate o at time t t The bias; σ represents the sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the Hadamard product;

[0102] S32, D j Historical photovoltaic data for the first 96 time steps in the region The input is fed into the LSTM network, and the h obtained from equation (7) is processed using equation (8). t After passing through a fully connected layer, the photovoltaic active power output trend characteristics related to time t+1 are generated. And Integrate into the local observations of agent j;

[0103]

[0104] In the formula, W p Indicates the weights of the fully connected layer; b P Indicates the bias of the fully connected layer;

[0105] S33, such as Figure 3 As shown, the connection topology graph G of the agents in the distribution network is obtained, and the shortest hop count dis of agents j and j' in the topology graph G is determined. jj' ;

[0106] S34. Construct the input {X,A} of the WGCN network, where the feature matrix... Adjacency matrix Based on the shortest hop count dis between agent j and agent j' jj' Let the j-th row and j' column of the adjacency matrix A be defined as

[0107] S35. Set the total number of layers in the WGCN network to L, and the propagation rule is as follows: Where F (0) =X;F (L) W represents the output of the L-th layer WGCN network. (L) σ represents the weight matrix of the Lth layer WGCN network; σ represents the sigmoid function. Represents the identity matrix; express The degree matrix.

[0108] S4. The Deep Deterministic Policy Gradient (DDPG) algorithm, based on reinforcement learning, is used to train the agent in the Markov decision problem described above to obtain the optimal voltage control strategy. The specific implementation steps are as follows:

[0109] S41. Construct an Actor network for agent j. A Critic network Among them o j Represents the local observations of agent j; a j ψ represents the output action of agent j; j and θ j These represent the parameters of the Actor network and the Critic network, respectively.

[0110] S42, Actor network μ based on agent j ψj (o j ) and Critic network Create each with ψ j 'Target Actor Network with parameters' With θ j 'Target Critic Network with parameters' And initialize its parameters to be the same as the original network, where o j ' represents the local observation of agent j at the next time step; a j 'Indicates the output action of agent j at the next moment;

[0111] S43. Add Gaussian noise to the output of the Actor network using equation (9).

[0112]

[0113] In the formula, ξt Let $\mathbf{a}$ be the standard deviation of the Gaussian distribution.

[0114] S44. During the training process, each agent j acquires an experience {o} through interaction with the power distribution network environment. j,t ,a j,t ,r j,t ,o j,t+1} and store it in the shared experience buffer M;

[0115] S45. During the network update phase, a batch of samples containing M experiences is randomly sampled from the experience buffer M. The learning process involves assembling the observations of each agent j into a feature matrix. Input into WGCN, and use the aggregated observations of each agent j for updates to their respective Actor and Critic networks;

[0116] S46. Update the Actor network and Critic network using equations (10) and (11) respectively:

[0117]

[0118] In the formula, ζ ψ The learning rate of the Actor network. Indicates the relationship between ψ j Find the gradient;

[0119]

[0120] In the formula, ζ θ γ is the learning rate of the Critic network, and γ is the discount rate.

[0121] S47. The target Actor and Critic network are tracked step by step using equation (12):

[0122]

[0123] In the formula, For soft update parameters,

[0124] S48. Following the process in S43-S47, adjust the Actor network parameters ψ of agent j. j and Critic network parameters θ j Perform iterative training to obtain the optimal parameters of agent j. The corresponding optimal Actor network and optimal parameters The corresponding optimal Critic network is used to form the optimal agent j, thereby obtaining all the optimal agents and outputting the optimal voltage control strategy.

[0125] This embodiment also includes simulation experiments to demonstrate the above method. The power distribution network environment was built using Python, and all multi-agent reinforcement learning algorithms were implemented using the Python deep learning framework PyTorch. For the LSTM network used, the time step was 96, the number of stacked layers was three, and the number of neurons in each layer was 256, 128, and 64 respectively. The Adam optimizer was used, and the learning rate was 5×10⁻⁶. -3 The weighted graph convolutional network used has one stacked layer, 128 neurons, and employs the Adam optimizer with a learning rate of 4×10⁻⁶. -3 The reinforcement learning algorithm used was trained for 3000 epochs, with 480 time steps per epoch. The Actor and Critic networks in the algorithm were trained using the Adam optimizer with learning rates of 8×10⁻⁶. -4 and 8×10 -4 Soft update rate The discount rate γ = 0.95.

[0126] The provided algorithm was applied to an IEEE-33 node distribution network, and the simulation results are as follows: Figure 4 As shown, in a typical summer day scenario, it can be seen that the voltage of all nodes is within the range of 0.95pu to 1.05pu throughout the day.

[0127] Example 2

[0128] It should be further explained that, based on the same inventive concept, this invention also provides a distribution network voltage control system that considers timing and topology characteristics. The system operates using the method described in Example 1, and includes the following modules:

[0129] The control model construction module is used to model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration. The specific operation mode of the control model construction module is as follows:

[0130] (1) Model the distribution network as a radial graph G=(V,L), where V={0,1,...,N} and L={1,2,...,N} represent sets with N+1 buses and N edges, respectively, and the transformer is connected to bus 0;

[0131] (2) Establish a mathematical model of photovoltaics using equation (1):

[0132]

[0133] In the formula, This represents the rated capacity of the photovoltaic system at bus i; This represents the active power generated by the photovoltaic system at bus i. This represents the reactive power generated by the photovoltaic system at bus i. This represents the maximum reactive power generated by the photovoltaic system at bus i. Let i represent the real-valued control variable for the photovoltaic system at bus i.

[0134] (3) Use equation (2) to establish the branch power flow balance equations of the distribution network:

[0135]

[0136] In the formula, This represents the active power consumed by the load at bus i; This represents the reactive power consumed by the load at bus i; v i and v i' θ represents the node voltage at bus i and bus i', respectively; ii' Indicates v i With v i' The phase angle between them; g ii' and b ii' Let i and i' represent the conductance and susceptance of the branch from bus i to bus i', respectively.

[0137] (4) The voltage control problem is expressed as equation (3):

[0138]

[0139] In the formula, v i,t vt represents the node voltage at bus i at time t; v0 represents the node voltage at bus 0, which is the reference voltage; β represents the weighting factor, used to measure the importance of line loss relative to voltage deviation. This represents the line loss on the l-th edge at time t; min indicates taking the minimum value.

[0140] The decision model construction module is used to formulate the voltage control problem as a Markov decision problem. The specific operation of the decision model construction module is as follows:

[0141] (1) Model the entire distribution network as an environment and divide it into K regions, denoted as D = {D1, D2, ... D}. j ...,D K}, D j Let j represent the j-th distribution network region, and let agent j be based on time t in D. j Information observed in the region j,t Control all photovoltaic power in this region;

[0142] (2) All local observations {o} of agent 1 to agent I at time t 1,t ,o 2,t ,...,o I,t}Composition of environmental state s t , where the local observation value o of agent j at time t j,t It can be defined as equation (4):

[0143]

[0144] In the formula, and D at time t j The active and reactive power consumed by all loads within the area; and D at time t j The active and reactive power of all photovoltaic systems within the region; |v t | and θ t D at time t j The magnitude and phase angle of the voltage at all nodes within the region; D represents time t+1. j Trends in the active power output of all photovoltaic systems within the region;

[0145] (3) Action a of agent j at time t j,t It can be defined as equation (5):

[0146]

[0147] In the formula, U represents the location of agent j in D. j The total number of photovoltaic cells in the region; D represents the control of agent j. j The action of the k-th photovoltaic unit within the region, when At one time, photovoltaic power injects reactive power into the power distribution network; when At that time, photovoltaic power absorbs reactive power from the distribution network;

[0148] (4) The state transition process of the entire system from time t to time t+1 is determined by the state transition function s t+1 =T(s) t ,o 1:I,t ,a 1:I,t ,χ t The decision is determined by the environmental state s at time t. t Local observations of all agents at time t 1:I,t and action a 1:I,t and environmental uncertainty at time t χ t The impact;

[0149] (5) At time t, after agent j performs the action and completes the state transition, it will receive the corresponding reward r. j,t The reward function is designed as shown in equation (6):

[0150]

[0151] In the formula, ψ v (v i,t ) represents the voltage barrier function, where a, b, c, d, and f are all hyperparameters; |v i,t When -v0|>0.05, the deviation between the node voltage and the reference voltage in the distribution network is large, resulting in a significant penalty; |v i,t When -v0|≤0.05, the deviation between the node voltage and the reference voltage in the distribution network is small, and the penalty is calculated using a Gaussian function with mean v0 and standard deviation d.

[0152] The feature processing module is used to process the temporal and topological features of the system using a Long Short-Term Memory (LSTM) network and a Weighted Graph Convolutional Network (WGCN). The specific operation of the feature processing module is as follows:

[0153] (1) Using equation (7), establish the forget gate f of the LSTM network at time t. t Input gate i t Memory gate c t and output gate o t :

[0154]

[0155] In the formula, x t Represents the input at time t; h t-1 This represents the hidden state at time t-1; W represents the candidate memory state at time t; f U f Both represent the forgetting gate f at time t. t weights; b f The forget gate f represents time t. t The bias; W i U i Both represent the input gate i at time t. t weights; b i Indicates the input gate i at time t t The bias; W c U c Both represent time t. weights; b c Represents time t The bias; W o U o Both represent the output gate o at time t. t weights; b c Indicates the output gate o at time t t The bias; σ represents the sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the Hadamard product;

[0156] (2) D j Historical photovoltaic data for the first 96 time steps in the region The input is fed into the LSTM network, and the h obtained from equation (7) is processed using equation (8). t After passing through a fully connected layer, the photovoltaic active power output trend characteristics related to time t+1 are generated. And Integrate into the local observations of agent j;

[0157]

[0158] In the formula, W p Indicates the weights of the fully connected layer; b P Indicates the bias of the fully connected layer;

[0159] (3) Obtain the topology G of the connection relationship between agents in the distribution network, and determine the shortest hop count dis of agent j and agent j' in the topology G. jj' ;

[0160] (4) Construct the input {X,A} of the WGCN network, where the feature matrix... Adjacency matrix Based on the shortest hop count dis between agent j and agent j' jj' Let the j-th row and j' column of the adjacency matrix A be defined as

[0161] (5) Set the total number of layers in the WGCN network to L, and the propagation rule to be: Where F (0) =X;F (L) W represents the output of the L-th layer WGCN network. (L) σ represents the weight matrix of the Lth layer WGCN network; σ represents the sigmoid function. Represents the identity matrix; express The degree matrix.

[0162] The reinforcement learning module is used to train the agent in the Markov decision problem described above using the Deep Deterministic Policy Gradient (DDPG) algorithm to obtain the optimal voltage control policy. The specific operation of the reinforcement learning module is as follows:

[0163] (1) Construct an Actor network for agent j. A Critic network Among them o j Represents the local observations of agent j; aj ψ represents the output action of agent j; j and θ j These represent the parameters of the Actor network and the Critic network, respectively.

[0164] (2) Actor network based on agent j and Critic Network Create each with ψ j 'Target Actor Network with parameters' With θ j 'Target Critic Network with parameters' And initialize its parameters to be the same as the original network, where o j ' represents the local observation of agent j at the next time step; a j 'Indicates the output action of agent j at the next moment;

[0165] (3) Use equation (9) to add Gaussian noise to the output of the Actor network.

[0166]

[0167] In the formula, ξ t Let $\mathbf{a}$ be the standard deviation of the Gaussian distribution.

[0168] (4) During the training process, each agent j acquires an experience {o} through interaction with the power distribution network environment. j,t ,a j,t ,r j,t ,o j,t+1} and store it in the shared experience buffer M;

[0169] (5) During the network update phase, a batch of samples containing M experiences is randomly sampled from the experience buffer M. The learning process involves assembling the observations of each agent j into a feature matrix. Input into WGCN, and use the aggregated observations of each agent j for updates to their respective Actor and Critic networks;

[0170] (6) Update the Actor network and Critic network using equations (10) and (11) respectively:

[0171]

[0172] In the formula, ζ ψ The learning rate of the Actor network. Indicates the relationship between ψ j Find the gradient;

[0173]

[0174] In the formula, ζ θ γ is the learning rate of the Critic network, and γ is the discount rate.

[0175] (7) The target Actor and Critic network are tracked step by step using equation (12):

[0176]

[0177] In the formula, For soft update parameters,

[0178] (8) Following the process of (3)-(7), adjust the Actor network parameters ψ of agent j. j and Critic network parameters θ j Perform iterative training to obtain the optimal parameters of agent j. The corresponding optimal Actor network and optimal parameters The corresponding optimal Critic network is used to form the optimal agent j, thereby obtaining all the optimal agents and outputting the optimal voltage control strategy.

[0179] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distribution network voltage control method based on timing and topology characteristics, used in scenarios where photovoltaic power is connected to the grid, characterized in that, include: S1. Model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration. S2. Formulate the voltage control problem as a Markov decision problem; S3. Use Long Short-Term Memory (LSTM) network and Weighted Graph Convolutional Network (WGCN) to process the temporal and topological characteristics of the system; S4. The agent in the above Markov decision problem is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm based on reinforcement learning to obtain the optimal voltage control strategy.

2. The distribution network voltage control method based on timing and topology characteristics according to claim 1, characterized in that, S1 includes the following steps: S11. Model the distribution network as a radial graph G = (V, L), where V = {0, 1, ..., N} and L = {1, 2, ..., N} represent sets with N+1 buses and N edges, respectively, and the transformer is connected to bus 0. S12. Establish a mathematical model for photovoltaics using equation (1): In the formula, This represents the rated capacity of the photovoltaic system at bus i; This represents the active power generated by the photovoltaic system at bus i. This represents the reactive power generated by the photovoltaic system at bus i. This represents the maximum reactive power generated by the photovoltaic system at bus i. Let i represent the real-valued control variable for the photovoltaic system at bus i. S13. Use equation (2) to establish the branch power flow balance equations of the distribution network: In the formula, This represents the active power consumed by the load at bus i; This represents the reactive power consumed by the load at bus i; v i and v i' θ represents the node voltage at bus i and bus i', respectively; ii' Indicates v i With v i' The phase angle between them; g ii' and b ii' Let i and i' represent the conductance and susceptance of the branch from bus i to bus i', respectively. S14. The voltage control problem is expressed as equation (3): In the formula, v i,t vt represents the node voltage at bus i at time t; v0 represents the node voltage at bus 0, which is the reference voltage; β represents the weighting factor, used to measure the importance of line loss relative to voltage deviation. This represents the line loss on the l-th edge at time t; min indicates taking the minimum value.

3. The distribution network voltage control method based on timing and topology characteristics according to claim 2, characterized in that, S2 includes the following steps: S21. Model the entire distribution network as an environment and divide it into K regions, denoted as D = {D1, D2, ... D}. j ...,D K }, D j Let j represent the j-th distribution network region, and let agent j be based on time t in D. j Information observed in the region j,t Control all photovoltaic power in this region; S22, All local observations from agent 1 to agent I at time t {o 1,t ,o 2,t ,...,o I,t }Composition of environmental state s t , where the local observation value o of agent j at time t j,t It can be defined as equation (4): In the formula, and D at time t j The active and reactive power consumed by all loads within the area; and D at time t j The active and reactive power of all photovoltaic systems within the region; |v t | and θ t D at time t j The magnitude and phase angle of the voltage at all nodes within the region; D represents time t+1. j Trends in the active power output of all photovoltaic systems within the region; S23. Action a of agent j at time t j,t It can be defined as equation (5): In the formula, U represents the location of agent j in D. j The total number of photovoltaic cells in the region; D represents the control of agent j. j The action of the k-th photovoltaic unit within the region, when At one time, photovoltaic power injects reactive power into the power distribution network; when At that time, photovoltaic power absorbs reactive power from the distribution network; S24. The state transition process of the entire system from time t to t+1 is determined by the state transition function s. t+1 =T(s) t ,o 1:I,t ,a 1:I,t ,χ t The decision is determined by the environmental state s at time t. t Local observations of all agents at time t 1:I,t and action a 1:I,t and environmental uncertainty at time t χ t The impact; S25. At time t, after agent j executes the action and completes the state transition, it will receive the corresponding reward r. j,t The reward function is designed as shown in equation (6): In the formula, ψ v (v i,t ) represents the voltage barrier function, where a, b, c, d, and f are all hyperparameters; |v i,t When -v0|>0.05, the deviation between the node voltage and the reference voltage in the distribution network is large, resulting in a significant penalty; |v i,t When -v0|≤0.05, the deviation between the node voltage and the reference voltage in the distribution network is small, and the penalty is calculated using a Gaussian function with mean v0 and standard deviation d.

4. The distribution network voltage control method based on timing and topology characteristics according to claim 3, characterized in that, S3 includes the following steps: S31. Using equation (7), establish the forget gate f of the LSTM network at time t. t Input gate i t Memory gate c t and output gate o t : In the formula, x t Represents the input at time t; h t-1 This represents the hidden state at time t-1; W represents the candidate memory state at time t; f U f Both represent the forgetting gate f at time t. t weights; b f The forget gate f represents time t. t The bias; W i U i Both represent the input gate i at time t. t weights; b i Indicates the input gate i at time t t The bias; W c U c Both represent time t. rights Heavy; b c Represents time t The bias; W o U o Both represent the output gate o at time t. t weights; b c Indicates the output gate o at time t t The bias; σ represents the sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the Hadamard product; S32, D j Historical photovoltaic data for the first 96 time steps in the region The input is fed into the LSTM network, and the h obtained from equation (7) is processed using equation (8). t After passing through a fully connected layer, the photovoltaic active power output trend characteristics related to time t+1 are generated. And Integrate into the local observations of agent j; In the formula, W p Indicates the weights of the fully connected layer; b P Indicates the bias of the fully connected layer; S33. Obtain the topology G of the connection relationships between agents in the distribution network, and simultaneously determine the shortest hop count dis of agent j and agent j' in the topology G. jj' ; S34. Construct the input {X,A} of the WGCN network, where the feature matrix... Adjacency matrix Based on the shortest hop count dis between agent j and agent j' jj' Let the j-th row and j' column of the adjacency matrix A be defined as S35. Set the total number of layers in the WGCN network to L, and the propagation rule is as follows: ), where F (0) =X;F (L) W represents the output of the L-th layer WGCN network. (L) σ represents the weight matrix of the Lth layer WGCN network; σ represents the sigmoid function. Represents the identity matrix; express The degree matrix.

5. The distribution network voltage control method based on timing and topology characteristics according to claim 4, characterized in that, S4 includes the following steps: S41. Construct an Actor network for agent j. A Critic network Among them o j Represents the local observations of agent j; a j ψ represents the output action of agent j; j and θ j These represent the parameters of the Actor network and the Critic network, respectively. S42, Actor Network Based on Agent j and Critic Network Create each with ψ j 'Target Actor Network with parameters' With θ j 'Target Critic Network with parameters' And initialize its parameters to be the same as the original network, where o j ' represents the local observation of agent j at the next time step; a j 'Indicates the output action of agent j at the next moment; S43. Add Gaussian noise to the output of the Actor network using equation (9). In the formula, ξ t Let $\mathbf{a}$ be the standard deviation of the Gaussian distribution. S44. During the training process, each agent j acquires an experience {o} through interaction with the power distribution network environment. j,t ,a j,t ,r j,t ,o j,t+1 } and store it in the shared experience buffer M; S45. During the network update phase, a batch of samples containing M experiences is randomly sampled from the experience buffer M. The learning process involves assembling the observations of each agent j into a feature matrix. Input into WGCN, and use the aggregated observations of each agent j for updates to their respective Actor and Critic networks; S46. Update the Actor network and Critic network using equations (10) and (11) respectively: In the formula, ζ ψ The learning rate of the Actor network. Indicates the relationship between ψ j Find the gradient; In the formula, ζ θ γ is the learning rate of the Critic network, and γ is the discount rate. S47. The target Actor and Critic network are tracked step by step using equation (12): In the formula, For soft update parameters, S48. Following the process in S43-S47, adjust the Actor network parameters ψ of agent j. j and Critic network parameters θ j Perform iterative training to obtain the optimal parameters of agent j. The corresponding optimal Actor network and optimal parameters The corresponding optimal Critic network is used to form the optimal agent j, thereby obtaining all the optimal agents and outputting the optimal voltage control strategy.

6. A distribution network voltage control system that considers timing and topology characteristics, used in photovoltaic grid connection scenarios, characterized in that, Includes the following modules: The control model building module is used to model the distribution network as a radial diagram and establish a voltage control mathematical model for the distribution network containing photovoltaic penetration. The decision model building module is used to formulate the voltage control problem as a Markov decision problem; The feature processing module is used to process the temporal and topological features of the system using a Long Short-Term Memory (LSTM) network and a Weighted Graph Convolutional Network (WGCN). The reinforcement learning module is used to train the agent in the above Markov decision problem using the Deep Deterministic Policy Gradient (DDPG) algorithm of reinforcement learning to obtain the optimal voltage control policy.

7. The distribution network voltage control system based on timing and topology characteristics according to claim 6, characterized in that, The specific operation mode of the control model construction module is as follows: (1) Model the distribution network as a radial graph G=(V,L), where V={0,1,...,N} and L={1,2,...,N} represent sets with N+1 buses and N edges, respectively, and the transformer is connected to bus 0; (2) Establish a mathematical model of photovoltaics using equation (1): In the formula, This represents the rated capacity of the photovoltaic system at bus i; This represents the active power generated by the photovoltaic system at bus i. This represents the reactive power generated by the photovoltaic system at bus i. This represents the maximum reactive power generated by the photovoltaic system at bus i. Let i represent the real-valued control variable for the photovoltaic system at bus i. (3) Use equation (2) to establish the branch power flow balance equations of the distribution network: In the formula, This represents the active power consumed by the load at bus i; This represents the reactive power consumed by the load at bus i; v i and v i' θ represents the node voltage at bus i and bus i', respectively; ii' Indicates v i With v i' The phase angle between them; g ii' and b ii' Let i and i' represent the conductance and susceptance of the branch from bus i to bus i', respectively. (4) The voltage control problem is expressed as equation (3): In the formula, v i,t vt represents the node voltage at bus i at time t; v0 represents the node voltage at bus 0, which is the reference voltage; β represents the weighting factor, used to measure the importance of line loss relative to voltage deviation. This represents the line loss on the l-th edge at time t; min indicates taking the minimum value.

8. The distribution network voltage control system based on timing and topology characteristics according to claim 7, characterized in that, The specific operation mode of the decision model construction module is as follows: (1) Model the entire distribution network as an environment and divide it into K regions, denoted as D = {D1, D2, ... D}. j ...,D K }, D j Let j represent the j-th distribution network region, and let agent j be based on time t in D. j Information observed in the region j,t Control all photovoltaic power in this region; (2) All local observations {o} of agent 1 to agent I at time t 1,t ,o 2,t ,...,o I,t }Composition of environmental state s t , where the local observation value o of agent j at time t j,t It can be defined as equation (4): In the formula, and D at time t j The active and reactive power consumed by all loads within the area; and D at time t j The active and reactive power of all photovoltaic systems within the region; |v t | and θ t D at time t j The magnitude and phase angle of the voltage at all nodes within the region; D represents time t+1. j Trends in the active power output of all photovoltaic systems within the region; (3) Action a of agent j at time t j,t It can be defined as equation (5): In the formula, U represents the location of agent j in D. j The total number of photovoltaic cells in the region; D represents the control of agent j. j The action of the k-th photovoltaic unit within the region, when At one time, photovoltaic power injects reactive power into the power distribution network; when At that time, photovoltaic power absorbs reactive power from the distribution network; (4) The state transition process of the entire system from time t to time t+1 is determined by the state transition function s t+1 =T(s) t ,o 1:I,t ,a 1:I,t ,χ t The decision is determined by the environmental state s at time t. t Local observations of all agents at time t 1:I,t and action a 1:I,t and environmental uncertainty at time t χ t The impact; (5) At time t, after agent j performs the action and completes the state transition, it will receive the corresponding reward r. j,t The reward function is designed as shown in equation (6): In the formula, ψ v (v i,t ) represents the voltage barrier function, where a, b, c, d, and f are all hyperparameters; |v i,t When -v0|>0.05, the deviation between the node voltage and the reference voltage in the distribution network is large, resulting in a significant penalty; |v i,t When -v0|≤0.05, the deviation between the node voltage and the reference voltage in the distribution network is small, and the penalty is calculated using a Gaussian function with mean v0 and standard deviation d.

9. The distribution network voltage control system based on timing and topology characteristics according to claim 8, characterized in that, The specific operation mode of the feature processing module is as follows: (1) Using equation (7), establish the forget gate f of the LSTM network at time t. t Input gate i t Memory gate c t and output gate o t : In the formula, x t Represents the input at time t; h t-1 This represents the hidden state at time t-1; W represents the candidate memory state at time t; f U f Both represent the forgetting gate f at time t. t weights; b f The forget gate f represents time t. t The bias; W i U i Both represent the input gate i at time t. t weights; b i Indicates the input gate i at time t t The bias; W c U c Both represent time t. weights; b c Represents time t The bias; W o U o Both represent the output gate o at time t. t weights; b c Indicates the output gate o at time t t The bias; σ represents the sigmoid activation function; tanh represents the hyperbolic tangent activation function; Represents the Hadamard product; (2) D j Historical photovoltaic data for the first 96 time steps in the region The input is fed into the LSTM network, and the h obtained from equation (7) is processed using equation (8). t After passing through a fully connected layer, the photovoltaic active power output trend characteristics related to time t+1 are generated. And Integrate into the local observations of agent j; In the formula, W p Indicates the weights of the fully connected layer; b P Indicates the bias of the fully connected layer; (3) Obtain the topology G of the connection relationship between agents in the distribution network, and determine the shortest hop count dis of agent j and agent j' in the topology G. jj' ; (4) Construct the input {X,A} of the WGCN network, where the feature matrix... Adjacency matrix Based on the shortest hop count dis between agent j and agent j' jj' Let the j-th row and j' column of the adjacency matrix A be defined as (5) Set the total number of layers in the WGCN network to L, and the propagation rule to be: ), where F (0) =X;F (L) W represents the output of the L-th layer WGCN network. (L) σ represents the weight matrix of the Lth layer WGCN network; σ represents the sigmoid function. Represents the identity matrix; express The degree matrix.

10. The distribution network voltage control system based on timing and topology characteristics according to claim 9, characterized in that, The reinforcement learning module operates as follows: (1) Construct an Actor network for agent j. A Critic network Among them o j Represents the local observations of agent j; a j ψ represents the output action of agent j; j and θ j These represent the parameters of the Actor network and the Critic network, respectively. (2) Actor network based on agent j and Critic Network Create each with ψ j 'Target Actor Network with parameters' With θ j 'Target Critic Network with parameters' And initialize its parameters to be the same as the original network, where o j ' represents the local observation of agent j at the next time step; a j 'Indicates the output action of agent j at the next moment; (3) Use equation (9) to add Gaussian noise to the output of the Actor network. In the formula, ξ t Let $\mathbf{a}$ be the standard deviation of the Gaussian distribution. (4) During the training process, each agent j acquires an experience {o} through interaction with the power distribution network environment. j,t ,a j,t ,r j,t ,o j,t+1 } and store it in the shared experience buffer M; (5) During the network update phase, a batch of samples containing M experiences is randomly sampled from the experience buffer M. The learning process involves assembling the observations of each agent j into a feature matrix. Input into WGCN, and use the aggregated observations of each agent j for updates to their respective Actor and Critic networks; (6) Update the Actor network and Critic network using equations (10) and (11) respectively: In the formula, ζ ψ The learning rate of the Actor network. Indicates the relationship between ψ j Find the gradient; In the formula, ζ θ γ is the learning rate of the Critic network, and γ is the discount rate. (7) The target Actor and Critic network are tracked step by step using equation (12): In the formula, For soft update parameters, (8) Following the process of (3)-(7), adjust the Actor network parameters ψ of agent j. j and Critic network parameters θ j Perform iterative training to obtain the optimal parameters of agent j. The corresponding optimal Actor network and optimal parameters The corresponding optimal Critic network is used to form the optimal agent j, thereby obtaining all the optimal agents and outputting the optimal voltage control strategy.

Images