A high proportion photovoltaic power distribution network voltage edge control method

Abstract

The present application belongs to the field of active power distribution network operation control technology and edge intelligence, and proposes a voltage edge control method for high-proportion photovoltaic power distribution network, which comprises: calculating node voltage sensitivity, defining electrical distance between nodes, and partitioning the active power distribution network through an improved modularity function; determining the elements of agent environment, agent number, state space, action space, reward function and state transition function according to the network partition result, and constructing a distributed voltage edge control model based on Markov game process; designing a multi-agent graph reinforcement learning algorithm embedded with graph neural network to iteratively solve the distributed voltage edge control model, introducing a physical auxiliary mechanism to generate reference experience to assist the agent in rapid optimization, and obtaining a voltage control strategy; and formulating a voltage edge control framework for agent offline learning and online application based on cloud-edge collaboration architecture. The present application can enhance the perception ability of the agent to dynamic topology reconstruction and its learning efficiency.

Description

Technical Field

[0001] This invention relates to active distribution network operation control technology and edge intelligence, and particularly to a voltage edge control method for a high-proportion photovoltaic distribution network. Background Technology

[0002] Driven by the energy crisis and environmental pressures, the penetration rate of distributed energy sources in active distribution networks continues to increase. However, the inherent randomness, volatility, and indirectness of distributed energy sources cause serious problems such as voltage fluctuations, flicker, and sags in distribution networks, further exacerbating the risk of voltage exceeding limits and posing a significant threat to power quality and system security. Addressing the voltage exceeding-limit problems caused by the widespread integration of distributed energy sources (such as photovoltaics) urgently requires the exploration of relevant theoretical innovations and technological support.

[0003] While previous voltage control techniques based on reinforcement learning have demonstrated superior performance, they are typically designed for specific topologies or make necessary approximations for fixed topologies. However, in practical distribution networks, dynamic topology reconfiguration is increasingly becoming a crucial means of ensuring power supply and mitigating grid congestion. Voltage control models need to be redesigned to address new topology scenarios; therefore, considering topology flexibility is key to improving the generalization ability of control models. Furthermore, especially in large-scale distribution networks, reinforcement learning training is difficult and time-consuming, and its low learning efficiency limits its engineering applicability. Therefore, improving the topology adaptability and learning efficiency of voltage control models is of significant research importance in addressing these challenges. Summary of the Invention

[0004] The purpose of this invention is to provide a voltage edge control method for high-proportion photovoltaic distribution networks, which aims to solve the technical problems of poor topology adaptability and low learning efficiency of traditional voltage control methods based on reinforcement learning.

[0005] The technical solution adopted by this invention to solve its technical problem is as follows:

[0006] A voltage edge control method for a high-proportion photovoltaic distribution network includes the following steps:

[0007] Calculate node voltage-active and voltage-reactive sensitivity, define electrical distances between nodes, and partition the active distribution network using an improved modularity function;

[0008] Based on the network partitioning results, the elements of agent environment, number of agents, state space, action space, reward function and state transition function are determined, and a distributed voltage edge control model based on Markov game process is constructed.

[0009] A multi-agent graph reinforcement learning algorithm with embedded graph neural network is designed to iteratively solve the distributed voltage edge control model. A physical auxiliary mechanism is introduced to generate reference experience to help the agent to quickly find the optimization and obtain the voltage control strategy.

[0010] A voltage edge control framework for intelligent agents' offline learning and online application is developed based on a cloud-edge collaborative architecture.

[0011] As a further optimization, the calculation of node voltage-active and voltage-reactive sensitivity, and the definition of electrical distance between nodes, specifically include the following steps:

[0012] For a radial distribution network with n nodes, the voltage drop between any two nodes can be approximated as:

[0013] V i -V j =r ij P ij +x ij Q ij

[0014] Among them, 0 <i,j≤n,V i and V j P represents the voltages at nodes i and j, respectively. ij and Q ij These represent the active power and reactive power flowing through the branches between nodes i and j, respectively. ij and x ij These are the resistance and reactance of the branch between nodes i and j, respectively;

[0015] Represent the voltage drop between any two nodes in matrix form:

[0016] ΔV=V i -V j =RP+XQ

[0017] Where ΔV is the voltage drop between any two nodes, and P = [P1, P2, ..., P n ] T and Q = [Q1, Q2, ..., Q n ] T Let R be the active power and reactive power injection vectors at the node, respectively; R is an n-dimensional resistance matrix representing the relationship between node voltage and active power injection; and X is an n-dimensional reactance matrix representing the relationship between node voltage and reactive power injection.

[0018] Based on the definition of voltage sensitivity, the voltage-active and voltage-reactive sensitivities of each node are obtained as follows:

[0019]

[0020]

[0021] in, and These are the voltage-active and voltage-reactive sensitivities, respectively. The voltage sensitivity does not require the calculation of the inverse of the Jacobian matrix.

[0022] Based on the voltage sensitivity calculation results, the electrical distance between any two nodes is defined as:

[0023]

[0024] Where, d ij This is the electrical matrix between nodes i and j, used to reflect the degree of electrical coupling between nodes.

[0025] As a further optimization, the active distribution network is partitioned using an improved modularity function, specifically as follows:

[0026] By replacing the adjacency matrix with electrical distance as the weight matrix in the modularity function, an improved modularity function is obtained and the distribution network is partitioned, as expressed below:

[0027]

[0028] Where ρ is the modularity index, used to measure the effectiveness of distribution network zoning; the larger the value, the better the zoning effect. m is the sum of the weights of all branches, and k... i and k j Let δ be the sum of the number of branches connected to nodes i and j, respectively, and let δ be the Kronecker function. The optimal partitioning result of the distribution network can be obtained by calculating the system modularity index through multiple traversals.

[0029] As a further optimization, the process of determining the agent environment, number of agents, state space, action space, reward function, and state transition function based on the network partitioning results specifically includes:

[0030] The intelligent agent environment, specifically referring to the distribution network environment, is defined as a power flow calculation model. OpenDSS is used as an environment simulator to calculate the node voltage and network loss of the distribution network.

[0031] The number of agents is assumed to be that each sub-network in the network partitioning result is an independent and interactive agent, that is, the number of partitions is the same as the number of agents N. Each sub-network includes some distribution network nodes, photovoltaics, energy storage and static var compensators. The agents corresponding to each sub-network provide active and reactive power support to the system by adjusting the distributed energy storage and static var compensators, so as to ensure voltage safety constraints while optimizing system network losses.

[0032] The state space is categorized through the global state space S. t The observable information describing all agents in time slot t is defined as follows:

[0033]

[0034] Wherein, the global state space S t Including active load of all nodes reactive load Photovoltaic power output Energy storage state of charge (SOC) j,t And the adjacency matrix A representing the topology t Local observation space o t The observable information describing a single agent in time slot t is defined as follows:

[0035]

[0036] Among them, local observation o t Including the active load of all nodes in the subnetwork reactive load Photovoltaic power output Energy storage state of charge (SOC) j,t And the adjacency matrix A representing the topology i,t ;

[0037] The action space refers to the control variables of the voltage edge control model, and is defined as follows:

[0038]

[0039] The operational space includes the active power provided by energy storage. The reactive power provided by the static var compensator The actual active power output of energy storage satisfies the following relationship:

[0040]

[0041] in, and These represent the charge and discharge efficiencies, E max For battery capacity, SOC max and SOC min These are the upper and lower limits of the state of charge, respectively;

[0042] The reward function, which is the optimization objective of the voltage control problem, is defined as follows:

[0043]

[0044]

[0045] Where, r tLet T be the reward function, T be the number of optimization slots, N:={1,…,n} be the set of nodes, E:={(i,j)}∈N×N be the set of branches, and l ij,t r is the square of the branch current. ij Let η be the resistance of the branch between nodes i and j, η be the penalty coefficient, and c be the resistance of the branch between nodes i and j. t Let v be the penalty function, 1(·) be the indicator function, and v j,t The square of the node voltage. Let them be their upper and lower limits, respectively;

[0046] The state transition function allows the agent to randomly transition from its current state to the next state when interacting with the power grid environment.

[0047] As a further optimization, the constructed distributed voltage edge control model based on Markov game process is represented by multivariate tuples:

[0048] <N,S t ,[o t ] N ,[a t ] N ,[r t r] N ,p>

[0049] This tuple includes the number of agents N and the global state space S. t Local observation space t Action space a t Reward function r t And the state transition function p, in each time slot t, the agent adjusts the state based on the local observation space o. t Execution action space a t The environment transitions to the next time slot t+1, and the corresponding reward function r is returned to the agent. t and local observation o t+1 And so on, in a continuous cycle.

[0050] As a further optimization, when designing the multi-agent graph reinforcement learning algorithm that embeds graph neural networks, the graph neural network is embedded in reinforcement learning to help the agent capture the spatial correlation and topological dependence between nodes, so that the agent can perceive the topological changes caused by dynamic reconstruction in real time.

[0051] As a further optimization, a graph attention network is embedded in a multi-agent soft actor critic reinforcement learning algorithm. Each agent includes two value networks, two soft Q networks, and one policy network. Each network consists of several graph attention layers and fully connected layers stacked together.

[0052] As a further optimization, the core of the graph attention layer is the multi-head attention coefficient;

[0053] The relationship between the inputs and outputs of the value network is represented as follows:

[0054] V = σ out (W out σ in (∑αW in (S t ||a t )))

[0055] Wherein, the global state space S t and action space a t As input, || represents the concatenation operation, and W in Let σ be the weight matrix of the graph attention layer. in Let LeakyReLU be the activation function for the graph attention layer, α be the multi-head attention coefficient, and W be the multi-head attention coefficient. out Let σ be the weight matrix of the fully connected layer. in ReLU is the activation function for the fully connected layer, and the state value V is the network output, which is used to evaluate the quality of the current state and actions, that is, the impact of the active power output of energy storage and the reactive power output of static var compensator on the future of the power grid under the current distribution network state.

[0056] The relationship between the input and output of the Q network is expressed as follows:

[0057] Q = σ out (W out σ in (∑αW in a t ))

[0058] Wherein, the global state space S t The input is the action value Q, which is the network output used to evaluate the quality of the current action, i.e., the impact of the active power output of the energy storage and the reactive power output of the static var compensator on the current state of the distribution network.

[0059] The relationship between the input and output of the policy network is represented as follows:

[0060] a t =σ out (W out σ in (∑αW in o t ))

[0061] Among them, the local observation space o t For input, the agent's action space a t For network output.

[0062] As a further optimization, the design incorporates a multi-agent graph reinforcement learning algorithm embedded with a graph neural network to iteratively solve the distributed voltage edge control model. The iterative update process is as follows:

[0063] The value network parameters are updated by minimizing the mean squared error:

[0064]

[0065] Among them, J V (ψ) represents the value network parameter optimization error, E is the mathematical expectation operation, log is the logarithmic operation, and V ψ Q θ and π φ These are the state value function, action value function, and policy function, respectively; ψ, θ, and φ are the parameters of the network mentioned above; and D is the experience pool.

[0066] The soft Q network parameters are updated by minimizing the Bellman mean square error:

[0067]

[0068]

[0069] Among them, J Q (θ) represents the optimization error of the soft Q network parameters, and γ is the discount factor. The target value network is used for stable training. Its parameters;

[0070] The policy network is updated by minimizing the KL divergence:

[0071]

[0072] Among them, J π (φ) represents the optimization error of the policy network parameters. This is a reparameterized sampling action.

[0073] As a further optimization, the voltage edge control framework includes a cloud learning layer and an edge control layer;

[0074] The edge control layer: This layer belongs to the distributed control layer. First, the distribution network is divided into several sub-networks. These sub-networks are modeled as independent and interactive intelligent agents. It is assumed that each sub-network is equipped with a specific edge controller, which is responsible for controlling the energy storage and static var compensators to provide dynamic voltage support to the system. The intelligent agents perform online reasoning and decision-making at the network edge and upload their experience of interacting with the environment to the cloud main station platform for training the control model, thus offloading the reasoning and decision-making tasks to the edge.

[0075] The cloud-based learning layer: This layer belongs to the centralized control layer. The agent performs offline training in the cloud. The cloud main station platform includes the operation and management functions of the distribution network dispatch center. The agent receives the experience uploaded by the edge terminal and stores it in the experience pool. It periodically samples to learn and update the control strategy. Then, the training converged control model is sent to the edge controller to ensure the agent's generalization ability to different operating conditions of the distribution network and to transfer the computationally intensive and time-consuming training tasks to the cloud.

[0076] The beneficial effects of this invention are:

[0077] 1. By proposing a voltage edge control technology based on a cloud-edge collaborative architecture, energy storage and static var compensators are rationally scheduled to provide fast and flexible voltage support to a high proportion of distributed photovoltaic power distribution networks. This ensures that the voltage operates within a safe range while optimizing system network losses. The improved modular partitioning method does not require pre-setting the number of partitions and depends only on the distribution network topology, independent of the system operating status.

[0078] 2. Embedding the graph attention network into the multi-agent soft actor critic reinforcement learning algorithm helps the agent capture the spatial correlation and topological dependence between distribution network nodes, enabling the agent to perceive the topological changes caused by dynamic reconstruction in real time, enhancing the agent's perception and generalization ability of dynamic topological reconstruction, and avoiding the need to redesign the agent network structure under new topological scenarios.

[0079] 3. Introducing a physical assistance mechanism: A relatively accurate physical model is used to generate quantitative reference experience, which is stored in an experience pool. This assists the agent in quickly determining effective actions during training, improving the learning efficiency of voltage regulation strategies. The reference experience contains near-optimal voltage control strategies, which can guide the agent to quickly obtain the optimal control strategy while avoiding unnecessary trial and error and exploration in the power distribution network environment.

[0080] Therefore, the present invention can enhance the agent's ability to perceive dynamic topology reconfiguration, improve the recognition speed of effective actions and the learning efficiency of voltage control strategies, and ensure the safe and economical operation of active distribution networks. Attached Figure Description

[0081] Figure 1 This is a flowchart of a high-proportion photovoltaic power distribution network voltage edge control method in an embodiment of the present invention;

[0082] Figure 2 This is a radial distribution network topology diagram of a high-proportion photovoltaic distribution network voltage edge control method in an embodiment of the present invention;

[0083] Figure 3This is a diagram of a physical-aided graph reinforcement learning architecture for a high-proportion photovoltaic distribution network voltage edge control method in an embodiment of the present invention.

[0084] Figure 4 This is a voltage edge control framework diagram based on a cloud-edge collaborative architecture for a high-proportion photovoltaic distribution network voltage edge control method in an embodiment of the present invention. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0086] Example

[0087] This embodiment provides a voltage edge control method for a high-proportion photovoltaic distribution network, the flowchart of which is shown below. Figure 1 The method includes the following steps:

[0088] S1. Calculate the node voltage-active and voltage-reactive sensitivity, define the electrical distance between nodes, and partition the active distribution network using an improved modularity function;

[0089] S2. Based on the network partitioning results, determine the elements of the agent environment, number of agents, state space, action space, reward function and state transition function, and construct a distributed voltage edge control model based on Markov game process;

[0090] S3. Design a multi-agent graph reinforcement learning algorithm with embedded graph neural network to iteratively solve the distributed voltage edge control model. Introduce a physical auxiliary mechanism to generate reference experience to help the agent to quickly find the optimization and obtain the voltage control strategy.

[0091] S4. Develop a voltage edge control framework for intelligent agents' offline learning and online applications based on a cloud-edge collaborative architecture.

[0092] Reference Figure 2-4 The high-proportion photovoltaic distribution network voltage edge control method proposed in this embodiment can include distribution network partitioning, edge control model, multi-agent graph reinforcement learning, physical auxiliary mechanism, edge control framework, etc. in practical application.

[0093] An improved modularity function can be used to rationally partition the active distribution network. For a radial distribution network with n nodes, such as... Figure 2As shown, the active / reactive power flowing through the branch is closely coupled with the branch's own resistance / reactance. Therefore, the voltage drop between any two nodes can be approximated as:

[0094] V i -V j =r ij P ij +x ij Q ij

[0095] Among them, 0 <i,j≤n,V i and V j P represents the voltages at nodes i and j, respectively. ij and Q ij These represent the active power and reactive power flowing through the branches between nodes i and j, respectively. ij and x ij Let be the resistance and reactance of the branch between nodes i and j, respectively. In a more general case, the voltage drop between any two nodes can be expressed in matrix form:

[0096] ΔV=V i -V j =RP+XQ

[0097] Where ΔV is the voltage drop between any two nodes, and P = [P1, P2, ..., P n ] T and Q = [Q1, Q2, ..., Q n ] T Let be the active power and reactive power injection vectors at the node, respectively. R is an n-dimensional resistance matrix representing the relationship between node voltage and active power injection, and X is an n-dimensional reactance matrix representing the relationship between node voltage and reactive power injection.

[0098] Based on the definition of voltage sensitivity, the voltage-active and voltage-reactive sensitivities at each node can be obtained as follows:

[0099]

[0100]

[0101] in, and These are the voltage-active and voltage-reactive sensitivities, respectively. The voltage sensitivity does not require the calculation of the inverse of the Jacobian matrix.

[0102] Based on the voltage sensitivity calculation results, the electrical distance between any two nodes can be defined as:

[0103]

[0104] Where, d ijThis is the electrical matrix between nodes i and j, reflecting the degree of electrical coupling between nodes.

[0105] In the above method, the electrical distance can be used instead of the adjacency matrix as the weight matrix in the modularity function, thus obtaining an improved modularity function for partitioning the distribution network, which can be expressed as:

[0106]

[0107] Where ρ is the modularity index, used to measure the effectiveness of distribution network zoning; the larger the value, the better the zoning effect. m is the sum of the weights of all branches, k i and k j Let δ be the sum of the number of branches connected to nodes i and j, respectively, and δ be the Kronecker function. The optimal partitioning result of the distribution network can be obtained by calculating the system modularity index through multiple traversals. This modularity partitioning method does not require pre-setting the number of partitions and depends only on the distribution network topology, independent of the system operating state.

[0108] When constructing a distributed voltage edge control model based on Markov game processes, the distributed voltage edge control model based on Markov game processes can be represented by multivariate tuples:

[0109] <N,S t ,[o t ] N ,[a t ] N ,[r t r] N ,p>

[0110] This tuple includes the number of agents N and the global state space S. t Local observation space t Action space a t Reward function r t And the state transition function p. In each time slot t, the agent adjusts its state based on local observations o. t Perform action a t The environment is moved to the next time slot t+1, and the corresponding reward r is returned to the agent. t and local observation o t+1 And so on, in a continuous cycle.

[0111] The specific definitions of the variables involved in the tuple are as follows:

[0112] Intelligent agent environment: It specifically refers to the distribution network environment and is defined as a power flow calculation model. It uses OpenDSS as an environment simulator to calculate the node voltage and network loss of the distribution network.

[0113] Number of agents: Assume that each subnetwork is an independent and interactive agent, i.e., the number of partitions is the same as the number of agents N. Each subnetwork includes some distribution network nodes, photovoltaics, energy storage, and static var compensators. The agents corresponding to each subnetwork provide active and reactive power support to the system by adjusting distributed energy storage and static var compensators, ensuring voltage safety constraints while optimizing system network losses.

[0114] State space: Global state S t It describes the observable information (global information) of all agents in time slot t, and is defined as:

[0115]

[0116] Wherein, the global state S t Including active load of all nodes reactive load Photovoltaic power output Energy storage state of charge (SOC) j,t And the adjacency matrix A representing the topology t Local observation space o t The observable information (sub-network information) of a single agent in time slot t is defined as follows:

[0117]

[0118] Among them, local observation o t Including the active load of all nodes in the subnetwork reactive load Photovoltaic power output Energy storage state of charge (SOC) j,t And the adjacency matrix A representing the topology i,t composition.

[0119] Action space: Essentially, it represents the control variables in the voltage edge control model. Since the agent provides voltage regulation services to the distribution network by adjusting energy storage and static var compensators, the action space can be defined as follows:

[0120]

[0121] The operational space includes the active power provided by energy storage. The reactive power provided by the static var compensator To avoid exceeding the state of charge limit of energy storage, the actual active power output of energy storage should satisfy the following relationship:

[0122]

[0123] in, and These represent the charge and discharge efficiencies, Emax For battery capacity, SOC max and SOC min These are the upper and lower limits of the state of charge. The above-mentioned action correction method can limit the state of charge of the energy storage to a reasonable range.

[0124] Reward function: Essentially, it is the optimization objective of the voltage control problem. This invention aims to minimize network losses and ensure that node voltages operate within a safe range. Therefore, the reward function can be defined as:

[0125]

[0126]

[0127] Where, r t Let T be the reward function, T be the number of optimization slots, N:={1,…,n} be the set of nodes, E:={(i,j)}∈N×N be the set of branches, and l ij,t r is the square of the branch current. ij Let η be the resistance of the branch between nodes i and j, η be the penalty coefficient, and c be the resistance of the branch between nodes i and j. t Let v be the penalty function, 1(·) be the indicator function, and v j,t The square of the node voltage. Let be its upper and lower limits, respectively.

[0128] State transition function: Due to the uncertainty of load demand and photovoltaic output, it is difficult to formulate an explicit environmental state transition function. That is, the state transition process is random. When the agent interacts with the environmental power grid, the current state will be randomly transferred to the next state.

[0129] When designing a multi-agent graph reinforcement learning algorithm that embeds a graph neural network, the specific structural design and iterative update process of the multi-agent graph reinforcement learning algorithm are as follows:

[0130] Graph neural networks are embedded in reinforcement learning to assist agents in capturing spatial correlations and topological dependencies between nodes, enabling agents to perceive topological changes caused by dynamic reconstruction in real time. This embodiment embeds a graph attention network into a multi-agent soft actor critic reinforcement learning algorithm. Each agent consists of two value networks, two soft Q-networks, and one policy network. Each network is composed of several stacked graph attention layers and fully connected layers, such as... Figure 3 As shown.

[0131] In this embodiment, the core of the graph attention layer is the multi-head attention coefficient, and the relationship between the input and output of the value network can be expressed as:

[0132] V = σ out (W out σ in (∑αWin (S t ||a t )))

[0133] Wherein, the global state space S t and action space a t As input, || represents the concatenation operation, and W in Let σ be the weight matrix of the graph attention layer. in Let LeakyReLU be the activation function for the graph attention layer, α be the multi-head attention coefficient, and W be the multi-head attention coefficient. out Let σ be the weight matrix of the fully connected layer. in ReLU is the activation function for the fully connected layer, and the state value V is the network output used to evaluate the quality of the current state and actions, that is, the impact of the active power output of energy storage and the reactive power output of static var compensators on the future of the power grid under the current distribution network state.

[0134] The relationship between the input and output of a soft Q network can be represented as:

[0135] Q = σ out (W out σ in (∑αW in a t ))

[0136] Wherein, the global state space S t The input is , and the action value Q is the network output, used to evaluate the quality of the current action, that is, the impact of the active power output of the energy storage and the reactive power output of the static var compensator on the current state of the distribution network.

[0137] The relationship between the input and output of a policy network can be represented as follows:

[0138] a t =σ out (W out σ in (∑αW in o t ))

[0139] Among them, local observation o t As input, agent action a t For network output,

[0140] It should be noted that each agent has its own emphasis on different methods for updating network parameters, and the value network parameters can be updated by minimizing the mean square error:

[0141]

[0142] Among them, J V (ψ) represents the value network parameter optimization error, E is the mathematical expectation operation, log is the logarithmic operation, and Vψ Q θ and π φ These are the state value function, action value function, and policy function, respectively. ψ, θ, and φ are the parameters corresponding to the above networks, and D is the experience pool.

[0143] The parameters of a soft Q network can be updated by minimizing the Bellman mean square error:

[0144]

[0145]

[0146] Among them, J Q (θ) represents the optimization error of the soft Q network parameters, and γ is the discount factor. The target value network is used for stable training. Its parameters.

[0147] The policy network can be updated by minimizing the KL divergence:

[0148]

[0149] Among them, J π (φ) represents the optimization error of the policy network parameters. This is a reparameterized sampling action.

[0150] In practical applications, a physical assistance mechanism can be introduced to generate reference experience to help the agent quickly find the optimal solution. This mechanism involves using a relatively accurate physical model (voltage control optimization model) to generate quantitative reference experience, which is then stored in an experience pool. This helps the agent quickly determine effective actions during training, improving the learning efficiency of voltage regulation strategies. The reference experience contains near-optimal voltage control strategies, guiding the agent to quickly obtain the optimal control strategy while avoiding unnecessary trial and error and exploration in the power distribution network environment.

[0151] When developing a voltage edge control framework based on a cloud-edge collaborative architecture, the framework can include two layers: a cloud learning layer and an edge control layer. Figure 4 As shown, where:

[0152] Edge Control Layer: This layer belongs to the distributed control layer. First, the distribution network is divided into several sub-networks. These sub-networks are modeled as independent and interactive agents. It is assumed that each sub-network deploys a specific edge controller responsible for controlling energy storage and static var compensators to provide dynamic voltage support to the system. The agents perform online inference and decision-making at the network edge and upload their experience interacting with the environment to the cloud master platform for training the control model. Offloading the inference and decision-making tasks to the edge reduces cloud communication overhead.

[0153] Cloud-based learning layer: This layer belongs to the centralized control layer. The agent is trained offline in the cloud, and the cloud master platform encompasses the operation and management functions of the distribution network dispatch center. The agent receives experience uploaded from the edge devices and stores it in an experience pool, periodically sampling to learn and update the control strategy. Then, the converged control model is distributed to the edge controller to ensure the agent's generalization ability to different operating conditions of the distribution network. The computationally intensive and time-consuming training tasks are transferred to the cloud to avoid exhausting the resources of edge devices.

[0154] In summary, cloud control centers and edge devices are complementary in terms of technology and mutually beneficial in terms of application. This centralized-distributed model can maximize the advantages of both the cloud and the edge to achieve reasonable scheduling of energy storage and static var compensators, while ensuring safe and effective voltage control.

[0155] Therefore, this embodiment proposes a high-proportion photovoltaic distribution network voltage edge control method, which can provide voltage support to the distribution network by reasonably calling energy storage and static var compensators, ensuring that the voltage operates within a safe range while optimizing system network losses, enhancing the agent's perception ability of dynamic topology reconfiguration, and improving the recognition speed of effective actions and the learning efficiency of voltage control strategies.

[0156] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A voltage edge control method for a high-proportion photovoltaic distribution network, characterized in that, Includes the following steps: Calculate node voltage-active and voltage-reactive sensitivity, define electrical distances between nodes, and partition the active distribution network using an improved modularity function; Based on the network partitioning results, the elements of agent environment, number of agents, state space, action space, reward function and state transition function are determined, and a distributed voltage edge control model based on Markov game process is constructed. A multi-agent graph reinforcement learning algorithm with embedded graph neural network is designed to iteratively solve the distributed voltage edge control model. A physical auxiliary mechanism is introduced to generate reference experience to help the agent to quickly find the optimization and obtain the voltage control strategy. A voltage edge control framework for intelligent agents' offline learning and online application was developed based on a cloud-edge collaborative architecture. The calculation of node voltage-active and voltage-reactive sensitivity, and the definition of electrical distance between nodes, specifically includes the following steps: For those with n For a radial distribution network with n nodes, the voltage drop between any two nodes can be approximated as: Where, 0 < i , j ≤ n , V i and V j They are nodes i and j voltage, P ij and Q ij The nodes that flow through are respectively i and j The active and reactive power of the intermediate branch circuit. r ij and x ij They are nodes i and j Resistance and reactance of intermediate branches; Represent the voltage drop between any two nodes in matrix form: Where, Δ V For the voltage drop between any two nodes P =[ P 1, P 2, …, P n ] T and Q =[ Q 1, Q 2, …, Q n ] T These are the active power and reactive power injection vectors for the nodes, respectively. R for n A 3D resistance matrix characterizes the relationship between node voltage and active power injection. X for n The 3D reactance matrix characterizes the relationship between node voltage and reactive power injection; Based on the definition of voltage sensitivity, the voltage-active and voltage-reactive sensitivities of each node are obtained as follows: in, SVPij and SVQij These are the voltage-active and voltage-reactive sensitivities, respectively. The voltage sensitivity does not require the calculation of the inverse of the Jacobian matrix. Based on the voltage sensitivity calculation results, the electrical distance between any two nodes is defined as: in, d ij For nodes i and j The electrical matrix between nodes is used to reflect the degree of electrical coupling between nodes; The method of partitioning the active distribution network using an improved modularity function is as follows: By replacing the adjacency matrix with electrical distance as the weight matrix in the modularity function, an improved modularity function is obtained and the distribution network is partitioned, as expressed below: in, ρ The modularity index is used to measure the effectiveness of power distribution network zoning; the higher the value, the better the zoning effect. m The sum of the weights of all branches. k i and k j They are respectively with nodes i and j The sum of the number of connected branches, δ is the Kronecker function, and the optimal partitioning result of the distribution network can be obtained by calculating the system modularity index through multiple traversals.

2. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 1, characterized in that, The process of determining the agent environment, number of agents, state space, action space, reward function, and state transition function based on network partitioning results specifically involves: The intelligent agent environment, specifically referring to the distribution network environment, is defined as a power flow calculation model. OpenDSS is used as an environment simulator to calculate the node voltage and network loss of the distribution network. The number of agents is assumed to be the number of independent and interacting agents in each subnetwork of the network partitioning result, i.e., the number of partitions and the number of agents. N Similarly, each sub-network includes some distribution network nodes, photovoltaics, energy storage and static var compensators. The corresponding intelligent agents of each sub-network provide active and reactive power support to the system by adjusting distributed energy storage and static var compensators, in order to ensure voltage safety constraints while optimizing system network losses. The state space, through the global state space S t The observable information describing all agents in time slot t is defined as follows: Among them, the global state space S t Including active load of all nodes Pd j , t reactive load Qd j , t Photovoltaic power output Ppv j , t Energy storage state of charge (SOC) j,t and the adjacency matrix representing the topological structure A t Local observation space o t The observable information describing a single agent in time slot t is defined as follows: Among them, local observation o t Including the active load of all nodes in the subnetwork Pd j , t reactive load Qd j , t Photovoltaic power output Ppv j , t Energy storage state of charge (SOC) j,t and the adjacency matrix representing the topological structure A i,t ; The action space refers to the control variables of the voltage edge control model, and is defined as follows: The operational space includes the active power provided by energy storage. Pess j , t The reactive power provided by the static var compensator Qsvc j , t The actual active power output of energy storage satisfies the following relationship: in, ηc j and ηdc j These are the charging and discharging efficiencies, respectively. E max For battery capacity, SOC max and SOC min These are the upper and lower limits of the state of charge, respectively; The reward function, which is the optimization objective of the voltage control problem, is defined as follows: in, r t For the reward function, T To optimize the number of time slots, :={1,…, n } is a set of nodes. :={( i , j )} For branch road collection, l ij,t The square of the branch current. r ij For nodes i and j The resistance of the intermediate branch, η The penalty coefficient is... c t For the penalty function, 1( ) is an indicator function. v j,t The square of the node voltage. v max j , t , v min j , t Let them be their upper and lower limits, respectively; The state transition function allows the agent to randomly transition from its current state to the next state when interacting with the power grid environment.

3. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 2, characterized in that, The constructed distributed voltage edge control model based on Markov game process is represented by multivariate tuples: This tuple includes the number of agents. N Global state space S t Local observation space o t Action space a t Reward function r t and state transition function p In each time slot t The intelligent agent is based on the local observation space o t Execution space a t The environment shifts to the next time slot. t +1, and return the corresponding reward function to the agent. r t and local observation o t+1 And so on, in a continuous cycle.

4. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 1, characterized in that, When designing the multi-agent graph reinforcement learning algorithm that embeds graph neural networks, the graph neural network is embedded in reinforcement learning to help the agent capture the spatial correlation and topological dependence between nodes, so that the agent can perceive the topological changes caused by dynamic reconstruction in real time.

5. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 4, characterized in that, The graph attention network is embedded in the multi-agent soft actor critic reinforcement learning algorithm. Each agent includes two value networks, two soft Q networks and one policy network. Each network is composed of several graph attention layers and fully connected layers stacked together.

6. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 5, characterized in that, The core of the graph attention layer is the multi-head attention coefficient; The relationship between the inputs and outputs of the value network is represented as follows: Among them, the global state space S t and action space a t As input, || represents the concatenation operation. W in The graph attention layer weight matrix, σ in Let LeakyReLU be the activation function for the graph attention layer. α Multi-head attention coefficient, W out This is the weight matrix of the fully connected layer. σ in For a fully connected layer, the activation function is ReLU, and the state value is... V The output is for the network to evaluate the current state and the quality of the actions, that is, the impact of the active power output of energy storage and the reactive power output of static var compensators on the future of the power grid under the current distribution network state. The relationship between the input and output of the Q network is expressed as follows: Among them, the global state space S t For input, action value Q This is the network output used to evaluate the quality of the current action, namely the impact of the active power output of the energy storage and the reactive power output of the static var compensator on the current state of the distribution network. The relationship between the input and output of the policy network is represented as follows: Among them, local observation space o t The actions performed by the agent as input. a t For network output.

7. A high-proportion photovoltaic distribution network voltage edge control method according to claim 6, characterized in that, The design incorporates a multi-agent graph reinforcement learning algorithm with embedded graph neural networks to iteratively solve the distributed voltage edge control model. The iterative update process is as follows: The value network parameters are updated by minimizing the mean squared error: in, J V ( ψ ) represents the value network parameter optimization error, E represents the mathematical expectation operation, and log represents the logarithmic operation. V ψ , Q θ and π These are the state value function, action value function, and policy function, respectively. ψ , θ and These are the parameters corresponding to the above networks. D For experience pool; The soft Q network parameters are updated by minimizing the Bellman mean square error: in, J Q ( θ The error represents the optimization error of the soft Q network parameters. γ This is the discount factor. The target value network is used for stable training. Its parameters; The policy network is updated by minimizing the KL divergence: in, J π ( ) represents the error in optimizing the policy network parameters. This is a reparameterized sampling action.

8. The voltage edge control method for a high-proportion photovoltaic distribution network according to claim 1, characterized in that, The voltage edge control framework includes a cloud learning layer and an edge control layer; The edge control layer: This layer belongs to the distributed control layer. First, the distribution network is divided into several sub-networks. These sub-networks are modeled as independent and interactive intelligent agents. It is assumed that each sub-network is equipped with a specific edge controller, which is responsible for controlling the energy storage and static var compensators to provide dynamic voltage support to the system. The intelligent agents perform online reasoning and decision-making at the network edge and upload their experience of interacting with the environment to the cloud main station platform for training the control model, thus offloading the reasoning and decision-making tasks to the edge. The cloud-based learning layer: This layer belongs to the centralized control layer. The agent performs offline training in the cloud. The cloud main station platform includes the operation and management functions of the distribution network dispatch center. The agent receives the experience uploaded by the edge terminal and stores it in the experience pool. It periodically samples to learn and update the control strategy. Then, the training converged control model is sent to the edge controller to ensure the agent's generalization ability to different operating conditions of the distribution network and to transfer the computationally intensive and time-consuming training tasks to the cloud.

Images