Method and device for evaluating distributed photovoltaic bearing capacity of power distribution network in real time

Abstract

The invention discloses a power distribution network distributed photovoltaic bearing capacity real-time evaluation method, which comprises the steps of determining a mathematical model for power distribution network photovoltaic bearing capacity evaluation, and constructing a Markov decision model for power distribution network photovoltaic bearing capacity dynamic evaluation based on the mathematical model; a soft actor-commentator SAC algorithm is used as a basic framework, graph sampling and aggregation GraphSAGE are used for improving the SAC algorithm, a graph deep reinforcement learning algorithm is constructed, the graph deep reinforcement learning algorithm is used for solving a Markov decision model, and a trained graph strategy network is obtained; carrying out the subsequent real-time evaluation of the distributed photovoltaic bearing capacity of the power distribution network through employing the trained graph strategy network; according to the method, the limitation that precise modeling is needed in a traditional method is effectively overcome, and a more accurate real-time evaluation result is provided.

Description

Technical Field

[0001] The present invention belongs to the field of photovoltaic carrying capacity assessment, and in particular relates to a real-time assessment method and device for distributed photovoltaic carrying capacity of a distribution network. Background Art

[0002] At present, the demand for distributed photovoltaic (PV) access in distribution networks has increased sharply. However, with the continuous increase in PV penetration, distribution networks face multiple challenges such as overvoltage, line overload, and abnormal power quality. In order to ensure the coordinated development of distribution networks and PV, accurately evaluating the photovoltaic carrying capacity (PVHC) of distribution networks has become the key to guiding PV access planning and construction.

[0003] PVHC is defined as the maximum photovoltaic capacity that the distribution network can accept while meeting the safety operation constraints. Its evaluation methods are mainly divided into two categories: mathematical optimization methods and random scenario methods. Mathematical optimization methods (such as particle swarm optimization and second-order cone optimization algorithm) transform PVHC evaluation into a constrained optimization problem with the goal of maximizing the carrying capacity; while random scenario methods (such as Monte Carlo simulation) determine the PV carrying capacity by generating a large number of photovoltaic access scenarios with specific probability distribution characteristics and combining multiple safety operation constraints. However, these traditional methods have three main problems: first, traditional evaluation methods rely on accurate mathematical models to describe the system, but the various uncertainties and complex relationships in the actual distribution network are difficult to represent with accurate mathematical models; second, the computational efficiency is low, especially when dealing with large-scale, multi-node distribution networks, the computational complexity grows exponentially; third, the real-time performance is insufficient, and it is difficult to adapt to the dynamics and uncertainty of the distribution network operation status.

[0004] Therefore, based on the above problems, it is necessary to study a dynamic evaluation method for distributed photovoltaic carrying capacity of active distribution network based on graph reinforcement learning, which can effectively overcome the shortcomings of traditional methods and provide a more flexible, efficient and intelligent solution for distribution networks with a high proportion of distributed photovoltaic access. Summary of the invention

[0005] In order to address the deficiencies in the prior art, the present invention provides a dynamic evaluation method for distributed photovoltaic carrying capacity of an active distribution network based on graph reinforcement learning. This method aims to solve several key problems existing in the existing photovoltaic carrying capacity evaluation technology, including high requirements for the accuracy of distribution network model parameters, too long evaluation time, and inability to effectively cope with drastic changes in system state caused by a high proportion of distributed photovoltaics. Through this method of the present application, the efficiency and accuracy of the photovoltaic carrying capacity evaluation of the distribution network can be significantly improved.

[0006] The present invention adopts the following technical solutions:

[0007] A real-time evaluation method for distributed photovoltaic carrying capacity of a distribution network, the method comprising the following steps:

[0008] Determine a mathematical model for evaluating the photovoltaic carrying capacity of the distribution network, and construct a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of the distribution network based on the mathematical model;

[0009] Taking the soft actor-critic SAC algorithm as the basic framework, the SAC algorithm is improved by using Graph Sample and Aggregate (GraphSAGE), and a graph deep reinforcement learning algorithm is constructed. The Markov decision model is solved by using the graph deep reinforcement learning algorithm to obtain the trained graph policy network.

[0010] The trained graph strategy network is used to carry out subsequent real-time evaluation of the distributed photovoltaic carrying capacity of the distribution network.

[0011] Furthermore, the method for determining the mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is:

[0012] The mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is expressed as an optimization problem with the goal of maximizing the carrying capacity, where the decision variable is the accessible capacity of the photovoltaic power generation at each node, which can be expressed as:

[0013]

[0014] Where N represents the node set, represents the photovoltaic access capacity of node i in period t;

[0015] The system also includes operation constraints of the distribution network, which include power balance constraints, node voltage constraints and line power constraints.

[0016] Furthermore, based on the mathematical model, a method for constructing a Markov decision model for dynamic evaluation of photovoltaic carrying capacity of distribution network is as follows:

[0017] The state space of the intelligent agent is constructed, which contains all the state information in the distribution network, including the active power and reactive power of the load and the timestamp of the system operation. In the period t, the observed state of the intelligent agent can be defined as:

[0018] S t =(P load,t ,Q load,t ,P pv,t ,Q pv,t ) (6)

[0019] Among them, P load,t ,Q load,t Respectively represent the active and reactive load demand vectors in period t; P pv,t ,Q pv,tThey represent the active and reactive output vectors of photovoltaic power in the distribution network during period t. If the node is not equipped with photovoltaic power, the position value of the corresponding node in the vector is always 0.

[0020] Construct the action space of the intelligent agent. If the number of distribution network nodes is N, the action of the intelligent agent can be defined as:

[0021] A t =[a 1,t ,a 2,t ,…,a N,t ] (7)

[0022] Among them, a N,t is a continuous variable, indicating the photovoltaic access capacity of the Nth node in period t;

[0023] According to the above state space and action space, the transfer function of the state variables of two adjacent time periods can be expressed as:

[0024]

[0025] Among them, S t+1 is the state vector at time period t, A t is the evaluation result of the photovoltaic installable capacity of each node in time period t;

[0026] Obtain the agent's reward function, which includes the sum of the photovoltaic configuration capacities of all nodes in the system and the penalty term for the violation of the system's operating constraints under the current photovoltaic configuration capacity, which can be expressed as:

[0027] R t =R(S t ,A t )=(e T A t )-(1-κ t )α (9)

[0028] In the formula, κ t is a Boolean variable, α is a large real number, which is used as a penalty term; in the photovoltaic configuration scenario A t Under this condition, the node voltage and current meet the safety constraints, then κ t =1, then the value of the reward function is equal to the sum of the photovoltaic configuration capacity of the node; otherwise, κ t =0, the value of the reward function is negative;

[0029] Based on the defined reward function, the goal of the Markov decision process is to find an optimal evaluation strategy π to maximize the expectation of the cumulative reward, which can be expressed as:

[0030]

[0031] Among them, G t represents the cumulative reward, G t =R t +γR t+1 +γ 2 R t+2 +…; In the framework of Markov decision process, the intelligent agent determines the state S of the distribution network (DN) at time period t. t Generate the photovoltaic access capacity A of each node t ; Then, A t is transmitted to the distribution network for power flow calculation, thereby obtaining a reward R t ;

[0032] The above operation is repeated at the next time step t+1. Finally, a series of strategies π that can maximize the action value are learned through closed-loop iteration. * .

[0033] Furthermore, the soft actor-critic SAC algorithm is used as the basic framework, and the SAC algorithm is improved by using graph sampling and aggregation (GraphSAGE). The graph policy network is introduced into the SAC algorithm. In the graph policy network, the system state S t Modeled as a property graph The vertex set represents the node set of the distribution network, the edge set ε represents the physical connection relationship between nodes, and the vertex attribute matrix X contains the real-time status information of each node, as shown in formula (6);

[0034] GraphSAGE builds the vertex representation vector layer by layer through the following aggregation and update steps:

[0035]

[0036] in, represents the representation vector of vertex v at the kth layer, represents the neighbor set of vertex v, W (k) represents the weight vector of the kth layer, AGG (k) is the aggregation function of the kth layer, and the weighted average based on the attention mechanism is used as the aggregation function:

[0037]

[0038] in, represents the attention weight, which can be calculated by learning the attention network;

[0039] The GraphSAGE network contains three layers of graph convolution, with hidden dimensions of each layer being 128, 256, and 256 respectively. After the last layer of graph convolution, a global average pooling layer is used to aggregate the representation vectors of all vertices into a full-graph representation vector:

[0040]

[0041] Full image representation vector Then it is processed by two fully connected layers of the SAC algorithm, and finally outputs the mean μ of the action distribution φ (S t ) and logarithmic standard deviation logσ φ (S t ); The sampling process uses the re-parameterization technique and can be expressed as:

[0042]

[0043] Among them, φ represents the parameter set of the policy network, ε represents Gaussian noise, represents a Gaussian distribution with a mean of 0 and a variance of β.

[0044] Furthermore, the Markov decision model is solved using the graph deep reinforcement learning algorithm to obtain the trained graph strategy network:

[0045] ① Initialization phase: First, initialize the parameters of the graph strategy network, cost evaluation network, and reward evaluation network; create an empty experience playback buffer Used to store experience data on the interaction between the intelligent agent and the distribution network environment;

[0046] ② Experience collection: Experience data is collected by executing actions output by the action network in the distribution network environment; including the distribution network operation status, the evaluation results of the photovoltaic output of each node, and the reward value, which can be expressed as {S t ,A t ,R t ,S t+1}, each set of experience data is stored in the buffer;

[0047] ③Data sampling: randomly extract a batch of experience data from the playback buffer for subsequent network training;

[0048] ④ Evaluation network update: Use the Q value network and the policy network to estimate the target Q value of each sampled experience, calculate the mean square error loss between the target Q value and the actual Q value, and update the parameters θ of the Q value evaluation network accordingly i ;

[0049] ⑤ Policy network update: Apply the gradient ascent method to update the parameter φ of the policy network to optimize the action selection of the policy network, that is, the photovoltaic access capacity of each node;

[0050] ⑥Soft update target network: Use soft update method to gradually adjust the parameters θ of the target Q value network i′ , to ensure the stability of network parameter updates;

[0051] ⑦ Training iteration: Repeat steps ② to ⑥ until the set number of training times is reached.

[0052] Furthermore, the Q-value network adopts a four-layer feedforward neural network structure, the input layer receives the concatenated vector of the state-action pair, and the dimension is the sum of the state dimension and the action dimension; the network includes two hidden layers and an output layer of a scalar output, and adopts a dual Q network architecture, and updates the target network parameters through Polyak averaging:

[0053] θ i′ ←τθ i +(1-τ)θ i′ ,i=1,2 (30)

[0054] Among them, θ i represents the parameter set of the i-th Q-value network, θ i′ represents the parameter set of the i-th target Q value network, and the soft update coefficient τ=0.005. During the training process, the algorithm maintains a capacity of 10 6 Experience Replay Buffer Used to store the transfer samples obtained by interactive sampling (S t ,A t ,R t ,S t+1 ), each time an update is performed, a mini-batch of size 256 is randomly sampled from the buffer for training.

[0055] Furthermore, when updating the network parameters of the graph deep reinforcement learning algorithm, the loss function of the policy network combines the maximization of Q value and the maximization of policy entropy:

[0056]

[0057] The temperature parameter α is automatically adjusted by the following objective function:

[0058]

[0059] Target entropy Set to the negative value of the action space dimension;

[0060] The loss function of the Q-value network uses the mean square error of the temporal difference error:

[0061]

[0062] The target value y t The calculation of is integrated into the maximum entropy framework:

[0063]

[0064] In the actual training process, in order to ensure the convergence of the algorithm, 10,000 steps of pre-training are first performed, and only the Q value network is updated. After the Q value estimation is relatively accurate, all networks are updated simultaneously. During the training process, a policy evaluation is performed after every 1,000 steps of environmental interaction. The training convergence is determined when the average return improvement for 5 consecutive evaluations is less than 1%. At the same time, in order to enhance the robustness of the algorithm, Gaussian noise is introduced when sampling actions. The standard deviation of the noise decays linearly with the training process. The initial value is 0.3 and eventually decays to 0.1.

[0065] The real-time evaluation device for distributed photovoltaic carrying capacity of distribution network includes:

[0066] A Markov decision model building module is used to determine a mathematical model for evaluating the photovoltaic carrying capacity of a distribution network, and based on the mathematical model, to build a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of a distribution network;

[0067] The module for acquiring the trained graph policy network is used to use the soft actor-critic SAC algorithm as the basic framework, and improve the SAC algorithm by using graph sampling and aggregation (GraphSAGE), construct a graph deep reinforcement learning algorithm, and use the graph deep reinforcement learning algorithm to solve the Markov decision model to obtain the trained graph policy network;

[0068] The photovoltaic carrying capacity real-time assessment module is used to use the trained graph strategy network to perform subsequent real-time assessment of the distributed photovoltaic carrying capacity of the distribution network.

[0069] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network as described above.

[0070] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network as described above is implemented.

[0071] The beneficial technical effects of the present invention are:

[0072] 1) Compared with traditional capacity methods (Monte Carlo method, genetic algorithm, etc.), the method proposed in the present invention makes decisions in a data-driven and model-free manner, that is, in view of the real-time and accurate estimation of system parameters;

[0073] 2) The proposed method uses neural networks to extract features and optimize evaluation results. Compared with traditional photovoltaic carrying capacity evaluation methods, it greatly reduces the computational complexity in practical applications. Even for large-scale systems, it can achieve extremely fast evaluation, making it more capable of responding to fluctuations in distribution network status in real time;

[0074] 3) The proposed method is a dynamic assessment method for the carrying capacity of high-penetration photovoltaic access distribution networks. Compared with the traditional assessment method that only focuses on a single time section, it can dynamically adjust the photovoltaic carrying capacity assessment results according to the operating status of the distribution network (node ​​load demand and photovoltaic output), providing support for relevant departments to timely grasp the development trend of distributed photovoltaic carrying capacity of distribution networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a schematic diagram of the process of the real-time evaluation method of the distributed photovoltaic carrying capacity of the distribution network in an embodiment of the present invention;

[0076] Figure 2 The following is a comparison of the training processes of different algorithms in the embodiments of the present invention. DETAILED DESCRIPTION

[0077] The following is a further clear and complete description of the method and device for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network provided by the present invention in conjunction with the accompanying drawings:

[0078] like Figure 1 As shown, a real-time evaluation method for distributed photovoltaic carrying capacity of a distribution network comprises the following steps:

[0079] Determine a mathematical model for evaluating the photovoltaic carrying capacity of the distribution network, and construct a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of the distribution network based on the mathematical model;

[0080] Taking the soft actor-critic SAC algorithm as the basic framework, the SAC algorithm is improved by using graph sampling and aggregation (Graph Sample and aggregate, GraphSAGE), and a graph deep reinforcement learning algorithm is constructed. The Markov decision model is solved by the graph deep reinforcement learning algorithm to obtain the trained graph policy network.

[0081] The trained graph strategy network is used to carry out subsequent real-time evaluation of the distributed photovoltaic carrying capacity of the distribution network.

[0082] The method for determining the mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is:

[0083] The mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is expressed as an optimization problem with the goal of maximizing the carrying capacity, where the decision variable is the accessible capacity of the photovoltaic power generation at each node, which can be expressed as:

[0084]

[0085] Where N represents the node set, represents the photovoltaic access capacity of node i in period t;

[0086] While maximizing the photovoltaic carrying capacity, the following distribution network operation constraints must be met; distribution network operation constraints mainly include power balance constraints, node voltage constraints and line power constraints. Among them, power balance constraints are the basic constraints for distribution network flow calculations, and node voltage and line power constraints are key technical indicators to ensure reliable operation of the system. The core is to ensure that each node voltage and line power are within the safe allowable range; they can be expressed as:

[0087] 1) Power balance constraints

[0088]

[0089] Among them, t represents the time period, P t,i and Q t,i is the injected active power and reactive power at node i, U t,i is the voltage amplitude of node i, G ij and B ij are the conductance and susceptance of branch ij, respectively, ij is the voltage phase angle difference between nodes i and j;

[0090] 2) Node voltage constraints

[0091] U imin ≤U i,t ≤U imax (4)

[0092] Among them, U imin and U imax Respectively represent the upper and lower limits of the node voltage;

[0093] 3) Branch current constraints

[0094] I ij,min ≤I ij,t ≤I ij,max (5)

[0095] Among them, I ij,min and I ij,max Respectively represent the upper and lower limits of the line current, I ij,tRepresents the current flowing through line ij in time period t.

[0096] Based on the mathematical model, the method of constructing a Markov decision model for dynamic evaluation of photovoltaic carrying capacity of distribution network is as follows:

[0097] Markov Decision Process (MDP) is a mathematical framework for modeling sequential decision problems, which provides a theoretical basis for reinforcement learning. Formally, MDP can be represented as a five-tuple M = (S, A, P, R, γ) agent, where S is the state space, A is the action space, P: S × A × S → [0, 1] is the state transition function, which represents the probability P(s′|s, a) of taking action a in state s to transition to state s'. is the reward function, and γ∈[0,1] is the discount factor. The core feature of MDP is to satisfy the Markov property, that is, the next state of the system depends only on the current state and action, and has nothing to do with the historical trajectory. The goal of the agent is to find an optimal strategy π * :S→A, which maximizes the expected cumulative discounted reward obtained from any initial state.

[0098] Based on the Markov decision process, the PV carrying capacity assessment problem is modeled as follows:

[0099] The state space of the intelligent agent is constructed, which contains all the state information in the distribution network, including the active power and reactive power of the load and the timestamp of the system operation. In the period t, the observed state of the intelligent agent can be defined as:

[0100] S t =(P load,t ,Q load,t ,P pv,t ,Q pv,t ) (6)

[0101] Among them, P load,t ,Q load,t Respectively represent the active and reactive load demand vectors in period t; P pv,t ,Q pv,t They represent the active and reactive output vectors of photovoltaic power in the distribution network during period t. If the node is not equipped with photovoltaic power, the position value of the corresponding node in the vector is always 0.

[0102] Construct the action space of the intelligent agent, that is, the photovoltaic access capacity of each node in the distribution network. If the number of distribution network nodes is N, the action of the intelligent agent can be defined as:

[0103] A t =[a 1,t ,a 2,t ,…,a N,t] (7)

[0104] Among them, a N,t is a continuous variable, indicating the photovoltaic access capacity of the Nth node in period t;

[0105] State transfer function: According to the above state space and action space, the transfer function of the state variables in two adjacent time periods can be expressed as:

[0106]

[0107] Among them, S t+1 is the state vector at time period t, A t is the evaluation result of the photovoltaic installable capacity of each node in time period t;

[0108] Reward function: R is the reward function of the agent, which is the feedback given by the environment after the action is performed; in the carrying capacity assessment problem, the reward function contains two parts, one is the sum of the photovoltaic configuration capacity of all nodes in the system, and the other is the penalty term for the violation of the operating constraints of the system under the current photovoltaic configuration capacity, which can be expressed as:

[0109] R t =R(S t ,A t )=(e T A t )-(1-κ t )α (9)

[0110] In the formula, κ t is a Boolean variable, α is a large real number, which is used as a penalty term; in the photovoltaic configuration scenario A t Under this condition, the node voltage and current meet the safety constraints, then κ t =1, then the value of the reward function is equal to the sum of the node photovoltaic configuration capacity. Otherwise, κ t =0, the value of the reward function is negative. By introducing the penalty term, the risk of system safety operation caused by overestimation of photovoltaic load capacity can be avoided during the strategy learning process.

[0111] Based on the defined reward function, the goal of the Markov decision process is to find an optimal evaluation strategy π to maximize the expectation of the cumulative reward, that is, the action value function. It can be expressed as:

[0112]

[0113] Among them, G t represents the cumulative reward, G t =R t +γR t+1 +γ 2 R t+2+…; In the framework of Markov decision process, the intelligent agent determines the state S of the distribution network (DN) at time period t. t Generate the photovoltaic access capacity A of each node t Then, A t is transmitted to the distribution network for power flow calculation, thereby obtaining a reward R t .

[0114] The above operation is repeated at the next time step t+1. Finally, a series of strategies π that can maximize the action value are learned through closed-loop iteration. * .

[0115] The present invention uses the Soft Actor-Critic (SAC) algorithm as the basic framework to realize the construction of a deep reinforcement learning algorithm. The SAC algorithm achieves a balance between exploration and utilization by introducing the maximum entropy framework, and adopts an off-policy training method to improve sample utilization efficiency. In order to further improve the performance of the algorithm, this method introduces GraphSAGE to improve the SAC algorithm. It has the following advantages: ① The graph neural network can adaptively learn the relationship between the operating states of each node in the distribution network, overcoming the limitation that traditional neural networks are difficult to process structured data; ② The introduction of the attention mechanism enables the algorithm to dynamically adjust the weights of different electrical connection nodes, improving the perception of key operating characteristics of the distribution network; ③ The use of the full graph representation enables the model to consider the operating state of the entire distribution network when making control decisions, rather than just focusing on the local characteristics of a single feeder or region.

[0116] In the specific implementation process, the algorithm includes a graph strategy network π φ (A t |S t ), Double Q value network and Target Q-value network and and core components such as the experience replay buffer;

[0117] (1) Graph Strategy Network

[0118] In the policy network, the system state S t Modeled as a property graph The vertex set represents the node set of the distribution network, the edge set ε represents the physical connection relationship between nodes, and the vertex attribute matrix X contains the real-time status information of each node, as shown in formula (6). GraphSAGE constructs the vertex representation vector layer by layer through the following aggregation and update steps:

[0119]

[0120] in, represents the representation vector of vertex v at the kth layer, represents the neighbor set of vertex v, W (k) represents the weight vector of the kth layer, AGG (k) is the aggregation function of the kth layer. In this embodiment, the weighted average based on the attention mechanism is used as the aggregation function:

[0121]

[0122] in, represents the attention weight, which can be calculated by learning the attention network.

[0123] The GraphSAGE network in this embodiment contains three layers of graph convolution, and the hidden dimensions of each layer are 128, 256, and 256 respectively. After the last layer of graph convolution, the representation vectors of all vertices are aggregated into a full-graph representation vector through a global average pooling layer:

[0124]

[0125] Full image representation vector It is then processed through two fully connected layers (256 neurons each, using the Leaky ReLU activation function) and finally outputs the mean μ of the action distribution φ (S t ) and logarithmic standard deviation logσ φ (S t ). The process of sampling action adopts the reparameterization technique and can be expressed as:

[0126]

[0127] Among them, φ represents the parameter set of the policy network, ∈ represents Gaussian noise, represents a Gaussian distribution with a mean of 0 and a variance of β.

[0128] (2) Q-value network

[0129] The Q-value network adopts a four-layer feedforward neural network structure. The input layer receives the concatenated vector of the state-action pair, and the dimension is the sum of the state dimension and the action dimension. The network includes two hidden layers (256 neurons each, using the Leaky ReLU activation function) and an output layer with scalar output. In order to improve the training stability, the present invention adopts a dual Q network architecture and updates the target network parameters through Polyak averaging:

[0130] θ i′ ←τθ i +(1-τ)θ i′ ,i=1,2 (16)

[0131] Among them, θ i represents the parameter set of the i-th Q-value network, θ i′ represents the parameter set of the i-th target Q value network, the soft update coefficient τ=0.005, during the training process, the algorithm maintains a capacity of 10 6 Experience Replay Buffer Used to store the transfer samples obtained by interactive sampling (S t ,A t ,R t ,S t+1 ). Each time an update is performed, a mini-batch of size 256 is randomly sampled from the buffer for training.

[0132] (3) Network parameter update process

[0133] The core of the SAC algorithm lies in the design of its objective function. The loss function of the policy network combines the maximization of Q value and the maximization of policy entropy:

[0134]

[0135] The temperature parameter α is automatically adjusted by the following objective function:

[0136]

[0137] Target entropy Set to the negative value of the action space dimension. The loss function of the Q-value network uses the mean square error of the temporal difference (TD) error:

[0138]

[0139] The target value y t The calculation of is integrated into the maximum entropy framework:

[0140]

[0141] The method of solving the Markov decision model using the graph deep reinforcement learning algorithm and obtaining the trained graph strategy network is as follows:

[0142] ① Initialization phase: First, initialize the parameters of the graph strategy network, cost evaluation network, and reward evaluation network; create an empty experience playback buffer Used to store experience data on the interaction between the intelligent agent and the distribution network environment;

[0143] ② Experience collection: Experience data is collected by executing actions output by the action network in the distribution network environment; including the distribution network operation status, the evaluation results of the photovoltaic output of each node, and the reward value, which can be expressed as {S t,A t ,R t ,S t+1}, each set of experience data is stored in the buffer;

[0144] ③Data sampling: randomly extract a batch of experience data from the playback buffer for subsequent network training;

[0145] ④ Evaluation network update: Use the Q value network and the policy network to estimate the target Q value of each sampled experience, calculate the mean square error loss between the target Q value and the actual Q value, and update the parameters θ of the Q value evaluation network accordingly i ;

[0146] ⑤ Policy network update: Apply the gradient ascent method to update the parameter φ of the policy network to optimize the action selection of the policy network, that is, the photovoltaic access capacity of each node;

[0147] ⑥Soft update target network: Use soft update method to gradually adjust the parameters θ of the target Q value network i′ , to ensure the stability of network parameter updates;

[0148] ⑦ Training iteration: Repeat steps ② to ⑥ until the set number of training times is reached.

[0149] Finally, the trained photovoltaic carrying capacity dynamic assessment strategy is implemented in the distribution system through the online deployment of the strategy network; specifically, after the algorithm training is completed, the parameters of the action network are fixed and deployed to the distribution network dispatching cloud platform. In the actual photovoltaic carrying capacity assessment process, the algorithm only provides the calculation results of the photovoltaic carrying capacity based on the real-time observation status of the distribution network, without relying on complex calculations. This method enables the carrying capacity results to respond quickly to changes in the operating status of the distribution network, improves real-time performance and reliability, and reduces the impact of distributed photovoltaic and load uncertainties on the assessment results.

[0150] As an example, in this embodiment, an IEEE 33-node distribution system is taken as an example to illustrate a real-time evaluation method for distributed photovoltaic carrying capacity of an active distribution network based on graph reinforcement learning described in the present invention, wherein the system includes 32 load nodes and 9 photovoltaic power generation systems.

[0151] (1) First, we construct an MDP for the dynamic evaluation of the PV carrying capacity of the distribution network, which mainly includes the following elements:

[0152] State space: S is the state space of the agent, which contains all the state information in the environment (i.e., the actual distribution network), including the active power and reactive power of the load and the timestamp of the system operation. In time period t, the observed state of the agent can be defined as:

[0153] S t =(Pload,t ,Q load,t ,P pv,t ,Q pv,t ) (twenty one)

[0154] Among them, P load,t ,Q load,t They represent the active and reactive load demand vectors of the 33-bus system in period t, which are two 33×1 vectors; P pv,t ,Q pv,t They represent the active and reactive output of photovoltaic power in the distribution network in period t, which are two 33×1 vectors. If the node is not equipped with photovoltaic power, the position value of the corresponding node in the vector is always 0.

[0155] Action space: A is the action space of the agent, that is, the photovoltaic access capacity of each node in the distribution network. If the number of distribution network nodes is N, the action of the agent can be defined as:

[0156] A t =[a 1,t ,a 2,t ,...,a N,t ] (twenty two)

[0157] Among them, a N,t is a continuous variable, indicating the photovoltaic access capacity of the Nth node in period t; in this embodiment, N is 33;

[0158] State transfer function: According to the above state space and action space, the transfer function of the state variables in two adjacent time periods can be expressed as:

[0159]

[0160] Among them, S t+1 is the state vector at time period t, A t is the evaluation result of the PV installable capacity of each node in period t, i.e., the action vector in formula (22);

[0161] Reward function: R is the reward function of the agent, which is the feedback given by the environment after the action is performed; in the carrying capacity assessment problem, the reward function contains two parts, one is the sum of the photovoltaic configuration capacity of all nodes in the system, and the other is the penalty term for the violation of the operating constraints of the system under the current photovoltaic configuration capacity, which can be expressed as:

[0162] R t =R(S t ,A t )=(e T A t )-(1-κ t )α (24)

[0163] In the formula, κ t is a Boolean variable. α is a large real number, which is used as a penalty term. In this embodiment, α=1000. t Under this condition, the node voltage and current meet the safety constraints, then κ t =1, then the value of the reward function is equal to the sum of the node photovoltaic configuration capacity. Otherwise, κ t =0, the value of the reward function is negative. By introducing the penalty term, the risk of system safety operation caused by overestimation of photovoltaic load capacity can be avoided during the strategy learning process.

[0164] Based on the defined reward function, the goal of the Markov decision process is to find an optimal evaluation strategy π to maximize the expectation of the cumulative reward, that is, the action value function. It can be expressed as:

[0165]

[0166] Among them, G t represents the cumulative reward, G t =R t +γR t+1 +γ 2 R t+2 +…. Under the MDP framework, the intelligent agent is based on the state S of the distribution network (DN) at time period t t+1 Generate the photovoltaic access capacity A of each node t Then, A t is transmitted to the distribution network for power flow calculation, thereby obtaining a reward R t The above operation is repeated at the next time step t+1. Finally, a series of strategies π that can maximize the action value are learned through closed-loop iteration. * .

[0167] It is worth noting that in the actual distribution network, the change of operating state between adjacent time periods is an uncertain random process, which is affected by weather and user electricity consumption behavior. It is difficult to give a clear mathematical expression for the state transition probability in MDP. Therefore, next, this embodiment adopts a deep reinforcement learning (DRL) algorithm based on a neural network (NN) to learn the optimal strategy for photovoltaic carrying capacity evaluation.

[0168] (2) Construct a graph deep reinforcement learning algorithm to solve the MDP:

[0169] In the specific implementation process, the algorithm includes a graph strategy network π φ (A t |S t ), Double Q value network and Target Q-value network and And core components such as the experience replay buffer.

[0170] (1) Graph Strategy Network

[0171] In the policy network, the system state S t Modeled as a property graph The vertex set represents the node set of the distribution network, the edge set ε represents the physical connection relationship between nodes, and the vertex attribute matrix X contains the real-time status information of each node, as shown in formula (21). GraphSAGE constructs the vertex representation vector layer by layer through the following aggregation and update steps:

[0172]

[0173] in, represents the representation vector of vertex v at the kth layer, represents the neighbor set of vertex v, W (k) represents the weight vector of the kth layer, AGG (k) is the aggregation function of the kth layer. In this invention, the weighted average based on the attention mechanism is used as the aggregation function:

[0174]

[0175] in, represents the attention weight, which can be calculated by learning the attention network.

[0176] The GraphSAGE network in the present invention contains three layers of graph convolution, and the hidden dimensions of each layer are 128, 256, and 256 respectively. After the last layer of graph convolution, a global average pooling layer is used to aggregate the representation vectors of all vertices into a full-graph representation vector:

[0177]

[0178] Full image representation vector It is then processed through two fully connected layers (256 neurons each, using the Leaky ReLU activation function) and finally outputs the mean μ of the action distribution φ (S t ) and logarithmic standard deviation logσ φ (S t ). The process of sampling action adopts the reparameterization technique and can be expressed as:

[0179]

[0180] Among them, φ represents the parameter set of the policy network, ∈ represents Gaussian noise, represents a Gaussian distribution with a mean of 0 and a variance of β.

[0181] (2) Q-value network

[0182] The Q-value network adopts a four-layer feedforward neural network structure. The input layer receives the concatenated vector of the state-action pair, and the dimension is the sum of the state dimension and the action dimension. The network contains two hidden layers (256 neurons each, using the Leaky ReLU activation function) and an output layer with scalar output. In order to improve the training stability, the present invention adopts a dual Q network architecture and updates the target network parameters through Polyak averaging:

[0183] θ i′ ←τθ i +(1-τ)θ i′ ,i=1,2 (30)

[0184] Among them, θ i represents the parameter set of the i-th Q-value network, θ i′ represents the parameter set of the i-th target Q value network, and the soft update coefficient τ=0.005. During the training process, the algorithm maintains a capacity of 10 6 Experience Replay Buffer Used to store the transfer samples obtained by interactive sampling (S t ,A t ,R t ,S t+1 ). Each time an update is performed, a mini-batch of size 256 is randomly sampled from the buffer for training.

[0185] (3) Network parameter update process

[0186] The core of the SAC algorithm lies in the design of its objective function. The loss function of the policy network combines the maximization of Q value and the maximization of policy entropy:

[0187]

[0188] The temperature parameter α is automatically adjusted by the following objective function:

[0189]

[0190] Target entropy Set to the negative value of the action space dimension. The loss function of the Q-value network uses the mean square error of the temporal difference (TD) error:

[0191]

[0192] The target value y t The calculation of is integrated into the maximum entropy framework:

[0193]

[0194] In the actual training process, in order to ensure the convergence of the algorithm, 10,000 steps of pre-training are first performed, and only the Q value network is updated. After the Q value estimation is relatively accurate, all networks are updated simultaneously. During the training process, a policy evaluation is performed after every 1,000 steps of environmental interaction. The training convergence is determined when the average return improvement for 5 consecutive evaluations is less than 1%. At the same time, in order to enhance the robustness of the algorithm, Gaussian noise is introduced when sampling actions. The standard deviation of the noise decays linearly with the training process. The initial value is 0.3 and eventually decays to 0.1.

[0195] In summary, the training process of the proposed graph deep reinforcement learning algorithm can be divided into the following core steps:

[0196] ① Initialization phase: First, initialize the parameters of the graph policy network, cost evaluation network, and reward evaluation network. Create an empty experience playback buffer Used to store experience data on the interaction between the intelligent agent and the distribution network environment;

[0197] ② Experience collection: Collect experience data by executing the actions output by the action network in the distribution network environment. It mainly includes the distribution network operation status, the evaluation results and reward values ​​of the photovoltaic output of each node, etc., which can be expressed as {S t ,A t ,R t ,S t+1}, each set of experience data is stored in the buffer;

[0198] ③Data sampling: randomly extract a batch of experience data from the playback buffer for subsequent network training;

[0199] ④ Evaluation network update: Use the Q value network and the policy network to estimate the target Q value for each sampled experience. Calculate the mean square error loss between the target Q value and the actual Q value, and update the parameters θ of the Q value evaluation network accordingly i ;

[0200] ⑤ Policy network update: Apply the gradient ascent method to update the parameter φ of the policy network to optimize the action selection of the policy network, that is, the photovoltaic access capacity of each node;

[0201] ⑥Soft update target network: Use soft update method to gradually adjust the parameters θ of the target Q value network i′ , to ensure the stability of network parameter updates;

[0202] ⑦ Training iteration: Repeat steps ② to ⑥ until the set number of training times is reached.

[0203] Finally, the trained photovoltaic carrying capacity dynamic assessment strategy is implemented in the distribution system through the online deployment of the strategy network.

[0204] In order to verify the accuracy of the evaluation method proposed in this patent, it is compared and analyzed with the traditional Monte Carlo evaluation algorithm and the standard SAC algorithm. Figure 2 The training curves of different algorithms are given. (Since the Monte Carlo method is a non-learning algorithm, its evaluation result is shown as a horizontal straight line in the figure) Figure 2 It can be observed that compared with the standard SAC algorithm, the load-bearing capacity assessment curve of the method proposed in the present invention can converge to a higher level. This shows that the present invention significantly improves the accuracy of photovoltaic load-bearing capacity assessment by introducing GraphSAGE to improve the SAC algorithm. At the same time, the load-bearing capacity assessment result of the traditional Monte Carlo random scenario method is 164.12MW, which is only 0.69% higher than that of this method, further verifying that the method proposed in the present invention is highly consistent with the actual load-bearing capacity of the system.

[0205] In order to further demonstrate the advantages of the present invention in terms of computational efficiency, Table 1 shows the computational time comparison of each algorithm. (The training time refers to the CPU time required for the deep reinforcement learning algorithm training phase, and the test time refers to the CPU time required for the photovoltaic carrying capacity assessment.)

[0206] Table 1 Comparison of computational efficiency of different algorithms

[0207]

[0208] In terms of training time, since the present invention introduces the strategic network structure of GraphSAGE, its training time is increased compared with the standard SAC algorithm. Figure 2 As shown in Figure 2, the proposed method is significantly better than the traditional SAC algorithm in performance and overcomes the problem of slow convergence of traditional SAC algorithm training. It is worth noting that the traditional Monte Carlo method takes 891.16 seconds to complete the calculation, and its solution efficiency is significantly lower than that of the method based on deep reinforcement learning.

[0209] The advantage of the present invention is that the trained evaluation algorithm can directly evaluate the photovoltaic carrying capacity without complex iterative calculations. Remarkably, a single evaluation cycle takes only 4.23 milliseconds. This efficient calculation speed enables the evaluation method to quickly give evaluation results based on real-time load and distributed generation data, greatly improving the evaluation efficiency.

[0210] In contrast, deep reinforcement learning algorithm, as an artificial intelligence method that combines deep learning and reinforcement learning, has the following significant advantages in PVHC assessment: (1) Strong model adaptability: Deep reinforcement learning learns the optimal PVHC assessment strategy through continuous interaction with the distribution network environment, without the need to explicitly construct an accurate mathematical model, effectively overcoming the limitation of traditional methods that require accurate modeling; (2) High computational efficiency: Compared with traditional optimization algorithms, the trained deep reinforcement learning model can quickly evaluate the PV carrying capacity of the distribution network. It avoids the complex iterative optimization process in traditional methods. Even when new PV access points are added to a large-scale distribution network, its evaluation speed can remain basically stable, showing superior computational efficiency; (3) Possessing online learning capabilities: Deep reinforcement learning can continuously learn from new operating data of the distribution network and continuously optimize its PVHC assessment strategy. This feature enables it to timely adapt to dynamic factors such as changes in distribution network load characteristics and fluctuations in PV output, providing more accurate real-time assessment results.

[0211] Example 2

[0212] This embodiment provides a real-time evaluation device for distributed photovoltaic carrying capacity of a distribution network, including:

[0213] A Markov decision model building module is used to determine a mathematical model for evaluating the photovoltaic carrying capacity of a distribution network, and based on the mathematical model, to build a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of a distribution network;

[0214] The module for acquiring the trained graph policy network is used to use the soft actor-critic SAC algorithm as the basic framework, and improve the SAC algorithm by using graph sampling and aggregation (GraphSAGE), construct a graph deep reinforcement learning algorithm, and use the graph deep reinforcement learning algorithm to solve the Markov decision model to obtain the trained graph policy network;

[0215] The photovoltaic carrying capacity real-time assessment module is used to use the trained graph strategy network to perform subsequent real-time assessment of the distributed photovoltaic carrying capacity of the distribution network.

[0216] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network as described above.

[0217] Furthermore, the present invention adopts the following technical solutions:

[0218] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network as described above is implemented.

[0219] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that the facilities of the present invention can be implemented by means of software plus the necessary general hardware platform. The embodiments of the present invention can be implemented using an existing processor, or by a dedicated processor used for this purpose or other purposes for an appropriate system, or by a hard-wired system. The embodiments of the present invention also include a non-transitory computer-readable storage medium, which includes a machine-readable medium for carrying or having a machine-executable instruction or data structure stored thereon; such a machine-readable medium can be any available medium that can be accessed by a general or special-purpose computer or other machine with a processor. For example, such a machine-readable medium can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures, and can be accessed by a general or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hard wiring, or wireless, or a combination of hard wiring and wireless), the connection is also considered a machine-readable medium.

[0220] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A real-time evaluation method for distributed photovoltaic carrying capacity of a distribution network, characterized in that: The method comprises the following steps: Determine a mathematical model for evaluating the photovoltaic carrying capacity of the distribution network, and construct a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of the distribution network based on the mathematical model; Taking the soft actor-critic SAC algorithm as the basic framework, the SAC algorithm is improved by using graph sampling and aggregation GraphSAGE, and a graph deep reinforcement learning algorithm is constructed. The Markov decision model is solved by the graph deep reinforcement learning algorithm to obtain the trained graph policy network. The trained graph strategy network is used to carry out subsequent real-time evaluation of the distributed photovoltaic carrying capacity of the distribution network.

2. The method for real-time evaluation of distributed photovoltaic carrying capacity of distribution network according to claim 1, characterized in that: The method to determine the mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is: The mathematical model for evaluating the photovoltaic carrying capacity of the distribution network is expressed as an optimization problem with the goal of maximizing the carrying capacity, where the decision variable is the accessible capacity of the photovoltaic power generation at each node, which can be expressed as: Where N represents the node set, represents the photovoltaic access capacity of node i in time period t; The system also includes operation constraints of the distribution network, which include power balance constraints, node voltage constraints and line power constraints.

3. The real-time evaluation method for distributed photovoltaic carrying capacity of distribution network according to claim 2 is characterized in that: Based on the mathematical model, the method of constructing a Markov decision model for dynamic evaluation of photovoltaic carrying capacity of distribution network is as follows: The state space of the intelligent agent is constructed, which contains all the state information in the distribution network, including the active power and reactive power of the load and the timestamp of the system operation. In the period t, the observed state of the intelligent agent can be defined as: S t =(P load,t ,Q load,t ,P pv,t ,Q pv,t ) (6) Among them, P load,t ,Q load,t Respectively represent the active and reactive load demand vectors in period t; P pv,t ,Q pv,t They represent the active and reactive output vectors of photovoltaic power in the distribution network during period t. If the node is not equipped with photovoltaic power, the position value of the corresponding node in the vector is always 0. Construct the action space of the intelligent agent. If the number of distribution network nodes is N, the action of the intelligent agent can be defined as: A t =[a 1,t ,a 2,t ,…,a N,t ] (7) Among them, a N,t is a continuous variable, indicating the photovoltaic access capacity of the Nth node in period t; According to the above state space and action space, the transfer function of the state variables of two adjacent time periods can be expressed as: Among them, S t+1 is the state vector at time period t, A t is the evaluation result of the photovoltaic installable capacity of each node in time period t; Obtain the agent's reward function, which includes the sum of the photovoltaic configuration capacities of all nodes in the system and the penalty term for the violation of the system's operating constraints under the current photovoltaic configuration capacity, which can be expressed as: R t =R(S t ,A t )=(e T A t )-(1-k t )a (9) In the formula, κ t is a Boolean variable, α is a large real number, which is used as a penalty term; in the photovoltaic configuration scenario A t Under this condition, the node voltage and current meet the safety constraints, then κ t =1, then the value of the reward function is equal to the sum of the photovoltaic configuration capacity of the node; otherwise, κ t =0, the value of the reward function is negative; Based on the defined reward function, the goal of the Markov decision process is to find an optimal evaluation strategy π to maximize the expectation of the cumulative reward, which can be expressed as: Among them, G t represents the cumulative reward, G t =R t +γR t+1 +γ 2 R t+2 +…; In the framework of Markov decision process, the intelligent agent determines the state S of the distribution network in time period t. t Generate the photovoltaic access capacity A of each node t ; Then, A t is transmitted to the distribution network for power flow calculation, thereby obtaining a reward R t ; The above operation is repeated at the next time step t+1. Finally, a series of strategies π that can maximize the action value are learned through closed-loop iteration. * .

4. The real-time evaluation method for distributed photovoltaic carrying capacity of distribution network according to claim 3 is characterized in that: Taking the soft actor-critic SAC algorithm as the basic framework, and using graph sampling and aggregation GraphSAGE to improve the SAC algorithm, the graph policy network is introduced into the SAC algorithm. In the graph policy network, the system state S t Modeled as a property graph The vertex set represents the node set of the distribution network, the edge set ε represents the physical connection relationship between nodes, and the vertex attribute matrix X contains the real-time status information of each node, as shown in formula (6); GraphSAGE builds the vertex representation vector layer by layer through the following aggregation and update steps: in, represents the representation vector of vertex v at the kth layer, represents the neighbor set of vertex v, W (k) represents the weight vector of the kth layer, AGG (k) is the aggregation function of the kth layer, and the weighted average based on the attention mechanism is used as the aggregation function: in, represents the attention weight, which can be calculated by learning the attention network; The GraphSAGE network contains three layers of graph convolution, with hidden dimensions of each layer being 128, 256, and 256 respectively. After the last layer of graph convolution, a global average pooling layer is used to aggregate the representation vectors of all vertices into a full-graph representation vector: Full image representation vector h g Then it is processed by two fully connected layers of the SAC algorithm, and finally outputs the mean μ of the action distribution φ (S t ) and logarithmic standard deviation logσ φ (S t ); The sampling process uses the re-parameterization technique and can be expressed as: Among them, φ represents the parameter set of the policy network, ε represents Gaussian noise, represents a Gaussian distribution with a mean of 0 and a variance of β.

5. The real-time evaluation method for distributed photovoltaic carrying capacity of distribution network according to claim 4 is characterized in that: The method of solving the Markov decision model using the graph deep reinforcement learning algorithm and obtaining the trained graph strategy network is as follows: ① Initialization phase: First, initialize the parameters of the graph strategy network, cost evaluation network, and reward evaluation network; create an empty experience playback buffer Used to store experience data on the interaction between the intelligent agent and the distribution network environment; ② Experience collection: Experience data is collected by executing actions output by the action network in the distribution network environment; including the distribution network operation status, the evaluation results of the photovoltaic output of each node, and the reward value, which can be expressed as {S t ,A t ,R t ,S t+1 }, each set of experience data is stored in the buffer; ③Data sampling: randomly extract a batch of experience data from the playback buffer for subsequent network training; ④ Evaluation network update: Use the Q value network and the policy network to estimate the target Q value of each sampled experience, calculate the mean square error loss between the target Q value and the actual Q value, and update the parameters θ of the Q value evaluation network accordingly i ; ⑤ Policy network update: Apply the gradient ascent method to update the parameter φ of the policy network to optimize the action selection of the policy network, that is, the photovoltaic access capacity of each node; ⑥Soft update target network: Use soft update method to gradually adjust the parameter θ of the target Q value network i′ , to ensure the stability of network parameter updates; ⑦ Training iteration: Repeat steps ② to ⑥ until the set number of training times is reached.

6. The method for real-time evaluation of distributed photovoltaic carrying capacity of distribution network according to claim 5, characterized in that: The Q-value network adopts a four-layer feedforward neural network structure. The input layer receives the concatenated vector of the state-action pair, and the dimension is the sum of the state dimension and the action dimension. The network contains two hidden layers and an output layer with a scalar output. It adopts a dual Q network architecture and updates the target network parameters through Polyak averaging: i i′ ←tth i +(1-τ)θ i′ ,i=1.2 (30) Among them, θ i represents the parameter set of the i-th Q-value network, θ i′ represents the parameter set of the i-th target Q value network, and the soft update coefficient τ = 0.005; during the training process, the algorithm maintains a capacity of 10 6 Experience Replay Buffer Used to store the transfer samples obtained by interactive sampling (S t ,A t ,R t ,S t+1 ), each time an update is performed, a mini-batch of size 256 is randomly sampled from the buffer for training.

7. The method for real-time evaluation of distributed photovoltaic carrying capacity of distribution network according to claim 5, characterized in that: When updating the network parameters of the graph deep reinforcement learning algorithm, the loss function of the policy network combines the maximization of Q value and the maximization of policy entropy: The temperature parameter α is automatically adjusted by the following objective function: Target entropy Set to the negative value of the action space dimension; The loss function of the Q-value network uses the mean square error of the temporal difference error: The target value y t The calculation of is integrated into the maximum entropy framework:

8. A real-time evaluation device for distributed photovoltaic carrying capacity of a distribution network, characterized in that: include: A Markov decision model building module is used to determine a mathematical model for evaluating the photovoltaic carrying capacity of a distribution network, and based on the mathematical model, to build a Markov decision model for dynamic evaluation of the photovoltaic carrying capacity of a distribution network; The module for obtaining the trained graph policy network is used to use the soft actor-critic SAC algorithm as the basic framework, and improve the SAC algorithm by using graph sampling and aggregation GraphSAGE, to construct a graph deep reinforcement learning algorithm, and to solve the Markov decision model by using the graph deep reinforcement learning algorithm to obtain the trained graph policy network; The photovoltaic carrying capacity real-time assessment module is used to use the trained graph strategy network to perform subsequent real-time assessment of the distributed photovoltaic carrying capacity of the distribution network.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for real-time evaluation of distributed photovoltaic carrying capacity of a distribution network as described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the real-time evaluation method for distributed photovoltaic carrying capacity of the distribution network as described in any one of claims 1 to 7 is implemented.

Images