SAC-HELM-based power grid load flow convergence adjustment method

Abstract

The invention provides a power grid power flow convergence adjustment method based on SAC-HELM, and the method comprises the following steps: 1, building a power grid model through a power flow calculation method based on a full-pure function embedding power flow calculation method, and introducing a voltage stability judgment index and a voltage collapse point index; and step 2, adding an index for judging power flow convergence, and screening out generators and loads participating in power flow adjustment. Step 3, establishing an SAC model based on HELM, wherein the reward of the SAC is set by combining a voltage stability judgment index and a voltage collapse point index; and step 4, training and testing the model to obtain a load flow convergence adjustment result. According to the method, the power grid can quickly find a convergence solution, and the workload is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of electric power information technology, and in particular to a power grid power flow convergence adjustment method based on SAC-HELM. Background Art

[0002] The adjustment of power flow convergence is the basis for ensuring the safe and stable operation of the power grid. Through reasonable power flow calculation and adjustment, the power system can be ensured to remain stable under different operating conditions, thereby avoiding potential risks such as overload and instability. Due to the complexity and uncertainty of the power system, the problem of power flow non-convergence may be caused by two reasons: one is that there is no solution in the system itself, and the other is that the algorithm cannot effectively find a solution. Therefore, when solving the problem of power flow non-convergence, it is necessary to comprehensively consider factors such as algorithm improvement and optimization of power grid operation mode, and adopt appropriate methods and technical means to improve the stability and reliability of power grid power flow convergence.

[0003] At present, many solutions have been proposed to optimize traditional power flow calculation methods. References [1] Z. Liu, X. Zhang, M. Su, Y. Sun, H. Han, P. Wang, Convergence Analysis of Newton-Raphson Method in Feasible Power-Flow for DC Network, IEEE Trans. Power Syst. 35 (2020) 4100–4103. https: / / doi.org / 10.1109 / TPWRS.2020.2986706 Based on the Newton-Raphson method, a simple initial value setting method is proposed to improve the convergence of power flow. Reference [2] K. Tang, S. Dong, J. Shen, C. Zhu, Y. Song, A Robust and Efficient Two-Stage Algorithm for Power Flow Calculation of Large-Scale Systems, IEEE Trans. Power Syst. 34 (2019) 5012–5022. https: / / doi.org / 10.1109 / TPWRS.2019.2914431 , a two-stage power flow calculation method for large systems based on Newton's method is proposed. Reference [3] L. Guo, Y. Zhang, X. Li, Z. Wang, Y. Liu, L. Bai, C. Wang, Data-Driven Power Flow Calculation Method: A Lifting Dimension Linear Regression Approach, IEEE Trans. Power Syst. 37 (2022) 1798–1808. https: / / doi.org / 10.1109 / TPWRS.2021.3112461, a dimensional linear regression method is proposed, which can significantly improve the calculation accuracy by converting the nonlinear relationship in the power flow calculation into a linear mapping in the high-dimensional state space. Reference [4] Y. Zhou, J. Zhang, W. Gao, Fast Calculation Method for Continuous Power Flow of Microgrid Based on Levenberg-Marquardt Algorithm, IEEE Access 11 (2023) 49578–49586. https: / / doi.org / 10.1109 / ACCESS.2023.3249781, a fast calculation method for continuous power flow of microgrid based on Levenberg-Marquardt algorithm is proposed. This method not only ensures the convergence of power flow calculation, but also ensures the efficiency of calculation. Reference [5] Y.Xu, L.Mili, J.Zhao, Probabilistic Power Flow Calculation and Variance Analysis Based on Hierarchical Adaptive Polynomial Chaos-ANOVA Method, IEEE Trans. Power Syst. 34 (2019) 3316–3325. https: / / doi.org / 10.1109 / TPWRS.2019.2903164, a hierarchical adaptive polynomial chaos variance analysis method is proposed, which has excellent performance, including accuracy, efficiency, rationality and adaptability. Reference [6] T. Wang, S. Wang, S. Ma, J. Guo, X. Zhou, An Extended Continuation Power Flow Method for Static Voltage Stability Assessment of Renewable Power Generation-Penetrated Power Systems, IEEE Trans. Circuits Syst. II Express Briefs 71 (2024) 892–896. https: / / doi.org / 10.1109 / TCSII.2022.3209335, an extended continuation power flow method is proposed to evaluate the static voltage stability of renewable power generation-penetrated power systems.Reference [7] Z. Fan, Z. Yang, J. Yu, K. Xie, G. Yang, Minimize Linearization Error of Power Flow Model Based on Optimal Selection of Variable Space, IEEE Trans. Power Syst. 36 (2021) 1130–1140. https: / / doi.org / 10.1109 / TPWRS.2020.3012894, proposed a linear power flow model with minimized linear error, which can effectively reduce the linearization error under large-scale load fluctuations.

[0004] However, the above traditional power flow methods are not suitable for complex power systems. Therefore, the literature [8–11][8]T.Wang,H.-D.Chiang,On the Holomorphic and Conjugate Properties forHolomorphic Embedding Methods for Solving Power Flow Equations,IEEE Trans.Power Syst.35(2020)2506–2515.https: / / doi.org / 10.1109 / TPWRS.2019.2958870.

[0005] [9] C. Liu, N. Qin, K. Sun, CLBak, Remote Voltage Control Using the Holomorphic Embedding Load Flow Method, IEEE Trans. Smart Grid 10 (2019) 6308–6319. https: / / doi.org / 10.1109 / TSG.2019.2901865.

[0006]

[10] S.Zhou, M.Wang, J.Wang, M.Yang,

[0007]

[11] R.Yao, K.Sun, D.Shi, X.Zhang, Voltage Stability Analysis of Power Systems With Induction Motors Based on Holomorphic Embedding, IEEE Trans.PowerSyst.34(2019)1278–1288.https: / / doi.org / 10.1109 / TPWRS.2018.2876429, proposed a Holomorphic embedding method (HELM). HELM is based on the theory of holomorphic functions. It embeds the nonlinear power flow equation of the power system into a complex space and converts it into a linear equation for solution. The specific steps include: first, the power flow equation is converted into a holomorphic function by embedding parameters; then, the equation is expanded into a power series to convert the complex nonlinear problem into a series of linear equations; then, the power series is solved term by term to gradually approach the true solution of the system; the good convergence of the power series is used to avoid the divergence problem in the traditional method; finally, the node voltage and power flow distribution of the system are restored by the solution of the power series.

[0008] For power flow methods that require iterative calculations, the advantage of the holomorphic embedded power flow method is that it avoids the iterative process, thereby greatly reducing the computational complexity and convergence problems. In contrast, the holomorphic embedded method directly solves the power flow equation analytically, so it is more reliable.

[0009] At the same time, the application of power flow calculation in artificial intelligence is also increasing. Reference

[12] P. Li, W. Wu, X. Wang, B. Xu, A Data-Driven Linear Optimal Power Flow Model for Distribution Networks, IEEE Trans. Power Syst. 38 (2023) 956–959. https: / / doi.org / 10.1109 / TPWRS.2022.3216161, proposed a data-driven linear optimal power flow model for distribution networks, combining support vector regression (SVR) and ridge regression (RR) algorithms, which is robust to bad data in measurement. Reference

[13] M.Ahmad, N.Javaid, IANiaz, I.Ahmed, MAHashmi, An Orthogonal Learning Bird Swarm Algorithm for Optimal Power Flow Problems, IEEE Access 11 (2023) 23659–23680. https: / / doi.org / 10.1109 / ACCESS.2023.3253796, proposed an orthogonal learning bird swarm algorithm, which can be used to solve the optimal power flow problem. It can effectively reduce the number of experimental test cases by predicting the decision parameter values ​​of the best combination. Reference

[0010]

[14] X.Hu,H.Hu,S.Verma,Z.-L.Zhang,Physics-Guided Deep Neural Networksfor Power Flow Analysis,IEEE Trans.Power Syst.36(2021)2082–2092.https: / / doi.org / 10.1109 / TPWRS.2020.3029557,A physics-guided neural network is proposed to solve the power flow problem with high solution accuracy and good physical consistency. References

[15] Y.Tao,J.Qiu,S.Lai,A Supervised-Learning Assisted Computation Method for Power System Planning,IEEE Trans.Artif.Intell.3(2022)819–832.https: / / doi.org / 10.1109 / TAI.2021.3133821,Supervised learning method is used for power flow calculation. With the help of data-driven methods, the amount of calculation can be reduced and the power flow can be calculated more accurately than the traditional linear model. However, the above algorithms have limited generalization capabilities and may perform poorly for new and unseen situations, which requires further improvement and optimization. In addition, in complex power systems, it is crucial to ensure the accuracy and convergence of power flow calculations, but the effect is still poor at this stage. Summary of the invention

[0011] In view of the deficiencies of the prior art, the present invention proposes a power grid power flow convergence adjustment method based on SAC-HELM, which can enable the power grid to quickly find a convergence solution and greatly reduce the workload.

[0012] In order to solve the above technical problems, the technical solution of the present invention is:

[0013] A method for adjusting power flow convergence based on SAC-HELM, the method comprising the following steps:

[0014] Step (1) Use the power flow calculation method based on the Holomorphic embedding method (HELM) to establish a power grid model and introduce the voltage stability index VSI HELM1i and Voltage Collapse Indicator VSI HELM2i Then, calculate the VSI of each node in the power grid HELM1i , and find the average VSI of these nodes ave . Calculate the VSI of each node in the power grid HELM2i value, and find the maximum VSI among them max .

[0015] Step (2) Add an indicator L to judge the convergence of the power flow ij , select the generators and loads that participate in power flow adjustment.

[0016] Step (3) establishing a SAC model based on HELM;

[0017] Step (4) trains and tests the model to obtain the result of the power flow convergence adjustment.

[0018] As a preferred method, the sensitivity of each order of node voltage is calculated based on HELM, thereby introducing the voltage stability index VSI HELM1i and Voltage Collapse Indicator VSI HELM2i Then, calculate the VSI of each node in the power grid HELM1i , and find the average VSI of these nodes ave . Calculate the VSI of each node in the power grid HELM2i value, and find the maximum VSI among them max At the same time, an indicator L is added to judge the convergence of the power flow ij , check the stable status of each transmission line and select the generators and loads involved in power flow adjustment.

[0019] As a preferred method, a HELM-based SAC environment is established, the power flow calculation is performed using the HELM method, and a voltage stability index L based on the power flow solution is introduced. ij , and the voltage stability index VSI based on sensitivity analysis HELM1i and Voltage Collapse Indicator VSI HELM2i Then, calculate the VSI of each node in the power grid HELM1i , and find the average VSI of these nodes ave . Calculate the VSI of each node in the power grid HELM2i value, and find the maximum VSI among them max Calculate the voltage stability index L of each node ij , filter out nodes where voltage collapse occurs, thereby reducing the number of generators and loads involved in the adjustment, shortening the unit output and load active power adjustment time, and improving grid stability. Then, construct a SAC model and combine it with VSI ave and VSI max The trained model can monitor the power grid flow data in real time, make intelligent adjustments based on real-time changes, and quickly respond to the impact of power flow fluctuations to ensure the continuous and stable operation of the power grid.

[0020] The present invention has the following characteristics and beneficial effects:

[0021] Using the above technical solution, it is proposed to combine the Holomorphic embedding method (HELM) with the deep reinforcement learning algorithm of Soft Actor Critic (SAC) to deal with the power grid power convergence problem. The power grid power convergence adjustment method based on HELM and SAC, the voltage collapse point index and voltage stability index calculated by HELM, can be used to evaluate whether the power flow solution of the system is in a stable state, and can guide the power grid power flow to adjust in a stable direction. The SAC algorithm makes full use of the advantages of deep learning and reinforcement learning algorithms. It can not only use neural networks to learn the complex nonlinear characteristics of the power grid, but also effectively deal with high-dimensional problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 This is a schematic diagram of a power flow convergence adjustment method based on SAC deep reinforcement learning in an embodiment of the present invention.

[0024] Figure 2 FIG. 4 is a diagram of a SAC network structure in an embodiment of the present invention.

[0025] Figure 3 This is a training flow chart of an embodiment of the present invention.

[0026] Figure 4 Comparison of IEEE14 adjustment results.

[0027] Figure 5 Comparison of IEEE57 adjustment results. DETAILED DESCRIPTION

[0028] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0029] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0030] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0031] Example 1

[0032] The present invention provides a power grid power flow convergence adjustment method based on SAC-HELM, and proposes to combine the Holomorphic embedding method (HELM) with the deep reinforcement learning algorithm of Soft Actor Critic (SAC) to deal with the power grid power flow convergence problem. The power grid power flow convergence adjustment method based on HELM and SAC, the voltage collapse point index and voltage stability index calculated by HELM can be used to evaluate whether the power flow solution of the system is in a stable state, and can guide the power grid power flow to adjust in a stable direction. The SAC algorithm makes full use of the advantages of deep learning and reinforcement learning algorithms, and can not only use neural networks to learn the complex nonlinear characteristics of the power grid, but also can effectively deal with high-dimensional problems.

[0033] Specifically, the present invention proposes to use the holomorphic embedded power flow method to establish a power grid model. The power flow calculation is performed using the HELM method, and a voltage stability index L based on the power flow solution is introduced. ij , and the voltage stability index VSI based on sensitivity analysis HELM1i and Voltage Collapse Indicator VSI HELM2iThen, calculate the VSI of each node in the power grid HELM1i , and find the average VSI of these nodes ave . Calculate the VSI of each node in the power grid HELM2i value, and find the maximum VSI among them max Calculate the voltage stability index L of each node ij , filter out nodes where voltage collapse occurs, thereby reducing the number of generators and loads involved in the adjustment, shortening the unit output and load active power adjustment time, and improving grid stability. Then, construct a SAC model and combine it with VSI ave and VSI max The trained model can monitor the power grid flow data in real time, make intelligent adjustments based on real-time changes, and quickly respond to the impact of power flow fluctuations to ensure the continuous and stable operation of the power grid.

[0034] In this embodiment, firstly, based on the SAC environment of HELM, a 14-node and 57-node power grid model is established to initialize the power grid flow data. Initialize network parameters ψ, φ and θ.

[0035] In the process of the SAC algorithm, at the beginning of training, the HELM method is first used to analyze the current state s t The power flow data of the power grid in the current state can be obtained by calculating the power flow model under the current power grid. According to formulas (3)-(8), the stability index is calculated, and action a t In order to determine the generators and loads involved in power flow adjustment, a t Get t+1 and reward R, then (s t , a t ,s t+1 , R) and store it in the experience pool , which is used for subsequent training. As the data in the experience pool is continuously updated, the network parameters will also be continuously adjusted and updated. The training process will continue until the stopping condition is met.

[0036] The specific steps are as follows:

[0037] Firstly, the power grid model is established by using the HELM-based power flow calculation method, and the voltage stability index VSI is introduced. HELM1i and Voltage Collapse Indicator VSI HELM2i .

[0038] The sensitivity of each order of node voltage is calculated based on HELM, and the voltage stability index and voltage collapse point index are introduced.

[0039] Furthermore, the sensitivity of each order of node voltage can be calculated according to HELM. First, an embedded pure virtual function is constructed, as shown in formula (1):

[0040]

[0041] Where V i [n] represents the nth voltage component of the voltage at node i in the HELM power flow calculation, s n represents the nth term of the frequency domain operator s;

[0042] Thus, the nonlinear total sensitivity of voltage to injected power can be obtained: The expression is as follows:

[0043]

[0044] In the formula, Q j represents the reactive power of node j, P j represents the reactive power of node j, The nth coefficient term representing the voltage at node i is used to find the partial derivative of the reactive power at node j. The nth coefficient term representing the voltage at node i is used to obtain the partial derivative of the active power at node j, where: and is the sensitivity of node voltage at each level.

[0045] Then we can get the value of voltage stability index VSI HELM1i The calculation formula is:

[0046]

[0047] Calculate the VSI of each node HELM1i Values, and calculate the average VSI of these nodes ave .

[0048]

[0049] Where: VSI HELM1i is the value of the voltage stability index, node i is the bus number of the distribution network (i∈2~T), node j is the bus number of reactive power injection, and real refers to the real part of the corresponding complex number. Q j represents the reactive power of node j, V i [n] represents the nth voltage component of the voltage at node i in the HELM power flow calculation. T represents the total number of nodes. The nth coefficient term representing the voltage at node i is used to calculate the partial derivative of the reactive power at node j. HELM1i The smaller the value, the better the voltage stability of the system.

[0050] In this embodiment, the IEEE14 power grid model is constructed using a power flow calculation method based on holomorphic function embedding. The power flow data of each node and each branch obtained by the two methods are compared by using HELM and pandapower to calculate the power flow of the 14-node power grid model. The results show that the power flow calculation results of the two methods are consistent.

[0051] Furthermore, in order to determine the voltage stability of the system as a whole, an indicator VSI is added HELM2i .

[0052]

[0053] When VSI HELM2i <0, the system is stable, when VSI HELM2i >0, the distribution network system voltage has reached the voltage collapse point.

[0054] Calculate the VSI of each node HELM2i Values, and find the maximum value VSI max .

[0055]

[0056] Based on the above technical solution, in this embodiment, the IEEE14 power grid model is constructed using a power flow calculation method based on holomorphic function embedding. The power flow data of each node and each branch obtained by the two methods are compared by using HELM and pandapower to calculate the power flow of the 14-node power grid model. The results show that the power flow calculation results of the two methods are consistent.

[0057] Furthermore, an indicator L is added to judge the convergence of the power flow ij , and screen out the generators and loads participating in power flow adjustment.

[0058] Understandable, VSI HELM1i and VSI HELM2i The data output by the indicator is node data, so increase L ij Indicators to check the stability of each transmission line. Add an indicator L to judge the convergence of power flow ij , L ij The calculation formula is as follows:

[0059]

[0060] Where: X ij R represents the reactance of the branch connecting node i and node j; ij Represents the resistance of the branch connecting node i and node j; represents the square of the voltage at node i; represents the fourth power of the voltage at node i;

[0061] When L ij When ≤1, the distribution network is in a voltage stable state, L ij The smaller the index, the higher the voltage stability; when L ij When the index is close to 1, it means that the voltage stability of the system decreases and there is a risk of voltage collapse. ij When ≥1, the system voltage has lost stability and entered a voltage collapse state, and the generators and loads participating in the power flow adjustment are screened out.

[0062] Then the L of each line in the power grid can be calculated ij value, and select the maximum value as L max .

[0063]

[0064] In the current non-convergent state, use HELM to calculate L ij Indicators, find out the voltage stability index L ij ≥1. The selected L ij The values ​​are sorted from large to small, and a certain proportion of L is selected based on the sorting results and the total number of grid nodes. ij value.

[0065] Based on the above technical solution, in this embodiment, for the current power grid flow not converging, the L of each line is calculated. ij Indicator. Output L ij ≥1, the calculation results are shown in Table 1: For a power grid with fewer nodes (such as the IEEE 14-node system), select all L ij ≥1 corresponding to the node i and node j; for the case of a large number of grid nodes, for a grid with a large number of nodes (such as the IEEE57 node system), select the L in the top 30% or so ij The corresponding nodes are nodes i and j. Then, the power flow is adjusted for the generators and loads at these nodes.

[0066] Table 1 L of each branch of IEEE14 power grid model ij Value

[0067]

[0068] According to the calculation results, the loads and nodes corresponding to node numbers 2, 3, 4, and 5 are selected for power flow adjustment.

[0069] The second step is to establish a SAC model based on HELM. The SAC model includes the environment and the agent.

[0070] Specifically, we need to build the environment first.Gi , Active power of load P Li , VSI of each node HELM1i The mean of the values ​​and the VSI in each node HELM2i The maximum value of state s t :

[0071]

[0072] In the formula, Represents the generator G i The active power, Indicates load L i Active power, VSI ave Indicates the VSI of each node HELM1i The mean of the values, VSI max Indicates the VSI in each node HELM2i The maximum value in .

[0073] Set the upper limit of generator power Gs max and lower limit Gs min And the upper limit of load power Ls max and lower limit Ls min :

[0074]

[0075] After the environment is created, the intelligent agent is built.

[0076] Through the voltage stability indicator L ij The nodes selected for power flow adjustment use the changes in the active power output of the generators and the active power changes of the loads at these nodes as action a t .

[0077]

[0078] In the formula, Represents the generator G i The change in active output, Indicates load L i The change in active power.

[0079] Understandably, a t It is a continuous space, and upper and lower limits must be set for the action value space:

[0080]

[0081] Where ΔP Gmin is the lower limit of the generator output, ΔP Gmax is the upper limit of the generator output; ΔP Lminis the minimum value of load active power, ΔP Lmax It is the maximum value of load active power.

[0082] The next state after taking the action is:

[0083] s t+1 ~f(s t+1 |s t ,a t ) (15)

[0084] According to the status t+1 , perform power flow calculation and obtain the VSI of the power grid in the current state max and VSI ave The value of .

[0085] After completing the construction of the environment and the agent, the interaction principle between the environment and the agent is as follows: Figure 1 As shown, the environment gives the state s t , the agent takes action a t And feed the result back to the environment, which then generates a new state s t+1 and reward R, and the experience tuple (s t ,a t ,s t+1 ,R) Deposit into experience pool Then, the experience pool Sampling is performed to train the agent. The agent learns the optimal strategy through interaction with the environment. Figure 2 As shown in Figure 1, this interactive process includes two key networks: the Actor network and the Critic network. The goal of the Actor network is to maximize the expected return and exploration of the strategy π, and the goal of the Critic network is to optimize the action value function and the state value function by minimizing the loss function. The Actor network only contains one strategy network π(s), while the Critic network includes four sub-networks: a state value network Two Action-Value Networks and a target value network The SAC algorithm combines the advantages of deep learning and reinforcement learning, using deep neural networks to approximate value functions and policy functions, enabling intelligent agents to estimate these functions more accurately, thereby improving learning efficiency and performance.

[0086] Specifically, in this embodiment, the deep neural network structure of SAC deep reinforcement learning is as follows: Figure 3 As shown in the figure. In this structure, the deep neural network uses the MLP multi-layer perceptron. The MLP neural network consists of three layers: input layer, hidden layer and output layer. The hidden layer consists of 256 fully connected layers, each of which contains 256 neurons. The policy network receives the current state st As input, the output is the parameters of the action probability distribution, and the action a is generated by random sampling t The state value network receives the current state s t As input, output the state value corresponding to the current state The action value network receives the current state s t and action a t As input, output the action value function corresponding to the current state and action The target value network receives the state s at the next moment t+1 As input, the output is the state value The SAC algorithm implements the action value network by performing stochastic gradient descent training updates on the parameters ψ, φ, and θ. State Value Network Policy Network π φ (s t ) training and optimization.

[0087] Furthermore, the reward mechanism is set up: when the status is within the valid range, that is, when it does not exceed the preset upper and lower limits, according to VSI max and VSI ave To set the reward function.

[0088] The reward function is set as follows:

[0089] 1. VSI max When it is a positive number, according to VSI max Initial value VSI ini To set the reward function. The details are as follows:

[0090] a. If 0 <VSI max ≤(1.5*VSI ini ), when VSI max In this range, set the continuous reward R = -(VSI max / (3*VSI ini ).

[0091] b.VSI max >(1.5*VSI ini ), when VSI max If it is within this range, a penalty R=-1 is given.

[0092] 2. VSI max When it is a non-positive number:

[0093] a. at -0.1 <VSI max ≤0, give reward R = abs(VSI max )*5.

[0094] b. In VSI max ≤-0.1 and VSI ave When ≤0.1, reward R=1 is given.

[0095] Based on the above technical solution, in this embodiment, the active output of the generator and the active power of the load are used as the actions of the intelligent agent. First, according to the calculation results, the generators and loads with node numbers 2, 3, 4, and 5 are determined as adjustable variables. Then, the active power of the generator is used as the active power of the load. Active power of load VSI of each node HELM1i The mean of the values ​​and the VSI in each node HELM2i The maximum value of the current state s t , the change in active power of the generator and load is taken as action a t According to formula (15), we can get the state s at the next moment: t+1 . Using VSI-based max and VSI ave The reward R is determined by the calculation formula, and the size of the reward is used to evaluate the performance of the model.

[0096] After the SAC model is built, it is trained and optimized. The SAC algorithm is a reinforcement learning strategy optimization algorithm based on the maximum entropy principle. The goal of the SAC algorithm is to optimize the strategy π * , to maximize the future cumulative rewards and maintain the high entropy of the strategy in the process, so as to ensure that the strategy is more random, that is, the action probability is evenly distributed rather than overly concentrated on a specific action.

[0097]

[0098] s t , a t is the state s of the agent at time t t and the action taken t ; π * represents the optimal strategy; refers to the maximum strategy for obtaining the corresponding function; ρ π is the (s t ,a t ) Trajectory distribution; To find the expectation of the accumulated reward and maximum entropy; R(s t ,a t ) means in state s t , action a t α is the discount factor; in formula (17), H(π(·|s t )) indicates that the strategy π is in state s t The entropy under .

[0099] In the SAC algorithm, maximum entropy is an important policy goal, so the concept of entropy is introduced as part of the reward in the state value function and action value function that represent the expected return. The state value function and action value function are expressed by formula (18) and formula (19) respectively:

[0100]

[0101] In formula (15), As the true value of the action-value network, It is obtained based on the target value network, γ is the discount factor, (s t ,a t ,s t+1 ,R) is from the experience pool The empirical samples obtained in . In formula (16), As the true value of the state-value network, It is re-output from the policy network, not the experience pool α is the discount factor. SAC has two action value networks, and the one with the smaller output is selected as

[0102] The training and optimization of the policy network, action value network, and state value network are all done through their respective loss functions.

[0103] Loss function of policy network π (φ) is shown in formula (20):

[0104]

[0105] in, It is obtained based on the action value network, and α is the discount factor.

[0106] Loss function of action value network θ (Q) As shown in formula (21):

[0107]

[0108] in, is the predicted value of the action value network, and is the true value of the action-value network.

[0109] Loss function of state value network V (ψ) is shown in formula (22):

[0110]

[0111] in, As the predicted value of the state value network, As the true value of the state-value network.

[0112] After obtaining the loss function of each network, start calculating the gradient of each network loss function Use these gradient information to update the network parameters so that the network is optimized in the direction of reducing the error. V ,λ Q ,λ π Represent the learning rates of the value function network, action value network and policy network respectively.

[0113]

[0114] Target Value Network The parameters of the state value network The update is performed using the parameter ψ, and the update process is affected by the learning factor τ, which is a number less than 1 that controls the speed and amplitude of the update.

[0115]

[0116] The Soft Actor-Critic algorithm process is as follows:

[0117]

[0118]

[0119] Specifically, in this embodiment, a neural network is built using the TensorFlow framework and trained using the Adam optimizer. Adam is a commonly used adaptive gradient descent algorithm that can usually converge to better results in a shorter time when training a neural network. When using the SAC algorithm based on the TensorFlow framework for training, the SAC algorithm parameter settings are shown in Table 2:

[0120] Table 2 SAC algorithm parameter settings

[0121]

[0122]

[0123] Two experimental schemes are set up in the IEEE14 power grid model. In scheme 1, VSI is used HELM2i Indicators are used to set the reward mechanism of the algorithm. Then, in scheme 2, L max The reward mechanism of the indicator reset algorithm, the reward function is set as follows:

[0124] 1.Lmax When ≤1:

[0125] a.0.1≤L max ≤1, give reward R=1 / (L max *20).

[0126] b. In L max When <0.1, a reward R=1 is given.

[0127] 2.L max >1, according to L max The initial value L ini The reward function is set by the size. The details are as follows:

[0128] a.1 <L max ≤(0.85*L ini ), when L max When in this range, reward R = -L is given max / (1.7*L ini ).

[0129] b. If L max >(0.85*L ini ), when L max When in this range, a reward R=-1 is given.

[0130] In the IEEE14 power grid model, the total number of training steps is 30,000. In the historical training data of the first experimental scheme, VSI max <0 occurred 16579 times, while in the historical training data of the second experimental scheme, L max <1 did not occur. This shows that Scheme 1 successfully adjusted the power flow to a convergence state, while Scheme 2 failed to achieve this. Subsequently, the trained models were tested. The cumulative rewards of the two experimental schemes in the IEEE14 power grid model in the training set and the change curves of the indicators before and after the power flow adjustment in the test set are shown in Figure 2. Figure 4 shown.

[0131] according to Figure 4 According to the analysis of the cumulative reward curve in , in Scheme 1, it is found that the reward gradually increases in the first 420 rounds, and after more than 420 rounds, the curve fluctuates slightly, but the overall cumulative reward remains at a high level. In contrast, in Scheme 2, it is observed from the curve that the reward rises rapidly in the first 130 rounds, then increases and stabilizes, but the reward curve remains at a low level, which indicates that the reward setting in Scheme 1 has certain limitations.

[0132] The trained model is applied to the test set. The load level and generator output are adjusted to different percentages, generator output (110%, 120%, 130%, 140%, 150%) and load level (102%, 104%, 106%, 108%, 110%). In this way, the performance of the model in dealing with load and generator changes can be evaluated. Figure 4 It can be seen that in scheme 1, the VSI of each test sample before adjustment is max The values ​​were all greater than 0, but after adjustment they all dropped to less than 0. In addition, after adjustment, VSI ave The value of L is also reduced. This shows that the power flow has successfully converged after the power flow is adjusted. According to the test results in the figure, in scheme 2, the L of the first 5 samples is adjusted. max The index values ​​are all greater than 1. After the flow adjustment, L max It has decreased but is still greater than 1. This shows that after adjustment, the power flow in Scheme 2 still does not converge and the adjustment fails. The above results show that the introduction of VSI max and VSI ave The power flow adjustment results after L max The result after.

[0133] In view of the small number of nodes in the IEEE14 power grid model, an example of an IEEE57 power grid model is added, and the reward setting in Scheme 2 is consistent with the IEEE14 node model. Table 3 shows the L of each branch of the IEEE57 power grid model. ij The value of .

[0134] Table 3 L of each branch of IEEE57 power grid model ij Value

[0135]

[0136] According to the calculation results, the generators and loads with node numbers 12, 13, 15, and 17 are selected to participate in the power flow adjustment. In the IEEE57 power grid model, in the historical training data of the first experimental scheme, VSI max <0 occurred 18641 times, while in the historical training data of the second experimental scheme, L did not appear max < 1. This shows that Scheme 1 can effectively adjust the power flow to a convergent state, while Scheme 2 has not been able to achieve convergence throughout the entire adjustment process.

[0137] Finally, the optimized SAC model is applied to obtain the result of power flow convergence adjustment.

[0138] Specifically, when using HELM for power flow calculation, a non-convergent power flow state can be generated by adjusting the load and generator output in the power grid as a training sample for the model. The load level and generator output are set to different percentages and their convergence is observed. If none of these situations converge, they are used as test sets to evaluate the generalization ability of the model.

[0139] In this embodiment, the detailed flow chart is as follows Figure 3 shown.

[0140] The power flow convergence adjustment method is as follows:

[0141]

[0142] Specifically, in this embodiment, after the training is completed, the model is tested. Under different grid load levels and generator outputs, the power flow does not converge, and then the model is evaluated. The cumulative rewards of the two experimental schemes in the IEEE57 power grid model in the training set, and the change curves of various indicators before and after the power flow adjustment in the test set are shown in Figure 2. Figure 5 shown.

[0143] from Figure 5 It can be seen that the rewards of Scheme 1 increased rapidly in the first 150 rounds, and although there were slight fluctuations afterwards, the overall cumulative rewards basically remained near the 0 scale line. The rewards of Scheme 2 gradually increased in the first 180 rounds and then stabilized, but compared with Scheme 1, the fluctuations were still large. In addition, the cumulative reward curve always remained near the -4 scale line, and the reward level was low. In Scheme 1, the VSI of each test sample before adjustment max The values ​​were all greater than 0, but after adjustment they all dropped below 0. At the same time, the adjusted VSI ave The value has also decreased. Figure 5 The test results show that in scheme 2, the L of the 5 samples before adjustment ij The index values ​​are all greater than 1, and after the tidal current adjustment, L ij Although the value has decreased, it is still greater than 1, which indicates that the power flow has not converged after the adjustment and the adjustment is unsuccessful. This is consistent with the debugging results of the IEEE14 node. max and VSI ave After the index is introduced, the adjustment results of the five samples are always better than those of the L max The above results show that the introduction of L ij The index participating in the convergence adjustment of the power flow cannot effectively adjust the non-convergent power flow to a convergent state, and the introduction of VSI max and VSI ave As an indicator of power flow convergence adjustment, the power flow can be successfully adjusted to a convergent state. In addition, from the adjustment results of the test set, the introduction of VSImax and VSI ave The experimental results show good generalization ability.

[0144] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A method for adjusting power flow convergence based on SAC-HELM, characterized in that: The steps include: Step 1: Use the HELM power flow calculation method to establish a power grid model and calculate the sensitivity of each order of node voltage in the power grid model; Step 2: Introduce voltage stability index, voltage collapse point index and power flow convergence index into the established power grid model; Step 3: Establish and train a HELM-based SAC model, wherein the SAC model includes an environment and an agent, wherein the environment gives a state s according to the power grid model. t , the agent takes action a t And feed the result back to the environment, which then generates a new state s t+1 and reward R, and the experience tuple (s t , a t ,s t+1 , R) deposited into the experience pool Then the experience pool Perform sampling to train the agent; Step 4: Apply the trained SAC model to obtain the result of power flow convergence adjustment.

2. A method for adjusting power flow convergence based on SAC-HELM according to claim 1, characterized in that: The calculation method of each order sensitivity of the node voltage is: First, construct a built-in pure virtual function with the following expression: Where V i [n] represents the nth voltage component of the voltage at node i in the HELM power flow calculation, s n represents the nth order term of the frequency domain operator s; The nonlinear total sensitivity of voltage to injected power is obtained and The expression is as follows: In the formula, Q j represents the reactive power of node j, P j represents the reactive power of node j, The nth coefficient term representing the voltage at node i is used to find the partial derivative of the reactive power at node j. The nth coefficient term representing the voltage at node i is used to obtain the partial derivative of the active power at node j, where: and is the sensitivity of node voltage at each level.

3. A method for adjusting power flow convergence based on SAC-HELM according to claim 2, characterized in that: The voltage stability indicator value VSI HELM1i The calculation method is: Where: node i is the bus number of the distribution network, i∈2~T, node j is the bus number of reactive power injection, and real refers to the real part of the corresponding complex number; Q j represents the reactive power of node j, V i [n] represents the nth voltage sub-item of the voltage at node i in the HELM power flow calculation; T represents the total number of nodes; The nth coefficient term representing the voltage at node i is used to obtain the partial derivative of the reactive power at node j; VSI HELM1i The smaller the value, the better the voltage stability of the system.

4. A method for adjusting power flow convergence based on SAC-HELM according to claim 3, characterized in that: The value of the voltage collapse point indicator VSI HELM2i The calculation method is: When VSIH ELM2i <0, the system is stable. When VSIH ELM2i >0, the distribution network system voltage has reached the voltage collapse point.

5. A method for adjusting power flow convergence based on SAC-HELM according to claim 4, characterized in that: The power flow convergence index L ij The calculation formula is as follows: Where: X ij R represents the reactance of the branch connecting node i and node j; ij Represents the resistance of the branch connecting node i and node j; represents the square of the voltage at node i; represents the fourth power of the voltage at node i; When L ij When ≤1, the distribution network is in a voltage stable state, L ij The smaller the index, the higher the voltage stability; when L ij When the index is close to 1, it means that the voltage stability of the system decreases and there is a risk of voltage collapse. ij When ≥1, the system voltage has lost stability and entered a voltage collapse state, and the generators and loads participating in the power flow adjustment are screened out.

6. A method for adjusting power flow convergence based on SAC-HELM according to claim 1, characterized in that: The agent learns the optimal strategy through interaction with the environment.

7. A method for adjusting power flow convergence based on SAC-HELM according to claim 6, characterized in that: During the interaction between the agent and the environment, the state s given by the environment t ,expression: In the formula, Indicates the generator G i The active power, Indicates load L i Active power, VSI ave Indicates the VSI of each node HELM1i The mean of the values, VSI max Indicates the VSI in each node HELM2i The maximum value in .

8. A method for adjusting power flow convergence based on SAC-HELM according to claim 7, characterized in that: During the interaction between the agent and the environment, the agent takes action a t ,expression: In the formula, Indicates the generator G i The change in active power is Indicates load L i The change in active power.

9. A method for adjusting power flow convergence based on SAC-HELM according to claim 6, characterized in that: The environment regenerates the new state s t+1 ,expression: s t+1 ~f(s t+1 |s t ,a t ) According to the status t+1 , perform power flow calculation and obtain the current state s t+1 Down-grid VSI max and VSI ave The value of .

10. A method for adjusting power flow convergence based on SAC-HELM according to claim 9, characterized in that: According to the current status t+1 Down-grid VSI max and VSI ave The value of sets the reward function: (1) VSI max When it is a positive number, according to VSI max Initial value VSI ini To set the reward function, it is as follows: a. If 0 <VSI max ≤(1.5*VSI ini ), when VSI max In this range, set the continuous reward R = -(VSI max / (3*VSI ini ); b.VSI max >(1.5*VSI ini ), when VSI max If it is within this range, a penalty R=-1 is given; (2) VSI max For non-positive numbers: a. at -0.1 <VSI max ≤0, give reward R=abs(VSI max )*5; b. In VSI max ≤-0.1 and VSI ave When ≤0.1, reward R=1 is given.

11. A method for adjusting power flow convergence based on SAC-HELM according to claim 10, characterized in that: In the process of interaction between the agent and the environment, the Actor network and the Critic network are applied. The goal of the Actor network is to maximize the expected return and exploratory power of the strategy π, and the goal of the Critic network is to optimize the action value function and the state value function by minimizing the loss function.

12. A method for adjusting power flow convergence based on SAC-HELM according to claim 11, characterized in that: The Actor network only contains one policy network π(S), while the Critic network includes four sub-networks: a state value network Two Action-Value Networks and a target value network 13. A method for adjusting power flow convergence based on SAC-HELM according to claim 12, characterized in that: The Actor network and the Critic network are optimized through a deep application network. The optimization method is: the policy network receives the current state s t As input, the output is the parameters of the action probability distribution, and the action a is generated by random sampling t ; The state value network receives the current state s t As input, output the state value corresponding to the current state The action value network receives the current state s t and action a t As input, output the action value function corresponding to the current state and action The target value network receives the state s at the next moment t+1 As input, the output is the state value The action value network is updated by performing stochastic gradient descent training on the parameters ψ, φ, and θ. State Value Network Policy Network π φ (s t ) training and optimization.

Images