Power system load flow calculation convergence improvement method and device based on deep reinforcement learning multi-agent cooperation

Abstract

The invention provides a power system load flow calculation convergence improvement method based on deep reinforcement learning multi-agent cooperation, and the method comprises the steps: constructing a plurality of reinforcement learning agent models, and enabling each agent to correspond to one region or one device in a power system; and realizing collaboration and information sharing among the intelligent agents by using a deep reinforcement learning algorithm. Each agent rapidly evaluates the operation state of the power system through real-time communication and cooperation, and carries out dynamic adjustment according to the distribution condition of active power and reactive power so as to optimize the convergence of the global power flow. Specifically, under different load and power generation conditions, the intelligent agent continuously improves the decision-making process by learning feedback and reward mechanisms and aiming at the adjustment requirements of active power and reactive power, so that the convergence speed and precision of load flow calculation are remarkably improved. The method provided by the invention has relatively strong robustness, can effectively cope with the complexity change of the power system, and provides an efficient and intelligent scheduling solution for an extreme scene with non-convergent power flow.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system simulation analysis, and relates to a method for improving the convergence of power system power flow calculation based on the collaboration of multiple deep reinforcement learning agents. Background Art

[0002] Power grid flow calculation is the basis for medium- and long-term planning and real-time dispatch analysis and decision-making of power systems. Due to the rapid access of high-proportion new energy and electric vehicle charging loads during the construction of new power systems, the operation characteristics of the power grid are highly uncertain and random. The problem of non-convergence of power flow calculation for the medium- and long-term planning operation mode of the power grid has long existed. In the actual planning of the power grid, the convergence adjustment of the power flow is highly dependent on manual experience, and the trial and error method is mainly used, which consumes a lot of time and cost, and the convergence after adjustment cannot be guaranteed. The efficiency is very low, making the non-convergence problem of power flow one of the important problems that plague power operation and dispatch, and it is difficult to cope with complex and changing scenarios. Power flow calculation is essentially the solution of the nonlinear equation group of the power grid model. Therefore, the initial value setting and initial mode arrangement have a great influence on the convergence of power flow calculation, and may eventually fall into a pathological state that makes it difficult to converge the solution of the power flow equation with feasible solutions or directly lead to no solution for the power flow. Both situations show the feedback of the calculation results of non-convergence of power flow calculation. The problem of non-convergence of power flow calculation after large-scale distributed energy is long-term. The variable dimension of the power flow equation of the large power grid is high, and there are many adjustable parameters, which makes convergence adjustment more difficult and less efficient.

[0003] Regarding the problem of non-convergence of power flow, it can be classified into two types: power flow adjustment without solution and ill-conditioned power flow adjustment. Ill-conditioned power flow problems often focus on the improvement of power flow calculation methods, or focus on the improvement of Newton-Raphson method and the setting of initial values. In the actual power system power flow adjustment process, the power flow algorithm and initial values ​​are often solved by encapsulated solvers, so the practical application of this part of the improvement is poor; and for the adjustment of power flow without solution, the earliest indicators reflecting the degree of power flow without solution were studied based on the characteristics of the system boundary with solution, and a method for power flow adjustment without solution was proposed.

[0004] With the rise of artificial intelligence technology, some scholars have made preliminary attempts to apply artificial intelligence technology to power flow analysis. Cluster analysis, expert system and other methods have been proposed to solve the power flow convergence problem, but the initial methods lack flexibility and self-exploration mechanism. Summary of the invention

[0005] The purpose of the present invention is to solve the defects existing in the prior art, to provide a method for improving the convergence of power system flow calculation based on deep reinforcement learning multi-agent collaboration, and to solve the problem of non-convergence of flow in the prior art by using a multi-agent system.

[0006] The present invention adopts the following technical solution: a method for improving the convergence of power system power flow calculation based on the collaboration of multiple deep reinforcement learning agents, comprising the following steps:

[0007] S10: Acquire grid data of the grid system, wherein the grid data includes grid active power, reactive power balance, generator output, switching status of node capacitors and reactors, and balancing machine output;

[0008] S20: Determine the active power adjustment range of the power plant units in the region and the switching range of node capacitors and reactors through a deep reinforcement learning algorithm according to the power grid data;

[0009] S30: Calculate the power flow of the power grid system, and determine the active power balance and reactive power balance;

[0010] S40: If the calculation of the power grid system does not converge, the deep reinforcement learning multi-agent adjusts the active power of the power plant units in the region and switches the reactive compensation devices on the reactive compensation nodes in the region through collaborative adjustment;

[0011] Among them, the deep reinforcement learning algorithm constructs a single agent state space according to the power grid system flow state space, active power flow adjustment state space, capacitor and reactor switching state, balancing machine state space, important variables of power grid active power, reactive power balance, and generator output state, and the states of multiple agents together constitute a joint state space; the rewards of multiple agents constitute a comprehensive reward; according to the output change or effectiveness of the generator and the effectiveness of specific capacitors and reactors, an action space is constructed; the specific capacitors and reactors are capacitors and reactors that are beneficial to regulating the reactive imbalance of the system.

[0012] The present invention constructs multiple reinforcement learning agent models, each agent corresponds to an area or a type of equipment in the power system, and uses deep reinforcement learning algorithms to achieve collaborative work and information sharing between agents. The present invention utilizes the autonomous learning ability of deep reinforcement learning agents to accumulate experience in the continuously accumulated optimization strategies and effectively improve the power flow convergence of the power grid system. At the same time, it utilizes multi-agent real-time communication and collaboration to quickly evaluate the operating status of the power system, and dynamically adjusts the distribution of active and reactive power to further optimize the convergence of the global power flow.

[0013] Furthermore, the above step S20 includes:

[0014] S210: Using the power grid data as input data for the deep reinforcement learning multi-agent, and setting an initial state set;

[0015] S220: Set the joint state space and comprehensive reward value of multiple agents in the deep reinforcement learning algorithm;

[0016] Among them, the state space of a single intelligent agent includes the power grid system flow state space, the active power flow adjustment state space, the reactive power flow adjustment state space and the balancing machine state space; the comprehensive reward value includes the calculation result reward value and the constraint condition reward value;

[0017] S230: According to the initial state set, determine the range of power plant units for regional active power adjustment, reactive power compensation nodes, and the number of reactive power compensation devices switched on and off.

[0018] More specifically, the setting of the joint state space is as follows:

[0019] (1) Power grid system power flow state space:

[0020] P L = {P L1 ,P L2 ,P L3 ,...,P Li}

[0021] Where: P L is the active power of all power grid lines, P Li is the actual active power output of the line, i is the line number;

[0022] (2) Active power flow adjustment state space:

[0023] P G = {P G1 ,P G2 ,P G3 ,...,P Gj}

[0024] Where: P G is the active output of all power plant units, P Gj is the active output of a single generator, j is the generator number;

[0025] (3) Reactive power flow adjustment state space:

[0026]

[0027] Where: CZ is the set of reactive capacitor and reactance switching quantities at the nodes, is the number of capacitors and reactance Z switched at the kth node;

[0028] (4) The state space of the balancing machine:

[0029] P s = {P s1,P s2 ,P s3 ,...,P sh}

[0030] Where: P s is the output of all balancing units, P sh is the active output of a single balancing machine, and h is the balancing machine number;

[0031] The state space of a single agent is:

[0032] S n = {P L ,P G ,δ CZ ,P s};

[0033] The joint state space of multiple agents is:

[0034] S={S 1 ,S 2 ,S 3 ,...,S n ,..,S N};

[0035] Where: S is the set of all agent states, n is the agent number, and N is the total number of multiple agents.

[0036] The above comprehensive reward value is set as follows:

[0037] (1) Calculation result bonus value: The bonus value is set according to the power flow calculation result. The specific parameter settings are as follows:

[0038]

[0039] (2) Constraint reward value: The reward value is set based on the constraint of rated power limit. The specific parameter settings are as follows:

[0040]

[0041] Where: P sh is the active output of a single balancing machine, P Gj is the active output of a single generator, P Li is the actual active power output of the line; P Gjmax , P Gjmin P is the upper and lower limits of the generator output; shmax , P shmin The upper and lower limits of the balancing machine's output; P Limin , P Limax It is the upper and lower limits of the active power output of the line;

[0042] The comprehensive reward value R of the multi-agent deep reinforcement learning algorithm is:

[0043] R=r1+r2+r3+r4+r5.

[0044] Furthermore, before performing step S30, the following operations are performed: simplifying the network structure data of the power grid system, and retaining the network structure data of the power grid system in the area to be analyzed.

[0045] More specifically, the coordinated regulation in step S40 is the coordinated regulation of active power and reactive power compensation, and the active output of power plants in the region and the number of connected capacitors and inductors (reactances) are adjusted according to the balanced output;

[0046] Among them, the action space of deep reinforcement learning multi-agent is as follows:

[0047]

[0048]

[0049] Where: ΔP G is the change in the active power of the generator; β is the effectiveness of the capacitor and reactor, β = 1 means the capacitor is switched; β = -1 means the reactor is switched; N G is the total number of generators; is the reactive power change, C is the capacitor model, and Z is the reactance model;

[0050] Discretizing the generator output, we can get:

[0051]

[0052] Where: P i GM is the upper limit of generator i output, ΔP G is the unit size of the generator adjustment amount, and k is the multiple of the adjustment amount of each step of the power plant.

[0053] The present invention also provides a device for improving convergence of power system power flow calculation, comprising:

[0054] The data acquisition module acquires the grid data of the grid system, including the grid active power, reactive power balance, generator output, switching status of node capacitors and reactors, and balancing machine output;

[0055] The environment construction module sets up the computing environment of the deep reinforcement learning algorithm according to the power grid data, including the state space construction unit, the action space construction unit and the reward value construction unit;

[0056] The power flow calculation module performs power flow calculation on the power grid system and makes judgments on the active power balance and reactive power balance. If the power grid system calculation does not converge, the deep reinforcement learning multi-agent will adjust the active power of the power plant units in the area and switch the reactive compensation devices on the reactive compensation nodes in the area through collaborative adjustment.

[0057] Among them, the state space construction unit constructs a single agent state space according to the power grid system flow state space, active power flow adjustment state space, capacitor and reactor switching state, balancing machine state space, important variables of power grid active power and reactive power balance, and generator output state, and the states of multiple agents together constitute a joint state space;

[0058] An action space construction unit constructs an action space according to the output change or effectiveness of the generator and the effectiveness of specific capacitors and reactors; the specific capacitors and reactors are capacitors and reactors that are beneficial to regulating the reactive power imbalance of the system;

[0059] The reward value construction unit is used to construct the comprehensive reward value of multiple agents in the deep reinforcement learning algorithm, including the calculation result reward value and the constraint condition reward value.

[0060] Compared with the prior art, the present invention has the following advantages:

[0061] Through deep reinforcement learning technology, the present invention enables each agent to intelligently allocate tasks to optimize the adjustment strategy, handle complex nonlinear relationships and dynamic changes, and thus achieve accurate power flow regulation. The collaborative mechanism of the multi-agent system is used to allow information sharing and decision-making results to be integrated, significantly improving the system coordination and global optimization capabilities. In addition, the present invention obtains the power flow data of the power grid system, which can make the active power flow adjustment and reactive power flow adjustment directional.

[0062] The method proposed in the present invention has strong robustness, can effectively cope with the complex changes of the power system, and provides an efficient and intelligent scheduling solution for extreme scenarios where power flow does not converge, which has important theoretical and practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the steps of a method for improving convergence of power system power flow calculations according to the present invention;

[0064] Figure 2 It is an overall flow chart of the method for improving the convergence of power system power flow calculation of the present invention;

[0065] Figure 3 The influence of load parameter changes on the convergence rate of power flow calculation in Example 2 of the present invention;

[0066] Figure 4 This is the convergence curve of the reward function in Example 2 of the present invention as the number of training rounds increases. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0068] Example 1

[0069] In the examples of the present invention, Figure 1 and Figure 2 The specific steps and detailed flow chart of a method for improving the convergence of power system flow calculation based on deep reinforcement learning multi-agent collaboration according to the present invention are as follows: Figure 1 and Figure 2 As shown, the present invention includes:

[0070] S10: Obtaining grid data of the grid system, wherein the grid data includes grid active power and reactive power balance, and involves some regional characteristics: generator output status, capacitor / reactor switching status, and inter-regional exchange power, etc.;

[0071] The power grid data is acquired in real time by the DSP software from the power grid operation system and then exported according to the byte method.

[0072] S20: Determine the regional active power adjustment range and reactive power compensation nodes according to the power grid data through a deep reinforcement learning algorithm;

[0073] The details are as follows:

[0074] S210: Using the power grid data as input data for deep reinforcement learning multi-agents, and setting an initial state set;

[0075] S220: Setting the state space and reward value of the deep reinforcement learning algorithm;

[0076] The state space includes important variables of active power and reactive power balance of the power grid, and involves some regional characteristics: generator output status, capacitor / reactor switching status, and inter-regional exchange power, etc. The power flow states of multiple agents together constitute the state space, as follows:

[0077] (1) Power grid system power flow state space:

[0078] P L = {P L1 ,P L2 ,P L3 ,...,P Li},

[0079] Among them, P Lis the active power of all power grid lines, P Li is the actual active power output of the line, i is the line number;

[0080] (2) Active power flow adjustment state space:

[0081] P G = {P G1 ,P G2 ,P G3 ,...,P Gj},

[0082] Among them, P G is the active output of all power plant units, P Gj is the active output of a single generator, j is the generator number;

[0083] (3) Reactive power flow adjustment state space:

[0084]

[0085] Where: CZ is the set of reactive capacitor / reactance switching quantities of all nodes, is the number of capacitors / reactors switched on and off at the kth node;

[0086] (4) The state space of the balancing machine is:

[0087] P s = {P s1 ,P s2 ,P s3 ,...,P sh},

[0088] Where: P s is the output of all balancing units, P sh is the active output of a single balancing machine, and h is the balancing machine number;

[0089] The single agent space is:

[0090] S n = {P L ,P G ,δ CZ ,P s},

[0091] The multi-agent state space is:

[0092] S={S 1 ,S 2 ,S 3 ,...,S n ,..,S N},

[0093] Where: S is the set of all agent states, n is the agent number, and N is the total number of multiple agents.

[0094] Reward values ​​include:

[0095] (1) Calculation result reward value: There are only two calculation results of the power flow, convergence and non-convergence. The convergence state is the adjustment target. When the convergence is not adjusted, the calculation result of the power flow is non-convergence in most cases. Therefore, when the convergence is achieved, the reward value is set to a large positive number; when the non-convergence is achieved, the reward value is set to a small negative number. The specific parameter settings are as follows:

[0096]

[0097] (2) Constraint reward value: The constraint condition is mainly the rated power limit, including the following two points (the specific parameter settings are shown in the following formula): ① Rated power constraint of the generator: If the output power exceeds the limit, a certain negative reward value will be obtained. ② Output constraint of the balancing machine: If the exchange power exceeds the limit, a certain negative reward value will be obtained.

[0098]

[0099] Among them, P sh is the active output of a single balancing machine, P Gj is the active output of a single generator, P Li is the actual active power output of the line; P Gjmax , P Gjmin P is the upper and lower limits of the generator output; shmax , P shmin The upper and lower limits of the balancing machine's output; P Limin , P Limax It is the upper and lower limits of the active power output of the line;

[0100] In summary, the trend convergence improves the agent's final reward value R to:

[0101] R=r1+r2+r3+r4+r5.

[0102] After the multi-agents obtain the initial state space, the flow calculation converges after multiple steps of adjustment of the scenario, indicating that the algorithm has reached the final adjustment target state of the scenario.

[0103] S230: The multi-agent determines the scope of power plants for regional active power adjustment and reactive power compensation nodes based on the initial state set.

[0104] The initial state set usually includes the following information:

[0105] The load of the system; the power generation capacity, operating status and cost characteristics of each power plant; the reactive power demand and the status of the compensation equipment; the topology of the system, including the connection relationship between nodes and lines.

[0106] When determining the power plants that can be adjusted in the region, calculate the required active power based on the load demand. If the load demand exceeds the power generation capacity of a power plant, the power plant may be selected as the object of regulation. Analyze the power generation capacity of each power plant, including the minimum / maximum output, to determine which power plants can participate in the regulation. According to the topology of the power system, ensure that the selected power plants can effectively meet the load demand in the region through the network.

[0107] Determine the nodes for reactive power compensation Analyze the reactive power demand of each node and identify the nodes that need compensation, such as nodes with lower voltage. Use a multi-agent model to select the best compensation nodes and equipment configuration to minimize voltage deviation or improve system stability.

[0108] Establish a dynamic adjustment mechanism to adapt to load changes and system status changes to ensure real-time compensation of reactive power.

[0109] In a multi-agent environment, each power plant and compensation node can be considered as an independent agent. They need to be coordinated in the following ways:

[0110] Agents share information such as load demand, generation capacity, and compensation status to facilitate better decision making. Design coordination protocols to ensure that agents can coordinate with each other to avoid conflicts when adjusting active and reactive power. Introduce reward mechanisms to encourage agents to maximize load demand and optimize operating costs while maintaining system stability. Once the scope of power plants and reactive compensation nodes is determined and the control strategy is implemented, the operating status of the system needs to be monitored and fed back for real-time adjustment and optimization.

[0111] Through the above steps, the power plant range for regional active power adjustment and reactive power compensation nodes can be effectively determined in the multi-agent system.

[0112] S30: Calculate the power flow of the power grid system and make judgments on the active power balance and reactive power balance. The details are as follows:

[0113] Active power balance is a global balance, which requires ensuring the active power balance of all generators and loads, and that the transmission channels do not exceed the limit, and is easier to adjust.

[0114] S310: Count active power in each partition and check active power between regions. Check whether the exchange is too large. If the exchange power exceeds the limit, increase the power-on of the receiving partition and reduce the power-on of the sending partition, thereby reducing the transmission power of the tie line.

[0115] S320: Check whether the active power borne by the balancing machine is reasonable. If the active power output of the balancing machine exceeds the tolerance range, increase the generator output to make the active power output of the balancing machine within a reasonable range.

[0116] S330: During adjustment, it is also necessary to check whether the generator output exceeds the limit.

[0117] Reactive power balance is a layered and partitioned on-site balance. It is difficult to accurately locate the location of reactive power imbalance. Only areas where reactive power may be missing can be considered, so it is difficult to adjust. The specific methods are as follows:

[0118] S340: Estimate reactive power shortage according to voltage level or zone, switch on capacitors or reactors nearby, and adjust reactive power balance.

[0119] S350: Heavy-load substations with many incoming and outgoing lines and large active power exchange, new energy access substations, DC converter stations, and inter-regional transmission channels are locations where reactive power may be unbalanced. Check whether the reactive compensation configuration at these locations is sufficient. If the reactive compensation configuration is insufficient, switch on nearby capacitors or reactors.

[0120] S40: If the calculation of the power grid system does not converge, the reinforcement learning multi-agents adjust the active power in the area and switch the reactive compensation devices on the reactive compensation nodes in the area through collaborative adjustment. Specifically, it includes:

[0121] S410: Count active power in each partition and check active power between different regions. Check whether the exchange is too large. If the exchange power exceeds the limit, increase the power-on of the receiving partition and reduce the power-on of the sending partition, thereby reducing the transmission power of the tie line.

[0122] S420: Check whether the active power borne by the balancing machine is reasonable. If the active power output of the balancing machine exceeds the tolerance range, increase the generator output to make the active power output of the balancing machine within a reasonable range.

[0123] S430: During adjustment, it is also necessary to check whether the generator output exceeds the limit.

[0124] S440: Estimate reactive power shortage according to voltage level or zone, switch on capacitors or reactors nearby, and adjust reactive power balance.

[0125] Furthermore, the active power adjustment method includes active power output data of regional generator sets, and the switching method of reactive power compensation devices includes the number of capacitors / reactors connected;

[0126] The action space of deep reinforcement learning multi-agent is expressed as: The action space is the reinforcement learning agent adjustment variable, including the variables of the adjustable active power and reactive power in the system. In order to narrow the action space, the range of active and reactive power adjustment is specified. The adjustable variables mainly include the output change or effectiveness of the generator, and the effectiveness of specific capacitors / reactors. Here, specific capacitors / reactors refer to capacitors and reactors that are beneficial to regulating the reactive imbalance of the system, specifically unused capacitors and put-in reactors near heavy loads and multi-outlet buses. Therefore, the action space can be constructed as shown in the formula:

[0127]

[0128] Where: ΔP G is the change in active power of the generator; N G is the total number of generators; Change in reactive power;

[0129]

[0130] Where: β is the effectiveness of capacitors and reactors, β = 1 means capacitor switching, β = -1 means reactor switching; C is the capacitor model, Z is the reactor model;

[0131] The present invention obtains the power flow data of the power grid system, and can make the active power flow adjustment and the reactive power flow adjustment directional.

[0132] Example 2

[0133] Case Analysis

[0134] (1) Description of the scenario non-convergence

[0135] This experiment uses BPA for power flow calculation, takes a regional power grid as the experimental scenario, and generates a large number of scenarios where power flow calculation does not converge as training data sets.

[0136] As the number of scenarios increases, the convergence success rate of the strategy-adjusted power flow calculation shows a downward trend. After preliminary analysis, it is speculated that this phenomenon may be closely related to the complexity of the scenario. In some complex scenarios, the overall load level is too high, and the power generation output of local power plants is unreasonable, which may lead to a source-load imbalance. This imbalance increases the difficulty for the agent to achieve convergence of the power flow calculation through strategy adjustment. In some complex scenarios, the agent's strategy adjustment failed, probably because the constraints and operating environment in these scenarios put higher demands on the agent's decision-making ability.

[0137] During the test, 20 scenarios with different total load sizes with the same parameter settings were analyzed in depth to evaluate the impact of load parameter changes on the convergence rate of power flow calculation. The results are as follows: Figure 3 as shown.

[0138] As can be seen from the figure, the change of the overall load factor has a significant impact on the convergence performance of the system: when the load factor is close to 1, the system shows the highest convergence rate, indicating that within this range, the stability and efficiency of the system are optimized. However, when the load factor is higher than 1.1, the convergence rate of the system drops significantly. This trend shows that excessively high or low load levels will bring greater challenges to the flow calculation and increase the difficulty of the system to reach a steady state. Therefore, maintaining the load factor within an appropriate range can ensure that the power system can operate efficiently and reliably.

[0139] (2) Reward Function and Number of Rounds

[0140] During the training process of this study, the dataset was divided into a training set and a test set for effective model evaluation. Specifically, the training set contains 400 different scenarios, which have undergone 80,000 rounds of iterative training to ensure that the model can learn from diverse data. When training the agent, the two agents perform active and reactive adjustments respectively, and each part is adjusted for 10 steps. If the active or reactive imbalance number is zero, the adjustment is stopped and proceeds to the next step until the power flow calculation converges. If the agent converges the scenario adjustment within the specified number of steps, the round ends early; if not, the agent receives a penalty for unsuccessful strategy adjustment. According to Figure 4 The results shown in the figure are two reward function curves representing the multi-agent system. The active regulation agent converges in about 27,000 rounds, and the reactive regulation agent converges in about 36,000 rounds. This phenomenon shows that the agent has found an effective strategy at this stage, which enables the power flow calculation to converge stably.

[0141] (2) Test results of some test sets

[0142] Table 1 Test results of some test sets

[0143]

[0144]

[0145] In the performance evaluation stage of this study, 100 typical scenarios in which the initial power flow calculation did not converge in the test set were selected for testing. Table 1 above shows some of the test results. These scenarios represent the complex situations that may be encountered in practical applications, especially under the condition that the load fluctuation range is between 0.95 and 1.15 parameters, to test the adaptability and adjustment efficiency of the agent under different overall load conditions. The experimental results show that in each selected scenario, the agent system can achieve the convergence of the power flow calculation through decision adjustments of no more than 20 steps in one round. In addition, after the optimization adjustment of the agent, the active power imbalance and reactive power imbalance in all scenarios were successfully adjusted to zero. This result verifies the effectiveness and reliability of the multi-agent system in a complex power system environment, and the ability of the multi-agent system in responding to and solving the source-load imbalance problem. This ability is of great significance to improving the stability and efficiency of the power system.

Claims

1. A method for improving the convergence of power system power flow calculation based on deep reinforcement learning multi-agent collaboration, characterized in that: The following steps are involved: S10: Acquire grid data of the grid system, wherein the grid data includes grid active power, reactive power balance, generator output, switching status of node capacitors and reactors, and balancing machine output; S20: Determine the active power adjustment range of the power plant units in the region and the switching range of node capacitors and reactors through a deep reinforcement learning algorithm according to the power grid data; S30: Calculate the power flow of the power grid system, and determine the active power balance and reactive power balance; S40: If the calculation of the power grid system does not converge, the deep reinforcement learning multi-agent adjusts the active power of the power plant units in the region and switches the reactive compensation devices on the reactive compensation nodes in the region through collaborative adjustment; Among them, the deep reinforcement learning algorithm constructs a single agent state space according to the power grid system flow state space, active power flow adjustment state space, capacitor and reactor switching state, balancing machine state space, important variables of power grid active power, reactive power balance, and generator output state, and the states of multiple agents together constitute a joint state space; the rewards of multiple agents constitute a comprehensive reward function; according to the output change or effectiveness of the generator and the effectiveness of specific capacitors and reactors, an action space is constructed; the specific capacitors and reactors are capacitors and reactors that are beneficial to regulating the reactive imbalance of the system.

2. The method for improving convergence of power system power flow calculation according to claim 1, characterized in that: The step S20 comprises: S210: Using the power grid data as input data for deep reinforcement learning multi-agents, and setting an initial state set; S220: Set the joint state space and comprehensive reward value of multiple agents in the deep reinforcement learning algorithm; Among them, the state space of a single intelligent agent includes the power grid system flow state space, the active power flow adjustment state space, the reactive power flow adjustment state space and the balancing machine state space; the comprehensive reward value includes the calculation result reward value and the constraint condition reward value; S230: According to the initial state set, determine the range of power plant units for regional active power adjustment, reactive power compensation nodes, and the number of reactive power compensation devices switched on and off.

3. The method for improving convergence of power system power flow calculation according to claim 2, characterized in that: The setting of the joint state space is as follows: (1) Power grid system power flow state space: P L ={P L1 ,P L2 ,P L3 ,...,P Li } Where: P L is the active power of all power grid lines, P Li is the actual active power output of the line, i is the line number; (2) Active power flow adjustment state space: P G ={P G1 ,P G2 ,P G3 ,...,P Gj } Where: P G is the active output of all power plant units, P Gj is the active output of a single generator, j is the generator number; (3) Reactive power flow adjustment state space: Where: CZ is the set of reactive capacitor and reactance switching quantities of all nodes, is the number of capacitors and reactors switched on and off at the kth node; (4) The state space of the balancing machine: P s ={P s1 ,P s2 ,P s3 ,...,P sh } Where: P s is the output of all balancing units, P sh is the active output of a single balancing machine, and h is the balancing machine number; The state space of the single agent is: S n ={P L ,P G ,δ CZ ,P s }; The joint state space of the multi-agents is: S={S 1 ,S 2 ,S 3 ,...,S n ,..,S N }; Where: S is the set of all agent states, n is the agent number, and N is the total number of multiple agents.

4. The method for improving convergence of power system power flow calculation according to claim 3, characterized in that: The comprehensive reward value is set as follows: (1) Calculation result bonus value: The bonus value is set according to the power flow calculation result. The specific parameter settings are as follows: (2) Constraint reward value: The reward value is set based on the constraint of rated power limit. The specific parameter settings are as follows: Where: P sh is the active output of a single balancing machine, P Gj is the active output of a single generator, P Li is the actual active power output of the line; P Gjmax , P Gjmin P is the upper and lower limits of the generator output; shmax , P shmin P is the upper and lower limits of the balancing machine’s output; Limin , P Limax Output upper and lower limits of the line The comprehensive reward value R of the multi-agent in the deep reinforcement learning algorithm is: R＝r1+r2+r3+r4+r5。 5. The method for improving convergence of power system power flow calculation according to claim 4, characterized in that: Before performing step S30, the following operations are performed: simplifying the network structure data of the power grid system, and retaining the network structure data of the power grid system in the area to be analyzed.

6. The method for improving convergence of power system power flow calculation according to claim 5, characterized in that: The coordinated regulation in step S40 is the coordinated regulation of active power and reactive power compensation, and the active output of power plants in the region and the number of connected capacitors and inductors (reactances) are adjusted according to the balanced output; The action space of the deep reinforcement learning multi-agent is as follows: Where: ΔP G is the change in the active power of the generator; β is the effectiveness of the capacitor and reactor, β = 1 means the capacitor is switched, β = -1 means the reactor is switched; N G is the total number of generators; is the reactive power change, C is the capacitor model, and Z is the reactance model; Discretizing the generator output, we can get: Where: is the upper limit of generator i output, ΔP G is the unit size of the generator adjustment amount, and k is the multiple of the adjustment amount of each step of the power plant.

7. A device for improving convergence of power system flow calculation, characterized in that: include: The data acquisition module acquires the grid data of the grid system, including the grid active power, reactive power balance, generator output, switching status of node capacitors and reactors, and balancing machine output; The environment construction module sets up the computing environment of the deep reinforcement learning algorithm according to the power grid data, including the state space construction unit, the action space construction unit and the reward value construction unit; The power flow calculation module performs power flow calculation on the power grid system and makes judgments on the active power balance and reactive power balance. If the power grid system calculation does not converge, the deep reinforcement learning multi-agent will adjust the active power of the power plant units in the area and switch the reactive compensation devices on the reactive compensation nodes in the area through collaborative adjustment. The state space construction unit constructs a single agent state space according to the power grid system power flow state space, active power flow adjustment state space, capacitor and reactor switching states, balancing machine state space, important variables of power grid active power and reactive power balance, and generator output state, and jointly constructs a joint state space with the states of multiple agents; The action space construction unit constructs the action space according to the output change or effectiveness of the generator and the effectiveness of specific capacitors and reactors; the specific capacitors and reactors are capacitors and reactors that are conducive to regulating the reactive power imbalance of the system; The reward value construction unit is used to construct the comprehensive reward value of the multi-agent of the deep reinforcement learning algorithm, including the calculation result reward value and the constraint condition reward value.

8. The device for improving convergence of power system power flow calculation according to claim 7, characterized in that: The construction of the joint state space satisfies the following formula: (1) Power grid system power flow state space: P L ={P L1 ,P L2 ,P L3 ,...,P Li } Where: P L is the active power of all power grid lines, P Li is the actual active power output of the line, i is the line number; (2) Active power flow adjustment state space: P G ={P G1 ,P G2 ,P G3 ,...,P Gj } Where: P G is the active output of all power plant units, P Gj is the active output of a single generator, j is the generator number; (3) Reactive power flow adjustment state space: Where: CZ is the set of reactive capacitor and reactance switching quantities of all nodes, is the number of capacitors and reactors switched on and off at the kth node; (4) The state space of the balancing machine: P s ={P s1 ,P s2 ,P s3 ,...,P sh } Where: P s is the output of all balancing units, P sh is the active output of a single balancing machine, and h is the balancing machine number; The state space of the single agent is: S n ={P L ,P G ,δ CZ ,P s }; The joint state space of the multi-agents is: S={S 1 ,S 2 ,S 3 ,...,S n ,..,S N }; Where: S is the set of all agent states, n is the agent number, and N is the total number of multiple agents.

9. The device for improving convergence of power system power flow calculation according to claim 8, characterized in that: The construction of the comprehensive reward value satisfies the following formula: (1) Calculation result bonus value: The bonus value is set according to the power flow calculation result. The specific parameter settings are as follows: (2) Constraint reward value: The reward value is set based on the constraint of rated power limit. The specific parameter settings are as follows: Where: P sh is the active output of a single balancing machine, P Gj is the active output of a single generator, P Li is the actual active power output of the line; P Gjmax , P Gjmin P is the upper and lower limits of the generator output; shmax , P shmin The upper and lower limits of the balancing machine's output; P Limin , P Limax It is the upper and lower limits of the active power output of the line; The comprehensive reward value R of the multi-agent in the deep reinforcement learning algorithm is: R＝r1+r2+r3+r4+r5。 10. The device for improving convergence of power system power flow calculation according to claim 9, characterized in that: The construction of the action space satisfies the following formula: Where: ΔP G is the change in the active power of the generator; β is the effectiveness of the capacitor and reactor, β = 1 means the capacitor is switched, β = -1 means the reactor is switched; N G is the total number of generators; is the reactive power change, C is the capacitor model, and Z is the reactance model; Discretizing the generator output, we can get: Where: is the upper limit of generator i output, ΔP G is the unit size of the generator adjustment amount, and k is the multiple of the adjustment amount of each step of the power plant.

Images