Multi-agent self-discipline collaborative optimization method and system for multi-region reactive voltage control

Abstract

The invention relates to the technical field of voltage optimization control of a power system, in particular to a multi-agent self-discipline collaborative optimization method and system for multi-region reactive voltage control, and provides an optimal reactive voltage optimization decision method for establishing a data interaction model, setting a data interaction mode and researching a power distribution network. By optimizing distribution of reactive power, the active loss of the power distribution network is reduced, the economical efficiency of the power system is improved, and the service life of equipment is prolonged; by using a deep reinforcement learning algorithm, effective adjustment of voltage is realized, and node voltage is stabilized; a deep reinforcement learning method is adopted, so that the influence of model errors is reduced, the calculation speed and the real-time response capability are improved, and the calculation cost is reduced; through continuous interaction with actual power grid data, the intelligent agent can cope with the uncertainty of distributed new energy, energy storage and flexible load power generation and power utilization, and the stability and reliability of the system under various complex conditions are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power system voltage optimization control, and in particular to a multi-agent autonomous collaborative optimization method and system for multi-region reactive voltage control. Background Art

[0002] According to the on-site investigation of several pilot projects in Yunnan, most areas of Yunnan currently have low voltage problems, which are caused by long lines and large load fluctuations. At the same time, charging piles as a special load also affect the voltage of the system to a large extent. With the implementation of the whole-county photovoltaic policy, large-scale photovoltaic (PV) is connected to the distribution network. However, the output of distributed power sources (DG) such as photovoltaics is intermittent and volatile, and their large-scale access has brought huge challenges to the stable operation of the distribution network. This challenge is reflected in the fact that the access of a high proportion of photovoltaics has changed the operation mode of the distribution network, and there may be a reverse flow of power, resulting in node voltage violations and increased active power losses.

[0003] To address the above issues, the dispatch center of the power grid will integrate reactive voltage control to optimize the power flow distribution of the distribution network. At the same time, in order to solve the uncertainty of distributed renewable energy, energy storage and flexible load generation and consumption in the distribution network, and to improve the efficiency of solving reactive voltage strategies, this project proposes a method based on deep reinforcement learning to solve the distributed autonomous collaborative control method in multiple regions of the distribution network. Summary of the invention

[0004] In view of the above problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the technical problem to be solved by the present invention is: the document covers two major technical fields: reactive voltage optimization control of distribution networks and the application of deep reinforcement learning in power systems, emphasizing the important role of these technologies in improving the stability and operating efficiency of power systems; through detailed mathematical models and algorithm analysis, the document deeply explores how to use modern optimization algorithms to solve practical problems in power systems.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control, comprising: establishing a data interaction model and setting a data interaction mode; and studying the optimal reactive voltage optimization decision method for the distribution network.

[0007] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control described in the present invention, the data interaction model is included in reactive power optimization and voltage optimization to describe the data exchange and coordination mechanism between the regional controller and the local controller.

[0008] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control described in the present invention, the data interaction mode includes a specific collaboration method between the regional controller and the local controller during the reactive power optimization and voltage optimization process.

[0009] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control described in the present invention, the reactive voltage optimization decision method includes methods for optimizing the reactive power and voltage levels of the power grid to ensure the stability and efficiency of the system.

[0010] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control described in the present invention, the data interaction model also includes a reactive power optimization data interaction model and a voltage optimization data interaction model.

[0011] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control described in the present invention, the interactive mode also includes a first mode of voltage regulation linkage and a second mode of reactive power linkage.

[0012] As a preferred solution of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control described in the present invention, the reactive voltage optimization decision method also includes reactive voltage control based on traditional mathematical optimization algorithms, reactive voltage control based on heuristic algorithms, reactive voltage control based on deep reinforcement learning algorithms, and a distribution network on-site voltage reactive power control method based on deterministic optimization;

[0013] Among them, the reactive power and voltage control based on deep reinforcement learning algorithm includes DON algorithm and discrete reactive power optimization of distribution network, DDPG algorithm and continuous reactive power optimization of distribution network, PPO algorithm and continuous reactive power optimization of distribution network, SAC algorithm and continuous reactive power optimization of distribution network, distribution network voltage reactive power control method based on DDQ algorithm, distribution network voltage reactive power control method based on double-delay deep deterministic policy gradient algorithm, distribution network voltage reactive power control method based on secure reinforcement learning CPO algorithm and distribution network reactive power optimization method based on time-space perception technology.

[0014] Another object of the present invention is to provide a multi-agent autonomous collaborative optimization system for multi-region reactive voltage control. By introducing multi-agent system (MAS) technology and deep reinforcement learning algorithm, reactive voltage optimization control in different regions of the distribution network is realized, aiming to solve the limitations of traditional centralized control methods in dealing with complex power grid environments, and utilize the distributed characteristics and autonomous learning capabilities of the multi-agent system to improve the stability and operation efficiency of the power system; each agent is responsible for reactive voltage regulation in a specific area, and shares information and coordinates actions through a communication network to achieve global optimization goals; this multi-agent autonomous collaborative optimization system can not only effectively respond to dynamic changes and uncertain factors in the power grid, but also improve the flexibility and adaptability of the system, ensuring that a stable voltage level is maintained under different load conditions.

[0015] In order to solve the above technical problems, the present invention provides the following technical solutions: a multi-agent autonomous collaborative optimization system for multi-region reactive power and voltage control, comprising: a data interaction module and an optimization decision module;

[0016] The data interaction module establishes a data interaction model and sets a data interaction mode;

[0017] The optimization decision module studies the optimal reactive power voltage optimization decision method of the distribution network.

[0018] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control as described above are implemented.

[0019] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control as described above are implemented.

[0020] Beneficial effects of the present invention: Improve system stability: Reduce active power loss: By optimizing the distribution of reactive power, the active power loss of the distribution network is effectively reduced. This not only improves the economy of the power system, but also extends the service life of the equipment. Stabilize node voltage: Using the deep reinforcement learning algorithm, the voltage is effectively regulated, so that the node voltage is kept within a reasonable range, avoiding the safety hazards caused by excessive voltage fluctuations. Improve computing efficiency: No model dependence: Unlike traditional methods based on mathematical models, the present invention adopts a deep reinforcement learning method, which does not need to rely on accurate physical models, reduces the impact of model errors, and improves computing speed and real-time response capabilities. Reduce computing costs: Since the deep reinforcement learning algorithm can automatically adjust the strategy during the training process, the need for manual intervention is reduced, thereby greatly reducing computing costs. Enhance system robustness: The uncertainty of energy storage and flexible load power generation and power consumption improves the handling of uncertainty: The present invention can cope with the stability and reliability of distributed new energy and systems in the face of various complex situations. Adapt to dynamic changes: Through continuous interaction with actual power grid data, the intelligent agent can optimize strategies in real time, adapt to the dynamic changes in the operating status of the power grid, and ensure the efficient operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the relationship between regional and local controllers of a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control provided by an embodiment of the present invention.

[0023] Figure 2 A diagram of the interaction process between the agent and the distribution system of the multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control provided by an embodiment of the present invention.

[0024] Figure 3 A nine-area diagram model diagram and a seventeen-area diagram model diagram of a multi-agent autonomous collaborative optimization method for multi-area reactive voltage control provided by an embodiment of the present invention.

[0025] Figure 4 A seventeen-zone diagram heavy load mode diagram and a seventeen-zone diagram light load mode diagram of a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention.

[0026] Figure 5A diagram of the switchable values ​​in different regions after actual switching and a diagram of the switchable values ​​in the same region after actual switching of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention.

[0027] Figure 6 A diagram of the DQN network training process of a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention.

[0028] Figure 7 A diagram of the average instantaneous reward convergence process of the DDPG algorithm of a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control provided by an embodiment of the present invention.

[0029] Figure 8 A modified IEEE 33-node system diagram of a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention, a diagram of the cumulative reward convergence of the 33-node system during training, and a diagram of the number of voltage out-of-bounds times of the 33-node system during training.

[0030] Fig. 9 An improved 12-node system diagram of a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention, a DG active output diagram, and a cumulative reward diagram during the training process.

[0031] Fig.10 A test system diagram of IEEE13 nodes and power loss diagrams of baseline and DDQN algorithms for a multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control provided for an embodiment of the present invention.

[0032] Fig.11 A multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention provides a deviation diagram of the lowest voltage in an IEEE-34 bus system using three methods and the total active power loss in 24 time periods using the three methods.

[0033] Fig.12 The convergence diagram of the average reward value of each round and the number of nodes with the cumulative voltage exceeding the limit of the method proposed in the training process of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention and the average active power loss in the test day compared with the PSO algorithm diagram.

[0034] Fig.13 An Actor and Critic network structure diagram of GCN-PPO, a clip truncation function diagram of PPO, and a distribution network VVO flow chart based on GCN-PPO of a multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control provided in one embodiment of the present invention.

[0035] Fig.14 A diagram of PV and load data for two typical days, a diagram of reward values ​​and voltage over-limit times during training for a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention.

[0036] Fig.15 A graph of the loss values ​​of the Critic and Actor networks during the training of a multi-agent autonomous collaborative optimization method for multi-region reactive voltage control provided by an embodiment of the present invention, a graph of the network loss of a method on a test set, and a comparison graph of the number of voltage over-limits in three scenarios. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0038] Example 1, reference Figure 1 , is an embodiment of the present invention, which provides a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control, including:

[0039] S1: Establish a data interaction model and set the data interaction mode.

[0040] It should be noted that the data interaction model between the regional controller and the local controller during reactive power optimization is as follows: Figure 1 As shown in S1; the controlled devices supported by the local controller include capacitors, dynamic reactive compensation, inverters, etc. When optimizing reactive power, it is necessary to count the current capacitor and dynamic reactive compensation capacity that has been invested and the remaining capacity that has not been invested, the current active output of the inverter and the power factor target value, and finally summarize them into the current reactive regulation upper limit and reactive regulation lower limit (i.e. reactive regulation capability) of the local controller. The calculation formula is as follows:

[0041] Q adj_h =Q C_h +Q S_h +Q PV_h

[0042] Q adj_l =Q C_l +Q S_l +Q PV_l

[0043] Among them, Q C_h is the upper limit of the current reactive power input of the capacitor, Q S_hThe capacitive reactive power upper limit of the dynamic reactive power compensation, Q PV_h is the upper limit of the capacitive reactive power currently invested by the inverter, Q adj_h is the upper limit of reactive power regulation, Q C_l is the lower limit of the current reactive power of the capacitor, Q S_l is the current inductive lower limit of dynamic reactive power compensation, Q PV_l is the lower limit of the inductive reactive power currently put into operation by the inverter, Q adj_l It is the lower limit of reactive power regulation;

[0044] The current upper and lower limits of capacitor reactive power are calculated as follows:

[0045]

[0046] Among them, U is the current voltage, U N is the rated voltage of the capacitor, q i is the rated capacity of the capacitor in group i, St i is the input and output status of the i-th group of capacitors (1 for input and 0 for output), and n is the total number of capacitor groups currently connected to the controller;

[0047] Calculation of the upper and lower limits of the current reactive power input for dynamic reactive compensation:

[0048] Q S_h =q N -qin

[0049] Q S_l =q N -qout

[0050] Among them, q N is the rated capacity of dynamic reactive power compensation, qin is the capacitive capacity of the current equipment, and qout is the inductive capacity of the current equipment;

[0051] Calculation of the upper and lower limits of the inverter's current reactive power input:

[0052]

[0053] Wherein, p is the current inverter active output, pf is the current power factor (the power factor is defined as positive when it indicates reactive power lags and is inductive, and negative when it indicates reactive power leads and is capacitive);

[0054] The reactive power regulation target sent by the regional controller to the local controller is calculated based on the optimal reactive power value and the reactive power data of the current node. The calculation formula is as follows:

[0055] Q tar =Q opt -Q

[0056]

[0057] Among them, Q opt is the reactive power optimization target of the regional control node, Q is the reactive power of the regional control node, Q tar is the reactive power regulation target, q tar_i is the reactive power regulation target sent by the regional controller to the i-th local controller, n is the number of local controllers under the regional controller, Q adj_i is the reactive power regulation capability of the ith controller (the upper limit is used when capacitive reactive power is required, and the lower limit is used when inductive reactive power is required).

[0058] Furthermore, there are two main data interaction models between the regional controller and the local controller during voltage optimization, as follows: Figure 1 As shown in S2, the first mode is voltage regulation linkage, and the regional controller is equipped with a voltage regulator to adjust the voltage range of the lower-level area; the second mode is reactive power linkage, and the regional controller is not equipped with a voltage regulator. According to the voltage regulation requirements in the area, the reactive power compensation equipment in the area is used to optimize the voltage.

[0059] Furthermore, in the first mode, voltage regulation linkage, the regional controller can perform voltage optimization linkage with the local controller. During voltage regulation linkage, the regional controller calculates the voltage regulation capability of the current node, i.e., the voltage control space. The local controller feedbacks the voltage regulation demand based on the voltage regulation capability of the regional controller and the comprehensive assessment of the monitoring point data. The regional controller summarizes the lower-level controllers and analyzes the decisions for optimization.

[0060] The voltage regulation capability of the voltage regulator includes the voltage optimization upper limit and voltage optimization lower limit of the regional controller. The calculation formula is as follows:

[0061]

[0062] Among them, d N_h is the upper limit of the voltage regulator gear, d N_l is the lower limit of the voltage regulator gear, u% is the voltage regulator gear step, q N is the rated capacity of the voltage regulator compensation module, qin is the capacitive capacity of the voltage regulator compensation module, qout is the inductive capacity of the voltage regulator compensation module, X is the current line equivalent reactance, U is the voltage at the regional controller assessment point, U% opt_h is the voltage optimization upper limit, U% opt_l Optimize the lower limit for voltage;

[0063] Local controller voltage regulation requirement calculation:

[0064]

[0065] Among them, U optis the local controller voltage optimization target, U is the local controller test point voltage, U% need For voltage regulation needs;

[0066] The second mode is reactive linkage. The regional controller can conduct reactive joint adjustment with the local controller to achieve voltage optimization. During reactive linkage, the local controller calculates the voltage regulation demand. After receiving the regulation demand, the regional controller requires the lower-level controller to upload their reactive regulation capabilities. The regional controller calculates the reactive regulation target based on the feedback and distributes it to each controller. The voltage regulation demand, reactive regulation capability, and reactive regulation target are the same as above. Figure 1 As shown in S3.

[0067] Example 2, reference Figure 2-Figure 5 , is an embodiment of the present invention, which provides a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control, including:

[0068] S2: Study the optimal reactive power and voltage optimization decision method for distribution network.

[0069] It should be noted that the reactive power optimization model of the power system can be uniformly described as:

[0070]

[0071] Where: x 1 is the control variable; u k is the state variable; f is the optimization objective function, which can be the system active network loss, node voltage deviation, static voltage stability margin, reactive power compensation equipment investment cost, etc.; g is the equation constraint, that is, the node power flow balance equation; h is the condition that the control variable and the state variable must meet;

[0072] According to different optimization requirements, different objective functions can be selected as multi-objective optimization functions: for example, from the perspective of system economy, safety, stability and investment;

[0073] Further, according to the seventeen-zone diagram model: The seventeen-zone diagram model adds voltage optimization upper and lower limits and reactive power optimization upper and lower limits on the basis of the traditional nine-zone diagram, and combines the voltage measurement effective interval to form the following structure: According to the definition of the optimization interval in the limit model, the optimization upper and lower limits are not effective at the same time. There is always an optimization limit and two assessment limits that divide the qualified interval into an optimization interval and a non-optimization interval. Therefore, the overall division is 16 intervals, plus the part divided by the voltage effective boundary (the purple dotted line V_H_Border upward part and V_L_Border downward part), a total of 17 zones;

[0074] The 17-zone diagram has two modes, heavy load mode and light load mode; the reactive power assessment upper and lower limits and optimization upper and lower limits are indirect parameters, which need to be calculated based on the current active power and power factor assessment upper and lower limits, and power factor optimization upper and lower limits. The specific calculation rules are as follows:

[0075]

[0076] Among them: the upper limit of reactive power corresponds to the lower limit of power factor (lagging, insufficient reactive power); the lower limit of reactive power corresponds to the upper limit of power factor (leading, excess reactive power);

[0077] Overload mode: Figure 4 As shown in Table 1:

[0078] The voltage is required to operate at the upper limit, and the power factor is required to operate at the upper limit. The position of zone 17 is omitted in the figure;

[0079] Table 1 Voltage priority mode

[0080]

[0081]

[0082] Light load mode: Figure 4 As shown in Table 2:

[0083] The voltage is required to operate at the lower limit, and the power factor is required to operate at the lower limit. The position of zone 17 is omitted in the figure;

[0084] Table 2 Voltage priority mode

[0085]

[0086] Furthermore, according to the reactive voltage control process of the 17-zone diagram algorithm: ① Initially send the parameter value APP according to the configuration table; ② Determine whether the parameter initialization is successful, that is, whether the APP can correctly read all the configuration parameters. If there are parameters that are not read and there are exceptions, return to ①, and if the parameter initialization is successful, proceed to the next step; ③ Determine whether the SVG is in normal state. If there are exceptions, return to continue to judge. If normal, determine whether the SVG function is put into use. If not, exit. If put into use, execute the SVG control strategy; Determine the priority of reactive voltage and three-phase unbalance. If put into use, enter the corresponding control strategy sub-process according to the priority order, otherwise exit;

[0087] According to the static VAR compensator sub-process and Table 3: the SVG voltage qualified upper and lower limits and power factor are set according to the required power factor and voltage, and sent to the device;

[0088] Table 3 Data requirements

[0089]

[0090]

[0091] According to the reactive voltage seventeen-zone diagram sub-process:

[0092] ① Read the voltage, current, active power and reactive power of each phase through the exchange acquisition APP; ② Determine whether the read data is valid, including: whether the voltage and current exceed the effective limit (adjustable according to the limit parameter details), whether the data is consistent for 5 consecutive times. If any problem occurs, the data is considered invalid, and then return to ①, otherwise enter ②; ③ Determine the seventeen-zone diagram model according to the load rate and limit value. The load rate calculation is detailed in the following indicator calculation load rate. The limit value is selected according to the limit mechanism in the governance target limit value, and refer to the seventeen-zone diagram model. In the ultra-light load mode, the reactive equipment is not adjusted; ④ Determine the state of each phase, collect data according to the real-time data refresh cycle (see the APP data reporting model), and if the average value of 5 times is taken in a certain area for 5 consecutive times, it is taken as the current system area; ⑤ Determine whether the area where each phase is located needs to act, and the regional action strategy is shown in the seventeen-zone diagram model. If action is required, enter ⑥, otherwise return to ①; ⑥ According to the different areas where the system state is located, select the action priority of the corresponding area, such as voltage regulation priority or reactive compensation priority. The one that ranks first in the regional action strategy will take priority; ⑦ Determine whether there is a voltage regulation action in each phase action strategy. If yes, enter the on-load voltage regulation and gear adjustment sub-process, otherwise enter the reactive power compensation equipment sub-process. If there is both voltage increase and voltage reduction in each phase action strategy, no on-load voltage regulation and gear adjustment will be performed. If the on-load voltage regulation and gear adjustment fails, enter the reactive power compensation equipment sub-process;

[0093] According to the reactive power compensation equipment sub-process:

[0094] ① Calculate the reactive capacity that needs to be compensated for each phase at present, see the reactive compensation capacity calculation below; ② Compensation equipment priority selection: Different reactive compensation equipment has different priorities, which are set by the user in the fixed value. 0 is exit, 1 is the highest priority, the higher the value, the lower the priority; set it with the compensation equipment priority parameter; ③ Equipment adjustment capability: make an adjustment capability judgment before entering the compensation sub-process. If the communication is normal, there is no operation alarm or abnormal information, the number of daily actions is not exceeded, and there is a compensation margin, then enter ④. If the compensation equipment at the current priority level does not have the adjustment capability, then enter ⑤; ④ enters the sub-process of each compensation equipment; ⑤ Determine the remaining compensation capacity: the difference between the target compensation capacity and the current compensation capacity. That is, when the compensation target value is not completed, enter ② to select the next level of compensation equipment according to the compensation equipment priority, and enter the next level of compensation sub-process until the compensation target value is completed;

[0095] Low voltage capacitor sub-process: such as Figure 5 As shown in Table 4:

[0096] Determine the reactive power adjustment range of each phase capacitor and the combination of capacitor compensable capacity;

[0097] Because the actual capacity of the capacitor will change with the change of voltage, it is necessary to convert the actual switchable capacity of the capacitor according to the current voltage;

[0098] The actual switchable combination of the current capacitor and the corresponding reactive power are recorded to facilitate the subsequent selection of the corresponding action capacitor; the subsequent selection of capacitor compensation method and capacity selection and capacitor switching;

[0099] Table 4 Data requirements

[0100] name unit Remark Capacitor status information Capacitor capacity kvar Capacitor compensation capacity kvar Capacitor switching status Capacitor operation times Second-rate Capacitor input time s

[0101] According to the energy storage converter sub-process and Table 5:

[0102] ① Calculate the current reactive power compensation range of the energy storage converter, that is, how much inductive / capacitive reactive power the current device can generate at most; ② Determine whether the reactive power requirements of the three phases are consistent, that is, whether they all need to increase or reduce reactive power; if the reactive power requirements of the three phases are inconsistent, exit the process, otherwise enter ③; ③ Find the minimum value Qreal_min of reactive power compensation in the three phases; ④ Select 3 times Qreal_min for compensation. If the maximum compensation amount of PCS is less than the amount to be compensated, compensate with the maximum compensable value; ⑤ Determine the area where the system is located before and after the action. If the area score after the action is lower than before the action, no action will be taken;

[0103] Table 5 Data requirements

[0104] name unit Remark Device Status Working Mode 0: Standby, 1: PCS mode, 2: EPS mode AC reactive power Positive numbers are inductive, negative numbers are capacitive

[0105] According to the PV inverter sub-process:

[0106] ① Calculate the adjustable capacity range. The reactive power adjustment range of the inverter can be calculated according to the rated power and power factor range of the inverter; ② Determine whether the reactive power requirements of the three phases are consistent, that is, whether they all need to increase or reduce reactive power; if so, enter ④, otherwise enter ③; ③ Determine whether there is a single-phase photovoltaic inverter available for each phase, including whether the device communicates normally, whether there is an alarm signal, and whether there is a compensation margin. If so, enter ⑦, otherwise exit the sub-process; ④ Determine whether there is a three-phase photovoltaic inverter available, including whether the device communicates normally, whether there is an alarm signal, and whether there is a compensation margin. If so, enter ⑤, otherwise enter ③; ⑤ Find the minimum value Qreal_min of reactive power compensation in the three phases, and select 3 times Qreal_min for compensation; ⑥ Calculate the reactive power that needs to be compensated for each phase after the three-phase inverter is compensated, which is Qneed-Qreal_min; ⑦ Compensate for the reactive power that needs to be compensated for each phase. If the maximum compensation amount of the inverter is less than the amount to be compensated, compensate with the maximum compensable value. If there are multiple inverters of the same type at the same time, the capacity is evenly distributed for compensation;⑧ Determine the area where the system is located before and after the action. If the area score after the action is lower than before the action, no action will be taken;

[0107] According to the on-load voltage regulation transformer process and Table 6:

[0108] ①Confirm the action requirements of each phase, that is, whether to increase or decrease the voltage; ②Judge whether the on-load voltage regulator has the ability to adjust, including whether the equipment communication is normal, whether there is an alarm signal, whether there is a margin for adjustment, and whether the number of daily actions exceeds the limit. If it has the ability to adjust, enter ③, otherwise exit the sub-process; ③According to the optimal action target voltage value U of each phase aft_opt , select the one with the smallest difference from the voltage limit as the action target, and the difference is U aft_opt -U cur , take its absolute value and select the one closest to U aft_opt ④Estimate the voltage regulation effect and record the sum of the scores of the three-phase area before and after the action. If the score of the three-phase area after the action is higher than that before the action, the action will be executed, otherwise no action will be taken;

[0109] Table 6 Data requirements

[0110] name unit Remark Device Status Current gear position of transformer Transformer daily operation times

[0111] The above is a schematic scheme of a multi-agent self-discipline collaborative optimization method for multi-region reactive voltage control of this embodiment. It should be noted that the technical solution of the system of the multi-agent self-discipline collaborative optimization method for multi-region reactive voltage control and the technical solution of the multi-agent self-discipline collaborative optimization method for multi-region reactive voltage control mentioned above belong to the same concept. For the details not described in detail in the technical solution of the multi-agent self-discipline collaborative optimization system for multi-region reactive voltage control in this embodiment, please refer to the description of the technical solution of the multi-agent self-discipline collaborative optimization method for multi-region reactive voltage control mentioned above.

[0112] Example 3, reference Figure 6-Figure 15 An embodiment of the present invention provides a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0113] Algorithm 1: Figure 6 As shown in Table 7:

[0114] Table 7 Main steps of the DQN algorithm

[0115]

[0116] The IEEE 12-node system was used to simulate the real distribution network for simulation testing. The distributed photovoltaic and wind turbines were trained using data from northern Jiangsu in 2019. Each unit produced power once every hour.

[0117] The hyperparameters for training the DQN agent for online discrete reactive power optimization are designed as follows: the memory set size is set to 2000, the discount factor γ is set to 0.9, the batch size is 32, the learning rate is 0.005, and the target network is updated every 200 steps. e is 50, the unit regulation cost of OLTC and SCB is 1, and the voltage over-limit penalty coefficient is 100; use the data of each hour for each round, and train 50 steps in one round, where cost is the average network loss of each round, reward is the average immediate reward of each round, and average reward is the average reward of each round;

[0118] The loss value of the Q function finally converges to a stable state. The reward value is relatively low at the beginning of training because of the random exploration of actions. As the exploration factor gradually decreases, actions are selected according to the value function, the strategy tends to be stable, and the reward value also reaches a relatively high value. The inequality constraint is put into the reward function as a penalty. The final reward value can show that the actions taken by the intelligent agent can ensure that the voltage is within the normal range.

[0119] Algorithm 2: Figure 7As shown:

[0120] The IEEE 12-node example system and DG output data are used, and two static VAR generators (SVG) are used as reactive compensation equipment. The hyperparameters for training DDPG agents for online continuous reactive power optimization are designed as follows: the learning rates of the Actor and Critic networks are 0.001 and 0.002 respectively, the discount rate γ is 0.9, the soft update coefficient τ is 0.01, the memory set size is 2000, the batch size is 64, and the reward function is the sum of the network loss cost and the voltage limit penalty.

[0121] The DDPG network is trained for 8000 rounds, with 50 steps per round to obtain the reward convergence process. It can be seen that the final convergence effect of the DDPG algorithm is very good, and after convergence, the voltage will no longer exceed the limit, which greatly improves the safety of the distribution network.

[0122] Algorithm 3: Figure 8 As shown in Table 8-Table 10:

[0123] Table 8 Offline training process of proximal strategy optimization algorithm

[0124]

[0125] The trained PPO agent strategy network is used as the decision-making unit for online voltage and reactive power control of the distribution network. Whenever the agent observes the state information s of the distribution network, t , the optimal gear position a of the voltage regulating device can be selected according to the current strategy function t , so as to achieve the purpose of voltage reactive power control;

[0126] The effectiveness is verified on IEEE-12 node and modified IEEE33 node systems;

[0127] like Figure 8 As shown in the upper part, the base voltage of the feeder is 12.66kV, the three-phase base power is 10MVA, and the node load level is consistent with the standard system; a photovoltaic is connected at nodes 8, 18, and 31, and a wind turbine is connected at nodes 22 and 26. The data of photovoltaic and wind turbines are derived from one year of real data in a certain area in northern Jiangsu, and their rated capacity is 750kW; an OLTC is installed at node 0, which has 9 gears and can adjust the voltage to 0.9 to 1.1pu (1±4×2.5%), and an SCB is installed at nodes 16, 20, 24, and 30, which has 4 gears, each gear provides 200kVar reactive capacity, and all parameters in the simulation system are converted to per unit values;

[0128] The detailed parameter settings of the PPO agent training process are as follows:

[0129] Table 9 Detailed parameter settings of the PPO agent training process

[0130]

[0131]

[0132] In order to demonstrate the effectiveness of the proposed PPO control scheme, the DQN algorithm is selected as a benchmark. The MDP settings of PPO and DQN are the same, which ensures the comparability of quantitative results such as cumulative rewards;

[0133] Both PPO and DQN reach convergence after 200 trainings. The dashed area around the curve is calculated by averaging 5 repetitions. The variance of PPO’s reward curve is small, indicating that the training process is relatively smooth. In contrast, DQN’s strategic decisions fluctuate quickly, which increases its fragility in actual deployment.

[0134] In addition, PPO outperforms DQN in quantitative indicators such as voltage deviation and number of voltage violations. The voltage violation number is defined as the average number of node voltage violations in an event. Under the control of the PPO agent, fewer voltage violations occurred in the distribution system. Although the voltage violations have been greatly improved, the frequency of about 3 times a day is still high. This may be caused by the lack of sufficient voltage regulators in the network. In order to study the impact of different system sizes, two DRL methods were tested in two distribution networks. It can be seen that PPO has better performance in all cases. In addition, when DQN is changed to a larger system, the cumulative reward of DQN declines by dozens of times, while PPO is relatively mild.

[0135] Table 10 Numerical results

[0136]

[0137] Algorithm 4: Fig. 9 As shown in Table 11-12:

[0138] Table 11 Offline training process of SAC algorithm

[0139]

[0140]

[0141] The SAC algorithm is more suitable for processing continuous action spaces. We use the SAC algorithm to control continuous devices DGs and SVCs, and use the IEEE-12 node system to analyze the effectiveness and feasibility of the scheme;

[0142] In the 12-node system, two SVCs are used, located at nodes 5 and 8, and two distributed generation units with inverters are located at nodes 3 and 12, respectively. The SVC is (-600 to 600) kVar, and the reactive power regulation range of the smart inverter is (-300 to 300) kVar;

[0143] Table 12 Algorithm parameter settings

[0144]

[0145] The data used in the example is the output compression of photovoltaic power sources and wind turbines in northern Jiangsu for one year. We randomly selected the active power output of 5 generators per hour within a week to observe the fluctuation of photovoltaic power output and the uncertainty of wind power generation. The rated capacity of each DG power source is 750kW;

[0146] In order to verify the superiority of this method over the traditional method, it is compared with the method of continuous reactive power optimization of the power system using the particle swarm optimization (PSO) algorithm. For the PSO algorithm, we first use a random function to generate the position of the first generation of ions (that is, the reactive output of SVC and DG); then calculate the fitness of each particle (the network loss calculated by taking the position of the particle as the decision variable) to find the global optimal solution and the local optimal solution; then calculate the speed of each particle so that each particle moves in the direction of the global optimal and local optimal solution; calculate the position of the next generation of particles based on the current position and speed of each particle, and repeat the above process until the preset maximum number of iterations is reached;

[0147] In order to verify the superiority of the SAC algorithm over other RL algorithms, it is compared with the Deep Deterministic Policy Gradient (DDPG) algorithm. The DDPG algorithm is a method with a deterministic policy. It does not have an entropy term. At the same time, it uses the same number of policies and value function models as the SAC algorithm, and the hyperparameters of the neural network are also set to the same as the SAC algorithm.

[0148] Algorithm 5: Fig.10 As shown in Table 13-14:

[0149] The reactive voltage control process based on FBS power flow calculation is as follows:

[0150] Initialize the voltage and power of each node; use the voltage and power values ​​to calculate the current of each node; forward process: start from the end node of the feeder and calculate the current of all branches; backward process: start from the root node and calculate the node voltage; determine whether the voltage of all nodes meets the formula, if not, execute the forward process, if satisfied, end and output the power flow calculation results;

[0151] Table 13 DDQN agent learning process

[0152]

[0153] To verify the effectiveness of the proposed method, it was verified on a test system of IEEE13 nodes, two photovoltaic units were added to bus 675 and 684, and a wind turbine was added to bus 680. For the DDQN algorithm, the NN used had three fully connected layers, and the learning rate was selected as 0.0001;

[0154] Table 14 Hyper parameters of the DDQN algorithm used in the test system

[0155]

[0156] The DDQN algorithm enables the power grid to self-learn through the cognitive functions of VVO control by imitating human thinking. Ultimately, these well-trained agents can implement effective control strategies when faced with new operating conditions. In 50 test cases randomly selected from 13 bus systems, the power loss of the baseline and DDQN algorithms was compared. It was concluded that the DDQN algorithm can effectively reduce power loss.

[0157] Algorithm 6: Fig.11 As shown in Table 15:

[0158] The simulation experiment uses the standard IEEE-34 node distribution system as the simulation environment for DRL agent interaction. There are 9 PVs and 4 BESs at different nodes of the main feeder in this system. The PV and load curves are taken from a real data set of a PV installation point in Kunming, Yunnan. The curve considers 12 hours with an interval of 30 minutes, starting from 8:00 in the morning and ending at 7:30 in the evening. In the 34-bus test case, 9 PV inverters with a maximum power generation of 55% of the total load and 4 BES inverters with a maximum capacity of 35% of the total PV power generation are installed on the main feeder.

[0159] The TD3 agent was trained 500 times to learn the best strategy and provide the best action for controlling voltage and minimizing active power loss. Each Actor network has two Critic networks. The Actor network learns the policy function of the best action, while the Critic network learns the Q value to evaluate the strategy. The Actor network and the Critic network are neural networks composed of fully connected layers. The hidden layer has a rectified linear unit (ReLU) activation function and the output layer has a tanh activation function. The following table shows the hyperparameter settings of the TD3 algorithm.

[0160] Table 15 Hyperparameter settings of TD3 algorithm

[0161]

[0162]

[0163] The two Critic networks return Q values ​​for these specific states and actions. Our goal is to minimize the difference between the Q values ​​returned by the Critic networks. At the beginning of the training process, the agent randomly explores in the environment decision space. It eventually converges and finds the best action in the design space to reduce voltage fluctuations and active power losses. The cumulative reward for each round is calculated by adding up the rewards for each of the 24 time periods before the training process steps into the next episode. The reward at the beginning is very low because initially, the agent has zero previous knowledge of how to adjust the grid voltage. As learning progresses, the agent learns from previous experience, so the reward increases. After a certain number of rounds, the reward converges and reaches a maximum value.

[0164] To reduce undervoltage and overvoltage problems in the distribution network, the VVO method based on the TD3 algorithm coordinates between the aggregated PV and BES inverters to control the reactive set point of the smart inverter, as well as the charge and discharge rate of the BES. From the beginning of the morning to the early afternoon, the BES charges to avoid overvoltage problems, because during this time period, the PV generation is high, while the load demand is relatively low. From the early afternoon to the end of the dispatch time, the BES discharges to avoid grid overvoltage problems, because during this time period, the load demand is high, while the PV generation begins to decrease; Deviation of the minimum voltage in the IEEE-34 bus system under three conditions, it is shown that in the basic case, when the agent does not take any action, that is, the inverter does not provide reactive power, the battery is heuristically charged and discharged according to the pre-set rules;

[0165] The power loss of the distribution network increases with the inaccurate power injection or absorption of DERs. The TD3 algorithm trained agent of the present invention reduces power loss while minimizing voltage fluctuations by controlling the reactive power of BES and the reactive power of photovoltaic inverters. The total active power loss in 24 time periods. The trained DRL agent can effectively control the output power of DER inverters to reduce power loss. Compared with droop control, the power loss of the VVO method based on DRL agent is smaller, which proves the performance of the proposed method.

[0166] Algorithm 7: Fig.12 As shown in Table 16-19:

[0167] Table 16 Algorithm flow of CPO algorithm during VVC

[0168]

[0169]

[0170] The standard IEEE-34 node distribution system is used as the simulation environment for CPO agent interaction. There are 9 PVs and 4 BSSs at different nodes of the main feeder in this system.

[0171] Table 17 Specific parameters of simulation environment

[0172]

[0173] Table 18 Installed capacity of distributed resources

[0174]

[0175] The historical operation data used in the simulation environment comes from the distribution transformer load and the real PV power generation curve in a certain area of ​​Yunnan. The time interval of each decision step is 30 minutes. The charging and discharging of BSSs depends on the load demand and PV power generation, as well as the decision of the CPO agent. In the simulation experiment, the CPO agent was trained 1,000 times to learn the appropriate neural network parameters in the best way to adjust the voltage and minimize the network loss.

[0176] Table 19 Algorithm hyperparameter settings

[0177]

[0178]

[0179] As shown in the figure, the average reward value of each round and the number of nodes with cumulative voltage exceeding the limit of the proposed method during the training process converge. Due to the randomness of the DRL algorithm, this example uses three random seeds to train the CPO agent. The solid line represents the average of the reward value and constraint value of the three training links, and the shadow represents the variance of the reward value and constraint value of the three training links. In the early stage of the training phase, it is in the exploration stage, the reward value is relatively low, and the cumulative number of overvoltage nodes in each round is also large. Through continuous interactive training, under the guidance of the reward value and under the constraint of the logarithmic barrier function, the DRL agent gradually learns from the sample experience a strategy that meets the safety constraints of the distribution system, which is specifically manifested in that the number of voltage crossings in each round is close to 0;

[0180] In addition, the reward value of the strategy shows a fluctuating convergence process, because it is easy to explore strategies that do not meet the safety constraints in the early stage of training, and it is necessary to immediately abandon the strategy and adjust the strategy to optimize the cumulative reward value while meeting the safety constraints; after 450 rounds, the strategy converges and tends to be stable, indicating that while meeting the system node voltage constraints, the network active power loss is gradually reduced; the results show that the proposed CPO method effectively meets the formulated CMDP constraint problem and successfully achieves the minimization of network losses;

[0181] In order to verify the reactive power optimization strategy of the distribution network based on the CPO algorithm proposed in this invention, this example compares it with the heuristic improved particle swarm optimization algorithm (PSO) and the model-based second-order cone programming (SOCP) algorithm; the three algorithms are compared in terms of system network loss on the same test day. The SOCP algorithm accurately predicts various data and shows very excellent performance in reactive power optimization of the distribution network. Its active power loss is slightly better than the method proposed in this article. However, due to the randomness in the actual scenario, the theoretical optimal result cannot be reproduced in the operation and scheduling; the proposed CPO algorithm shows similar performance to SOCP under the condition of meeting the voltage constraint, which can effectively reduce the active power loss of the distribution network, and the average active power loss on the test day is reduced by 18.4kW compared with the PSO algorithm.

[0182] Algorithm 8: Figure 13-15 As shown in Table 20-23:

[0183] As shown in the figure, the GCN-PPO algorithm proposed in this paper adds a graph convolution layer in front of the Multi-Layer Perception (MLP) neural network, which improves the PPO agent's perception of graph data. The structure of the Actor network and the Critic network in the GCN-PPO algorithm. The Actor network architecture we use consists of two graph convolution layers and three MLP layers. Each layer is accompanied by a ReLU activation function, and the sum pooling function is used to aggregate the output of the graph convolution layer on adjacent nodes, and then pass it to the MLP layer to output a strategy; the architecture of the Critic network that defines the value function is roughly the same as that of the Actor network. The main difference is that a global sum pooling function is added after its graph convolution layer, so that the value function can aggregate the information of all nodes in the graph, thereby calculating the estimated value of the entire network;

[0184] As shown in the figure, when the estimate of the advantage function is negative, it means that the current strategy is negative, and its probability should be reduced (with 1-ε as the limit); when the estimate of the advantage function is positive, it means that the current strategy is positive, and its probability should be increased within a certain range (with 1+ε as the limit);

[0185] Table 20 Offline training process of GCN-PPO algorithm

[0186]

[0187] This project uses the IEEE 33-node test system to verify the effectiveness of the proposed method. The test system has 33 nodes and 37 lines (including 32 normally closed lines and 5 normally open lines). In the simulation stage, the test system uses the PyPower toolkit in Python to perform power flow calculations. The design and training of all DRL agents are completed by the PyTorch toolkit on a computer equipped with 16GB memory and 2.50GHz Intel(R) Core(TM) i7-11700.

[0188] The 4 PVs in the test system are installed at nodes 13, 22, 25, and 33, with installed capacities of 0.8MW, 1.2MW, 1.5MW, and 0.5MW, respectively. The 2 SVCs are installed at nodes 7 and 18, with reactive compensation capacities of 1Mvar each. This paper uses two years of measured distributed PV output and load data from a certain area in Yunnan Province to train the DRL agent. The output interval of PV and load data is 3 minutes, so the agent will capture the uncertainty of the problem and the dynamic characteristics of the environment from 240,000 (500×480) sets of data. At the same time, the data from the summer solstice and the winter solstice are selected as the test set to observe the effect of training, including the PV and load data of these two typical days.

[0189] In the offline training phase, the DRL agent was trained for 500 rounds, with 480 steps per round. The voltage limit was set to 0.97-1.07, and if the voltage exceeded the limit, it would be punished accordingly. If the power flow did not converge during the training process, the termination command would be returned;

[0190] Table 21 Hyperparameters of Actor network and Critic network and weights of reward function

[0191]

[0192] In order to verify the effectiveness of the proposed GCN-PPO algorithm, we compared it with the PPO algorithm without graph neural network. The Actor network and Critic network of the two algorithms use the same design and hyperparameters and are trained for 500 rounds on the same IEEE-33 node test system. The cumulative reward value and voltage limit times of the training rounds are calculated. Due to the randomness of the DRL algorithm, we use different random seeds to conduct three simulation experiments on each algorithm. The average value and error bound of the training results are represented by the solid line and the filled area.

[0193] At the beginning of training, the randomly initialized parameters made the agent show low performance, which was manifested as serious voltage over-limit. At this time, the action taken by the agent was wrong, and the Q value output by the critic network was quite different from the label y. As the training progressed, the agent began to learn the correct strategy from experience under the guidance of the reward value. At the same time, according to the gradient provided by the loss function, the parameters of the critic network were continuously optimized, and the loss was continuously reduced. After 200 rounds of training, the agent could make correct actions. At this time, the reward value and loss began to converge, and the number of voltage over-limits per round also approached 0. The performance of GCN-PPO after convergence was slightly better than that of PPO algorithm, but the fluctuation of the reward value of GCN-PPO algorithm during training was large, and it was not as stable as PPO algorithm. In order to further prove the advantages of the proposed GCN-PPO algorithm in VVO, the trained model was tested on the test set and compared with the centralized control OPF algorithm and the heuristic-based PSO algorithm. The active power loss of the three algorithms on the test set of summer solstice and winter solstice was compared. The centralized OPF is the optimization result obtained through a large amount of calculation under accurate grid parameters and data, so it has the best VVO performance. The GCN-PPO algorithm in this paper shows similar performance to OPF, and under the condition of meeting the voltage constraint, the average network loss of GCN-PPO on the summer solstice and winter solstice test sets is reduced by 37kW and 16.2kW compared with the PSO algorithm;

[0194] The time for solving the strategy in each decision step of the GCN-PPO algorithm and the PSO algorithm is that since the trained reinforcement learning agent can be regarded as a controller with a neural network model, when making decisions, it only needs to input the current state to get a set of strategy actions. However, the PSO algorithm needs to iterate and optimize again at the beginning of each decision, and its solution speed is not of the same order of magnitude as the DRL algorithm. At the same time, since the graph convolution layer of the proposed GCN-PPO algorithm needs to perform message passing, its training and execution speed is slightly slower than that of the PPO algorithm.

[0195] Table 22 Comparison of single-step decision time

[0196]

[0197] In order to verify the scalability of the proposed algorithm when the distribution network topology changes, this paper saves the trained GCN-PPO and PPO agent models, and randomly extracts 10 days of PV and load operation data for testing on the three designed scenarios; in order to simulate the scenario of fault maintenance, we use closing the tie switch and opening the section switch as a set of topological transformation methods, and design three scenarios according to the degree of topological transformation: 1) Scenario 1 is to close the tie switch between nodes 12 and 22, and open the section switch between nodes 10 and 11; 2) Scenario 2 is to close the tie switch between nodes 12 and 22, 8 and 21, 25 and 29, and then open the section switch between nodes 10 and 11, 4 and 5, 6 and 26; 3) Close the tie switch between nodes 12 and 22, 8 and 21, 25 and 29, 9 and 15, 18 and 33, and then open the section switch between nodes 10 and 11, 4 and 5, 6 and 26, 8 and 9, 30 and 31;

[0198] The number of voltage overruns of the GCN-PPO algorithm and the PPO algorithm in 10 days under three scenarios. Since the ordinary PPO agent does not consider the topological connection relationship during the training process, as the degree of change of the distribution network topology continues to increase, its voltage overrun rate continues to increase. However, the proposed GCN-PPO agent still has good voltage control performance when the topology changes greatly;

[0199] In power systems, it is not uncommon to lose measurement data due to sensor communication failures, so we studied the robustness of the proposed algorithm when measurement data is lost; in order to simulate the situation of measurement data loss, we randomly selected some nodes in the distribution network and set their voltage to zero during the test. This caused some values ​​of the state vector input to the PPO algorithm to be zero, and some node features in the graph data input to the GCN-PPO algorithm to be zero. At the same time, we set up 3 groups of experimental scenarios, representing 25%, 50%, and 75% of the node voltage measurement values ​​lost.

[0200] From the table, we can see that although the performance of both algorithms is affected by data loss, the impact of data loss on the PPO algorithm is much more serious than that on the GCN-PPO, and this impact increases with the increase in the amount of data loss. In the worst case (when the data is lost by 75%), the performance of PPO is nearly 3.6 times worse than that of GCN-PPO. Therefore, GCN-PPO has better robustness than PPO when measuring data loss.

[0201] Table 23 Impact of data loss on algorithm performance

[0202]

[0203] In practice, grid dispatchers often use the average value of neighboring node voltages to fill in missing data, which is similar to the principle of message passing in the graph convolutional layer used in GCN-PPO, but we believe that the GCN-PPO algorithm is a more principled way to achieve this robustness because GCN-PPO learns a (possibly nonlinear) aggregation strategy for neighboring values ​​rather than a strategy that needs to be manually formulated.

[0204] Embodiment 4 is an embodiment of the present invention, which provides a multi-agent autonomous collaborative optimization system for multi-region reactive power voltage control, including: a data interaction module and an optimization decision module;

[0205] The data interaction module establishes a data interaction model and sets a data interaction mode;

[0206] The optimization decision module studies the optimal reactive power voltage optimization decision method of the distribution network.

[0207] This embodiment further provides a computing device, which is applicable to a multi-agent autonomous collaborative optimization method for multi-region reactive power voltage control, and includes:

[0208] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control as proposed in the above embodiment.

[0209] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control proposed in the above embodiment is implemented.

[0210] The storage medium proposed in this embodiment and the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0211] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0212] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0213] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0214] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control, characterized by: include: Establish data interaction model and set data interaction mode; The optimal reactive power and voltage optimization decision method of distribution network is studied.

2. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 1, characterized in that: The data interaction model is used to describe the data exchange and coordination mechanism between the regional controller and the local controller in reactive power optimization and voltage optimization.

3. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 2, characterized in that: The data interaction mode includes a specific cooperation mode between the regional controller and the local controller during the reactive power optimization and voltage optimization process.

4. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 3, characterized in that: The reactive power and voltage optimization decision-making method includes methods for optimizing reactive power and voltage levels of a power grid to ensure system stability and efficiency.

5. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 4, characterized in that: The data interaction model also includes a reactive power optimization data interaction model and a voltage optimization data interaction model.

6. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 5, characterized in that: The interactive mode also includes a first mode of voltage regulation linkage and a second mode of reactive power linkage.

7. The multi-agent autonomous collaborative optimization method for multi-region reactive power and voltage control according to claim 6, characterized in that: The reactive voltage optimization decision method also includes reactive voltage control based on traditional mathematical optimization algorithm, reactive voltage control based on heuristic algorithm, reactive voltage control based on deep reinforcement learning algorithm, and distribution network on-site voltage reactive power control method based on deterministic optimization; Among them, the reactive power and voltage control based on deep reinforcement learning algorithm includes DON algorithm and discrete reactive power optimization of distribution network, DDPG algorithm and continuous reactive power optimization of distribution network, PPO algorithm and continuous reactive power optimization of distribution network, SAC algorithm and continuous reactive power optimization of distribution network, distribution network voltage reactive power control method based on DDQ algorithm, distribution network voltage reactive power control method based on double-delay deep deterministic policy gradient algorithm, distribution network voltage reactive power control method based on secure reinforcement learning CPO algorithm and distribution network reactive power optimization method based on time-space perception technology.

8. A multi-agent autonomous collaborative optimization system for multi-region reactive power and voltage control based on any one of claims 1 to 7, characterized in that: include: Data interaction module and optimization decision module; The data interaction module establishes a data interaction model and sets a data interaction mode; The optimization decision module studies the optimal reactive power voltage optimization decision method of the distribution network.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-agent autonomous collaborative optimization method for multi-region reactive voltage control according to any one of claims 1 to 7 are implemented.

Images