An optimization method and system for distribution voltage regulation considering an incentive microgrid

Abstract

The present invention proposes an optimization method and system for a distribution voltage regulation considering an incentive microgrid, belonging to the technical field of distribution network voltage control, including: obtaining data; inputting the obtained data into the optimal operation model of the microgrid before establishing the voltage regulation incentive mechanism and the reinforcement learning agent; calculating the operating voltage of the distribution network when the microgrid optimizes the purchased and sold electric powers without considering the voltage regulation incentive based on the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism, and using the operating voltage at this time as the reference voltage; the reinforcement learning agent makes a decision on the current operating state according to the input data, outputs the power of each device, and then obtains the actual operating voltage through the distribution network power flow calculation; obtaining the voltage regulation compensation, updating the agent strategy based on the voltage regulation compensation, and the reinforcement learning agent continuously updates its own strategy to make a decision on the current operating state and outputs the optimized operating power of each device.

Description

Technical Field

[0001] The present invention belongs to the technical field of distribution network voltage control, and particularly relates to an optimization method and system for distribution voltage regulation considering an exciting microgrid. Background Art

[0002] The statements in this part merely provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] The global energy and environmental crises have accelerated the large-scale development of renewable energy and distributed energy. Therefore, microgrid technology has received extensive attention in the power industry. A microgrid is a small power system that includes distributed energy and loads and can operate in grid-connected mode or island mode. The operation of a microgrid has great flexibility. The advantages of integrating a microgrid into a distribution system are manifold: First, the distributed energy units in the microgrid can meet local energy demands, reduce dependence on the superior power grid, and improve the reliability of power supply; Second, the microgrid can promote the full utilization of new energy through renewable energy such as wind turbines and photovoltaics; Moreover, through the microgrid, on-site energy production and consumption can be achieved, reducing long-distance power transmission losses and the investment in large transformers and transmission lines.

[0004] Regarding the optimal operation of microgrids, domestic and foreign scholars have conducted a large number of studies, but mainly focused on the optimal scheduling problems of single microgrids, multi-microgrids, and microgrid-distribution network, with the emphasis on solving the problems of how to improve the benefits of microgrids or reduce the operating costs of microgrids. However, with the continuous increase in the microgrid penetration rate, the peak loads and peak reverse energy flows during off-peak / peak periods caused by the mismatch between the new energy generation characteristics and the load electricity consumption characteristics will violate the distribution network voltage limits, and most studies have ignored the consideration of distribution network voltage constraints.

[0005] For example, in the prior art, CN118157148A, an intelligent multi-agent collaborative voltage control method for a microgrid group. This method first establishes a master-slave game model between the distribution network and the multi-microgrid, and uses a reasonable economic incentive mechanism to encourage the multi-microgrid to autonomously participate in the distribution network voltage control; To ensure the privacy of multiple agents, the multiple agents perform dynamic game decisions based on incomplete information and deep reinforcement learning algorithms, enabling them to maximize their own interests while providing voltage control auxiliary services for the distribution network; The proposed method can enable the distribution network, under a reasonable economic incentive mechanism, to encourage multiple interest agents to autonomously participate in the operation optimization of the distribution network, so as to reduce the additional voltage regulation equipment configured in the distribution network, thereby improving the economy and energy utilization efficiency of the distribution network.

[0006] The incentive problem proposed in the above patent is mainly the voltage regulation subsidy for the active power change of the microgrid. The electricity trading price is determined through the master-slave game bidding between the distribution network and the microgrid. However, the voltage regulation subsidy in the above patent is only measured by the active power change of the microgrid, and the actual voltage regulation effect of the distribution network nodes is not considered as an influence on the incentive mechanism. In fact, when the distribution network does not consider setting voltage regulation subsidies or incentives, the microgrid often maximizes its own benefits. The operation plan in this state may often lead to a series of problems such as voltage over-limit. By measuring the actual voltage regulation effect, it can better reflect the contribution of the microgrid in the voltage regulation process. By converting the actual voltage regulation effect into voltage regulation compensation and jointly representing it with the operation cost of the microgrid through the economic objective function, the enthusiasm of the microgrid for voltage regulation can be fully stimulated.

[0007] The traditional equipment participating in the voltage regulation of the distribution network mainly includes on-load tap-changing transformers, capacitor banks, photovoltaics, energy storage systems, etc. As a small system containing sources, loads, and energy storage, the microgrid can flexibly adjust active and reactive power to contribute to solving the voltage regulation problem of the distribution network. However, in actual operation, the microgrid and the distribution network belong to different operators. The voltage regulation of the microgrid may cause an increase in operation costs or a decrease in benefits. There are many practical problems in the direct control and scheduling of the distribution network. The main problems solved by the present invention are: how to fully mobilize the enthusiasm of the microgrid for voltage regulation, compensate for the increased costs caused by voltage regulation of the microgrid, and mobilize different microgrids to minimize the voltage regulation cost, etc. Summary of the Invention

[0008] To overcome the deficiencies of the above prior art, the present invention provides an optimization method for considering the incentive of the microgrid for distribution voltage regulation, which mobilizes the enthusiasm of the microgrid to participate in the voltage regulation of the distribution network and is conducive to building a more secure and reliable distribution system.

[0009] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0010] In the first aspect, an optimization method for considering the incentive of the microgrid for distribution voltage regulation is provided, including:

[0011] Obtain the load data of the distribution network, photovoltaic predicted output data, wind turbine predicted output data, energy storage system capacity, micro gas turbine capacity, electric boiler capacity and time-of-use electricity price predicted the day before;

[0012] Input the obtained data into the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism and the reinforcement learning agent;

[0013] Based on the input data, the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism calculates the operating voltage of the distribution network when the microgrid optimizes the purchased and sold electric power without considering the voltage regulation incentive. The operating voltage at this time is used as the reference voltage;

[0014] The reinforcement learning agent makes decisions on the current operating state based on the input data, outputs the power of each device, and then obtains the actual operating voltage through the power flow calculation of the distribution network;

[0015] Calculate the voltage regulation compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives, obtain the voltage regulation compensation, update the agent's policy based on the voltage regulation compensation, and the reinforcement learning agent continuously updates its own policy to make decisions on the current operating state and output the optimized operating power of each device.

[0016] As a further technical solution, the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism aims to minimize the operating cost of the microgrid, mainly including equipment operating cost, fuel cost, and power purchase and sale cost;

[0017] Consider the operating constraints of the energy storage system, the operating constraints of the micro gas turbine, the operating constraints of the electric boiler, and the power balance constraints of the microgrid.

[0018] As a further technical solution, various set voltage regulation incentives include:

[0019] The reduced voltage regulation cost in the distribution network;

[0020] The reduced DG disconnection risk cost in the distribution network;

[0021] The reduced electricity cost of voltage-related loads in the microgrid.

[0022] As a further technical solution, calculate the voltage regulation compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives, obtain the voltage regulation compensation, specifically including:

[0023] The compensation obtained from the three types of incentives includes the following three parts:

[0024] ;

[0025] ;

[0026] ;

[0027] Therefore, the total voltage regulation compensation obtained by the microgrid at moment is:

[0028] .

[0029] As a further technical solution, regarding the reinforcement learning agent, each microgrid is represented by an agent, and the agent continuously repeats the exploration of the sequential decision-making process to learn the optimal policy.

[0030] As a further technical solution, the elements of the agent all include , where: is the state set that the agent can perceive, is the policy of the agent; is the action made by the agent according to the policy in the current state; is the state transition function, that is, the probability of transitioning to the next state under the current state and action; is the reward set that the agent can obtain after executing an action in the environment;

[0031] State space: The agent learns and makes decisions on behalf of the microgrid, and observes the current environment at each decision moment to obtain the current state , including photovoltaic power, wind turbine power, electrical load power, thermal load power, energy storage charge and discharge power, micro gas turbine power, electric boiler power, electricity purchase and sale price, and the current decision moment;

[0032] Action space: According to the state and policy at the current moment, the agent makes decisions and obtains the actual actions of each device after linear mapping , including the charge and discharge power of the energy storage, the change in the power of the micro gas turbine, and the power of the electric boiler;

[0033] Reward space: Applying the actual actions of each device in the simulation environment can obtain the reward at this moment , including voltage regulation compensation, the revenue from purchasing and selling electricity from the superior power grid, the charge and discharge costs of the energy storage, the fuel costs of the micro gas turbine, and the reactive power regulation costs.

[0034] In a second aspect, an optimization system for distribution voltage regulation considering the incentive of the microgrid is provided, including:

[0035] A data acquisition module, configured to: acquire the load data of the distribution network predicted in advance, the predicted output data of photovoltaic power, the predicted output data of wind turbines, the capacity of the energy storage system, the capacity of the micro gas turbine, the capacity of the electric boiler, and the time-of-use electricity price;

[0036] Input the acquired data into the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism and the reinforcement learning agent;

[0037] A reference voltage determination module, configured to: The optimal operation model of the microgrid before considering the voltage regulation incentive mechanism calculates the operating voltage of the distribution network when the microgrid optimizes the purchased and sold electric power without considering the voltage regulation incentive based on the input data, and this operating voltage is used as the reference voltage;

[0038] An actual operating voltage determination module, configured to: the reinforcement learning agent makes a decision on the current operating state according to the input data, outputs the power of each device, and then obtains the actual operating voltage through the power flow calculation of the distribution network;

[0039] A voltage regulation and compensation module, configured to: calculate the voltage regulation and compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives, obtain the voltage regulation and compensation, update the agent policy based on the voltage regulation and compensation, and the reinforcement learning agent continuously updates its own policy to make a decision on the current operating state and outputs the optimized operating power of each device.

[0040] The above one or more technical solutions have the following beneficial effects:

[0041] In the technical solution of the present invention, the incentives for the microgrid in the voltage control problem are fully considered, the voltage regulation effect of the microgrid is measured through the voltage regulation incentive mechanism, and its enthusiasm for participating in voltage regulation is effectively improved. The voltage regulation incentives include the direct incentives of the distribution network for the microgrid to participate in voltage regulation: the reduction of the voltage regulation cost of the distribution network, and the reduction of the risk cost of disconnecting the DG of the distribution network due to the solution of the overvoltage problem; the indirect incentives obtained from the reduction of the load power consumption cost in the microgrid: the reduction of the power consumption cost of the voltage-related loads in the microgrid. Through the above voltage regulation incentives, a reasonable and effective voltage regulation incentive mechanism is constructed to measure the voltage regulation contribution of the microgrid. And the SQDDPG algorithm introducing the Shaply Q value is used to distribute the voltage regulation rewards of different microgrids, realizing the fair credit distribution among agents. In the optimization strategy considering the incentive for the microgrid to regulate the distribution voltage, the microgrid can obtain benefits by providing voltage regulation support for the distribution network on the basis of ensuring its own operating economy, improving the overall operating economy, and solving the voltage over-limit problem of the distribution network, which is beneficial to constructing a safe and reliable power distribution system.

[0042] The voltage regulation incentives proposed in the technical solution of the present invention consider that the microgrid can adjust the reactive power. The reactive power has a lower adjustment cost and a more remarkable voltage regulation effect, and it is more reasonable to consider the active power and the reactive power together.

[0043] The advantages of the additional aspects of the present invention will be partially given in the following description, partially will become obvious from the following description, or will be understood through the practice of the present invention. Brief Description of the Drawings

[0044] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0045] Figure 1 It is a schematic diagram of the distribution network structure;

[0046] Figure 2Schematic diagram of voltage regulation incentive mechanism;

[0047] Figure 3 Schematic diagram of the optimization strategy considering the incentive of the microgrid for distribution voltage regulation;

[0048] Figure 4 Schematic diagram of the voltage distribution of the distribution network before considering the voltage regulation incentive mechanism;

[0049] Figure 5 Schematic diagram of the voltage distribution of the distribution network after considering the voltage regulation incentive mechanism;

[0050] Figure 6 Flowchart of the method of the embodiment of the present invention. Detailed implementation mode

[0051] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0052] It should be noted that the terms used herein are only for describing specific implementation modes and are not intended to limit the exemplary implementation modes according to the present invention.

[0053] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0054] When the voltage regulation compensation is not set, the microgrid always makes full use of its flexibility resources. On the basis of meeting the local load, it optimizes the purchase and sale of electric power according to the time-of-use electricity price and optimizes its operation plan with the goal of minimizing costs, without considering the operation state of the distribution network. However, when operating according to this operation plan in practice, the distribution network will inevitably have problems of node voltage over-limit, resulting in increased operation and control difficulties. The technical solution of the embodiment of the present invention proposes an incentive strategy for the microgrid to regulate the voltage of the distribution network. The proposed invention fully considers the incentive factors for the microgrid to regulate the voltage, reasonably measures the contribution to the voltage regulation of the distribution network and fairly distributes the rewards. The microgrid can improve its own operation economy while providing voltage support for the distribution network, which is of great significance for building a safe and reliable operation distribution network.

[0055] Embodiment 1

[0056] See Appendix Figure 6 As shown, this embodiment discloses an optimization method considering the incentive of the microgrid for distribution voltage regulation, including:

[0057] Step 1: Input the load data of the distribution network predicted on the day before, the predicted output data of the photovoltaic, the predicted output data of the wind turbine, the capacity of the energy storage system, the capacity of the micro gas turbine, the capacity of the electric boiler, and the time-of-use electricity price;

[0058] Step 2: In the optimal operation model of the microgrid before introducing the voltage regulation incentive mechanism, the operation cost of the microgrid is minimized, mainly including equipment operation cost, fuel cost, and power purchase and sale cost; the operation constraints of the energy storage system, the operation constraints of the micro gas turbine, the operation constraints of the electric boiler, and the power balance constraints of the microgrid are considered;

[0059] Step 3: In the voltage regulation incentive mechanism, reducing the voltage regulation cost of the distribution network, reducing the DG disconnection risk cost of the distribution network due to overvoltage problems, and reducing the electricity cost of voltage-related loads in the microgrid are considered as the incentives for the microgrid voltage regulation. The voltage regulation compensation can be obtained by comparing the node voltage operation states before and after participating in voltage regulation.

[0060] The voltage regulation incentive mechanism is attached Figure 2 All the content in it is essentially a calculation formula for calculating the voltage regulation compensation of the microgrid attached Figure 2 in it, and three parts are required for the calculation of this formula:

[0061] (1) Various voltage regulation incentives include three types: reducing the voltage regulation cost of the distribution network, reducing the DG disconnection risk cost of the distribution network due to overvoltage problems, and reducing the electricity cost of voltage-related loads in the microgrid;

[0062] (2) The voltage operation state of the distribution network is obtained through the optimal operation plan of the microgrid without considering the voltage regulation incentive and used as the reference voltage;

[0063] (3) The calculation formula of the voltage regulation compensation, and the voltage regulation compensation of the microgrid is obtained through the reference voltage, the actual operation voltage, and this calculation formula;

[0064] The voltage regulation incentive mechanism is all the above content, and the voltage regulation incentive is the three types of voltage regulation compensation set in (1).

[0065] Step 4: Establish a reinforcement learning agent based on the SQDDPG algorithm, construct an optimization problem of the microgrid based on the voltage regulation incentive mechanism as the interaction environment, construct the state space, action space, and reward space of the agent, calculate the reward for the agent through the node voltage of the system after the agent's decision and the reference voltage before considering the voltage regulation incentive mechanism, and obtain the optimal strategy of the agent through continuous learning and iteration.

[0066] The data and parameters in Step 1 are the input data of the "optimal operation model of the microgrid before introducing the voltage regulation incentive mechanism" in Step 2 and the "optimization problem of the microgrid based on the voltage regulation incentive mechanism" in Step 4 respectively.

[0067] The model constructed in Step 2 is optimized according to the data and parameters in Step 1, that is, the data and parameters in Step 1 are known data, and the optimization variable is the active power of each device.

[0068] Step 4 constructs the interaction environment required for the reinforcement learning agent based on the data and parameters in Step 1. Similarly, the optimization variables controlled by the agent are the powers of various devices, and the data and parameters in Step 1 are known data used to construct the state space of the agent.

[0069] The operation plan of the microgrid, i.e., the power purchase and sale electricity, can be obtained through the optimization model in Step 2. The distribution network can obtain the node voltage by substituting the power purchase and sale electricity into the power flow calculation. Since this voltage is obtained through the operation plan of the microgrid before considering the voltage regulation incentive, this voltage is used as the reference voltage.

[0070] The agent in Step 4 makes decisions based on the current operating state and controls the powers of various devices. Since the voltage regulation incentive is considered in the objective function in Step 4, in addition to the power purchase and sale revenue, voltage regulation compensation can also be obtained. This voltage regulation compensation is calculated through the actual operating voltage of the distribution network at this time, the reference voltage, and the voltage regulation incentive mechanism set in Step 3. Through the calculated voltage regulation compensation, the agent in Step 3 updates its parameters to obtain higher operating revenue.

[0071] The "optimal operation model of the microgrid before the voltage regulation incentive mechanism" in Step 2 is used to calculate the operating voltage of the distribution network when the microgrid optimizes the obtained power purchase and sale electricity before considering the voltage regulation incentive. This operating voltage is used as the reference voltage and compared with the actual operating voltage after considering the voltage regulation incentive mechanism later to measure the contribution of the microgrid to voltage regulation, and the voltage regulation compensation of the microgrid is calculated in combination with the voltage regulation incentive mechanism to stimulate its enthusiasm for participating in voltage regulation.

[0072] The voltage regulation incentive mechanism in Step 3 calculates the voltage regulation compensation based on the reference voltage in Step 2, the actual operating voltage in Step 4, and various set voltage regulation incentives. After obtaining the voltage regulation compensation, the microgrid can not only change the power to obtain revenue through power purchase and sale, but also obtain revenue through voltage regulation by changing the power. The reinforcement learning agent continuously updates its own strategy to obtain higher overall operating revenue.

[0073] In Step 4, operating data, device parameters, and other known data are obtained through Step 1 and used to construct the interaction environment of reinforcement learning; voltage regulation compensation can be obtained through Steps 2 and 3 and used to update its own strategy to obtain higher revenue.

[0074] Through the continuous update of the agent, stable control strategies of each agent can be obtained. These strategies can control the powers of various devices in the microgrid and output the optimized operating powers of various devices, as well as the obtained power purchase and sale revenue and voltage regulation compensation.

[0075] The agent adjusts the interactive power between the microgrid and the distribution network by controlling the active power and reactive power of the device. This interactive power not only includes the revenue obtained from buying and selling electricity with the distribution network, but also includes the voltage regulation compensation obtained by adjusting the node voltage. Through the optimized strategy, the operating power of the microgrid can be obtained. This operating power can improve the overall operating revenue of the microgrid while reducing the operating voltage deviation of the distribution network and ensuring the safety of operation.

[0076] As Figure 2 shown, to construct the voltage regulation incentive mechanism, first, without considering the voltage regulation incentive, the optimal operating plan of the microgrid is obtained and the power flow calculation is carried out to obtain the operating state of each node. The operating state of the node voltage of the system at this time is used as the reference voltage, and the voltage regulation compensation of the microgrid is obtained by comparing it with the actual operating voltage after the incentive is introduced. The microgrid can adjust the node voltage of the distribution network by changing the operating plan or adjusting the interactive power. In this process, the voltage regulation incentive is used to mobilize the enthusiasm of the microgrid to participate in the voltage regulation of the distribution network. The voltage regulation incentive consists of the following three parts: the reduction of the voltage regulation cost of the distribution network, the reduction of the DG disconnection risk cost of the distribution network due to the solution of the overvoltage problem, and the reduction of the electricity cost of the voltage-related loads in the microgrid. Among them, the first two are the direct incentives of the distribution network for the microgrid to participate in voltage regulation, and the third is the indirect incentive obtained by reducing the load electricity demand cost in the microgrid.

[0077] The voltage regulation compensation of the microgrid can be calculated through the reference voltage, actual voltage, and voltage regulation incentive. As shown in Equations 19 - 22, the reference voltage and actual voltage are the corresponding voltage values in the equations. Each voltage regulation incentive corresponds to a calculation formula. Substituting the reference voltage and actual voltage into the corresponding voltage regulation incentive calculation formula can obtain the corresponding voltage regulation compensation.

[0078] The optimization strategy considering the incentive for the microgrid to regulate the distribution voltage is as Figure 3 shown. The optimal operating strategy of the microgrid considering the voltage regulation incentive is obtained through the SQDDPG algorithm. Each agent in the algorithm represents a microgrid to control the power of internal devices, and the global reward distribution is carried out by introducing the Shapley Q value to measure the contributions of different microgrids. The algorithm adopts a centralized training and distributed execution framework. Through continuous exploration and learning, the agent gradually converges to the optimal operating strategy. The strategy can perform fast control only through local information and the trained local strategy during intraday operation, effectively reducing data transmission and calculation.

[0079] In this implementation case, the structure of the distribution network is as Figure 1As shown in the figure. The voltage regulation incentive mechanism of the microgrid includes voltage regulation incentive setting, the optimal operation model of the microgrid without considering voltage regulation incentives, and voltage regulation compensation calculation. The optimal operation model of the microgrid without considering voltage regulation incentives takes into account the operation constraints of the energy storage system, the operation constraints of the micro gas turbine, the operation constraints of the electric boiler, and the power balance constraints, and uses the Gurobi solver in Python to solve the problem, obtaining the optimal operation plan of the microgrid without considering voltage regulation incentives, that is, the output power of each device. The voltage regulation strategy of the microgrid to the distribution network is incentivized. Through the power flow calculation of the distribution network and the voltage regulation incentive mechanism as the interaction environment, the reinforcement learning agent is trained through deep reinforcement learning, and the operation strategy of the microgrid is obtained through iterative optimization between the agent and the environment, including the charge and discharge power of the energy storage, the output power of the gas turbine, the power of the electric boiler, the reactive power of the fan, and the reactive power of the photovoltaic.

[0080] The voltage regulation incentive mechanism is attached Figure 2 All the content in it is used to calculate the voltage regulation compensation of the microgrid attached Figure 2 in which the formula calculation requires three parts:

[0081] (1) Various voltage regulation incentive settings, including three types: the reduction of the voltage regulation cost of the distribution network, the reduction of the DG disconnection risk cost of the distribution network due to the solution of overvoltage problems, and the reduction of the electricity cost of voltage-related loads in the microgrid;

[0082] (2) Obtain the voltage operation state of the distribution network through the optimal operation model of the microgrid without considering voltage regulation incentives and use it as the reference voltage;

[0083] (3) Voltage regulation compensation calculation, obtaining the voltage regulation compensation of the microgrid through the reference voltage, the actual operating voltage, and the voltage regulation incentive calculation formula.

[0084] Voltage regulation incentive setting: the reduction of the voltage regulation cost in the distribution network, the reduction of the DG disconnection risk cost in the distribution network, and the reduction of the electricity cost of voltage-related loads in the microgrid.

[0085] Regarding the reduction of the voltage regulation cost in the distribution network: The distribution network usually adjusts the voltage by setting devices such as on-load tap-changer transformers, capacitor banks, and static var compensators. The configuration, operation, and maintenance costs of these devices can be considered as the voltage regulation cost of the distribution network. When the node voltage is closer to the rated value, the voltage regulation cost required by the distribution network is smaller, which is expressed by the following formula:

[0086] (1);

[0087] In the formula, is the voltage regulation cost of the distribution network at time is the total number of nodes in the distribution network; The voltage regulation cost coefficient of the distribution network representing the unit node voltage deviation; is the voltage magnitude of node at time ;

[0088] Regarding the reduction of the DG disconnection risk cost in the distribution network: When no voltage regulation measures are taken or the voltage regulation is improper, overvoltage phenomena will occur in the distribution network, causing unnecessary economic losses to the distribution network. For example, DGs connected through inverters need to be temporarily disconnected from the grid to ensure operation safety, and the disconnection of DGs will increase the power purchase cost from the superior grid. The DG disconnection risk cost caused by overvoltage in the distribution network is expressed by the following formula:

[0089] (2);

[0090] In the formula, is the DG disconnection risk cost at time ; represents the DG disconnection risk cost coefficient per unit node overvoltage;

[0091] Regarding the reduction of the electricity consumption cost of voltage-related loads in the microgrid: From the static voltage characteristics of the load, it can be known that the active power of the load will change with the slow change of the node voltage. Under overvoltage conditions, the load usually consumes more electric energy. Therefore, if the node voltage can be effectively regulated, for the microgrid, while the load power decreases, more electric energy is allowed to be fed into the grid to increase the revenue, effectively stimulating its enthusiasm for participating in voltage regulation. The ZIP model is used to model the load in the microgrid, which consists of three parts: constant impedance (Z), constant current (I), and constant power (P).

[0092] (3);

[0093] In the formula, is the active power of the load in the microgrid when the voltage of the access node at time is ; represents the active power of the load when the rated voltage ; and respectively represent the proportions of the three parts of constant impedance, constant current, and constant power in the total node load. Formula (3) is used to calculate the voltage regulation compensation of the third type of voltage regulation incentive in subsequent formula (21).

[0094] In Step 2, the optimal operation model of the microgrid without considering voltage regulation incentives: The microgrid mainly consists of distributed power sources, energy storage systems, electric boilers, and user loads (electric load, heat load), etc. Among them, the distributed power sources can be divided into uncontrollable units and controllable units. The uncontrollable units include wind turbines and photovoltaics, while the controllable units include micro gas turbines. When not considering voltage regulation incentives, the operating costs of the microgrid mainly include the operating costs of the above-mentioned equipment, fuel costs, and power purchase and sale costs. The optimal operation model includes an objective function and constraint conditions.

[0095] Objective function: The microgrid is optimized with the goal of minimizing operating costs, and the objective function is as follows:

[0096] (4);

[0097] In the formula, is the set of optimization time periods; is the set of microgrids; is the charging and discharging cost of the energy storage system in the microgrid at time is the operating cost of the micro gas turbine in the microgrid at time and are the power purchase and sale power of the microgrid at time and are the power purchase and sale electricity price at time

[0098] The constraint conditions include: energy storage system operation constraints, micro gas turbine operation constraints, electric boiler operation constraints, and microgrid power balance constraints.

[0099] Regarding the energy storage system operation constraints: Considering the service life, energy efficiency of the energy storage system, and the discharge depth and capacity attenuation of electrochemical energy storage technology, etc., the charging and discharging cost of the energy storage system can be obtained by performing a levelized calculation on the costs and power generation during the entire life cycle of the energy storage system, also known as the cost per kilowatt-hour. Then, the charging and discharging cost of the energy storage system can be expressed by the following formula, where is the charging and discharging cost coefficient of the energy storage system:

[0100] (5);

[0101] (6);

[0102] (7);

[0103] (8);

[0104] (9);

[0105] In the formula, is the charging power of the energy storage system in the microgrid at time ; is the discharging power of the energy storage system in the microgrid at time ; Formula (6) is used to calculate the State of Charge (SOC) of the energy storage system, and are the SOC states of the energy storage system in the microgrid at time and time respectively; and are 0-1 variables, indicating the charging and discharging states of the energy storage system in the microgrid at time ; and are the charging and discharging efficiencies of the energy storage system in the microgrid at time ; is the rated capacity of the energy storage system in the microgrid ; is the optimization time interval. Formula (7) represents the nuclear power state constraint of the energy storage system, and are the minimum and maximum allowable SOC states of the energy storage system respectively; Formula (8) is the charging and discharging power constraint of the energy storage system, and are the maximum charging and discharging powers of the energy storage system respectively; Formula (9) indicates that the energy storage system can only operate in one state of charging or discharging.

[0106] Regarding the operation constraints of the micro gas turbine: The micro gas turbine generally forms a combined heat and power generation device with a bromine chiller. It uses the high-grade heat energy during natural gas combustion to drive the micro gas turbine to generate electricity, and the discharged high-temperature waste heat flue gas is used for heating and supplying domestic hot water after passing through the bromine chiller. Its operating cost and the mathematical model of the heat-electricity relationship are as follows:

[0107] (10);

[0108] (11);

[0109] In the formula, is the unit price of natural gas; is the power generation efficiency of the micro gas turbine; is the lower heating value of natural gas; is the microgrid at time the heating power of MT in; is the heat dissipation loss; is the flue gas recovery rate of the absorption chiller; is the heating coefficient of the absorption chiller.

[0110] The operating constraints of the micro gas turbine are as follows:

[0111] (12);

[0112] (13);

[0113] In the formula, is the power limit of the micro gas turbine; and are the ramp power limits of the micro gas turbine.

[0114] Regarding the operating constraints of the electric boiler: The electric boiler converts electrical energy into heat energy to meet the heat load, and the operating constraints are:

[0115] (14);

[0116] (15);

[0117] In the formula, is the microgrid at time the electrical power consumed by EB in; is the microgrid at time the heat power converted by EB in; is the electro-thermal conversion efficiency of EB; is the power limit of the electric boiler.

[0118] Regarding the power balance constraints of the microgrid: The power balance constraints of electricity and heat in the microgrid are as follows:

[0119] (16);

[0120] (17);

[0121] (18);

[0122] In the formula, and are the microgrid at time the active power of PV and wind turbines in; is Momentary microgrid The power of the medium - electrical load; Equation (17) indicates that the microgrid can only operate in one of the states of purchasing electricity or selling electricity. and is a 0 - 1 variable, indicating the purchasing and selling electricity state of the momentary microgrid ; is the momentary microgrid the power of the thermal load.

[0123] Step 3: Voltage regulation compensation calculation: After knowing the optimal operation plan of each microgrid, the distribution network obtains the voltage state of the system through power flow calculation as the reference voltage, and by comparing with the actual voltage after considering the voltage regulation incentive, the voltage regulation compensation of the microgrid is obtained. The compensation calculated by the three types of incentives includes the following three parts:

[0124] (19);

[0125] (20);

[0126] (21);

[0127] Therefore, the total voltage regulation compensation obtained by the microgrid at moment is:

[0128] (22).

[0129] Step 4: Consider the optimization strategy of the incentive microgrid for distribution voltage regulation: Each microgrid in the SQDDPG algorithm is represented by an agent. The agent continuously repeats the exploration of the sequential decision - making process to learn the optimal strategy. Usually, multi - agent reinforcement learning needs to be described by the Markov game process (MGP).

[0130] The elements of the agent all include , where: is the state set that the agent can perceive, is the strategy of the agent; is the action taken by the agent according to the strategy in the current state; is the state transition function, that is, the probability of transitioning to the next state under the current state and action; is the reward set that the agent can obtain after executing the action in the environment. For each microgrid, the corresponding relationship between each element and the actual problem is as follows.

[0131] State space: Agent Instead of learning and making decisions for the microgrid, the current environment is observed at each decision moment to obtain the state at this time , including photovoltaic power, wind turbine power, electrical load power, heat load power, energy storage charge and discharge power, micro gas turbine power, electric boiler power, electricity purchase and sale price, and the current decision moment:

[0132] (23);

[0133] In the formula, and are the active power of the photovoltaic and wind turbines in the microgrid at time ; is the electrical load power in the microgrid at time ; is the active power of the load in the microgrid when the voltage of the access node is ; is the charging power of the energy storage system in the microgrid at time , is the discharging power of the energy storage system in the microgrid at time ; is the power of the micro gas turbine in the microgrid at time ; is the power of the electric boiler in the microgrid at time ; and are the electricity purchase and sale price at time ;

[0134] Action space: According to the state and strategy at the current moment, the agent makes a decision and obtains the actual actions of each device after linear mapping , including the charge and discharge power of the energy storage, the change in the power of the micro gas turbine, and the power of the electric boiler.

[0135] (24);

[0136] In the formula, is the charging power of the energy storage system in the microgrid at time , is the discharging power of the energy storage system in the microgrid at time ; is the power change of the micro gas turbine in the microgrid at time ; is the power of the electric heating boiler in the microgrid at time ; and is the reactive power of the PV and wind turbines in the microgrid at time .

[0137] Since the idle reactive power capacity regulation ability of the PV and wind turbine inverters is strong and the regulation cost is low, they are high-quality voltage regulation resources. Therefore, they are part of the decision variables in the strategy considering voltage regulation incentives. The reactive power output constraints and reactive power costs of the PV and wind turbine converters are as follows:

[0138] (25);

[0139] (26);

[0140] In the formula, is the reactive power of the PV and wind turbines in the microgrid at time ; and are the minimum and maximum power factor angles of the inverter; is the utilization rate of the capacity occupied by the reactive power regulation of the converter. The higher the utilization rate under the same occupied capacity, the higher the reactive power regulation cost, generally 5% - 10%; is the capacity of the PV and wind turbine grid-connected converters.

[0141] Reward space: The actual actions of each device can be applied in the simulation environment to obtain the reward at this time , including voltage regulation compensation, the revenue from buying and selling electricity from the superior grid, the charge and discharge costs of energy storage, the fuel cost of the micro gas turbine, and the reactive power regulation cost:

[0142] (27);

[0143] In the formula, is the reward obtained by the agent at time is the voltage regulation compensation obtained by the agent at time and is the electricity price for buying and selling electricity at time and is the power of buying and selling electricity in the microgrid at time is The operating cost of energy storage in the microgrid at moment is the operating cost of energy storage in the microgrid at moment is the operating cost of energy storage in the microgrid at moment

[0144] Since the overall optimization strategy of the agent is to obtain the maximum reward, the revenue of the microgrid is used as the reward instead of the operating cost.

[0145] The technical solution of the present invention sets up a voltage regulation incentive mechanism. The microgrid obtains corresponding compensation by actively participating in voltage regulation. The incentives include three parts: the reduction of voltage regulation cost in the distribution network, the reduction of the risk cost of DG disconnection in the distribution network, and the reduction of the electricity cost of voltage-related loads in the microgrid. Without considering the voltage regulation incentive condition, the overall planned maximum revenue optimization operation plan of the microgrid is calculated. By substituting into the distribution network power flow calculation for verification, the voltage of each node is obtained as the reference value in the incentive mechanism. The voltage regulation compensation of the microgrid can be obtained by comparing the reference voltage and the actual voltage after introducing the voltage regulation incentive. Secondly, the multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to learn the optimization strategy of the microgrid considering the voltage regulation incentive. And to measure the contributions of different microgrids in the voltage regulation process, the Shapley Q value is introduced into the traditional MADDPG algorithm for global reward distribution, which solves the problem of fair credit distribution among microgrids.

[0146] As shown in the Figure 3 appendix, during the update process of the reinforcement learning algorithm, after calculating the Shapley Q value, the parameters of the agent are updated through this value, which is used in the update process of the algorithm to ensure that the agent can obtain higher rewards and improve the optimization performance of the algorithm.

[0147] Based on the voltage regulation incentive mechanism and the Shapley Q-value deep deterministic policy gradient (SQDDPG) algorithm introducing the Shapley Q value, the enthusiasm of the microgrid to participate in the voltage regulation of the distribution network is mobilized, which is beneficial to constructing a more secure and reliable distribution system. The voltage distributions of the distribution network before and after considering the voltage regulation incentive mechanism are as Figure 4 and Figure 5 shown, and the distribution network operates in a safer state.

[0148] Embodiment 2

[0149] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0150] Embodiment III

[0151] The purpose of this embodiment is to provide a computer-readable storage medium.

[0152] A computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are executed.

[0153] Embodiment IV

[0154] The purpose of this embodiment is to provide an optimization system for distribution voltage regulation considering an incentive microgrid, including:

[0155] A data acquisition module, configured to: acquire the load data of the distribution network, the predicted output power data of photovoltaic power generation, the predicted output power data of wind turbines, the capacity of the energy storage system, the capacity of the micro gas turbine, the capacity of the electric boiler, and the time-of-use electricity price predicted for the day ahead;

[0156] Input the acquired data into the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism and the reinforcement learning agent;

[0157] A reference voltage determination module, configured to: the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism calculates the operating voltage of the distribution network when the optimal power purchase and sale electricity of the microgrid is optimized without considering the voltage regulation incentive based on the input data, and this operating voltage is used as the reference voltage;

[0158] An actual operating voltage determination module, configured to: the reinforcement learning agent makes a decision on the current operating state according to the input data, outputs the power of each device, and then obtains the actual operating voltage through the distribution network power flow calculation;

[0159] A voltage regulation compensation module, configured to: calculate the voltage regulation compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives, obtain the voltage regulation compensation, update the agent strategy based on the voltage regulation compensation, and the reinforcement learning agent continuously updates its own strategy to make a decision on the current operating state and output the optimized operating power of each device.

[0160] Embodiment V

[0161] The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any one of the above embodiments.

[0162] The steps involved in the devices of the above embodiments correspond to those of the first method embodiment. For specific implementation manners, reference may be made to the relevant description part of the first embodiment. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to execute any method in the present invention.

[0163] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device for execution by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0164] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, this is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. An optimization method for a distribution voltage regulation considering an incentive microgrid, characterized by comprising: Obtaining the load data of the distribution network, the predicted output data of the photovoltaic power generation, the predicted output data of the wind turbines, the capacity of the energy storage system, the capacity of the micro gas turbine, the capacity of the electric boiler, and the time-of-use electricity price predicted on the previous day; Inputting the obtained data into the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism and the reinforcement learning agent; The optimal operation model of the microgrid before considering the voltage regulation incentive mechanism calculates the operating voltage of the distribution network when the microgrid optimizes the purchase and sale of electric power without considering the voltage regulation incentive based on the input data, and this operating voltage is used as the reference voltage; The reinforcement learning agent makes a decision on the current operating state according to the input data, outputs the power of each device, and then obtains the actual operating voltage through the distribution network power flow calculation; Calculating the voltage regulation compensation according to the reference voltage, the actual operating voltage and various set voltage regulation incentives, obtaining the voltage regulation compensation, updating the agent strategy based on the voltage regulation compensation, and the reinforcement learning agent continuously updates its own strategy to make a decision on the current operating state and outputs the optimized operating power of each device; The calculation of the voltage regulation compensation according to the reference voltage, the actual operating voltage and various set voltage regulation incentives to obtain the voltage regulation compensation specifically includes: The compensation calculated by the three types of incentives includes the following three parts: ； ； ； Among them, is the total number of nodes in the distribution network; represents the voltage regulation cost coefficient of the distribution network for the unit node voltage deviation; is the voltage amplitude of node at time is the rated voltage; represents the DG disconnection risk cost coefficient for the unit node overvoltage; is the upper limit of the distribution network node voltage; and respectively represent the proportions of the constant impedance and the constant current in the total node load; is the electricity selling price at time Therefore, the total voltage regulation compensation obtained by the microgrid at is: 。 2. The optimization method for distribution voltage regulation considering the incentive microgrid according to claim 1, characterized in that, The optimal operation model of the microgrid before considering the voltage regulation incentive mechanism aims to minimize the operating cost of the microgrid, mainly including the equipment operating cost, the fuel cost, and the purchase and sale electricity cost; Considering the operating constraints of the energy storage system, the operating constraints of the micro gas turbine, the operating constraints of the electric boiler, and the power balance constraints of the microgrid.

3. The optimization method for distribution voltage regulation considering the incentive of a microgrid according to claim 1, characterized in that, Various set voltage regulation incentives include: The reduced voltage regulation cost in the distribution network; The reduced DG disconnection risk cost in the distribution network; The reduced electricity cost of the voltage-related loads in the microgrid.

4. The optimization method for distribution voltage regulation considering the incentive of a microgrid as described in claim 1, characterized in that the objective Function: The microgrid is optimized with the goal of minimizing the operating cost, and the objective function is as follows: (28)； In the formula, is the set of optimized periods; is the set of microgrids; is the charge and discharge cost of the energy storage system in the microgrid at time is the operating cost of the micro gas turbine in the microgrid at time and are the power purchase and sale electricity of the microgrid at time and are the electricity price of power purchase and sale at time 5. The optimization method for distribution voltage regulation considering the incentive microgrid as described in claim 1, characterized in that Regarding the reinforcement learning agent, each microgrid is represented by an agent, and the agent continuously repeats the exploration of the sequential decision-making process to learn the optimal strategy.

6. The optimization method for distribution voltage regulation considering an incentive microgrid as claimed in claim 1, characterized in that The elements of the agent all include , where: is the set of states that the agent can perceive, is the policy of the agent; is the action taken by the agent according to the policy in the current state; is the state transition function, that is, the probability of transitioning to the next state under the current state and action; is the set of rewards that the agent can obtain after performing an action in the environment; State space: Agent Instead of the microgrid for learning and decision-making, the current environment is observed at each decision moment to obtain the state at this time , including photovoltaic power, wind turbine power, electrical load power, heat load power, energy storage charge and discharge power, micro gas turbine power, electric boiler power, electricity purchase and sale price, and the current decision moment; Action space: Based on the state and policy at the current moment, the agent makes a decision and obtains the actual actions of each device after linear mapping , including the charging and discharging power of the energy storage, the change in the power of the micro gas turbine, and the power of the electric boiler; Reward space: The reward at a certain moment can be obtained by applying the actual actions of each device in the simulation environment , including voltage regulation compensation, the revenue from buying and selling electricity from the superior power grid, the charge and discharge costs of energy storage, the fuel cost of the micro gas turbine, and the reactive power regulation cost.

7. An optimization system for a distribution voltage regulation considering an incentive microgrid, characterized by comprising: A data acquisition module configured to: obtain the load data of the distribution network, the predicted output data of the photovoltaic power generation, the predicted output data of the wind turbines, the capacity of the energy storage system, the capacity of the micro gas turbine, the capacity of the electric boiler, and the time-of-use electricity price predicted on the previous day; Inputting the obtained data into the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism and the reinforcement learning agent; A reference voltage determination module configured to: the optimal operation model of the microgrid before considering the voltage regulation incentive mechanism calculates the operating voltage of the distribution network when the microgrid optimizes the purchase and sale of electric power without considering the voltage regulation incentive based on the input data, and this operating voltage is used as the reference voltage; An actual operating voltage determination module configured to: the reinforcement learning agent makes a decision on the current operating state according to the input data, outputs the power of each device, and then obtains the actual operating voltage through the distribution network power flow calculation; The voltage regulation compensation module is configured to: calculate voltage regulation compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives, obtain the voltage regulation compensation, update the agent policy based on the voltage regulation compensation, and the reinforcement learning agent continuously updates its own policy to make decisions on the current operating state and output the optimized operating power of each device; Calculating the voltage regulation compensation according to the reference voltage, the actual operating voltage, and various set voltage regulation incentives to obtain the voltage regulation compensation specifically includes: The compensation supplies calculated by the three types of incentives include the following three parts: ； ； ； Among them, is the total number of distribution network nodes; represents the voltage regulation cost coefficient of the distribution network for the unit node voltage deviation; is the voltage amplitude of node at time is the rated voltage; represents the DG disconnection risk cost coefficient for the unit node overvoltage; is the upper limit of the distribution network node voltage; and respectively represent the proportions of the constant impedance and constant current in the total node load; is the electricity selling price at time Therefore, the total voltage regulation compensation obtained by the microgrid at is: 。 8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of the above claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it executes the steps of the method described in any one of the above claims 1-6.

Images