Two-stage multi-equipment participation power distribution system deduction simulation control strategy and device

Abstract

The invention discloses a two-stage multi-equipment participation power distribution system deduction simulation control strategy and device, and relates to the technical field of new energy photovoltaic power generation and power distribution system optimization operation. The strategy comprises the following steps of initializing a power flow model and collecting information, loading key data, building a power flow equation, regulating and controlling OLTC discrete actions in a first stage, optimizing by adopting a DQN algorithm, judging discrete control ending, and regulating reactive power of a photovoltaic inverter in a second stage. Modeling a PV agent, and adopting a multi-agent depth deterministic strategy gradient algorithm for cooperative control and overvoltage node judgment and ending; according to the invention, coordination and advantage complementation of various adjustable devices can be realized while the problems of aggravation of fluctuation risk and voltage out-of-limit of the power distribution system caused by distributed photovoltaic grid connection are solved, and the utilization efficiency of distributed energy is improved while safe and stable operation of the power distribution system is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of new energy photovoltaic power generation and distribution system optimization operation, and more specifically, to a two-stage multi-device participation distribution system simulation control strategy and device. Background Art

[0002] With the development of distributed new energy application technology, distributed photovoltaic power generation technology has gradually become an important form of energy supply in the power system. However, the strong volatility and randomness of photovoltaic power generation have also brought many challenges to the safe and stable operation of the distribution system. Especially in the context of large-scale distributed photovoltaic access to the distribution system, it is easy to cause frequent system voltage fluctuations and even voltage exceeding the limit, which is not conducive to the stable operation of the distribution system.

[0003] At present, the main controllable devices in the distribution system include discrete regulation devices such as on load tap changer (OLTC), shunt capacitor bank (SCB) and continuous regulation devices such as photovoltaic inverter and static var generator (SVG). Among them, OLTC mainly maintains the system voltage by adjusting the tap position, while SCB, photovoltaic inverter and SVG can assist in voltage control by adjusting the reactive power compensation amount. However, the OLTC action has discrete characteristics, while the control of photovoltaic inverter and SVG is continuous action. The differences in action characteristics and control objectives further increase the complexity of coordinated control of equipment. Therefore, it is necessary to study how to achieve coordinated optimization among multiple devices, comprehensively consider the action constraints of the equipment and the system status, so as to effectively solve the voltage limit problem in the distribution system. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention proposes a two-stage multi-device participation distribution system simulation control strategy and device, which comprehensively considers the action differences of discrete devices OLTC and continuous devices photovoltaic inverter in the distribution system, and optimizes the tap position of OLTC and the output of photovoltaic inverter, so as to achieve the purpose of coordinated control.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a two-stage multi-device participation distribution system simulation control strategy and device, the improvement of which is that it includes:

[0006] Collect key data on distribution system topology information, OLTC tap position, photovoltaic power generation forecast, and power load demand;

[0007] Establish a power distribution system flow model based on the collected topology information and key data

[0008] Perform discrete action regulation on the OLTC, and determine whether the discrete action regulation effect of the OLTC reaches the predetermined regulation target. If the predetermined regulation target is not reached, then use the photovoltaic inverter in the distribution system power flow model to perform continuous action regulation;

[0009] Check whether there are overvoltage nodes in the distribution system. If there are, continue to use the reactive power regulation ability of the continuous photovoltaic inverter to regulate the distribution system until there are no overvoltage nodes in the distribution system.

[0010] Furthermore, the specific steps for performing discrete action regulation on the OLTC are as follows:

[0011] Describe the tap optimal gear problem of the on-load tap-changer transformer in the distribution system as a Markov decision process, define the state and state space of the distribution system, and design a reward function to minimize the tap action frequency, variation range, and node voltage deviation;

[0012] Use the DQN algorithm in deep reinforcement learning to optimize the tap gear of the OLTC, and gradually find the tap gear setting that maximizes the reward function through trial-and-error learning;

[0013] The specific steps for regulating the distribution system using the continuous photovoltaic inverter are as follows:

[0014] Use the reactive power regulation ability of the photovoltaic inverter to adjust the voltage of the distribution system, and determine the regulation range and direction of its reactive power according to the actual operating conditions of the photovoltaic inverter and system requirements;

[0015] According to the multi-agent deep deterministic policy gradient algorithm, model multiple photovoltaic inverters as independent agents respectively, and through autonomous decision-making and collaborative optimization strategies, achieve collaborative control between devices, solve the problem of voltage over-limit, and maintain the voltage stability of the distribution system;

[0016] Furthermore, the specific steps for establishing the distribution system power flow model are as follows:

[0017] Determine all the nodes in the distribution system and form a node set N = {0, 1,..., N};

[0018] Identify all the feeders in the distribution system. Each feeder l ∈ L is formed by connecting two nodes i and j, forming an ordered real number pair (i, j), and form a feeder set L = {0, 1,..., L};

[0019] Based on the node set N and the feeder set L, construct a graph model D = (N, L) of the distribution system, where N is the vertex set of the graph and L is the edge set of the graph;

[0020] In the node set N, node 0 is connected to the main network and serves as a balancing node. Node 0 is located on the high-voltage side of the substation, and the substation is connected to the distribution system through this node. The substation is equipped with a on-load tap-changer to regulate the voltage of the distribution system.

[0021] Based on the principle of power conservation, for each unbalanced node in the distribution system, active power and reactive power balance equations are established.

[0022] Integrate the node power balance equation, the feeder power flow equation, and the regulation effect of the OLTC together to form a distribution system power flow model.

[0023] Furthermore, the specific steps for collecting key data are as follows:

[0024] In a distribution system containing feeders, first determine the feeder connected to the on-load tap-changing transformer, and connect the tap of the on-load tap-changing transformer to the relevant line l of the feeder.

[0025] Set the number of tap adjustment positions of the OLTC to 17. Each position can be adjusted by 1.25%. The voltage regulation range is 0.9 p.u. - 1.1 p.u. The upper limit of the number of operations of the OLTC per day is set to 6 times, and the upper limit of the cumulative number of position changes is 8 positions.

[0026] According to the regulation range and the change amount per change, calculate the specific change range of the tap position. Specify the maximum change amount of the voltage during each regulation, and list the possible values of the tap position change amount between two moments.

[0027] Furthermore, the specific steps for describing the problem of the optimal tap position of the on-load tap-changing transformer in the distribution system as a Markov decision process, defining the state and state space of the distribution system, and designing a reward function to minimize the tap operation frequency, change range, and node voltage deviation are as follows:

[0028] Adjust the node voltage amplitude of the distribution system by setting appropriate OLTC tap positions.

[0029] The state s of the distribution system is the combination of the current tap situation T tap and the voltage set V, which does not include the voltage of the balancing node. The state space S of the distribution system is the set of all possible states s.

[0030] The action space is all optional on-load tap-changing transformer position changes. The action space is discrete. Select and execute appropriate actions to determine the new tap position and handle the high-dimensional action space to control the multi-OLTC system.

[0031] Record the frequency of OLTC tap position changes, calculate the action frequency by defining an indicator function, define a function to calculate the change range of the OLTC tap position, give a lower reward for large changes and a higher reward for small changes;

[0032] Use the norm to calculate the deviation degree of the node voltage from the reference voltage, and integrate the three objectives of minimizing the tap action frequency, change range and node voltage deviation through weighting, which is used to evaluate and optimize the OLTC action. By comprehensively considering the tap adjustment cost and voltage deviation penalty cost, a multi-objective optimization and effective voltage control strategy is realized;

[0033] Maximize the cumulative reward value from the current moment to the end within a given time period. The OLTC agent adjusts the strategy to maximize the objective function to achieve the global optimization of the distribution system.

[0034] Furthermore, the specific steps for optimizing the OLTC tap position of the on-load tap-changer transformer using the DQN algorithm are as follows:

[0035] Build a DRL framework and design an OLTC agent. The OLTC agent contains two neural networks, one of which is a DQN network and the other is a target network, and make it interact with the distribution system environment;

[0036] At each time step, the distribution system feeds back a new system state according to the tap position action of the OLTC agent; in the new state, the agent calculates the Q values of all possible actions in the state through the DQN network, and selects the action with the largest Q value with a certain probability according to the ε-greedy strategy for voltage control;

[0037] The DQN network approximates the optimal action-value function, inputs the state of the current distribution system, outputs the Q values corresponding to the adjustment actions of each OLTC tap position, calculates the loss function through continuously collected training samples, and updates the network parameters using the gradient descent method to gradually approximate the optimal Q value function and improve the optimality of the strategy;

[0038] Use the target network to reduce the bias propagation during the training process and improve stability. The target network is updated with the DQN network parameters after a certain time interval.

[0039] Furthermore, the specific steps for adjusting the distribution system voltage by utilizing the reactive power regulation ability of the photovoltaic inverter and determining the regulation range of its reactive power according to the actual operation situation of the photovoltaic inverter and the system requirements are as follows:

[0040] Each photovoltaic inverter is modeled as an independent PV agent, enabling it to make autonomous decisions based on local state information, including the voltage, active power, and reactive power at the connection point, which are obtained through the perception of the PV agent, ensuring that each PV agent can effectively interact with the distribution system environment;

[0041] Define the state s for the PV agent, including the active power, reactive power of the photovoltaic inverter and the load, the voltage magnitude and phase angle of the node; at the same time, based on the reactive power regulation ability of the photovoltaic inverter, define a continuous action set for each PV agent, representing its adjustable reactive power range;

[0042] Use the multi-agent deep deterministic policy gradient algorithm to update and iterate the policy of the PV agent, facilitating information sharing and policy optimization among agents.

[0043] Furthermore, the PV agent is defined as the state s:

[0044] s = [P PV , Q PV , P l , Q l , V, θ];

[0045] Among them, P PV and Q PV are the sets of active power and reactive power generated by the photovoltaic inverter, P l and Q l are the sets of active power and reactive power of the load, V is the set of voltage magnitudes of the nodes, and θ is the set of voltage phase angles;

[0046] In the distribution system environment, each PV agent is equipped with a continuous action set The continuous action represents the reactive power Q PV generated by the photovoltaic inverter with reactive power regulation ability, and the relationship between its adjustable reactive power capacity and the inverter capacity is:

[0047]

[0048] Among them, is the maximum reactive power output of the photovoltaic inverter, P PV is the active power output of the photovoltaic power generation, S PV is the capacity of the photovoltaic inverter; Q PV is the adjustable reactive power of the photovoltaic inverter, β ∈ [-1, 1]. When β > 0, the photovoltaic inverter absorbs reactive power from the distribution network; when β < 0, the photovoltaic inverter injects reactive power into the distribution network;

[0049] The strategy of the PV agent is updated and iterated by setting a reward function, and the reward function is as follows:

[0050]

[0051] where ΔV i is the voltage offset of node i, and P loss is the line loss. The purpose is to control the voltage within a safe range and minimize the line loss in the distribution network as much as possible; C1 and C2 are the weight coefficients of the voltage offset penalty cost and the distribution system line loss cost respectively, which are used to measure the importance between the two.

[0052] Furthermore, the set target function in step S4016 is the cumulative sum of all reward values from the current moment to the end within a given time period T; the OLTC agent finally finds an optimal strategy by maximizing the target function, and the target function is as follows:

[0053]

[0054] where α1, α2, and α3 are the weight coefficients of the tap action frequency, the tap action amplitude, and the node voltage deviation respectively.

[0055] Furthermore, the constraint conditions that need to be satisfied for the multi-objective optimization problem include:

[0056] Power flow constraint:

[0057]

[0058] where are the active power and reactive power of the unit connected at node i respectively, are the active power and reactive power consumed by the load connected at node i respectively, G ij and B ij are the conductance and susceptance between feeders (i, j) respectively, and θ ij is the voltage phase angle difference between nodes i and j;

[0059] Voltage constraint:

[0060] The node voltage of the distribution system needs to be maintained within a certain safe range:

[0061] V min ≤V i ≤V max ;

[0062] where V min and V max are the lower limit and upper limit of the allowable node voltage amplitude respectively;

[0063] Current Constraint:

[0064] During the power flow calculation of the distribution system, the current flowing through the feeder (i, j) cannot exceed the maximum current carrying capacity of the feeder:

[0065] 0≤I ij ≤I max ;

[0066] Among them, I max Related to ambient temperature and insulation materials, etc.;

[0067] OLTC discrete action constraints:

[0068] The tap position constraint of the discrete regulation device OLTC is as follows:

[0069]

[0070]

[0071] in, are the lower and upper limits of the tap position allowed by the OLTC connected to feeder l, It is the maximum cumulative tap position change allowed within the selected time period T.

[0072] The present invention also proposes a two-stage multi-device-participated power distribution system simulation control device, which is characterized by comprising:

[0073] Data acquisition module, used to collect key data on distribution system topology information, OLTC tap position, photovoltaic power generation prediction value and power load demand;

[0074] Model building module, used to establish the power flow model of the distribution system based on the collected data, including node power balance equations, feeder power flow equations and the regulation function of OLTC;

[0075] OLTC control module, used to optimize the OLTC tap position through discrete control mode, determine whether the predetermined target is reached, and continue to adjust if the target is not reached;

[0076] Photovoltaic inverter control module, used to control the power distribution system based on the reactive power regulation capability of the photovoltaic inverter, further reducing voltage deviation and line loss;

[0077] The voltage monitoring module is used to detect the voltage status in the power distribution system and determine whether there is an overvoltage node. If so, it returns to the photovoltaic inverter control module for control;

[0078] An optimization control module, which is used to optimize the control strategy through a deep reinforcement learning algorithm and a multi-agent deep deterministic policy gradient algorithm to achieve coordinated control of multiple devices;

[0079] In the above structure, the data acquisition module includes:

[0080] A distribution system topology information acquisition unit, which is used to acquire the node set and feeder set of the distribution system;

[0081] An OLTC tap position information acquisition unit, which is used to obtain the tap adjustment range, number of tap positions and action frequency of the OLTC;

[0082] A photovoltaic power generation power acquisition unit, which is used to predict and obtain the photovoltaic power generation power in real time;

[0083] An electricity load demand acquisition unit, which is used to obtain the electricity demand data of the distribution system.

[0084] The OLTC regulation module includes:

[0085] A state space definition unit, which is used to define the state and state space of the distribution system based on the Markov decision process;

[0086] A reward function design unit, which is used to minimize the change frequency, amplitude of the OLTC tap position and the node voltage deviation;

[0087] A DQN algorithm unit, which is used to optimize the OLTC tap position through deep reinforcement learning and gradually find the tap position setting that maximizes the reward function;

[0088] The photovoltaic inverter regulation module includes:

[0089] An agent modeling unit, which is used to model the photovoltaic inverter as an independent PV agent;

[0090] A state definition unit, which is used to define the state of the PV agent, including active power, reactive power, node voltage amplitude and phase angle;

[0091] An action set definition unit, which is used to define the continuous action set of the PV agent and determine the reactive power regulation range of the photovoltaic inverter;

[0092] A collaborative optimization unit, which is used to update and optimize the control strategy of the photovoltaic inverter by using the multi-agent deep deterministic policy gradient algorithm. Description of the Drawings

[0093] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0094] Figure 1 is the two-stage multi-device coordinated control framework of the present invention;

[0095] Figure 2 is the program flowchart of a two-stage multi-device participation in the distribution system deduction simulation control strategy of the present invention. Specific embodiments

[0096] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will detail the technical solutions of the present invention in conjunction with the drawings and specific embodiments. It should be noted that the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0097] The two-stage multi-device coordinated control framework of the present invention is as Figure 1 shown. Aiming at the voltage over-limit problem caused by the strong volatility and randomness of photovoltaic power generation in the distribution system, considering the action differences and constraint conditions of various controllable devices, in the first stage, taking the discrete device OLTC as the research target, minimizing the change of the OLTC tap position and the node voltage offset, and obtaining the preliminary optimized control result under the conditions of meeting the power flow balance, voltage and current limits, and OLTC tap position limits. In the second stage, taking the continuous device photovoltaic inverter as the research target, coordinating the output of multiple photovoltaic inverters in the system to further reduce the voltage deviation and line loss, so as to ensure the safe and stable operation of the distribution system and improve the utilization efficiency of distributed energy.

[0098] As Figure 2 shown, the two-stage multi-device participation in the distribution system deduction simulation control strategy of the present invention includes the following steps:

[0099] S10: Collect the topological information of the distribution system;

[0100] S20: Load the key data of the OLTC tap position, the predicted photovoltaic power generation, and the electricity demand into the distribution system;

[0101] S30: Based on the distribution system power flow model, establish a power flow equation including the influence of OLTC;

[0102] S40: Discretely control the OLTC in the first stage;

[0103] S50: Determine whether the discrete control of the OLTC reaches the predetermined goal. If it reaches the goal, enter the next stage; otherwise, continue with step S40;

[0104] S60: In the second stage, regulate the distribution system by utilizing the reactive power regulation ability of the continuous PV inverter;

[0105] S70: Check whether there are overvoltage nodes in the distribution system. If there are, return to step S60; if not, end.

[0106] Furthermore, the specific steps of step S40 are as follows:

[0107] S401: Describe the problem of the optimal tap position of the on-load tap-changer in the distribution system as a Markov decision process, define the state and state space of the distribution system, and design a reward function to minimize the tap action frequency, variation range, and node voltage deviation;

[0108] S402: Use the DQN algorithm in deep reinforcement learning to optimize the tap position of the OLTC, and gradually find the tap position setting that maximizes the reward function through trial-and-error learning;

[0109] The specific steps of step S60 are as follows:

[0110] S601: Adjust the voltage of the distribution system by utilizing the reactive power regulation ability of the PV inverter, and determine the regulation range and direction of its reactive power according to the actual operating conditions of the PV inverter and system requirements;

[0111] S602: According to the multi-agent deep deterministic policy gradient algorithm, model multiple PV inverters as independent agents respectively, and achieve collaborative control between devices through autonomous decision-making and collaborative optimization strategies to solve the problem of voltage over-limit and maintain the voltage stability of the distribution system.

[0112] In the present invention, by constructing a linearized power flow model and collecting detailed distribution system topology information, the modeling of the distribution system becomes more accurate, thereby improving the accuracy of simulation and control. Key data such as the tap position of the OLTC, the predicted value of photovoltaic power generation, and the load power demand are introduced, providing a solid data foundation for system regulation and facilitating accurate load forecasting and equipment scheduling. The optimal control is divided into two stages. In the first stage, the voltage is initially stabilized through the regulation of the OLTC. In the second stage, the reactive power regulation ability of the photovoltaic inverter is used to make finer adjustments to the voltage, improving the flexibility and effectiveness of control. The DQN algorithm and the multi-agent deep deterministic policy gradient algorithm in deep reinforcement learning are utilized. These algorithms can automatically optimize the control strategy through learning and experience accumulation, thereby maximizing the system's performance metrics. Through the design of the reward function, the frequent operation of the OLTC tap is reduced, extending the equipment life and reducing the maintenance cost. The photovoltaic inverter is modeled as an independent agent, enabling it to make autonomous decisions based on local information and achieving optimal coordination between devices through a collaborative algorithm, enhancing the overall synergy of the system. Through the multi-agent collaborative optimization strategy, the voltage over-limit problem in the distribution system is effectively addressed, maintaining the stability of the system voltage and reducing the risk of related faults. When an over-voltage node is detected, the strategy allows the system to return for further adjustment, thereby ensuring that the final grid state reaches the goals of stability and optimization. This method can adapt to different load and generation conditions in the power grid, has strong robustness, and can efficiently handle various dynamic changes in the power grid. This strategy helps to manage the voltage of the distribution system more efficiently and economically, supports the reliable access of distributed energy, and improves the overall operation efficiency of the system and the reliability of power supply.

[0113] Furthermore, distribution system topology information is collected, including distribution system feeder information, node information, OLTC device information in the substation, and DPV information, etc.

[0114] This model is a balanced radial system. The set of nodes in the distribution system is N = {0, 1,..., N}, and the set of feeders in the distribution lines is L = {0, 1,..., L}. Each feeder l ∈ L is connected by two nodes i and j, and these two nodes can form an ordered real pair (i, j) ∈ N × N. Therefore, the distribution system can be modeled as D = {N, L}.

[0115] Node 0 is connected to the main grid. As a balancing node, it can balance the active power and reactive power in the distribution system and can be regarded as an ideal voltage source. Node 0 is located on the high-voltage side of the substation and is connected to the distribution system through the substation at the other end. The substation can regulate the distribution system through the OLTC.

[0116] In the first stage of distribution system simulation control, the OLTC discrete action control mode is established:

[0117] In a distribution system with feeder L, the feeder connected to OLTC is L tap ={1,2,...,L tap For the discrete device OLTC that needs to be regulated, the tap is connected to the line l related to (i, j). The tap position can take 17 discrete values. The voltage regulation range of OLTC is 0.9pu-1.1pu. Each time the position changes, the change of 0.0125pu will be regulated. Therefore, the adjustment range of the tap is They correspond to 17 discrete gear values. It is stipulated that each adjustment can change at most 0.2pu, and the possible value of the tap adjustment change between two moments is:

[0118]

[0119] In order to ensure the rationality of the control results, in the actual control process, the tap adjustment change It is not a fixed value, but depends on the current tap position. This means that when the power distribution system is in different operating states, the tap position The adjustment range should be consistent with the current system load conditions, operating voltage and other parameters. Especially in the case of high power load or high distributed energy penetration, the tap position needs to be adjusted carefully to avoid excessive voltage fluctuations.

[0120] Based on the power flow model of the distribution system, consider the feeder l∈L tap Due to the influence of OLTC, the voltage relationship between nodes i and j at both ends of feeder l changes, and the power flow equation is established:

[0121]

[0122] Among them, V i and V j are the voltage amplitudes at nodes i and j, respectively, and R ij and X ij are the resistance and reactance of the feeder (i, j), P ij and Q ij They are respectively the active power and reactive power flowing through the feeder (i, j). When the power flows from node i to node j, the power value is positive, otherwise it is negative.

[0123] The optimal position problem of OLTC tap in the distribution system is described as a Markov decision process (MDP):

[0124] Adjust the node voltage amplitude of the distribution system by setting appropriate OLTC tap positions, and define the state s of the distribution system according to the obtained tap positions and voltage amplitude information of all nodes:

[0125] s = {T tap , V};

[0126] Among them, is the tap position situation of all connected OLTCs, and the voltage set V = {V1, V2,..., V N} does not include the voltage of the slack node. The state of the distribution system at each moment forms the state space S, and at any moment k, the relationship between the state and the state space is satisfied:

[0127]

[0128] Adjust the node voltage of the distribution system by setting appropriate OLTC tap positions:

[0129] In the distribution system, the action a is determined by the OLTC tap position , and the two satisfy the following relationship:

[0130]

[0131] This action is discrete because the OLTC controls the voltage by adjusting the tap position step by step, which may be to adjust the tap position upward, downward, or remain unchanged. The action space A of the distribution system is the set of all optional tap position actions of the OLTC. If the system selects an action a k at any moment k according to the current state s k , and the OLTC receives the instruction and executes the action, then the current tap position after executing the action a k , the new tap position at the k + 1 moment is:

[0132]

[0133] Since the tap position of the OLTC can only be adjusted between certain specific discrete values, the action space A is finite. However, if there are multiple OLTCs in the distribution system, the dimension of the action space A will increase exponentially with the increase in the number of OLTCs, that is, the "curse of dimensionality". At this time, an effective control algorithm is needed to handle the high-dimensional action space and solve the optimal action

[0134] Record the action frequency of the OLTC tap position:

[0135] When the distribution system executes the action from the current state s = {T tap , V} and transfer to the next state s′ = {T tap′ , V′}, it is necessary to evaluate the quality of this action by setting a reward function. The reward function aims to minimize the tap action frequency and the tap action amplitude, where the tap action frequency refers to the number of times the tap position changes, that is if it is not 0, it is recorded as a change in frequency. Therefore, an indicator function can be defined

[0136]

[0137] Based on the indicator function δ, a function is defined to calculate the change in the OLTC tap position:

[0138] The indicator function can be used to determine whether the tap position changes at the current time k and the next time k + 1. If they are different, it means a change has occurred. Based on the indicator function, a function is defined to calculate the number of changes in all OLTC tap positions at time k:

[0139]

[0140] The tap action amplitude refers to the magnitude of the change in the tap position. When the position change is large, a lower reward value is given, and when the position change is small, a higher reward value is given. Therefore, a function can be defined to calculate the number of changed tap positions of all OLTC tap positions at time k:

[0141]

[0142] Regarding the voltage fluctuation situation of the distribution system, a function is defined to measure the node voltage deviation:

[0143] Considering the voltage fluctuation situation of the distribution system, the L2 norm is used to measure the node voltage deviation, and a function h(V k+1 ) is defined as follows:

[0144]

[0145] where represents the reference voltage, generally the per-unit value 1.0 p.u., ||·|| represents the Euclidean distance, which can be calculated by the following formula:

[0146]

[0147] A reward function is defined to evaluate the quality of the OLTC tap action:

[0148] Based on the above-defined function, with the goal of minimizing the tap action frequency and tap action amplitude, and combining the node voltage deviation to achieve voltage regulation, the reward function is designed as the following formula:

[0149]

[0150] where α1, α2, and α3 are the weight coefficients of the tap action frequency, tap action amplitude, and node voltage deviation respectively, which are used to measure the importance among the three. The weight coefficient values can be adjusted according to different requirements to balance the attention degree of the distribution system to the tap adjustment times, adjustment amplitude, and voltage regulation accuracy. The first two terms in this formula represent the tap position adjustment cost of the OLTC, and the latter term represents the penalty cost caused by the node voltage deviating from the reference voltage. The reward function realizes a multi-objective optimization by comprehensively considering the tap adjustment cost and voltage penalty cost, which is used to measure the quality of taking a certain action in the current state, so as to learn a strategy that can effectively regulate the voltage, reduce frequent regulation, and maintain smooth regulation.

[0151] Maximize the cumulative reward value within a given time period T:

[0152] The objective function is the cumulative sum of all reward values from the current moment to the end within a given time period T; the OLTC agent finally finds an optimal strategy by maximizing the objective function, so as to achieve the global optimization of the distribution system. The objective function is as follows:

[0153]

[0154] Define the constraint conditions that need to be satisfied to solve the multi-objective optimization problem:

[0155] (1) Power flow constraint:

[0156]

[0157] where are the active power and reactive power of the generating unit connected at node i respectively, are the active power and reactive power consumed by the load connected at node i respectively, G ij and B ij are the conductance and susceptance between feeders (i, j) respectively, and θ ij is the voltage phase angle difference between nodes i and j.

[0158] (2) Voltage constraint:

[0159] The node voltage of the distribution system needs to be maintained within a certain safe range:

[0160] V min ≤V i ≤Vmax ;

[0161] Among them, V min and V max are respectively the lower limit value and the upper limit value of the allowable node voltage amplitude.

[0162] (3) Current constraint:

[0163] During the power flow calculation of the distribution system, the current flowing through the feeder (i, j) cannot exceed the maximum current-carrying capacity of the feeder:

[0164] 0 ≤ I ij ≤ I max ;

[0165] Among them, I max is related to the ambient temperature and insulation materials, etc.

[0166] (4) OLTC discrete action constraint:

[0167] The tap position constraint of the discrete regulating device OLTC is as follows:

[0168]

[0169]

[0170] Among them, are respectively the lower limit value and the upper limit value of the allowable tap positions of the OLTC connected to the feeder l, is the maximum allowable cumulative tap position change amount within the selected time period T.

[0171] Furthermore, design an OLTC agent to interact with the distribution system environment:

[0172] In the DRL framework, the distribution system constitutes the required environment, and the state in the environment includes the OLTC tap position information in the distribution system and the power flow information such as node voltage, load, and active and reactive powers of photovoltaic.

[0173] Design an OLTC agent to interact with the environment. The agent observes the state s of the environment and makes an action a. At this time, the environment inputs the state s and the action a into the state transition function p(s′|s, a) to obtain the probability values of all possible states. The environment makes a random sampling according to the probability values to obtain a new state s′, and feedbacks a reward value r to the OLTC agent.

[0174] Set the optimal tap position adjustment for voltage control:

[0175] The distribution system environment will make corresponding changes according to the tap position action of the OLTC agent at each moment and feedback the new distribution system state. The OLTC agent fits the optimal action-value function Q through the DQN network. * (s,a)(action-value function), estimates the Q value of each possible action under the new distribution system state, and the agent selects the appropriate tap position according to the action with the largest Q value under a certain probability to regulate the voltage of the distribution system. The expression of the action-value function is as follows:

[0176] Qπ(s k ,a k )=E[R k +γR k+1 +γ 2 R k+2 +…|s k ,a k ];

[0177] Among them, γ∈[0,1] is the discount rate, which is a hyperparameter that needs to be manually adjusted, indicating that the future reward value is discounted. The action-value function is to find the expectation of the accumulated discounted reward. The future state and action are random. k+1 ,s k+2 ,...) and other states and (a k+1 ,a k+2 ,...) and other actions are eliminated by integration, so that the action-value function Qπ(s k ,a k ) only with the current state s k 、Action a k And it is related to the strategy function π. In order to reduce the impact of multiple strategies on the action-value function, only the current state and action are evaluated, and the optimal action-value function can be found:

[0178]

[0179] During the interaction process, the OLTC agent continuously collects training samples (s k ,a k ,r k ,s k+1 ), where r k The rewards given for the three indicators of tap action frequency, tap action amplitude and node voltage deviation are used to update the parameters of the neural network through these samples, gradually approaching the optimal Q value function; after obtaining enough training samples, the OLTC agent finally learns to obtain the best tap position The optimal strategy π *, so as to make the best tap gear adjustment under different distribution system conditions and regulate the voltage of the distribution system to a certain extent.

[0180] The OLTC agent contains two neural networks, one is the DQN network and the other is the target network. The DQN network is denoted as Q(s,a;ω) and is used to approximate the optimal action-value function Q * (s,a), input the current distribution system state s, and output the Q value corresponding to the OLTC tap adjustment action a of different gears, where ω is the parameter of the network. The OLTC agent selects the tap gear action according to the ε-greedy strategy, which is specifically manifested as the agent selecting a random action for exploration with a certain probability ε, or selecting the action with the largest current Q value with probability 1-ε:

[0181]

[0182] When training the DQN network, the training samples (s k ,a k ,r k ,s k+1 ) to calculate the loss function and use the gradient descent method to update the parameters ω of the DQN network:

[0183]

[0184] Among them, α is the learning rate, which is a hyperparameter that needs to be manually adjusted.

[0185] In order to avoid the propagation of bias caused by "bootstrapping" during training and thus alleviate the overestimation problem of DQN, another neural network called the target network is used to calculate the target value of the Q value and maintain a certain stability during training.

[0186] The target network is denoted as Q(s,a;ω′), which has the same structure as the DQN network, but the network parameter ω′ is updated less frequently. It is not updated every time training. The DQN network parameter ω is used to update the target network only after a certain time interval:

[0187] ω′←τω+(1-τ)ω′;

[0188] In the second stage of the distribution system simulation control, based on the results of node voltage regulation by discrete device OLTC in the first stage, the reactive power regulation of the continuous device photovoltaic inverter is carried out on this basis;

[0189] Multiple DPVs in the distribution system are modeled as PV agents. The agents interact with the distribution system environment and make autonomous decisions based on local information measured at the connection points, including active power, reactive power, and node voltage information obtained through power flow calculations, and are defined as state s:

[0190] s = [P PV , Q PV , P l , Q l , V, θ];

[0191] Among them, P PV and Q PV are the sets of active power and reactive power output by the PV inverter, P l and Q l are the sets of active power and reactive power of the load, V is the set of voltage amplitudes of the nodes, and θ is the set of voltage phase angles.

[0192] Based on the reactive power regulation ability of the PV inverter, define the action set of the PV agent:

[0193] In the distribution system environment, each PV agent is equipped with a continuous action set The continuous action represents the reactive power Q PV generated by the PV inverter with reactive power regulation ability. The relationship between its adjustable reactive power capacity and the inverter capacity is:

[0194]

[0195] Among them, is the maximum reactive power output of the PV inverter, P PV is the active power output of the PV generation, S PV is the capacity of the PV inverter. Q PV is the adjustable reactive power of the PV inverter, β ∈ [-1, 1]. When β > 0, the PV inverter absorbs reactive power from the distribution network; when β < 0, the PV inverter injects reactive power into the distribution network.

[0196] Furthermore, update and iterate the strategy of the PV agent:

[0197] Aiming at the phenomenon of voltage over-limit caused by distributed PV accessing the distribution system, the goal of reactive power optimization in the distribution system is to reduce its voltage deviation and line losses by adjusting the reactive power distribution in the distribution system according to the node voltage and the actual operating conditions of the inverter equipment. Define the reward function of this process:

[0198]

[0199] Among them, ΔV iis the voltage offset of node i, P loss is the line loss, the purpose of which is to control the voltage within a safe range while minimizing the line loss in the distribution network; C1 and C2 are the weight coefficients of the voltage deviation penalty cost and the distribution system line loss cost, respectively, which are used to measure the importance between the two.

[0200] The present invention also includes a two-stage multi-device-participated power distribution system simulation control device, characterized in that it includes:

[0201] Data acquisition module, used to collect key data on distribution system topology information, OLTC tap position, photovoltaic power generation prediction value and power load demand;

[0202] Model building module, used to establish the power flow model of the distribution system based on the collected data, including node power balance equations, feeder power flow equations and the regulation function of OLTC;

[0203] OLTC control module, used to optimize the OLTC tap position through discrete control mode, determine whether the predetermined target is reached, and continue to adjust if the target is not reached;

[0204] Photovoltaic inverter control module, used to control the power distribution system based on the reactive power regulation capability of the photovoltaic inverter, further reducing voltage deviation and line loss;

[0205] The voltage monitoring module is used to detect the voltage status in the power distribution system and determine whether there is an overvoltage node. If so, it returns to the photovoltaic inverter control module for control;

[0206] The optimization control module is used to optimize the control strategy through deep reinforcement learning algorithm and multi-agent deep deterministic policy gradient algorithm to achieve multi-device collaborative control.

[0207] Furthermore, the data acquisition module includes:

[0208] A power distribution system topology information collection unit, used to collect node sets and feeder sets of the power distribution system;

[0209] OLTC gear information acquisition unit, used to obtain the tap adjustment range, gear number and action frequency of OLTC;

[0210] Photovoltaic power generation acquisition unit, used to predict and obtain photovoltaic power generation in real time;

[0211] The power load demand collection unit is used to obtain the power demand data of the power distribution system.

[0212] The OLTC control module includes:

[0213] A state space definition unit, used for defining the state and state space of the distribution system based on a Markov decision process;

[0214] Reward function design unit, used to minimize the OLTC tap position change frequency, amplitude and node voltage deviation;

[0215] The DQN algorithm unit is used to optimize the OLTC tap position through deep reinforcement learning, and gradually find the tap position setting that maximizes the reward function.

[0216] Furthermore, the photovoltaic inverter control module includes:

[0217] An agent modeling unit for modeling a photovoltaic inverter as an independent PV agent;

[0218] The state definition unit is used to define the state of the PV agent, including active power, reactive power, node voltage amplitude and phase angle;

[0219] An action set definition unit is used to define the continuous action set of the PV agent and determine the reactive power regulation range of the PV inverter;

[0220] A collaborative optimization unit is used to update and optimize the control strategy of the photovoltaic inverter using a multi-agent deep deterministic policy gradient algorithm.

[0221] The present invention proposes a two-stage multi-device participation distribution system simulation control strategy and device. Aiming at the voltage over-limit problem caused by the strong volatility and randomness of photovoltaic power generation in the distribution system, the action differences and constraints of various controllable devices are considered. In the first stage, the discrete device OLTC is taken as the research target, and the OLTC tap gear change and node voltage offset are minimized. Under the conditions of satisfying the power flow balance, voltage and current limits and OLTC tap gear restrictions, the preliminary optimization control results are obtained. Based on the optimization configuration results of the first stage, the continuous device photovoltaic inverter is taken as the research target in the second stage, and the output of multiple photovoltaic inverters in the system is coordinated and controlled to further reduce voltage deviation and line loss. This method can achieve the coordination of various adjustable devices while solving the problems of increased fluctuation risk and voltage over-limit of the distribution system caused by distributed photovoltaic grid connection, and improve the utilization efficiency of distributed energy while ensuring the safe and stable operation of the distribution system.

[0222] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. Two-stage multi-device distribution system simulation control strategy, characterized by: Including: Collect distribution system topology information 、 Key data such as the tap position of the OLTC, the predicted power generation of the PV, and the electricity load demand; Based on the collected topology information and key data, establish a power flow model for the distribution system; Perform discrete action control on the regulating device OLTC in the power flow model of the distribution system, and determine whether the discrete action control effect of the OLTC reaches the predetermined control target. If the predetermined control target is not reached, then use the photovoltaic inverter in the power flow model of the distribution system to perform continuous action control; Check whether there are overvoltage nodes in the distribution system. If so, continue to use the reactive power regulation ability of the continuous photovoltaic inverter to regulate the distribution system until there are no overvoltage nodes in the distribution system.

2. The two-stage multi-device distribution system simulation control strategy according to claim 1 is characterized in that: The specific steps for performing discrete action control on the OLTC are as follows: Describe the tap optimal position problem of the on-load tap-changer transformer in the distribution system as a Markov decision process, define the state and state space of the distribution system, and design a reward function to minimize the tap action frequency, variation range, and node voltage deviation; Use the DQN algorithm in deep reinforcement learning to optimize the tap position of the OLTC, and gradually find the tap position setting that maximizes the reward function through trial-and-error learning; The specific steps for regulating the distribution system using the continuous photovoltaic inverter are as follows: Use the reactive power regulation ability of the photovoltaic inverter to adjust the voltage of the distribution system, and determine the regulation range and direction of its reactive power according to the actual operating conditions of the photovoltaic inverter and system requirements; According to the multi-agent deep deterministic policy gradient algorithm, model multiple photovoltaic inverters as independent agents respectively, and through autonomous decision-making and collaborative optimization strategies, achieve collaborative control between devices, solve the problem of voltage over-limit, and maintain the voltage stability of the distribution system; The specific steps for establishing the power flow model of the distribution system are as follows: Determine all nodes in the distribution system and form a node set N = {0, 1,..., N}; Identify all feeders in the distribution system. Each feeder l ∈ L is formed by connecting two nodes i and j, forming an ordered real number pair (i, j). These feeders are formed into a feeder set L = {0, 1,..., L}; Based on the node set N and the feeder set L, construct a graph model D = (N, L) of the distribution system, where N is the vertex set of the graph and L is the edge set of the graph; In the node set N, node 0 is connected to the main grid and serves as a balancing node. Node 0 is located on the high-voltage side of the substation. The substation is connected to the distribution system through this node. The substation is equipped with an on-load tap-changer to regulate the voltage of the distribution system; Based on the principle of power conservation, establish active power and reactive power balance equations for each unbalanced node in the distribution system; Integrate the node power balance equation, the feeder power flow equation, and the regulation effect of the OLTC together to form a power flow model of the distribution system; The specific steps for collecting key data are as follows: In the distribution system containing feeders, first determine the feeder connected to the on-load tap-changer transformer, and connect the tap of the on-load tap-changer transformer to the line l related to the feeder; Set the number of tap adjustment positions of the OLTC to 17, with each position adjustable by 1.25%. The voltage regulation range is 0.9 p.u. - 1.1 p.u., the upper limit of the number of daily operations of the OLTC is set to 6 times, and the upper limit of the cumulative tap change is 8 positions; Based on the regulation range and the change amount per adjustment, calculate the specific change range of the tap positions. Specify the maximum voltage change amount during each regulation, and list the possible values of the tap position change amount between two moments.

3. The two-stage multi-device participation power distribution system deduction simulation control strategy according to claim 2, wherein, Describe the problem of the optimal tap position of the on-load tap-changer in the distribution system as a Markov decision process, define the state and state space of the distribution system, and design a reward function to minimize the tap operation frequency, change range, and node voltage deviation. The specific steps are as follows: Adjust the node voltage amplitude of the distribution system by setting appropriate OLTC tap positions; The state s of the distribution system is the current tap position T tap in combination with the voltage set V, which does not include the voltage of the slack node. The state space S of the distribution system is the set of all possible states s; The action space is all optional on-load tap-changer position changes. The action space is discrete. Select and execute appropriate actions to determine the new tap position, and handle the high-dimensional action space to control the multi-OLTC system; Record the frequency of OLTC tap position changes. Calculate the action frequency by defining an indicator function, define a function to calculate the change range of the OLTC tap positions, and give a lower reward for large changes and a higher reward for small changes; Use the norm to calculate the deviation degree between the node voltage and the reference voltage. Integrate the three objectives of minimizing the tap operation frequency, change range, and node voltage deviation through weighting, which is used to evaluate and optimize the OLTC operation. By comprehensively considering the tap adjustment cost and the voltage deviation penalty cost, a multi-objective optimization and effective voltage regulation strategy are achieved; Maximize the cumulative reward value from the current moment to the end within a given time period. The OLTC agent adjusts the strategy to maximize the objective function to achieve the global optimization of the distribution system.

4. The two-stage multi-device distribution system simulation control strategy according to claim 2 is characterized in that: The specific steps for optimizing the tap position of the on-load tap-changer OLTC using the DQN algorithm are as follows: Establish a DRL framework and design an OLTC agent. The OLTC agent contains two neural networks, one of which is a DQN network and the other is a target network, and enable the OLTC agent to interact with the distribution system environment; At each time step, the distribution system feeds back a new system state according to the tap position action of the OLTC agent; in the new state, the agent calculates the Q values of all possible actions in the state through the DQN network, and selects the action with the maximum Q value with a certain probability according to the ε-greedy strategy for voltage regulation; The DQN network approximates the optimal action-value function, inputs the current state of the distribution system, and outputs the Q values corresponding to the adjustment actions of each OLTC tap position. Calculate the loss function through continuously collected training samples, and use the gradient descent method to update the network parameters to gradually approximate the optimal Q value function and improve the optimality of the strategy; Use the target network to reduce the bias propagation during the training process and improve stability. The target network is updated with the DQN network parameters at a certain time interval.

5. The two-stage multi-device distribution system simulation control strategy according to claim 2 is characterized in that: The specific steps of utilizing the reactive power regulation capability of the photovoltaic inverter to adjust the voltage of the power distribution system and determining the reactive power regulation range according to the actual operation of the photovoltaic inverter and system requirements are as follows: Model each PV inverter as an independent PV agent, enabling it to make autonomous decisions based on local state information, including voltage, active power, and reactive power at the access point. This state information is obtained through the perception of the PV agent, ensuring that each PV agent can effectively interact with the distribution system environment. Define the state s for the PV agent, including the active power, reactive power, voltage amplitude and phase angle of the node of the PV inverter and load; at the same time, based on the reactive power regulation capability of the PV inverter, define a continuous set of actions for each PV agent to represent its adjustable reactive power range; The multi-agent deep deterministic policy gradient algorithm is used to update and iterate the strategy of the PV agent, facilitating information sharing and strategy optimization among the agents.

6. The two-stage multi-device participation power distribution system deduction simulation control strategy according to claim 7, characterized in that, The PV agent is defined as state s: s = [P PV , Q PV , P l , Q l , V, θ]; where P PV and Q PV are the sets of active power and reactive power generated by the photovoltaic inverter, P l and Q l are the sets of active power and reactive power of the load, V is the set of voltage amplitudes of the nodes, and θ is the set of voltage phase angles; In the distribution system environment, each PV agent is equipped with a continuous set of actions The continuous action represents the reactive power Q generated by a PV inverter with reactive power regulation capabilities PV , and the relationship between its adjustable reactive power capacity and the inverter capacity is as follows: Among them, is the maximum reactive power output of the PV inverter, and P PV is the active power output of the PV power generation, and S PV is the capacity of the PV inverter; Q PV is the adjustable reactive power of the PV inverter, β ∈ [-1, 1]. When β > 0, the PV inverter absorbs reactive power from the distribution network; when β < 0, the PV inverter injects reactive power into the distribution network. The strategy of the PV agent is updated and iterated by setting a reward function, which is: Among them, ΔV i is the voltage offset of node i, and P loss is the line loss. The purpose is to control the voltage within a safe range while minimizing the line loss in the distribution network as much as possible; C1 and C2 are the weight coefficients of the voltage offset penalty cost and the distribution system line loss cost respectively, which are used to measure the importance between the two.

7. The two-stage multi-device participation power distribution system deduction simulation control strategy according to claim 2, characterized in that, The objective function is set to be the cumulative sum of all reward values from the current moment to the end within a given time period T; the OLTC agent ultimately finds an optimal strategy by maximizing the objective function, which is as follows: Among them, α1, α2 and α3 are the weight coefficients of the tap action frequency, tap action amplitude and node voltage deviation respectively.

8. The two-stage multi-device distribution system simulation control strategy according to claim 5 is characterized in that: The constraints that need to be met in multi-objective optimization problems include: Power flow constraints: Among them, P i Gen and are the active power and reactive power of the unit connected at node i, respectively. P i Load and are the active power and reactive power consumed by the load connected at node i, respectively. G ij and B ij are the conductance and susceptance between the feeder (i, j), respectively. θ ij is the voltage phase angle difference between nodes i and j; Voltage Constraints: The node voltage of the power distribution system needs to be maintained within a certain safety range: V min ≤V i ≤V max ; where V min and V max are the lower and upper limit values of the allowable node voltage magnitudes, respectively; Current Constraint: During the power flow calculation of the distribution system, the current flowing through the feeder (i, j) cannot exceed the maximum current carrying capacity of the feeder: 0 ≤ I ij ≤ I max ; Among them, I max is related to the ambient temperature, insulating material, etc.; OLTC discrete action constraints: The tap position constraint of the discrete regulation device OLTC is as follows: wherein, and are the lower limit value and the upper limit value of the tap positions allowed by the OLTC of the access feeder l, respectively, is the maximum allowable cumulative tap position change amount within the selected time period T.

9. Two-stage multi-device participation distribution system deduction simulation control device, characterized in that, include: Data acquisition module, used to collect key data on distribution system topology information, OLTC tap position, photovoltaic power generation prediction value and power load demand; Model building module, used to establish the power flow model of the distribution system based on the collected data, including node power balance equations, feeder power flow equations and the regulation function of OLTC; OLTC control module, used to optimize the OLTC tap position through discrete control mode, determine whether the predetermined target is reached, and continue to adjust if the target is not reached; Photovoltaic inverter control module, used to control the power distribution system based on the reactive power regulation capability of the photovoltaic inverter, further reducing voltage deviation and line loss; The voltage monitoring module is used to detect the voltage status in the power distribution system and determine whether there is an overvoltage node. If so, it returns to the photovoltaic inverter control module for control; The optimization control module is used to optimize the control strategy through deep reinforcement learning algorithm and multi-agent deep deterministic policy gradient algorithm to achieve multi-device collaborative control.

10. The two-stage multi-device distribution system simulation control device according to claim 9, characterized in that: The data acquisition module comprises: The distribution system topology information acquisition unit is used to acquire the node set and feeder set of the distribution system; The OLTC tap position information acquisition unit is used to obtain the tap adjustment range, the number of tap positions and the operation frequency of the OLTC; The photovoltaic power generation power acquisition unit is used to predict and obtain the photovoltaic power generation power in real time; The electricity load demand acquisition unit is used to obtain the electricity demand data of the distribution system; The OLTC regulation module includes: The state space definition unit is used to define the state and state space of the distribution system based on the Markov decision process; The reward function design unit is used to minimize the tap position change frequency, amplitude and node voltage deviation of the OLTC; The DQN algorithm unit is used to optimize the OLTC tap position through deep reinforcement learning and gradually find the tap position setting that maximizes the reward function; The photovoltaic inverter regulation module includes: The agent modeling unit is used to model the photovoltaic inverter as an independent PV agent; The state definition unit is used to define the state of the PV agent, including active power, reactive power, node voltage amplitude and phase angle; The action set definition unit is used to define the continuous action set of the PV agent and determine the reactive power regulation range of the photovoltaic inverter; The collaborative optimization unit is used to update and optimize the control strategy of the photovoltaic inverter by using the multi-agent deep deterministic policy gradient algorithm.

Images