A real-time dispatching method and system for a high-proportion new energy power system under source-storage cooperation

Abstract

The application discloses a kind of source storage synergic under high proportion new energy power system real-time scheduling method and system, belong to power grid operation analysis and dispatching field.The method includes: collection and arrangement source and load operation scene data;Unit day-ahead dispatching plan solution;Markov decision process modeling in real-time scheduling process;Solution of static security domain model;Multi-agent deep deterministic policy gradient model training, debugging and application;To complete source storage synergic under high proportion new energy power system real-time scheduling strategy on-line solution, effectively improve the security and economy of system operation.

Description

TECHNICAL FIELD

[0001] The application relates to a high-proportion new energy power system real-time scheduling method under source-storage cooperation and belongs to the field of power grid operation analysis and scheduling. BACKGROUND

[0002] With the gradual advancement of new-type power system construction, the proportion of new energy output gradually increases, the uncertainty of the power system is further highlighted, and the traditional deterministic scheduling mode based on typical operation mode calculation and online matching of scheduling plans cannot cope with the severe fluctuations of new energy output in a short time, thereby causing power imbalance and serious wind and light curtailment problems, which seriously affect the safe and economic operation of the system. Therefore, in the real-time scheduling stage, the day-ahead scheduling plan needs to be further corrected in combination with the source-load ultra-short-term prediction value to eliminate the system power flow out-of-limit risk as much as possible.

[0003] The real-time scheduling of the power system is essentially an uncertain optimization problem, that is, given the load and new energy prediction value, the output plans of various types of generator units and energy storage and other flexible resources are solved to meet various safety constraints of the system and minimize the system operation cost by considering the uncertainty of the source and load. Such problems have the characteristics of non-convexity, nonlinearity, high dimension, and are difficult to solve. At present, in view of the uncertainty of the source and load of the system, the solving methods mainly include stochastic optimization (SO) considering random variables, robust optimization (RO) considering the range of uncertain variables, and artificial intelligence (AI) based methods.

[0004] The stochastic optimization method describes the uncertainty of the variable by assuming that the random variable obeys a certain probability distribution. The robust optimization algorithm is a method for solving the optimal strategy under the worst disturbance condition by using interval disturbance information. Compared with the stochastic optimization algorithm, the robust optimization does not need to know the specific probability distribution form of the random variable, but only focuses on the fluctuation boundary value of the uncertain variable. It cannot be ignored that the above algorithms need to simplify the nonlinear power flow equation and constraints of the power system, and this process may lose key information of the system and cannot accurately reflect the real operation of the system. Therefore, some scholars try to solve the optimization scheduling model based on heuristic algorithms such as genetic algorithm and particle swarm algorithm. However, such algorithms have the problems of low solving efficiency and easy falling into local optimal solution when solving high-dimensional problems. SUMMARY

[0005] Based on the above analysis, the application proposes a steady-state security region guided multi-agent deep deterministic policy gradient (SSRG-MADDPG) method. In the real-time scheduling stage, source and storage collaborative scheduling is used to cope with the impact of source and load uncertainty on the system, and the safety and economy of model decision-making are improved. Specifically, based on the day-ahead unit output plan, considering the energy storage resource, selecting the corresponding controlled thermal power unit, the online solution of the real-time scheduling strategy of the new energy power system under source and storage collaboration is carried out. Considering the growing scale of the power grid, the order of magnitude of the system state quantity and controllable resources is huge, and the single-agent reinforcement learning faces the problem of dimension disaster. The real-time scheduling problem is modeled as a decentralized partially observable Markov decision process (DPOMDP), and a new energy power system based on SSRG-MADDPG is proposed. Each controlled unit is modeled as an agent, and the corresponding observation space and action space are designed. Considering the system operation target, the cost of controlled units and energy storage scheduling, the model reward function is designed. Further, combined with the day-ahead unit output plan, the source and load ultra-short-term prediction value and the source and load prediction error prior distribution, the next time system operating point is sampled, and the next time system operating state is pre-judged combined with the solved static security region boundary model. In the multi-agent model training stage, the state prediction information is used as the model observation, which provides a reference for model decision-making and solution. In the multi-agent model deployment and application stage, the state prediction information is used as the model action output hard constraint, and unnecessary actions are shielded to reduce the system operation cost and improve the system safety.

[0006] The purpose of the application is to provide a high-proportion new energy power system real-time scheduling method under source and storage collaboration, which comprises:

[0007] Collect and organize source and load operation scenario data and perform preprocessing;

[0008] Combine the source and load day-ahead prediction scenario data to solve the day-ahead scheduling plan of the unit;

[0009] Modeling of Markov decision process in real-time scheduling process;

[0010] Solving the static security region model;

[0011] In combination with the solved static security domain model, the agent observation space, action space and reward function in the Markov decision process modeling, and the solved unit day-ahead scheduling plan, a multi-agent deep deterministic policy gradient model based on static security domain guidance is constructed, the input of the policy gradient model is the system state at each time step under each running scenario: including new energy output, load value, unit day-ahead output plan value, line load rate; the output is the action after observing the state, which is used to guide the output of the controlled unit; the source and load running scenario data is used to train the model, and the trained policy gradient model is used to realize the real-time scheduling of the new energy power system.

[0012] Further, collecting and organizing source and load running scenario data includes: setting an N-node power system, node 0 as the reference node, containing N G thermal power units, N R renewable energy units, N ESS energy storage devices, N L load nodes; a total of N S source and load running scenarios, a single day-ahead real scenario data is represented as S rq ={P r,rq ,P l,rq}, load data The day-ahead scenario time interval is 15 minutes, so there are 96 steps in the time sequence of a single day-ahead scenario; a single day-ahead predicted scenario data is represented as S rq,f ={P r,rq,f ,P l,rq,f}, A single intra-day real scenario data can be represented as S rn ={P r,rn ,P l,rn}, The intra-day scenario time interval is 5 minutes, so there are 288 steps in the time sequence of a single intra-day scenario; a single intra-day predicted scenario data is represented as S rn,f ={P r,rn,f ,P l,rn,f}, The preprocessing includes missing value supplement and outlier removal operation on the source and load running scenario data.

[0013] Further, the unit day-ahead scheduling plan solving includes:

[0014] Set a single day-ahead predicted scenario data scenario S rq,f The corresponding unit day-ahead scheduling plan is denoted as P g,rq , N GThe number of thermal power units is represented, and the system AC power flow equation is approximated to a second-order cone form to minimize the system generation cost. The mathematical model of the day-ahead dispatching plan is as follows:

[0015] (1) Objective function

[0016]

[0017] In the formula, T DA is the step of day-ahead dispatching plan, the day-ahead dispatching plan time interval is 15 minutes, that is, T DA = 96, N G is the number of system generators, P G,i,t is the output value of the i-th generator at time t, a i , b i , c i are cost coefficients of the i-th generator;

[0018] (2) Power balance constraint at node j

[0019]

[0020] In the formula, P G,k,t , Q G,k,t are the active and reactive power outputs of the generator set k at time t, k∈j indicates that the generator set k is connected to the power grid through node j; P b,ij,t and Q b,ij,t are the active and reactive power from node i to node j on branch i→j at time t; P b,je,t and Q b,je,t are the active and reactive power from node j to node e on branch j→e at time t; P L,j,t , Q L,j,t are the active and reactive loads of node j at time t; r ij and x ij are the resistance and reactance of branch i→j, I ij,t is the current module square flowing through branch i→j at time t;

[0021] (3) Voltage amplitude constraint of nodes at both ends of branch i, j

[0022]

[0023] In the formula, U i,t , U j,t are the voltage amplitudes of nodes i, j at time t;

[0024] (4) Power constraint of branch i, j

[0025]

[0026] (5) Line end node voltage phase angle constraint

[0027]

[0028] wherein, and respectively represent the variable considering the voltage amplitude of node i and j as a constant, which is not an optimization variable, and here takes 1; θ i,t , θ j,t are the phase angles of node i and j at time t respectively;

[0029] (6) Generator output constraint

[0030]

[0031] wherein, is the minimum and maximum output value of the i-th generator set, is the day-ahead planning value of the i-th generator set at time t;

[0032] (7) Generator ramp rate constraint

[0033]

[0034] wherein, is the maximum ramp power of the i-th generator set per unit time period;

[0035] (8) Branch transmission power constraint

[0036]

[0037] wherein, is the maximum transmission power of line i, j;

[0038] Therefore, the mathematical model of unit day-ahead scheduling plan is written as:

[0039]

[0040] Finally, the above optimization model is solved based on Gurobi optimization solver, and the unit day-ahead output plan is solved in combination with the source and load day-ahead prediction values in the scenario data.

[0041] Further, the process of Markov decision process modeling is as follows:

[0042] Agent i: Agent i is deployed at the node where the controlled generator set is located or the node where the energy storage device is located, to correct the output plan of the device at the node in combination with the source and load intra-day prediction values in the real-time scheduling stage;

[0043] State s t: The index reflecting the system running state at time t, including the actual output value of thermal power units at time t and the planned output value at time t+1, the actual value of new energy units and each node load at time t and the intraday prediction value at time t+1, the real-time SoC value of energy storage devices and the load rate of each line;

[0044] Observation o i,t : The system state value that each agent can observe, which is set as the state information of the node where the agent is located and the K-order neighbor nodes;

[0045] Action a i,t : The scheduling action that the agent can take; a discrete action space is adopted, and for the controlled thermal power units, the climbing power of each unit per unit time is normalized to [-1, 1] and then divided into N Pr equal parts, -1 indicating that the unit reduces the output and 1 indicating that the unit increases the output For energy storage devices, the maximum charging and discharging power of each unit per unit time is normalized to [-1, 1] and then divided into N Pr equal parts, -1 indicating that the energy storage device charging power is and 1 indicating that the energy storage device discharging power is Let there be n agents, then at time t, the environment receives the joint action of the agents A t =[a 1,t ,a 2,t ,...,a n,t ];

[0046] Reward: After the environment executes the joint action A t of the agents, the reward feedback is calculated by the set reward function and given to the multi-agent system.

[0047] Further, the reward function is designed as follows:

[0048] 1) If the following conditions exist, the scenario is terminated and a penalty r d is given: a) the system power flow does not converge; b) there is 1 and more line load rate ρ>2; c) there is a line load rate ρ>1 for 2 steps and more in succession; d) the balancing machine output is out of limits, r d The calculation formula is as follows:

[0049]

[0050] In the formula, r d,base is the benchmark penalty for the unfinished round, T d is the total step length of the round, T down is the current running step of the system, and r d,minis the minimum penalty for giving up the round, avoiding the penalty given to the later is too small to make the agent tend to inaction;

[0051] 2) If there is no above case, calculate the penalty r of overload line and overload line L,t :

[0052]

[0053] In the formula, ρ l,t is the load rate of line l at time t, respectively, the penalty coefficient for overload line and overload line, define the line load rate exceeds the first threshold, then the line is overload line, the load rate is between the second threshold and the first threshold, then the line is overload line, the system exists line overload or overload, need to give a certain penalty, to guide the agent to make the corresponding decision, optimize the system operation state, N Br is the total number of system lines;

[0054] 3) Give the system survival and operation state reward r B,t :

[0055]

[0056] In the formula, r b is the basic reward of system survival, λ r,b is the reward for system running step t, to guide the system to extend the running step as much as possible, λ l,1 , λ l,2 , λ l,3 is the reward coefficient for reducing the maximum load rate of system line, ρ l,max is the maximum load rate of system;

[0057] 4) Calculate the action cost of controlled generator unit and energy storage device:

[0058]

[0059] In the formula, N ctg is the number of controlled units, N stg is the number of system energy storage devices, is the generator g adjustment cost coefficient, ΔP ctg,g,t is the absolute value of generator g output adjustment, I(·) is the characteristic function, the number of energy storage device action, is the action cost coefficient of energy storage device s;

[0060] 5) Give the system the reward r C,t for running through the whole scene smoothly;

[0061] Therefore, the system reward at time t is:

[0062]

[0063] The cumulative reward value of the agent i at any time k is:

[0064]

[0065] In the formula, γ is a reward discount factor.

[0066] Further, the static security domain of the new energy power system is defined as a set of points satisfying the power flow equation and operation safety constraints, and the expression is formula (16), wherein the new energy unit and the thermal power unit are uniformly modeled as a generator node, and the installed capacity is the upper limit of the maximum output;

[0067]

[0068] In the formula, according to different constraints, different types of static security domains are divided, Ω U is a static voltage security domain meeting the voltage constraint; Ω P is a static generator active power security domain meeting the generator active power constraint; Ω Q is a static generator reactive power security domain meeting the generator reactive power constraint; is a branch thermal stability security domain meeting the system branch transmission power; Ω ss is the entire static security domain, composed of the intersection of Ω U , Ω P , Ω Q , SR is a system node voltage and phase vector; x SR is a node power injection vector; φ(x SR , y SR ) = 0 is an alternating current system power flow equation; N is a system bus node set; N G is a system generator node set; are respectively the upper and lower limits of the voltage of system node i; are respectively the upper and lower limits of the active power output of system generator node i; are respectively the upper and lower limits of the reactive power output of system generator node i; P b,ij , -P b,ij are respectively the forward and reverse transmission power limits of the connection node i and j in the system;

[0069] In the case of reactive power approximation local balance, only the static security domain under active power injection is considered, that is:

[0070]

[0071] In the formula, x SR = [x​P x Q ];x P is the active power injection of each node; x Q is the reactive power injection of each node, C is a constant; P G,i is the active power output of the i-th generator; P L,j is the active power value of the load at the j-th node.

[0072] Further, the overall processing process of the multi-agent deep deterministic policy gradient model based on static security domain guidance is as follows:

[0073] Randomly select an intra-day prediction scenario and a real scenario in the source load operation scenario set, load the actual data at the first time point in the intra-day real scenario and the data at the first time point of the unit day-ahead scheduling plan, perform alternating current power flow calculation, obtain the current state of the system, then provide the state value of the system at time T, the intra-day prediction value of the source load at time T+1, and the judgment result of the system state at time T+1 as observation data to the agent, the agent makes decisions according to the observation data, outputs decision actions, and after the action legality is verified, provides the power system calculation environment for power flow calculation to obtain the system state at time T+1, at this time, the reward value is calculated according to the agent action and the system current state information, thereby serving as feedback information to guide the agent to evolve the strategy.

[0074] Further, the judgment result of the system state at time T+1 is the solved static security domain boundary, that is, combining the unit day-ahead output value at time T+1 and the intra-day prediction value of the source load at time T+1, according to the source load prediction error, the possible intra-day output value of the source load at the next time is extracted by Latin hypercube sampling, and the unit day-ahead output value is combined to form the system operating point set at time T+1, and the result is judged online by the static security domain boundary, and the result is provided as observation information to the agent; specifically, the next time source load ultra-short-term prediction value of the system is P R,f , P L,f , the current controlled unit output value of the system is P g,sk , and the next time output plan value of the uncontrolled unit of the system is P g,fsk ; the source load ultra-short-term prediction error distribution is set to be normally distributed, N err groups of error P R,err , P L,err are constructed, N err groups of source load operation scenario values are constructed at the next time, and the observation x p of the static security domain model is constructed by combining the unit output value. g,sk , P g,fsk , P R,f +P R,err , P L,f +P L,err}, the static security domain model is used to determine the security distribution of the source-load operation scenario at the next moment, and the static security domain model is set to determine N err Group scene N' err For safety, the samples Provided to the agent as observation features at time T;

[0075] when When , it indicates that the system is safe at the next moment and no additional adjustment is required. Otherwise, it reminds the agent to adjust the controlled resources. S is the set threshold.

[0076] Furthermore, the training process of the multi-agent deep deterministic policy gradient model guided by the static safety domain is as follows:

[0077] First, initialize the power grid computing environment, training data set, agent parameters, experience pool, and set the number of training rounds to episode = m;

[0078] Then, within the set number of rounds, a scene is randomly selected each time. At T=0, the initial state x is calculated by combining the new energy output, load value, and unit planned output value. Then, the static safety domain model is used to predict the system state safety information at time T+1, and the observation value o is formed together with x. s ;

[0079] Then, for each agent i, in the scene step, from the global observation o s Constructing self-observation i , then for the observation value o i , select action a i =μ θ,i (o i )+ε,ε~N(0,σ 2 I), μ θ,i (o i ) represents the policy network, ε~N(0,σ 2 I) represents the random perturbation imposed on the policy network; the actions are discretized by combining the Gumbel distribution, and the compliance of the actions is verified. Then, the joint actions of each agent a=[a1,a2,...,a n ], get instant rewards r , the next moment state x′ of the system, and based on the static safety domain model, the next moment information is predicted, and together with x′, it forms the observation value o′ s The round ends with signal d, (x, a, x′, r, d) is stored in the experience pool D, and the system state x←x′ is updated; for the update of each agent, a batch of data B={(x j ,a j ,x′j r j d j x j a j x' j r j d j are the state, action, updated state after performing the action, reward value and episode termination symbol of the jth sample, respectively; and finally data is updated until the training round M is reached or the reward converges.

[0080] The application also provides a high-proportion new energy power system real-time scheduling system under source-storage coordination, comprising:

[0081] a processor and a memory, the memory being used for storing program instructions, and the processor being used for calling the stored instructions in the memory to execute the high-proportion new energy power system real-time scheduling method under source-storage coordination as described in the above technical solution.

[0082] Compared with the prior art, the application has the following beneficial effects:

[0083] Compared with the existing reinforcement learning method, the proposed method integrates the safety domain model into the training and deployment application process of the system, improves the state perception and prediction ability of the agent to the uncertainty of the system. Especially in the deployment decision process, the proposed model improves the system safety and reduces the system operation cost. Compared with the model-driven method, the proposed method improves the system safety and online solving efficiency of the model, and the operation cost is similar BRIEF DESCRIPTION OF DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction will be given below to the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained from the structures shown in these drawings without creative labor.

[0085] Figure 1 It is a flowchart of the method of the present application.

[0086] Figure 2 It is a high-proportion new energy power system real-time scheduling technical framework based on SSRG-MADDPG designed by the present application.

[0087] Figure 3 It is a pseudo code of the SSRG-MADDPG algorithm. DETAILED DESCRIPTION

[0088] The technical solutions of the present application will be further described in detail below in combination with the drawings and embodiments.

[0089] As Figure 1 shown, the application provides a high-proportion new energy power system real-time scheduling method under source-storage coordination, comprising the following steps:

[0090] 1. Collect and organize source-load operation scenario data;

[0091] Set an N-node power system, node 0 as the reference node, containing N G thermal power units, N R renewable energy units, N ESS energy storage devices, and N L load nodes. Collect N S group source-load operation scenarios, and a single day-ahead real scenario data can be represented as S rq ={P r,rq ,P l,rq}, wherein the new energy output data load data The day-ahead scenario time interval is 15 minutes, so there are 96 steps in the time sequence of a single day-ahead scenario. A single day-ahead predicted scenario data can be represented as S rq,f ={P r,rq,f ,P l,rq,f}, A single intra-day real scenario data can be represented as S rn ={P r,rn ,P l,rn}, The intra-day scenario time interval is 5 minutes, so there are 288 steps in the time sequence of a single intra-day scenario. A single intra-day predicted scenario data can be represented as S rn,f ={P r,rn,f ,P l,rn,f}, wherein the subscript r in the above symbols represents new energy, l represents load, rq represents day-ahead, rn represents intra-day, and f represents prediction. Collect and organize the above scenario data and perform data processing such as missing value supplementation and abnormal value elimination.

[0092] 2. Solve the unit day-ahead scheduling plan;

[0093] Combine N S S rq,f day-ahead predicted scenarios to solve the output plan of conventional thermal power units, and the corresponding unit day-ahead scheduling plan of a single scenario S rq,f is denoted as P g,rq , The mathematical model for solving the unit day-ahead scheduling plan is as follows:

[0094] Approximate the system alternating current flow equation to the second-order cone form, without considering the participation of energy storage resources in this stage, and minimize the system power generation cost as the target, and the mathematical model is as follows:

[0095] (1) Objective function

[0096]

[0097] where T DA is the day-ahead dispatch planning step, the day-ahead dispatch planning time interval is taken as 15 minutes, i.e. T DA = 96, N G is the number of system generators, P G,i,t is the output value of the i-th generator at time t, a i , b i , c i are cost coefficients of the i-th generator.

[0098] (2) Power balance constraint at node j

[0099]

[0100] where P G,k,t , Q G,k,t are the active and reactive power output of generator set k at time t, k∈j indicates that the generator set k is connected to the grid through node j; P b,ij,t and Q b,ij,t are the active and reactive power flowing from node i to node j on branch i→j at time t; P b,je,t and Q b,je,t are the active and reactive power flowing from node j to node e on branch j→e at time t; P L,j,t , Q L,j,t are the active and reactive load of node j at time t; r ij and x ij are the resistance and reactance of branch i→j, I ij,t is the current squared flowing through branch i→j at time t.

[0101] (3) Constraint on voltage amplitude of nodes at both ends of line i, j

[0102]

[0103] where U i,t , U j,t are the voltage amplitudes of nodes i, j at time t.

[0104] (4) Power constraint of line i, j

[0105]

[0106] (5) Constraint on phase angle of voltage at the ends of line

[0107]

[0108] where, and denote variables considering the voltage amplitude of nodes i and j as constants, and are not optimization variables, where 1 is taken; θ i,t , θ j,t are the phase angles of nodes i and j at time t.

[0109] (6) Generator output constraint

[0110]

[0111] where, is the minimum and maximum output value of the i-th generator set, is the day-ahead planning value of the i-th generator set at time t.

[0112] (7) Generator ramp rate constraint

[0113]

[0114] where, is the maximum ramp power per unit time period of the i-th generator set.

[0115] (8) Branch transmission power constraint

[0116]

[0117] where, is the maximum transmission power of line i, j.

[0118] Therefore, the mathematical model of the unit day-ahead dispatching plan can be written as:

[0119] min formula (1) (9)

[0120] s.t. formula (2)-(8)

[0121] Finally, the above optimization model is solved based on the Gurobi optimization solver, that is, based on the Gurobi optimization solver, formula (9) is solved combined with the source and load day-ahead prediction values in the scenario data, to realize the solution of the unit day-ahead output plan.

[0122] 3. Markov decision process modeling of real-time scheduling process;

[0123] Multi-Agent Reinforcement Learning (MARL) is used to solve the real-time scheduling problem of a high-proportion renewable energy power system under source-storage collaboration. Each controlled unit and energy storage device is modeled as a single agent, enabling coordinated scheduling of multiple agents within a region. In MARL, each agent follows the basic reinforcement learning paradigm and learns its own strategy based on a decentralized partially observed Markov decision process (DPOMDP). The main components of this process are:

[0124] (1) Agent i: Agent i is deployed at the node where the controlled generator set is located or the node where the energy storage device is located, so as to correct the output plan of the node device in the real-time scheduling stage based on the intraday forecast value of the source and load.

[0125] (2) State s t : Indicators reflecting the system's operating status at time t, including the actual output value of the thermal power unit at time t and the actual output value at time t+1

[0126] The planned output value at the moment, the actual value of the new energy unit and each node load at time t and the intraday forecast value at time t+1, the real-time SoC value of the energy storage equipment and the load rate of each line, etc.

[0127] (3) Observation value o i,t : The system state value that each agent can observe is set as the state information of the node where the agent is located and its K-order neighbor nodes;

[0128] (4) Action a i,t :The scheduling actions that the agent can take; using discrete action space, for the controlled thermal power units, the climbing power of each unit per unit time is Normalized to [-1,1] and then N Pr Equally divided, -1 means the unit reduces output 1 means the unit increases output For energy storage equipment, the maximum charge and discharge power per unit time of each unit is Normalized to [-1,1] and then N Pr Equally divided, -1 means the charging power of the energy storage device is 1 means the discharge power of the energy storage device is Assume there are n agents in total. At time t, the environment receives the joint action A of the agents. t =[a 1,t ,a 2,t ,...,a n,t ].

[0129] (5) Reward: The environment executes the joint action A of the agent t Finally, the reward is calculated through the set reward function and fed back to the multi-agent system.

[0130] The objective of the reward function is to balance the system power distribution, eliminate the risk of system flow out-of-limit, and avoid flow divergence according to the real-time running state of the system and the system intra-day prediction value. Therefore, the reward function is designed as follows for the above objective:

[0131] 1) If the following conditions exist, the scene terminates running and a penalty r is given d : a) the system flow is not convergent; b) there is 1 line and more with load rate ρ>2; c) there is a line with load rate ρ>1 for 2 steps and more in succession; d) the balancing machine output is out of limit. d The calculation formula is as follows:

[0132]

[0133] In the formula, r d,base is the benchmark penalty for the unfinished round, T d is the total step length of the round, and T down is the current running step length of the system. As the system running step length becomes more and more, the penalty for terminating the scene of the system is smaller and smaller. d,min r L,t is the minimum penalty for the unfinished running of the round, which avoids the tendency of the agent to inaction due to the too small penalty given in the later period.

[0134] 2) If there is no above condition, the penalty r L,t for the overloaded line and the overloading line is calculated:

[0135]

[0136] In the formula, ρ l,t is the load rate of the line l at time t, are the penalty coefficients for the overloaded line and the overloading line respectively, the line is defined as the overloading line if the load rate exceeds 1, and the line is defined as the overloaded line if the load rate is between 0.95 and 1, and a certain penalty is given to the system when there is a line overload or overload, so as to guide the agent to make corresponding decisions and optimize the system running state, and N Br is the total number of system lines.

[0137] 3) In order to further avoid the system running risk and optimize the system running index, and at the same time guide the system to survive more steps as much as possible, the survival and running state reward r B,t is given to the system:

[0138]

[0139] In the formula, r b is the basic reward for the survival of the system, and λ r,bThe reward for the system running step t is used to guide the system to extend the running step as much as possible, λ l,1 、λ l,2 、λ l,3 The reward coefficient for reducing the maximum load rate of the system line, ρ l,max The maximum load rate of the system.

[0140] 4) Calculate the action cost of the controlled generator set and the energy storage device:

[0141]

[0142] In the formula, N ctg The number of controlled units, N stg The number of energy storage devices in the system, The generator g adjustment cost coefficient, ΔP ctg,g,t The absolute value of the generator g output adjustment, I(·) is an indicative function, and the number of energy storage device actions is counted, The action cost coefficient of the energy storage device s.

[0143] 5) Give the system a reward r C,t for successfully running through the entire scene.

[0144] Therefore, the reward of the system at time t is:

[0145]

[0146] The cumulative reward value of the intelligent agent i at any time k is:

[0147]

[0148] In the formula, γ is the reward discount factor.

[0149] 4, Solution of static security domain model

[0150] The static security domain of new energy power system is defined as the set of points that meet the power flow equation and operation safety constraints, which can be expressed as formula (16). The new energy units and thermal power units are modeled as generator nodes, and the installed capacity is the upper limit of the maximum output.

[0151]

[0152] In the formula, according to different constraints, different types of static security domains are divided, Ω U The static voltage security domain that meets the voltage constraint; Ω P The static generator active power security domain that meets the generator active power constraint; Ω Q The static generator reactive power security domain that meets the generator reactive power constraint; Ω ss Ω U Ω P Ω Q Ω Ω SR Ω SR Ω SR Ω SR Ω G Ω Ω Ω Ω b,ij Ω b,ij Ω Ω

[0153] Ω Ω

[0154] Ω Ω Ω

[0155] Ω SR Ω P Ω Q Ω P Ω Q Ω G,i Ω L,j Ω Ω

[0156] Ω Ω

[0157] Ω Ω

[0158] 5. Training, debugging and application of multi-agent deep deterministic policy gradient models;

[0159] Aiming at the real-time dispatch problem of high-proportion renewable energy power system, a Steady-state Security Region Guided Multi-agent Deep Deterministic Policy Gradient (SSRG-MADDPG) method based on static security region guidance is proposed. Its technical route is shown in the attached figure. Figure 2 First, randomly select the intraday forecast scenario S from the scenario set rn,f With real scene S rn , at T=1, load S rn The actual data at the first time point in the day-ahead dispatch plan and the data at the first time point in the day-ahead dispatch plan are used to perform AC power flow calculation to obtain the current state of the system. Then, the system state value at time T, the source-load short-term forecast value at time T+1, and the system state judgment result at time T+1 are provided to the intelligent agent as observation data. The intelligent agent makes decisions based on the observation data and outputs decision actions. After the action legitimacy is verified, it is provided to the power system computing environment for power flow calculation to obtain the system state at time T+1. At this time, the reward value is calculated based on the intelligent agent action and the current system state information, which serves as feedback information to guide the intelligent agent to evolve its strategy.

[0160] Attachment Figure 2 In the above equation, the judgment result of the system state at time T+1 is obtained by solving the boundary of the static safety domain of the system. That is, combining the unit's day-ahead output value at time T+1, the source-load ultra-short-term forecast value at time T+1, and considering the source-load forecast error, Latin Hypercube sampling (LHS) is used to extract the possible source-load ultra-short-term output value at the next moment, which together with the unit's day-ahead output value constitutes the system operating point set at time T+1, and is judged online by the SSR. The result is provided to the intelligent agent as observation information. Specifically, let the source-load ultra-short-term forecast value of the system at the next moment be P R,f 、P L,f , the system's current controlled unit output value P g,sk , the planned output value of the system's non-controlled units at the next moment, P g,fsk Assuming that the source-load ultra-short-term forecast error distribution conforms to the normal distribution, according to the LHS sampling N err Group error P R,err 、P L,err , construct the next moment N err The source-load operation scenario value of the group is combined with the output of the unit to construct the observation x of the safety domain model p ={P g,sk ,Pg,fsk ,P R,f +P R,err ,P L,f +P L,err}, the static security domain model SSR is used to judge the security distribution of the source-load operation scenario at the next moment, and the static security domain model SSR is set to judge N err Group scene N' err For safety, the samples Provided to the agent as observation features at time T.

[0161] After the model training is completed, in the deployment and application stage, the judgment results of SSR are used as hard constraints for the output of intelligent action, shielding some unnecessary actions to reduce the action cost of the model and improve the economic efficiency of system operation. When , it indicates that the system is safe at the next moment and no additional adjustment is required. Otherwise, it reminds the agent that it needs to adjust the controlled resources. S is the set threshold.

[0162] The specific algorithm flow of SSRG-MADDPG is as follows Figure 3 shown.

[0163] The input to the multi-agent deep deterministic policy gradient model is the system state at each time step in each operating scenario (each scenario has 288 time steps, 96 on the day before, and will be expanded to 288). This includes renewable energy output, load value, unit output plan value on the day before, line load factor, etc.

[0164] The output is the action after observing the state, that is, how to dispatch the output of the controlled unit;

[0165] First, initialize the grid computing environment, training data set, agent parameters, experience pool, and set the number of training rounds to episode = m. Then, within the set number of rounds, randomly select a scene each time. At T = 0, combine the new energy output, load value, and unit planned output value to calculate the initial state x. Then, combine the SSR model to predict the system state security information at time T + 1, and combine it with x to form the observation value o. s Then, in this scenario step, for each agent i, from the global observation o s Constructing self-observation i , then for the observation value o i , select action a i =μ θ,i (o i )+ε,ε~N(0,σ 2 I), μ θ,i (o i ) represents the policy network, ε~N(0,σ 2I) represents imposing a random disturbance on the strategy network for exploration of actions, and discretizing actions in combination with a Gumbel distribution, and performing action compliance checking, and then performing joint actions a = [a1, a2,..., a n ] of each agent in the power grid computing environment to obtain immediate reward r , the system state x' at the next moment, and based on the SSR pre-judgment of the next moment information, together with x' to form the observation value o' s The round termination signal d. Store (x, a, x', r, d) in the experience pool D, and update the system state x <- x'. For the update of each agent strategy, randomly extract a batch of data B = {(x j , a j , x' j , r j , d j ) from D, x j , a j , x' j , r j , d j are the state, action, updated state after executing the action, reward value and round termination symbol of the jth sample respectively. Calculate Q network is represented. Update the parameters of the evaluator: Update the parameters of the actor: Then, update the parameters of the target network Where, φ i , θ i are the parameters of the online actor and evaluator, φ target,i , θ target,i are the parameters of the target actor and evaluator, and ρ is the weight of the updated parameters. Until the training round number M is reached, or the reward converges.

[0166] After the model training is completed, in order to further verify the effectiveness of the real-time scheduling strategy of the model, the trained multi-agent system is deployed in the power system computing environment to apply the model.

[0167] The effect of the present application is illustrated by a specific example as follows:

[0168] Based on the IEEE-118 node system, a power system computing environment is constructed, and the power flow calculation part is realized by PYPOWER. On the IEEE-118 node system, 9 wind turbines and 6 photovoltaic devices are connected, a total of 15 new energy units. Considering the IEEE118 node topology connection information, 9 thermal power units are selected as controlled generating units based on the day-ahead dispatching plan solution. And 5 energy storage devices are installed in the corresponding new energy unit access nodes to participate in the system real-time scheduling.

[0169] For the constructed system operation scenario dataset, first, the day-ahead unit output plan is arranged based on the proposed QCQP model, and then the training of the SSRG-MADDPG model is carried out in combination with the constructed training scenario set data. The reward function coefficient setting of the agent is shown in Table S1.

[0170] Table S1 Reward function related coefficient setting

[0171]

[0172] After the model training is completed, in order to further verify the effectiveness of the real-time scheduling strategy of the model, the trained multi-agent system is deployed in the power system calculation environment, the performance of the model is verified and analyzed on the test set, and the proposed SSRG-MADDPG algorithm is compared with the day-ahead unit plan, MADDPG and model-driven mixed integer second-order cone optimization model (MISOCP). Among them, the day-ahead unit plan model is that all generators are output according to the day-ahead scheduling plan, the energy storage does not participate in the real-time scheduling link, and the controlled unit does not perform additional adjustment, and the MADDPG model is that its action is not subject to the hard constraint of the safety domain in the model deployment stage. The performance of each model under 50 test scenarios is shown in Table S2.

[0173] In Table S2, when the scene runs 90% of the process, i.e. after 260 steps, it is determined to be a victory. It can be seen that when facing the ultra-short-term uncertainty of new energy, if the real-time scheduling strategy is not formulated, the victory rate is 0 under 50 test scenarios, and the average running step length of the system is only 54.51 steps. In these scenarios, continuous line over-limit often occurs when new energy is large, and in some scenarios, the balancing machine adjustment capacity is insufficient due to the fluctuation of new energy, and the balancing machine output over-limit. Therefore, it is necessary to solve and formulate the real-time scheduling strategy in the intraday stage.

[0174] Performance analysis of each model in test scenarios in Table S2

[0175]

[0176] Table S2 compares and analyzes the performance of SSRG-MADDPG, MADDPG, MISOCP and day-ahead unit plan in training time, single-scene average solving time, average running step length, victory rate, average maximum load rate of line and average running cost.

[0177] As can be seen from Table S2, in terms of solving efficiency, the data-driven model is significantly better than the model-driven model, and the data-driven model can be solved in less than 1 minute on average for a single scenario, while the model-driven model takes more than 2 minutes, and the solving efficiency is improved by 3 times. In terms of average running step and win rate, the MISOCP model is slightly better than the MADDPG model, but under the guidance of the SSR, the SSRG-MADDPG model is better than the comparative model in terms of running step and win rate, and the running cost is greatly reduced compared with the MADDPG model, and is basically the same as the MISOCP model.

[0178] MADDPG model, but under the guidance of the SSR, the SSRG-MADDPG model is better than the comparative model in terms of running step and win rate, and the running cost is greatly reduced compared with the MADDPG model, and is basically the same as the MISOCP model.

[0179] Under the action of the SSR, the model avoids a large number of unnecessary actions of the agent, on the one hand, reduces the running cost, and on the other hand, reserves sufficient flexible adjustable resources for the system, so that the agent can fully coordinate and dispatch the energy storage and controllable units when the system is in a real "crisis" moment.

[0180] Therefore, the SSRG-MADDPG model significantly improves the safety and economy of the system.

[0181] On the other hand, the embodiment of the present application also provides a high-proportion new energy power system real-time scheduling system under source-storage coordination, comprising:

[0182] A processor and a memory, the memory is used for storing program instructions, and the processor is used for calling the stored instructions in the memory to execute the high-proportion new energy power system real-time scheduling method under source-storage coordination as described in the above technical solution.

[0183] The above is only a preferred embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the scope disclosed by the present application, and it still belongs to the protection scope of the present application.

Claims

1. A real-time scheduling method for a high-proportion new energy power system under source-storage coordination, characterized in that, Comprise: Collect and organize source load operation scenario data and perform preprocessing; Combine source load day-ahead prediction scenario data to solve unit day-ahead scheduling plan; Model Markov decision process for real-time scheduling process; Solve static security domain model; Combine the solved static security domain model, the agent observation space, action space and reward function in Markov decision process modeling, and the solved unit day-ahead scheduling plan to construct a multi-agent deep deterministic policy gradient model based on static security domain guidance. The input of the policy gradient model is the system state at each time step under each operation scenario: including new energy output, load value, unit day-ahead output plan value, line load rate; The output is the action after observing the state, which is used to guide the output of the controlled unit; Train the model using source load operation scenario data, and use the trained policy gradient model to realize real-time scheduling of new energy power system; The overall processing process of the multi-agent deep deterministic policy gradient model based on static security domain guidance is as follows: Randomly select the intra-day prediction scenarios and real scenarios from the source load operation scenario set, and T When =1, load the actual data of the first time point in the intra-day real scenario and the data of the first time point in the unit day-ahead scheduling plan, perform AC power flow calculation to obtain the current state of the system, and then provide the system T time state value, T intra-day prediction value of the +1 time point source and load, T the +1 time point system state judgment result together as observation data to the agent, the agent makes decisions according to the observation data, outputs decision actions, and after the action legality verification, provides the power system calculation environment for power flow calculation to obtain T the system state at the +1 time point, at this time, according to the agent action and the system current state information, the reward value is calculated, thereby as feedback information to guide the agent to evolve the strategy. T The judgment result of the system state at time t+1 is the solved static security region boundary, i.e. combining T The unit day-ahead output value at time t+1, T The source load day-ahead prediction value at time t+1, according to the source load prediction error, combining the Latin hypercube sampling to extract the possible source load day-ahead output value at the next time, and the unit day-ahead output value together constitute T The system operating point set at time t+1, and the online judgment is performed by the static security region boundary, and the result is provided as observation information to the intelligent agent. 2.The real-time scheduling method of a high-proportion new energy power system under source-storage coordination according to claim 1, wherein Collecting and organizing source-load operation scenario data includes: setting a power system of nodes, node 0 is a reference node, containing thermal power units, renewable energy units, energy storage devices, load nodes; Collecting groups of source-load operation scenarios, a single day-ahead real scenario data is represented as , wherein the new energy output data , the load data , the day-ahead scenario time interval is 15 minutes, so there are 96 steps on the time sequence of a single day-ahead scenario; a single day-ahead prediction scenario data is represented as , , a single intra-day real scenario data can be represented as , , the intra-day scenario time interval is 5 minutes, so there are 288 steps on the time sequence of a single intra-day scenario; a single intra-day prediction scenario data is represented as , ; preprocessing includes missing value supplementing and outlier removing operations on source-load operation scenario data. 3.The real-time dispatching method of a high-proportion new energy power system under source-storage coordination according to claim 1, characterized in that: Solving unit day-ahead scheduling plan includes: Setting a single day-ahead forecast scenario data scene The corresponding unit day-ahead dispatching plan is denoted as , , where N is the number of thermal power units, the system AC power flow equation is approximated to a second-order cone form, and the mathematical model of the unit day-ahead dispatching plan is as follows: (1) Objective function (1) Where, is the day-ahead scheduling step size, and the day-ahead scheduling time interval is 15 minutes, that is, , is the number of generators in the system, It is i A generator in t The output value at the moment, 、 、 It is i Generator cost coefficient; (2) Node j at power balance constraints (2) wherein , are the active and reactive power output of the generator unit at time k , are the active and reactive power output of the generator unit k connected to the grid via node j ; and are the active and reactive power flowing from node t to node i on branch j → i at time j ; and are the active and reactive power flowing from node t to node j on branch e → j at time e ; , are the active and reactive load of node at time ; and are the resistance and reactance of branch , is the current squared flowing on branch at time ; (3) Line i , j Two end node voltage magnitude constraints (3) In the formula, , are the voltage amplitudes of the nodes t , i , j at the time (4) Line i , j power constraints (4) (5) Line end node voltage phase angle constraint (5) wherein, and denote variables that are constant with respect to the voltage amplitude of the nodes i and j , respectively, and are not optimization variables, here taken as 1 ; , are the phase angles of the nodes t at time i and j , respectively. (6) Generator output constraint (6) Where, 、 For the i The minimum and maximum output values ​​of each generator set, This is for the i Generator sets in The day-ahead planned value at the time; (7) Generator ramp rate constraint (7) Where, It is i The maximum ramp power per unit time period of each generator set; (8) Branch transmission power constraint (8) wherein is the maximum transmission power of the line i , j . Therefore, the mathematical model of unit day-ahead scheduling plan is written as: (9) Finally, based on the Gurobi optimizer, the above mathematical model is solved, and the source load day-ahead prediction value in the scenario data is used to solve the unit day-ahead output plan. 4.The real-time scheduling method of a high-proportion new energy power system under source-storage coordination according to claim 1, characterized in that: The process of Markov decision process modeling is as follows: Agent i : Agent i Deployed at the node where the controlled generator set is located or the node where the energy storage device is located to correct the output plan of the device at the node in combination with the source-load daily prediction value in the real-time scheduling stage; State : Reflecting system t The indicators of the moment operation state, including the thermal power unit t The actual output value of the moment and t+ The planned output value of the moment, new energy unit, and each node load t The actual value of the moment and t+ The intraday prediction value of the moment, real-time SoC value of the energy storage device, and each line load rate Observation : The system state value that each agent can observe, set as the state information of the node where the agent is located and the state information of the k-order neighbor nodes. K ​ action :The scheduling actions that the agent can take; using discrete action space, for the controlled thermal power units, the climbing power of each unit per unit time is Normalized to [ 1,1] and then equal parts, 1 means the unit reduces output , 1 means the unit increases output ; For energy storage equipment, the maximum charge and discharge power of each unit per unit time Normalized to [ 1,1] and then equal parts, 1 means the charging power of the energy storage device is , 1 means the discharge power of the energy storage device is ; Assume a total n intelligent agent, then t At this moment, the environment receives the joint action of the agent ; Reward: Environment performs joint action of agents After that, the reward feedback is calculated by the set reward function to the multi-agent system.

5. The real-time dispatching method of a high-proportion new energy power system under source-storage coordination according to claim 4, characterized in that: The reward function is designed as follows: 1) Consider the scenario terminates running, giving a penalty if the following conditions exist : a) System power flow does not converge; b) There is 1 or more line loadings ; c) There is a line with 2 or more consecutive loadings ; d) Out of balance machine output, The calculation formula is as follows: (10) wherein, is the unfinished benchmark penalty, is the total step of the episode, is the step that the system is currently running, is the minimum penalty given to the unfinished running of the episode, avoiding the unfinished penalty given to the later period too small leading to the agent tends to inaction; 2) If none of the above, then calculate penalties for overloaded lines and overloading lines : (11) In the formula, is the line l At time t The load rate, , The penalty coefficient for heavy load line and overload line respectively, define the line load rate exceeds the first threshold value, then the line is overload line, the load rate is between the second threshold value and the first threshold value, then the line is heavy load line, the system exists line overload or heavy load, need to give certain punishment, to guide the agent to make corresponding decision, optimize the system running state, is the total number of system lines; 3) Give the system survival and operational state rewards : (12) wherein is a system survival base reward, is a reward for system operation step t , , , is a reward coefficient for reducing the system line maximum load rate, is a system maximum load rate; 4) Calculate the action cost of the controlled generator unit and the energy storage device: (13) In the formula, is the number of controlled units, is the number of system energy storage devices, is the number of generators g is the adjustment cost coefficient, is the number of generators g is the absolute value of the output adjustment amount, is the indicator function, and the number of actions of the energy storage device is counted, is the energy storage device s is the action cost coefficient; 5) Reward the system for running smoothly through the entire scenario ; Thus, the system t The instant reward is: (14) Then the agent i At any k The cumulative reward value at any time is: (15) In the formula, is a reward discount factor.

6. The real-time dispatching method of a high-proportion new energy power system under source-storage coordination according to claim 1, characterized in that: The static security domain of new energy power system is defined as the set of points that satisfy the power flow equation and operation safety constraints, expressed as formula (16), wherein the new energy unit and the thermal power unit are modeled as generator nodes, and the maximum output is limited by the installed capacity; (16) In which, according to different constraints, different types of static security domains are divided, Static voltage security domain for meeting voltage constraints; Static generator active power security domain for meeting generator active power constraints; Static generator reactive power security domain for meeting generator reactive power constraints; Branch thermal stability security domain for meeting system branch transmission power constraints; The entire static security domain is composed of the intersection of , , , ; The system node voltage and phase vector; The node power injection vector; The AC system power flow equation; The system bus node set; The system generator node set; , The system node i voltage upper and lower limits; , The system generator node i active power upper and lower limits; , The system generator node i reactive power upper and lower limits; , The forward and reverse transmission power limits of the connection nodes i and j in the system; In the case of reactive power approximate local balance, only consider the static security domain under active power injection, that is: (17) Where, ; Inject active power into each node; Reactive power injection for each node, is a constant; For the i Active output of generators; For the j Active load value of each node.

7. The real-time scheduling method of a high-proportion new energy power system under source-storage coordination according to claim 1, characterized in that: T The judgment result of the system state at time t+1 is the solved static security region boundary, which is specifically, assuming that the source load ultra-short-term prediction value of the system at time t+1 is , the system current controlled unit output value is , and the system non-controlled unit output plan value at time t+1 is ; The source-load ultra-short-term prediction error distribution is set to conform to a normal distribution, and LHS sampling is used Group error , the next time Group source-load operation scene value is constructed, and the observation of the static security domain model is constructed in combination with the unit output , the safety distribution of the next time source-load operation scene is judged by the static security domain model, and the static security domain model is set In the group scene The sample is safe, and It is provided as an observation to the intelligent agent as T Observation features at the moment When is true, it indicates that the system is safe at the next time, without additional adjustment, otherwise, it reminds the agent to adjust the controlled resources, is the set threshold value. 8.The real-time dispatching method of a high-proportion new energy power system under source-storage coordination according to claim 1, wherein: The training process of the multi-agent deep deterministic policy gradient model based on static security domain guidance is as follows: First, initialize the power grid calculation environment, training data set, agent parameters, experience pool, and set the number of training rounds episode=m; Then, in the set number of rounds, a scene is randomly selected each time, and at T=0, the initial state is calculated in combination with the new energy output, load value, and unit planned output value , and then the system state safety information at T+1 is predicted in combination with the static security domain model to jointly constitute the observation value ; Then, for each agent i, in the scene step, from the global observation Constructing own observation , then for the observed value , select an action , represents the policy network, Indicates applying random perturbations to the policy network; and discretizes the actions in combination with the Gumbel distribution, performs action compliance verification, and then executes the joint actions of each agent in the power grid computing environment. , get instant rewards , the state of the system at the next moment , and predict the next moment information based on the static security domain model, and Together they form observations End of round signal ,Will Deposit into experience pool , and update the system status ; For the update of each agent, from Randomly extract a batch of data from , They are the state, action, updated state after executing the action, reward value and round terminator of the j-th sample respectively; finally, the data is updated until the number of training rounds M is reached or the reward converges.

9. A high-proportion new energy power system real-time dispatching system under source-storage cooperation, characterized in that, Comprise: A processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a real-time scheduling method for a high-proportion new energy power system under source storage cooperation as claimed in any one of claims 1-8.

Images