Intraday scheduling method of water, wind and solar power based on improved flexible strategy-value algorithm

Abstract

The present invention relates to the field of power grid operation control technology, and specifically to a method for intraday dispatching of water, wind and solar power based on an improved flexible strategy-value algorithm, the method comprising establishing a water, wind and solar power operation environment based on the multidimensional uncertainty of water, wind and solar power; establishing a safe reinforcement learning decision model considering the constraints of the water, wind and solar power system; and performing intraday dispatching based on an improved flexible strategy-evaluation algorithm based on a conditional variational autoencoder. The method can fully consider the multidimensional uncertainty of water, wind and solar power, ensure that the formulated dispatching strategy meets the constraints of the water, wind and solar power system, and ensure the safe and stable operation of the system; improve the flexible strategy-value algorithm based on the conditional variational autoencoder, use low-dimensional latent variables to effectively capture the characteristics of multidimensional complex variables, and improve the convergence speed and convergence accuracy of the algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid operation control, and in particular relates to a water-wind-solar daytime dispatching method based on an improved flexible strategy-value algorithm. Background Art

[0002] With the continuous increase in wind and solar penetration, the spatiotemporal decision variables of power system dispatch are becoming more complex and high-dimensional. The traditional power system dispatch mode is difficult to meet the real-time control needs of the power system under the influence of multiple uncertainties and coupling factors such as water inflow, wind and solar power generation. How to effectively extract the characteristics of high-dimensional complex decision variables and efficiently perform intraday dispatch while meeting the complex safety constraints of the system is a huge challenge facing the water-wind-solar complementary system.

[0003] The safe reinforcement learning method provides a new way to solve the problem of timely and rapid regulation of power systems. For the constraints of reinforcement learning, most of the existing research is to add penalty terms to the reward function. This method converges slowly and requires repeated manual debugging of the penalty coefficient to ensure the training effect. The present invention establishes the relationship between the reinforcement learning action volume and the dispatching instructions, and reasonably handles the upper and lower limit constraints of the unit output, the unit climbing constraints, and the upper and lower limit constraints of the water level in the water-wind-solar complementary system, thereby ensuring the strict satisfaction of the complex constraints of the water-wind-solar system and accelerating the convergence speed of reinforcement learning; by introducing the conditional variational autoencoder to improve the strategy network, the convergence accuracy of reinforcement learning is improved. Summary of the invention

[0004] The purpose of the present invention is to fully consider the multi-dimensional uncertainty of water, wind and solar power, and to ensure that a scheduling strategy that meets the complex constraints of the water, wind and solar power systems is quickly and efficiently formulated, and to provide a water, wind and solar power intraday scheduling method based on an improved flexible strategy-value algorithm.

[0005] The intraday scheduling method of water, wind and solar power based on the improved flexible strategy-value algorithm includes the following steps:

[0006] Step 1: Establish the water-wind-solar operation environment based on the multi-dimensional uncertainty of water-wind-solar;

[0007] Step 2: Consider the constraints of the water, wind and solar systems to establish a safety reinforcement learning decision model;

[0008] Step 3: Improve the flexible strategy-evaluation algorithm based on conditional variational autoencoder for intraday scheduling;

[0009] The step 1 comprises the following steps:

[0010] Step 1-1: Establish the status update environment of the water, wind and solar system;

[0011] ;

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] in, For the scene s Lower Hydropower Station i exist t +1 storage capacity at time; For the scene s Lower Hydropower Station i exist t Storage capacity at the time; For the scene s Lower Hydropower Station i exist t Inbound traffic at the time; For the scene s Lower Hydropower Station i exist t Outbound flow at the time; is the scheduling time interval; For hydropower station i -1 with hydroelectric power station i When the water flow is stagnant; For the scene s Lower Hydropower Station i -1 at time Traffic volume; For the scene s Lower Hydropower Station i -1 and hydropower station i Between t Interval flow at the time; For the scene s Xiadi Hydropower Stationi exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i The abandoned water flow at time t; For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; , , and For hydropower station i Polynomial fitting coefficients of the water level-reservoir capacity relationship; For the scene s Lower Hydropower Station i exist t Tailwater level at the time; , and For hydropower station i Polynomial fitting coefficients of the tailwater level-outflow relationship; For the scene s Down t Shike Hydropower Station i The crew n The generating head; For the scene s Lower power station i The crew n exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i exist t -1 Reservoir water level at time; For the scene s Down t Shike Hydropower Station i The crew n Head loss; For hydropower station i Head loss coefficient; For hydropower station i The head loss constant; For hydropower station i The crew n exist t The effort of the moment; The overall efficiency of hydropower generation; For hydropower station i exist t The effort of the moment; For hydropower station i The number of units; For the scene s Photovoltaic power stationg exist t The effort of the moment; For the scene s Photovoltaic power station g exist t Maximum output at any time; For the scene s Photovoltaic power station g exist t The abandoned optical power at the moment; For the scene s Downwind farm w exist t The effort of the moment; For the scene s Downwind farm w exist t Maximum output at any time; For the scene s Downwind farm w exist t The wind power curtailment at the moment.

[0024] Step 1-2: Establish the constraints of the water, wind and solar system:

[0025] (1) Upper and lower water level constraints:

[0026] ;

[0027] in, For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; For hydropower station i exist t The lower limit of water level at the moment; For hydropower station i exist t The upper limit of the water level at the moment;

[0028] (2) Upper and lower limit constraints of power generation flow:

[0029] ;

[0030] in, For the scene s Down t Shike Hydropower Station i The crew n The power generation flow; For hydropower station i The crew n The upper limit of power generation flow; For hydropower station i The crew n The lower limit of power generation flow;

[0031] (3) Upper and lower limits of unit output:

[0032] ;

[0033] in, For the scene s Down t Shike Hydropower Station i The crew n of efforts, For hydropower station i The crew n The output limit of For hydropower station i The crew n The lower limit of output;

[0034] (4) Unit output climbing constraints:

[0035] ;

[0036] in, For the scene s Down t +1 moment hydroelectric power station i unit n of efforts, For hydropower station i The crew n Climbing ability;

[0037] (5) Restrictions on abandoned hydropower and electricity:

[0038] ;

[0039] ;

[0040] ;

[0041] (6) Section transmission constraints:

[0042] ;

[0043] in, I o For cross section o The number of hydroelectric power stations included; W o For cross section o the number of wind farms included; G o For cross section o The number of PV plants included; For cross section o transmission capacity.

[0044] Step 1-3: Establish the objective function of the water-wind-solar system:

[0045] ;

[0046] ;

[0047] ;

[0048] in, is the system output adjustment cost function, indicating that the cost of adjusting the output of the hydropower, wind and solar power systems within a day is minimal; T is the total time of the scheduling cycle; For hydropower station i The crew n exist t The day-ahead plan effort at the moment; For hydropower station i The crew n Output adjustment costs; For the scene s Photovoltaic power station g exist t The day-ahead plan effort at the moment; For photovoltaic power plants g Output adjustment costs; For the scene s Downwind farm w exist t The day-ahead plan effort at the moment; For wind farms w Output adjustment costs; S is the total number of scenes; is the system abandonment of water and electricity function, indicating that the abandonment of water and electricity of the hydro-wind-solar system is minimal; I is the number of hydropower stations included in the system; G is the number of PV power stations included in the system; W is the number of wind farms included in the system; Adjust the weights corresponding to the costs for the system; is the weight corresponding to the system's wind and solar power abandonment, is the objective function of the water-wind-solar system.

[0049] The step 2 comprises the following steps:

[0050] Step 2-1: Calculate the corresponding relationship between the action amount and the scheduling instruction;

[0051] The corresponding relationship between the action amount and the dispatching instruction is established. The dispatching instruction of the water-wind-solar system can be obtained according to the action output by the intelligent agent. The dispatching instruction of the water-wind-solar system includes the output of cascade hydropower, outflow flow, wind power output and photovoltaic output:

[0052] (1) Relationship between the output action of cascade hydropower and dispatching instructions:

[0053] The output of cascade hydropower should meet the upper and lower limit constraints of unit output and the unit output climbing constraints. The boundary expression of hydropower output is:

[0054] ;

[0055] ;

[0056] in, For hydropower station i The crew n exist t The output of the moment is lower bound; For hydropower station i The crew n exist t The upper limit of output at any time; For hydropower station i The crew n exist t-1 The effort of the moment; Indicates that at the hydropower station i The crew n exist t-1 The maximum value is taken from the output minus the gradeability at the moment and the lower limit of the unit output; Indicates that at the hydropower station i The crew n exist t-1 The minimum value is taken from the output plus the gradeability at the moment and the upper limit of the unit output;

[0057] The unit output is calculated based on the unit output action volume. The calculation formula is as follows:

[0058] ;

[0059] in, For power station i The crew n exist t The unit output action amount at the moment; is the upper limit of the action amount, which is 1; is the lower limit of the action amount, take -1;

[0060] (2) Relationship between the outflow action volume of cascade hydropower and the dispatching instruction: The outflow of cascade hydropower is set between the upper and lower limits of the outflow and the upper and lower limits of the water level constraints that the cascade hydropower meets. The calculation is as follows:

[0061] ;

[0062] ;

[0063] in, For the scene s Lower power station i exist t The upper limit of outbound traffic at the time; For the scene s Lower power station i exist t The lower limit of outbound flow at the time;

[0064] Calculate the outbound flow according to the outbound flow action volume:

[0065] ;

[0066] in, For the scene s Lower Hydropower Station i exist t The outbound flow volume at the moment;

[0067] (3) Relationship between wind and solar power output and dispatching instructions: To meet the power curtailment constraint, the relationship between wind and solar power output and dispatching instructions is calculated as:

[0068] ;

[0069] ;

[0070] in, For the scene s Photovoltaic power station g exist t The amount of force output at any given moment; For the scene s Downwind farm w exist t The amount of force applied at any given moment.

[0071] Step 2-2: Calculate the corresponding relationship between the state quantity and the state;

[0072] The status of the water, wind and solar system includes water level, interval flow, maximum wind and solar output, planned output of hydropower units and planned output of wind and solar units. The state quantities of safety reinforcement learning are as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] in, is the system output adjustment cost function, indicating that the cost of adjusting the output of the hydropower, wind and solar power systems within a day is minimal; T is the total time of the scheduling cycle; For hydropower station i The crew n exist t The day-ahead plan effort at the moment; For hydropower station i The crew n Output adjustment costs; For the scene s Photovoltaic power station g exist t The day-ahead plan effort at the moment; For photovoltaic power plants g Output adjustment costs; For the scene s Downwind farm w exist t The day-ahead plan effort at the moment; For wind farms w Output adjustment costs; S is the total number of scenes; is the system abandonment of water and electricity function, indicating that the abandonment of water and electricity of the hydro-wind-solar system is minimal; I is the number of hydropower stations included in the system; G is the number of PV power stations included in the system; W is the number of wind farms included in the system; Adjust the weights corresponding to the costs for the system; is the weight corresponding to the system's wind and solar power abandonment, is the objective function of the water-wind-solar system.

[0081] Step 2-3: Calculate the corresponding relationship between the constraint quantity and the constraint condition;

[0082] In addition to the upper and lower limits of unit output, the unit output climbing constraints and the upper and lower limits of water level, the hydropower, wind and solar power system should meet the power generation flow constraints, water abandonment constraints and section output constraints. The upper limit constraint of power generation flow, the lower limit constraint of power generation flow, the water abandonment flow constraint and the section transmission constraint of safety reinforcement learning are calculated:

[0083] ;

[0084] ;

[0085] ;

[0086] ;

[0087] in, For the scene s Lower Hydropower Station i exist t The upper limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The lower limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The abandoned water flow constraint at the time; For the scene s Lower section o exist t The section transmission constraint at the time.

[0088] Step 2-4: Calculate the reward amount for secure reinforcement learning;

[0089] The reward amount of safety reinforcement learning is composed of the system output adjustment cost function, system regulation capability, and system abandonment of water and electricity:

[0090] ;

[0091] ;

[0092] ;

[0093] in, for t The amount of reward at the moment; is the adjusted cost reward at time t; is the reward amount for abandoning water and electricity at time t.

[0094] In step 3, the flexible strategy-evaluation algorithm based on the conditional variational autoencoder is improved as follows:

[0095] Step 3-1 Establish a policy network based on conditional variational autoencoder:

[0096] The conditional variational autoencoder consists of an encoder and a decoder. The encoder is composed of a multi-layer neural network whose input is the state quantity and action quantity, and whose output is the mean vector and covariance matrix of the latent variable; the decoder is composed of a multi-layer neural network whose input is the state quantity and latent variable, and whose output is the mean vector and variance vector of the action quantity. The steps to build an actor network using the conditional variational autoencoder are as follows:

[0097] 1) Take the action-state pair from the memory pool and use the mean vector and covariance matrix output by the encoder to form a multidimensional Gaussian distribution q ,right q Perform reparameterization sampling to obtain latent variables and calculate q With the multidimensional standard Gaussian distribution kl Divergence is calculated as follows:

[0098] ;

[0099] in, is a multidimensional standard Gaussian distribution. for q The covariance matrix of is the trace of the matrix; for q The mean vector of ; d is the dimension of the latent variable.

[0100] 2) Input the state quantity and latent variables into the decoder, output the mean vector and covariance matrix of the action quantity, and calculate the multidimensional Gaussian distribution they constitute p The reconstruction loss is:

[0101] ;

[0102] in, for t The amount of action at a moment.

[0103] 3) Obtain the state from the water, wind and solar operation environment and convert it into state quantity and d The latent variables obtained by sampling the standard Gaussian distribution are input into the decoder to obtain the mean vector and covariance matrix to form a multi-dimensional Gaussian distribution, and re-parameterized sampling is performed to obtain a new action volume.

[0104] The training objectives of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows:

[0105] ;

[0106] in, The parameters are Conditional variational autoencoder of ; is the parameter value that maximizes the objective function; For the moment t The action amount at the time is composed of all the action amounts obtained in step 2-1; The training step length is t The state quantity at the time is composed of all the state quantities obtained in step 2-2; Indicates that in a given state When the strategy The generated probability distribution is sampled to obtain the action amount ; is the discount factor; is the temperature coefficient; The status is When , the probability distribution generated by the policy network is sampled to obtain the action probability.

[0107] The constraints of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows:

[0108] ;

[0109] in, For the moment t The constraint amount at this time is composed of all the constraints obtained in steps 2-3.

[0110] The above constraints are transformed into the training objectives using the Lagrange multiplier method, and we get:

[0111] ;

[0112] in, For the l Lagrange multipliers; is the dimension of the constraint.

[0113] Step 3-2 proposes an update method to improve the flexible strategy-evaluation algorithm;

[0114] (1) Lagrange multiplier update method:

[0115] ;

[0116] in, is the learning rate of the Lagrange multiplier; for The gradient function of .

[0117] (2) Q Network update method:

[0118] Q The network consists of a multi-layer neural network whose input is the action-state pair extracted from the memory pool and whose output is the expected reward value of the action-state pair. softBellman The residual is used to update the parameters:

[0119] ;

[0120] ;

[0121] in, for Q The parameters of the network; for Q The learning rate of the network; for The gradient function of Indicates that from the memory pool D The action-state pairs extracted from for Q The network input is Output when For the goal Q value.

[0122] Target Q The value is calculated as follows:

[0123] ;

[0124] in, Represents a positive number; The state of the next moment; Indicates that from the memory pool D The state of the next moment extracted from The input is the action-state pairing target for the next moment Q The output of the network; The status is When the state is , the probability distribution generated by the policy network is sampled to obtain the action probability.

[0125] (3) Objectives Q Network update method:

[0126] Target Q Network and Q The network structure is the same, and the initialization parameters are also the same Q Same network, target Q The network is updated as follows:

[0127] ;

[0128] in, is the soft update coefficient;

[0129] (4) Conditional variational autoencoder update method:

[0130] ; ;

[0131] in, for The learning rate; for The gradient function of Represents the extraction of state quantity from the memory pool; The representative input is When , the action amount is extracted from the multidimensional probability distribution output by the decoder of the conditional variational autoencoder.

[0132] (5) Temperature coefficient update method:

[0133] ;

[0134] ;

[0135] in, for The learning rate; for The gradient function of is the expected entropy value.

[0136] Compared with the prior art, the present invention has the following beneficial effects:

[0137] 1) The present invention effectively reduces the constraint penalty items of safety reinforcement learning by establishing the relationship between action variables and dispatch instructions. The unit output action variables are reasonably calculated so that the unit output meets the upper and lower limit constraints of the unit output and the unit output climbing constraints; the upper and lower limits of the outbound flow are reasonably calculated so that the water, wind and solar systems meet the upper and lower limit constraints of the water level. The present invention can effectively reduce the constraint penalty items of safety reinforcement learning by reasonably processing action variables, ensure the strict satisfaction of constraint conditions, and accelerate the convergence of the model.

[0138] 2) The policy network established by the present invention can effectively capture the characteristics of high-dimensional complex variables. The conditional variational autoencoder is used to map the action-state pairs to a low-dimensional space, and the key features are captured from the high-dimensional decision variables. At the same time, the low-dimensional latent variables are used to generate high-dimensional decision variables under given state quantities, thereby improving the generation ability of the policy network. Examples show that the intraday scheduling method for water, wind and solar power based on the improved flexible strategy-value algorithm proposed in the present invention can improve the convergence accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0139] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0140] Figure 1 This is a flow chart of a method for daily scheduling of water, wind and solar power based on an improved flexible strategy-value algorithm according to an embodiment of the present invention;

[0141] Figure 2 This is a topological diagram of a water, wind and solar system according to an embodiment of the present invention;

[0142] Figure 3 is a comparison chart of the amount of rewards obtained during the training process before and after the improvement of the embodiment of the present invention;

[0143] Figure 4 is a diagram of the violation of the section constraint during the training process of the embodiment of the present invention;

[0144] Figure 5 is a diagram showing violation of the upper limit constraint of the power generation flow of the hydropower station 1 during the training process of the embodiment of the present invention;

[0145] Figure 6 is a diagram showing violation of the lower limit constraint of the power generation flow of the hydropower station 1 during the training process of the embodiment of the present invention;

[0146] Figure 7 is a diagram showing violation of the water abandonment flow constraint of the hydropower station 2 during the training process of the embodiment of the present invention;

[0147] Figure 8 This is a diagram of the daily dispatching output of a hydropower station obtained by testing an embodiment of the present invention;

[0148] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention.

[0149] Among them: 810 - processor, 820 - communication interface, 830 - memory, 840 - communication bus. DETAILED DESCRIPTION

[0150] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0151] The intraday scheduling method of water, wind and solar power based on the improved flexible strategy-value algorithm is as follows: Figure 1 shown.

[0152] The topology diagram of the water, wind and solar system corresponding to the embodiment is as follows: Figure 2 As shown in Figure 1, the power grid includes 4 cascade hydropower stations, 5 photovoltaic power stations and 3 wind farms. The historical operation data of the water-wind-solar system is divided into a training set and a test set at a ratio of 5:1, and a study on the intraday scheduling of water-wind-solar based on the improved flexible strategy-value algorithm is conducted.

[0153] Step 1: Establish the water-wind-solar operation environment based on the multi-dimensional uncertainty of water-wind-solar;

[0154] Step 1-1. Establish the status update environment of the water, wind and light system;

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] ;

[0167] in, For the scene s Lower Hydropower Station i exist t +1 storage capacity at time; For the scene s Lower Hydropower Station i exist t Storage capacity at the time; For the scene s Lower Hydropower Station i exist t Inbound traffic at the time; For the scene s Lower Hydropower Station i exist t Outbound flow at the time; is the scheduling time interval; For hydropower station i -1 with hydroelectric power station i When the water flow is stagnant; For the scene s Lower Hydropower Station i -1 at time Traffic volume; For the scene sLower Hydropower Station i -1 and hydropower station i Between t Interval flow at the time; For the scene s Xiadi Hydropower Station i exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i The abandoned water flow at time t; For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; , , and For hydropower station i Polynomial fitting coefficients of the water level-reservoir capacity relationship; For the scene s Lower Hydropower Station i exist t Tailwater level at the time; , and For hydropower station i Polynomial fitting coefficients of the tailwater level-outflow relationship; For the scene s Down t Shike Hydropower Station i The crew n The generating head; For the scene s Lower power station i The crew n exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i exist t -1 Reservoir water level at time; For the scene s Down t Shike Hydropower Station i The crew n Head loss; For hydropower station i Head loss coefficient; For hydropower station i The head loss constant; For hydropower station i The crew n exist t The effort of the moment; The overall efficiency of hydropower generation; For hydropower station i existt The effort of the moment; For hydropower station i The number of units; For the scene s Photovoltaic power station g exist t The effort of the moment; For the scene s Photovoltaic power station g exist t Maximum output at any time; For the scene s Photovoltaic power station g exist t The abandoned optical power at the moment; For the scene s Downwind farm w exist t The effort of the moment; For the scene s Downwind farm w exist t Maximum output at any time; For the scene s Downwind farm w exist t The wind power curtailment at the moment.

[0168] Step 1-2 Establish the constraints of the water, wind and solar system;

[0169] (1) Water level upper and lower limit constraints:

[0170] ;

[0171] in, For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; For hydropower station i exist t The lower limit of water level at the moment; For hydropower station i exist t The upper limit of the water level at the moment;

[0172] (2) Upper and lower limits of power generation flow:

[0173] ;

[0174] in, For the scene s Down t Shike Hydropower Station i The crew n The power generation flow; For hydropower station iThe crew n The upper limit of power generation flow; For hydropower station i The crew n The lower limit of power generation flow;

[0175] (3) Upper and lower limits of unit output:

[0176] ;

[0177] in, For the scene s Down t Shike Hydropower Station i The crew n of efforts, For hydropower station i The crew n The output limit of For hydropower station i The crew n The lower limit of output;

[0178] (4) Unit output climbing constraints:

[0179] ;

[0180] in, For the scene s Down t +1 moment hydroelectric power station i unit n of efforts, For hydropower station i The crew n Climbing ability;

[0181] (5) Restrictions on abandoned hydropower and electricity:

[0182] ;

[0183] ;

[0184] ;

[0185] (6) Section transmission constraints:

[0186] ;

[0187] in, I o For cross section o The number of hydroelectric power stations included; W o For cross section o the number of wind farms included; G oFor cross section o The number of PV plants included; For cross section o transmission capacity;

[0188] Step 1-3 establishes the objective function of the water-wind-solar system;

[0189] ;

[0190] ;

[0191] ;

[0192] in, is the system output adjustment cost function, indicating that the cost of adjusting the output of the hydropower, wind and solar power systems within a day is minimal; T is the total time of the scheduling cycle; For hydropower station i The crew n exist t The day-ahead plan effort at the moment; For hydropower station i The crew n Output adjustment costs; For the scene s Photovoltaic power station g exist t The day-ahead plan effort at the moment; For photovoltaic power plants g Output adjustment costs; For the scene s Downwind farm w exist t The day-ahead plan effort at the moment; For wind farms w Output adjustment costs; S is the total number of scenes; is the system abandonment of water and electricity function, indicating that the abandonment of water and electricity of the hydro-wind-solar system is minimal; I is the number of hydropower stations included in the system; G is the number of PV power stations included in the system; W is the number of wind farms included in the system; Adjust the weights corresponding to the costs for the system; is the weight corresponding to the system's wind and solar power abandonment, is the objective function of the water-wind-solar system.

[0193] Step 2 Considering the constraints of the water-wind-solar system, a safety reinforcement learning decision model is established:

[0194] Step 2-1 calculates the corresponding relationship between the action amount of safety reinforcement learning and the scheduling instruction;

[0195] The corresponding relationship between the action amount and the dispatching instruction is established, and the dispatching instruction of the water, wind and solar system can be obtained according to the action output by the intelligent agent. The dispatching instruction of the water, wind and solar system includes the output of cascade hydropower, outflow flow, wind power output and photovoltaic output.

[0196] The corresponding relationship between the action amount and the dispatching instruction is established, and the dispatching instruction of the water, wind and solar system can be obtained according to the action output by the intelligent agent. The dispatching instruction of the water, wind and solar system includes the output of cascade hydropower, outflow flow, wind power output and photovoltaic output.

[0197] (1) Relationship between the output action of cascade hydropower and dispatching instructions:

[0198] The output of cascade hydropower should meet the upper and lower limit constraints of unit output and the unit output climbing constraints. The boundary of hydropower output should be:

[0199] ;

[0200] ;

[0201] in, For hydropower station i The crew n exist t The output of the moment is lower bound; For hydropower station i The crew n exist t The upper limit of output at any time; For hydropower station i The crew n exist t-1 The effort of the moment; Indicates that at the hydropower station i The crew n exist t-1 The maximum value is taken from the output minus the gradeability at the moment and the lower limit of the unit output; Indicates that at the hydropower station i The crew n exist t-1 The minimum value is taken from the output plus the gradeability at the moment and the upper limit of the unit output;

[0202] The unit output is calculated based on the unit output action variable, and the calculation formula is as follows:

[0203] ;

[0204] in, For power station i The crew n exist t The unit output action amount at the moment; is the upper limit of the action amount, which is 1; is the lower limit of the action amount, which is -1.

[0205] (2) Relationship between the outflow action volume of cascade hydropower and the dispatching instruction: The outflow of cascade hydropower is set between the upper and lower limits of the outflow and the upper and lower limits of the water level constraints that the cascade hydropower meets. The calculation is as follows:

[0206] ;

[0207] ;

[0208] in, For the scene s Lower power station i exist t The upper limit of outbound traffic at the time; For the scene s Lower power station i exist t The lower limit of outbound flow at the time;

[0209] Calculate the outbound flow according to the outbound flow action volume:

[0210] ;

[0211] in, For the scene s Lower Hydropower Station i exist t The outbound flow volume at the moment.

[0212] (3) Relationship between wind and solar power output and dispatching instructions: To meet the power curtailment constraint, the relationship between wind and solar power output and dispatching instructions is calculated as follows:

[0213] ;

[0214] ;

[0215] in, For the scene s Photovoltaic power station g exist t The amount of force output at any given moment; For the scene s Downwind farm w exist t The amount of force applied at any given moment.

[0216] Step 2-2 calculates the corresponding relationship between the state quantity and the state;

[0217] The status of the water, wind and solar system includes water level, interval flow, maximum wind and solar output, planned output of hydropower units and planned output of wind and solar. The state quantities of safety reinforcement learning are as follows:

[0218]

[0219]

[0220]

[0221]

[0222]

[0223]

[0224] ;

[0225] in, For the scene s Lower Hydropower Station i exist t The water level status at the moment; is the upper limit of the state quantity, which is 1; is the lower limit of the state quantity, take -1; For the scene s Lower Hydropower Station i exist t The water supply state quantity of the interval at the time; For hydropower station i The maximum value of water inflow in the interval; m pvl,s,g,t For the scene s Photovoltaic power station g exist t The maximum output state at the moment; The photovoltaic power station under scenario s g exist t Maximum output at any time; For photovoltaic power plants g Capacity; m wtl,s,w,t For the scene s Downwind farm w exist t The maximum output state at the moment; is the maximum output of wind farm w at time t under scenario s; For wind farms w Capacity; For hydropower station i The crew n exist t Output state quantity at any moment; For the scene s Photovoltaic power station g exist t Output state quantity at the moment; For the scene s Downwind farm w exist t The output state quantity at the moment.

[0226] Step 2-3: Calculate the corresponding relationship between the constraints of safety reinforcement learning and the constraints of the water, wind and solar systems:

[0227] In addition to the upper and lower limits of unit output, the unit output climbing constraints, and the upper and lower limits of water level, the hydropower, wind and solar power system should meet the power generation flow constraints, water abandonment constraints and section output constraints. The upper limit constraint of power generation flow, the lower limit constraint of power generation flow, the water abandonment flow constraint and the section transmission constraint of safety reinforcement learning are calculated:

[0228] ;

[0229] ;

[0230] ;

[0231] ;

[0232] in, For the scene s Lower Hydropower Station i exist t The upper limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The lower limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The abandoned water flow constraint at the time; For the scene s Lower section o exist t Section transmission constraints at the time.

[0233] Step 2-4: Calculate the reward for secure reinforcement learning:

[0234] The reward amount of safety reinforcement learning is composed of the system output adjustment cost function, system regulation capability, and system abandonment of water and electricity:

[0235] ;

[0236] ;

[0237] ;

[0238] in, for t The amount of reward at the moment; is the adjusted cost reward at time t; is the reward amount for abandoning water and electricity at time t.

[0239] In step 3, the flexible strategy-evaluation algorithm based on the conditional variational autoencoder is improved as follows:

[0240] Step 3-1 Establish a policy network based on conditional variational autoencoder:

[0241] The conditional variational autoencoder consists of an encoder and a decoder. The encoder is composed of a multi-layer neural network whose input is the state quantity and action quantity, and whose output is the mean vector and covariance matrix of the latent variable; the decoder is composed of a multi-layer neural network whose input is the state quantity and latent variable, and whose output is the mean vector and variance vector of the action quantity. The steps to build an actor network using the conditional variational autoencoder are as follows:

[0242] 1) Take the action-state pair from the memory pool and use the mean vector and covariance matrix output by the encoder to form a multidimensional Gaussian distribution q ,right q Perform reparameterization sampling to obtain latent variables and calculate q With the multidimensional standard Gaussian distribution kl Divergence is calculated as follows:

[0243] ;

[0244] in, is a multidimensional standard Gaussian distribution. for q The covariance matrix of is the trace of the matrix; for q The mean vector of ; d is the dimension of the latent variable; for q The transpose of the mean vector of ; express q With the multidimensional standard Gaussian distribution kl Divergence;

[0245] 2) Input the state quantity and latent variables into the decoder, output the mean vector and covariance matrix of the action quantity, and calculate the multidimensional Gaussian distribution they constitute p The reconstruction loss of , the expression of reconstruction loss is:

[0246] ;

[0247] in, for t The amount of action at a moment; Indicates that the amount of action is distributed in a multidimensional Gaussian p The probability of the following; represents the reconstruction loss;

[0248] 3) Obtain the state from the water and wind power operation environment, convert it into a state quantity, and then compare the state quantity with the d The latent variables obtained by sampling the dimensional standard Gaussian distribution are input into the decoder to obtain the mean vector and covariance matrix; the multidimensional Gaussian distribution composed of the mean vector and the covariance matrix is ​​reparameterized and sampled to obtain a new action quantity.

[0249] The training objectives of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows:

[0250] ;

[0251] in, The parameters are Conditional variational autoencoder of ; is the parameter value that maximizes the objective function; For the moment t The action amount at the time is composed of all the action amounts obtained in step 2-1; The training step length is t The state quantity at the time is composed of all the state quantities obtained in step 2-2; Indicates that in a given state When the strategy The generated probability distribution is sampled to obtain the action amount ; is the discount factor; is the temperature coefficient; The status is When , the probability distribution generated by the policy network is sampled to obtain the action probability; It means to take expectations; is the discount factor raised to the power of t; is the reward amount at time t;

[0252] The constraints of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows:

[0253] ;

[0254] in, For the moment t The constraint amount at the time is composed of all the constraint amounts obtained in steps 2-3;

[0255] The above constraints are transformed into the training objectives using the Lagrange multiplier method, and we get:

[0256] ;

[0257] in, For the l Lagrange multipliers; is the dimension of the constraint.

[0258] Step 3-2 proposes an update method to improve the flexible strategy-evaluation algorithm;

[0259] (1) Lagrange multiplier update method:

[0260] ;

[0261] in, is the learning rate of the Lagrange multiplier; for The gradient function of To take a positive function, when season ;

[0262] (2) Q Network update method:

[0263] Q The network consists of a multi-layer neural network whose input is the action-state pair extracted from the memory pool and whose output is the expected reward value of the action-state pair. softBellman The residual is used to update the parameters:

[0264] ;

[0265] ;

[0266] in, for Q The parameters of the network; for Q The learning rate of the network; for The gradient function of for softBellman Residual; Indicates that from the memory pool D The action-state pairs extracted from It means to take expectations; for Q The network input is Output when For the goal Q value;

[0267] Target Q The value is calculated as follows:

[0268] ;

[0269] in, Represents a positive number; The state of the next moment; Indicates that from the memory pool D The state of the next moment extracted from The input is the action-state pairing target for the next moment Q The output of the network; The status is When the state is , the probability distribution generated by the policy network is sampled to obtain the action probability; represents the target Q value; It means to take expectations; is the amount of action at time t+1; Represents the state quantity at time t+1; is the dimension of the constraint quantity;

[0270] (3) Objectives Q Network update method:

[0271] Target Q Network and Q The network structure is the same, and the initialization parameters are also the same Q Same network, target Q The network is updated as follows:

[0272] ;

[0273] in, is the soft update coefficient; Indicates the target Q The parameters of the network;

[0274] (4) Conditional variational autoencoder update method:

[0275] ;

[0276] ;

[0277] in, for The learning rate; for The gradient function of Represents the extraction of state quantity from the memory pool; The representative input is When , the action amount is extracted from the multidimensional probability distribution output by the decoder of the conditional variational autoencoder; It means to take expectations; Demonstrate expectations;

[0278] (5) Temperature coefficient update method:

[0279] ;

[0280] ;

[0281] in, for The learning rate; for The gradient function of is the expected entropy value; represents the loss function of the temperature coefficient.

[0282] Furthermore, in order to verify the effectiveness of the intraday scheduling method of water, wind and solar power based on the improved flexible strategy-value algorithm proposed in the present invention. Figure 3 The comparison of the reward amount obtained by training before and after the improvement is shown. It can be seen from the figure that in the early stage of training, the reward amount obtained by the improved algorithm is significantly greater than that before the improvement; in the later stage of training, the convergence accuracy of the improved algorithm is higher than that before the improvement. Figure 4 It shows the violation of section constraints during training. No section constraints were applied during training. Figure 6 , 7 Figures 8 and 8 show the upper and lower limits of the power generation flow of hydropower station 1 and the water abandonment constraint violations of hydropower station 2 during the training process. In the early stage of training, there are many constraint violations, but as the training progresses, the constraint violations decrease rapidly. There are no constraint violations in the middle and late stages of training, which proves that the present invention can effectively ensure the safety constraints of the water-wind-solar system. Fig. 9 This is the output diagram of the hydropower station obtained through testing. The overall output fluctuation of the hydropower station is small, which is consistent with the actual operation situation.

[0283] In summary, the present invention can comprehensively consider the complex constraints of the water, wind and solar systems, effectively reduce the constraint penalty terms of safety reinforcement learning, ensure the strict satisfaction of the constraints, and accelerate the convergence speed of the model; use the conditional variational autoencoder to map the action-state pairs to a low-dimensional space, capture its key features from high-dimensional decision variables, and improve the convergence accuracy.

[0284] Fig. 9 The physical structure diagram of the electronic device is shown as an example. Fig. 9 As shown, the electronic device may include: a processor ( processor )810, communication interface ( CommunicationsInterface )820, memory ( memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute the intraday scheduling method of water, wind and solar power based on the improved flexible strategy-value algorithm.

[0285] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U Disk, mobile hard disk, read-only memory ( ROM , Read - OnlyMemory ), Random Access Memory ( RAM , RandomAccessMemory ), magnetic disks, optical disks, and other media that can store program codes.

[0286] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intraday scheduling method of water, wind and solar power based on the improved flexible strategy-value algorithm provided by the above-mentioned methods.

[0287] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the intraday scheduling method for water, wind and solar power based on the improved flexible strategy-value algorithm provided by the above-mentioned methods.

[0288] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0289] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM , disks, optical disks, etc., include several instructions for enabling a computer device, which may be a personal computer, a server, or a network device, etc., to execute the methods described in various embodiments or certain parts of the embodiments.

[0290] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for daily scheduling of water, wind and solar power based on an improved flexible strategy-value algorithm, characterized in that: The following steps are involved: Step 1: Establish the water-wind-solar operation environment based on the multi-dimensional uncertainty of water-wind-solar; Step 2: Consider the constraints of the water, wind and solar systems to establish a safety reinforcement learning decision model; Step 3: Improve the flexible strategy-evaluation algorithm based on conditional variational autoencoder for intraday scheduling; The step 1 comprises the following steps: Step 1-1: Establish the status update environment of the water, wind and solar system; Step 1-2: Establish the constraints of the water-wind-solar system, which include upper and lower limits of water level, upper and lower limits of power generation flow, upper and lower limits of unit output, unit output climbing constraints, water and power abandonment constraints and section transmission constraints: Step 1-3: Establish the objective function of the water-wind-solar system: The step 2 comprises the following steps: Step 2-1: Calculate the corresponding relationship between the action amount and the dispatching instruction, including establishing the corresponding relationship between the action amount and the dispatching instruction. The dispatching instruction of the water-wind-solar system can be obtained according to the action output by the intelligent agent. The dispatching instruction of the water-wind-solar system includes the output of cascade hydropower, outflow flow, wind power output and photovoltaic output; Step 2-2: Calculate the corresponding relationship between the state quantity and the state. The state of the water-wind-solar system includes water level, interval flow, maximum output of wind-solar system, planned output of hydropower units and planned output of wind-solar system; Step 2-3: Calculate the corresponding relationship between the constraint quantity and the constraint conditions, including that the hydropower, wind-solar and solar power systems should meet the power generation flow constraint, water abandonment constraint and section output constraint, in addition to the upper and lower limit constraints of the unit output, the unit output ramp constraint and the upper and lower limit constraints of the water level; Step 2-4: Calculate the corresponding relationship between the reward amount and the objective function; The step 3 comprises the following steps: Step 3-1: Establish a policy network based on conditional variational autoencoder; The conditional variational autoencoder is composed of an encoder and a decoder. The encoder is composed of a multi-layer neural network whose input is state quantity and action quantity and whose output is the mean vector and covariance matrix of latent variables; the decoder is composed of a multi-layer neural network whose input is state quantity and latent variables and whose output is the mean vector and variance vector of action quantity. Using the conditional variational autoencoder, the steps of establishing the actor network are as follows: 1) Take the action-state pair from the memory pool and use the mean vector and covariance matrix output by the encoder to form a multidimensional Gaussian distribution q ,right q Perform reparameterization sampling to obtain latent variables and calculate q With multidimensional standard Gaussian distribution kl Divergence is calculated as follows: ； in, is a multidimensional standard Gaussian distribution; for q The covariance matrix of is the trace of the matrix; for q The mean vector of ; d is the dimension of the latent variable; for q The transpose of the mean vector of ; express q With multidimensional standard Gaussian distribution kl Divergence; 2) Input the state quantity and latent variables into the decoder, output the mean vector and covariance matrix of the action quantity, and calculate the multidimensional Gaussian distribution they constitute p The reconstruction loss is: ； in, for t The amount of action at a moment; Indicates that the amount of action is distributed in a multidimensional Gaussian p The probability of the following; represents the reconstruction loss; 3) Obtain the state from the water, wind and solar operation environment and convert it into state quantity; then compare it with the d The latent variables obtained by sampling the multidimensional standard Gaussian distribution are input into the decoder to obtain the mean vector and covariance matrix; the multidimensional Gaussian distribution composed of the mean vector and the covariance matrix is ​​reparameterized and sampled to obtain a new action amount; The training objectives of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows: ； in, The parameters are Conditional variational autoencoder of ; is the parameter value that maximizes the objective function; For the moment t The action amount at the time is composed of all the action amounts obtained in step 2-1; The training step length is t The state quantity at the time is composed of all the state quantities obtained in step 2-2; Indicates that in a given state When the strategy The generated probability distribution is sampled to obtain the action amount ; is the discount factor; is the temperature coefficient; The status is When , the probability distribution generated by the policy network is sampled to obtain the action The probability of It means to take expectations; is the discount factor raised to the power of t; is the reward amount at time t; The constraints of the conditional variational autoencoder to improve the flexible strategy-evaluation algorithm are as follows: ； in, For the moment t The constraint amount at the time is composed of all the constraint amounts obtained in steps 2-3; The above constraints are transformed into training objectives using the Lagrange multiplier method, and we get: ； in, For the l Lagrange multipliers; is the dimension of the constraint quantity; Step 3-2: Propose an update method for improving the flexible strategy-evaluation algorithm, the update method comprising: (1) Lagrange multiplier update method: ； in, is the learning rate of the Lagrange multiplier; for The gradient function of To take a positive function, when season ; (2) Q Network update method: Q The network consists of a multi-layer neural network whose input is the action-state pair extracted from the memory pool and whose output is the expected reward value of the action-state pair. softBellman The residual is used to update the parameters: ； ； in, for Q The parameters of the network; for Q The learning rate of the network; for The gradient function of for softBellman Residual; Indicates that from the memory pool D The action-state pairs extracted from It means to take expectations; for Q The network input is Output when For the goal Q value; Target Q The value is calculated as follows: ； in, Represents a positive number; The state of the next moment; Indicates that from the memory pool D The state of the next moment extracted from The input is the action-state pairing target for the next moment Q The output of the network; The status is When the state is , the probability distribution generated by the policy network is sampled to obtain the action probability; represents the target Q value; It means to take expectations; is the amount of action at time t+1; Represents the state quantity at time t+1; is the dimension of the constraint quantity; (3) Objectives Q Network update method: Target Q Network and Q The network structure is the same, and the initialization parameters are also the same Q Same network, target Q The network is updated as follows: ； in, is the soft update coefficient; Indicates the target Q The parameters of the network; (4) Conditional variational autoencoder update method: ； ； in, for The learning rate; for The gradient function of Represents the extraction of state quantity from the memory pool; The representative input is When , the action amount is extracted from the multidimensional probability distribution output by the decoder of the conditional variational autoencoder; It means to take expectations; Demonstrate expectations; (5) Temperature coefficient update method: ； ； in, for The learning rate; for The gradient function of is the expected entropy value; represents the loss function of the temperature coefficient.

2. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 1 is characterized in that: The state update environment of the water-wind-solar system established in step 1-1 includes: ； ； ； ； ； ； ； ； ； ； ； ； in, For the scene s Lower Hydropower Station i exist t +1 storage capacity at time; For the scene s Lower Hydropower Station i exist t Storage capacity at a given moment; For the scene s Lower Hydropower Station i exist t Inbound traffic at the time; For the scene s Lower Hydropower Station i exist t Outbound flow at the time; is the scheduling time interval; For hydropower station i -1 with hydroelectric power station i When the water flow is stagnant; For the scene s Lower Hydropower Station i -1 at time Traffic volume; For the scene s Lower Hydropower Station i -1 and hydropower station i Between t Interval flow at the time; For the scene s Xiadi Hydropower Station i exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i The abandoned water flow at time t; For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; , , and For hydropower station i Polynomial fitting coefficients of the water level-reservoir capacity relationship; For the scene s Lower Hydropower Station i exist t Tailwater level at the time; , and For hydropower station i Polynomial fitting coefficients of the tailwater level-outflow relationship; For the scene s Down t Shike Hydropower Station i The crew n The generating head; For the scene s Lower power station i The crew n exist t The power generation flow at the moment; For the scene s Lower Hydropower Station i exist t -1 Reservoir water level at time; For the scene s Down t Shike Hydropower Station i The crew n Head loss; For hydropower station i Head loss coefficient; For hydropower station i The head loss constant; For hydropower station i The crew n exist t The effort of the moment; The overall efficiency of hydropower generation; For hydropower station i exist t The effort of the moment; For hydropower station i The number of units; For the scene s Photovoltaic power station g exist t The effort of the moment; For the scene s Photovoltaic power station g exist t Maximum output at any time; For the scene s Photovoltaic power station g exist t The abandoned optical power at the moment; For the scene s Downwind farm w exist t The effort of the moment; For the scene s Downwind farm w exist t Maximum output at any time; For the scene s Downwind farm w exist t The wind power curtailment at the moment.

3. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 2 is characterized in that: In step 1-2; (1) Upper and lower water level constraints: ； in, For the scene s Lower Hydropower Station i exist t Reservoir water level at the time; For hydropower station i exist t The lower limit of water level at the moment; For hydropower station i exist t The upper limit of the water level at the moment; (2) Upper and lower limit constraints of power generation flow: ； in, For the scene s Down t Shike Hydropower Station i The crew n The power generation flow; For hydropower station i The crew n The upper limit of power generation flow; For hydropower station i The crew n The lower limit of power generation flow; (3) Upper and lower limits of unit output: ； in, For the scene s Down t Shike Hydropower Station i The crew n of effort, For hydropower station i The crew n The output limit of For hydropower station i The crew n The lower limit of output; (4) Unit output climbing constraints: ； in, For the scene s Down t +1 moment hydroelectric power station i unit n of efforts, For hydropower station i The crew n Climbing ability; (5) Restrictions on abandoned hydropower and electricity: ； ； ； (6) Section transmission constraints: ； in, I o For cross section o The number of hydroelectric power stations included; W o For cross section o the number of wind farms included; G o For cross section o The number of PV plants included; For cross section o transmission capacity.

4. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 3 is characterized in that: The expression of the objective function of the water-wind-solar system established in step 1-3 is: ； ； ； in, is the system output adjustment cost function, indicating that the cost of adjusting the output of the hydropower, wind and solar power systems within a day is minimal; T is the total time of the scheduling cycle; For hydropower station i The crew n exist t The day-ahead plan effort of the moment; For hydropower station i The crew n Output adjustment costs; For the scene s Photovoltaic power station g exist t The day-ahead plan effort of the moment; For photovoltaic power plants g Output adjustment costs; For the scene s Downwind farm w exist t The day-ahead plan effort at the moment; For wind farms w Output adjustment costs; S is the total number of scenes; is the system abandonment of water and electricity function, indicating that the abandonment of water and electricity of the hydro-wind-solar system is minimal; I is the number of hydropower stations included in the system; G is the number of PV power stations included in the system; W is the number of wind farms included in the system; Adjust the weights corresponding to the costs for the system; is the weight corresponding to the system's wind and solar power abandonment, It is the objective function of the water-wind-solar system.

5. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 4 is characterized in that: In step 2-1: (1) Relationship between the output action of cascade hydropower and dispatching instructions: The output of cascade hydropower should meet the upper and lower limit constraints of unit output and the unit output climbing constraints. The boundary expression of hydropower output is: ； ； in, For hydropower station i The crew n exist t The output of the moment is lower bound; For hydropower station i The crew n exist t The upper limit of output at any time; For hydropower station i The crew n exist t-1 The effort of the moment; Indicates that at the hydropower station i The crew n exist t-1 The maximum value is taken from the output minus the gradeability at the moment and the lower limit of the unit output; Indicates that at the hydropower station i The crew n exist t-1 The minimum value is taken from the output plus the gradeability at the moment and the upper limit of the unit output; The unit output is calculated based on the unit output action volume. The calculation formula is as follows: ； in, For power station i The crew n exist t The unit output action amount at the moment; is the upper limit of the action amount, which is 1; is the lower limit of the action amount, take -1; (2) Relationship between the outflow action volume of cascade hydropower and the dispatching instruction: The outflow of cascade hydropower is set between the upper and lower limits of the outflow and the upper and lower limits of the water level constraints that the cascade hydropower meets. The calculation is as follows: ； ； in, For the scene s Lower power station i exist t The upper limit of outbound traffic at the time; For the scene s Lower power station i exist t The lower limit of outbound flow at the time; Calculate the outbound flow according to the outbound flow action volume: ； in, For the scene s Lower Hydropower Station i exist t The outbound flow volume at the moment; (3) Relationship between wind and solar power output and dispatching instructions: To meet the power curtailment constraint, the relationship between wind and solar power output and dispatching instructions is calculated as: ； ； in, For the scene s Photovoltaic power station g exist t The amount of force output at any given moment; For the scene s Downwind farm w exist t The amount of force at a given moment.

6. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 5 is characterized in that: The state quantities of safe reinforcement learning in step 2-2 include: scenario s Lower Hydropower Station i exist t Water level status and scene at the moment s Lower Hydropower Station i exist t Time interval water supply status and scene s Photovoltaic power station g exist t Maximum output state and scenario at the moment s Downwind farm w exist t Maximum output state quantity at the moment, hydropower station i The crew n exist t Output status and scene at all times s Photovoltaic power station g exist t Output state and scene at the moment s Downwind farm w exist t Output state quantity at the moment: ； ； ； ； ； ； ； in, For the scene s Lower Hydropower Station i exist t The water level status at the moment; is the upper limit of the state quantity, which is 1; is the lower limit of the state quantity, take -1; For the scene s Lower Hydropower Station i exist t The water supply state quantity of the interval at the time; For hydropower station i The maximum value of water inflow in the interval; m pvl,s,g,t For the scene s Photovoltaic power station g exist t The maximum output state at the moment; The photovoltaic power station under scenario s g exist t Maximum output at any time; For photovoltaic power plants g Capacity; m wtl,s,w,t For the scene s Downwind farm w exist t The maximum output state at the moment; is the maximum output of wind farm w at time t under scenario s; For wind farms w Capacity; For hydropower station i The crew n exist t Output state quantity at any moment; For the scene s Photovoltaic power station g exist t Output state quantity at the moment; For the scene s Downwind farm w exist t The output state quantity at the moment.

7. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 6 is characterized in that: In step 2-3: the formula for calculating the upper limit constraint of power generation flow, the lower limit constraint of power generation flow, the abandoned water flow constraint and the section transmission constraint of safety reinforcement learning is: ； ； ； ； in, For the scene s Lower Hydropower Station i exist t The upper limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The lower limit constraint of power generation flow at the moment; For the scene s Lower Hydropower Station i exist t The amount of water abandonment flow constraint at the time; For the scene s Lower section o exist t The section transmission constraint at the time.

8. The method for intraday scheduling of water, wind and solar power based on improved flexible strategy-value algorithm according to claim 7 is characterized in that: In step 2-4, the reward amount of safety reinforcement learning is calculated: the reward amount of safety reinforcement learning is composed of the system output adjustment cost function, the system regulation capability and the system abandonment of water and power: ； ； ； in, for t The amount of reward at the moment; is the adjusted cost reward at time t; is the reward amount for abandoning water and electricity at time t.

Images