Unit commitment generative adversarial imitation learning method considering participation of green generator set
The adversarial imitation learning method addresses inefficiencies in unit commitment by using a neural network-based expert system to learn optimal unit combinations, enhancing computational efficiency and stability for green power units.
Patent Information
- Application Number
- CN202411385176.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-12-27
AI Technical Summary
Existing methods for safety-constrained unit commitment with green power units face high computational complexity, large computational burden, and challenges in defining reward functions, leading to inefficient solutions.
An adversarial imitation learning method is employed to directly learn optimal unit combinations using a safety-constrained unit commitment expert system, incorporating a generator and discriminator neural network model, with a proximal policy optimization-based loss function to enhance training performance.
Enables rapid online calculation of unit commitment under uncertainty, avoiding manually designed reward functions and improving solution stability.
Smart Images
Figure CN119204115A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of power system operation and planning, and specifically relates to an adversarial imitation learning method for generating a unit combination taking into account the participation of green power units. Background Art
[0002] Safety-constrained unit commitment is a basic tool for clearing the modern electricity spot market. Its goal is to minimize the system operating cost under the premise of satisfying a series of physical and engineering constraints. From a mathematical point of view, it is essentially a large-scale mixed integer linear programming problem. With the increase in the number of green power units connected to the grid, the computational difficulty of the problem has been greatly increased due to its inherent power generation uncertainty. Existing solutions can be roughly divided into heuristic algorithms, mathematical optimization techniques, and data-driven methods. Heuristic algorithms simulate natural or human behavior to find the optimal solution, but when dealing with complex power system constraints, they usually face the problems of large computational burden and insufficient solution stability. In terms of mathematical optimization technology, the early safety-constrained unit commitment problem was mainly solved by Lagrangian relaxation method, and with the rise of commercial solvers, large-scale mixed integer linear programming methods began to be used. Although this method has a solid theoretical foundation and can provide a global optimal solution, it has high computational complexity and strict requirements for accurate modeling. Under the limitations of traditional methods, data-driven methods, especially reinforcement learning, have received more attention. Reinforcement learning has shown great potential in solving non-convex nonlinear problems, but it faces challenges such as lack of prior knowledge guidance and subjective definition of reward function when solving safety-constrained unit commitment problems.
[0003] In summary, the existing methods are gradually showing their shortcomings in meeting the needs of efficient solution of safety-constrained unit combinations under the access of large-scale green power units. Therefore, there is still room for improvement in the existing technologies. Summary of the invention
[0004] Taking into account the shortcomings of the existing technology and the development needs, the purpose of the present invention is to provide an adversarial imitation learning method for generating unit combinations taking into account the participation of green power units, and to directly learn the optimal unit combination strategy through expert strategy guidance, thereby solving the problem that traditional reinforcement learning methods rely on manually formulated reward functions. It has strong practical application significance and provides technical support for promoting the full opening of the electricity spot market and the green electricity trading market.
[0005] The purpose of the present invention can be achieved through the following technical solutions: A unit combination generative adversarial imitation learning method considering the participation of green power units includes the following steps: S1, establish a safety constraint unit combination expert system; S2, establish a safety constraint unit commitment generator network model; S3, establishing a safety constraint unit combination discriminator network model; S4, conducts training of the safety-constrained unit combination generative adversarial imitation learning network model.
[0006] Beneficial effects of the present invention: 1) A generative adversarial imitation learning method for safety-constrained unit commitment considering the participation of green power units is proposed, which can realize online rapid calculation of unit commitment under the interference of multiple uncertain factors; 2) A safety-constrained unit combination expert system was established to directly generate data to guide generative adversarial imitation learning and avoid artificially designed reward functions; 3) A generator neural network loss function based on proximal strategy optimization is proposed to enhance training performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0008] Figure 1 It is a framework diagram of the adversarial imitation learning method for generating a unit combination considering the participation of green power units in the present invention; Figure 2 The invention provides a flow chart for solving the safety constraint unit combination model driven by the power transfer distribution factor method. DETAILED DESCRIPTION
[0009] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0010] like Figure 1 As shown, a unit combination generative adversarial imitation learning method considering the participation of green power units includes the following steps: S1, establish a safety constraint unit combination expert system.
[0011] The security-constrained unit commitment (SCUC) expert system consists of two parts: one is the historical unit commitment operation database, and the other is the database generated by the model-driven method based on perfect prediction information; The data in the historical unit combination operation database comes from the unit combination operation records of the actual power grid, covering information such as the actual output of all units, the start and stop status of thermal power units, the maximum output of green power units, section flow and node voltage.
[0012] The data in the database generated by the model-driven approach based on perfect prediction information comes from solving the following optimization model: Objective function: ; Constraint 1: ; Constraint 2: ; Constraint three: ; In the formula, f is the optimization target, m and M represent thermal power unit m and unit M respectively. represents the operating cost associated with thermal power unit m, n and N represent green power unit n and unit N respectively, represents the operating cost associated with green power unit n; , , and The parameters here are the parameters of the abstract model of safety constraint unit combination. The specific settings can refer to the classic model of power system safety constraint unit combination. Indicates the output of thermal power units. Indicates the start and stop status of the thermal power unit; represents the output of the green power unit; represents the physical operation constraints related to thermal power unit m; Represents the physical operating constraints associated with green generator set n.
[0013] For the above safety-constrained unit commitment optimization model, a solution method based on the power transfer distribution factor (PTDF) method is adopted. The flow chart is as follows: Figure 2 As shown, the solution steps are: Step 1, establish a basic model (Base Case) of unit combination mixed integer programming, and solve it based on a commercial solver; Step 2, perform safety verification based on power flow calculation, and go to step 4 if the verification passes, otherwise go to step 3; Step 3, construct power flow constraints that violate safety requirements based on power transfer distribution factors and add them to the basic model of unit combination mixed integer programming in step 1, and go to step 1; Step 4, end; S2, establish a safety constraint unit combination generator network model. The safety constraint unit combination generator network model is a fully connected neural network model, which includes an input layer, a hidden layer, an activation function and an output layer.
[0014] Among them, the input of the input layer of the fully connected neural network model is: ; In the formula, represents the input state space set, Indicates that the green generator set 1: N is The output size at the time, Indicates that the green generator set 1: N is The maximum predicted output at the time, Indicates that thermal power unit 1:M is The output size at the time, Indicates that thermal power unit 1:M is The start and stop status at all times, and They represent the number of shutdown and startup steps required for maintenance of thermal power unit 1:M, Represents the power system The load size at any time, express The node voltage at the moment, express The current level of the line at any moment.
[0015] The output of the output layer of the fully connected neural network model is: ; In the formula, represents the set of output action spaces, For green generator set 1:N The output size at the time, For thermal power unit 1:M The output size at the moment.
[0016] The action space of thermal power units has the following constraints: First, when the output of thermal power unit m , if the number of downtime steps required for maintenance , the upper boundary of the action space is the rated minimum output of the unit, and the lower boundary is 0, otherwise the upper and lower boundaries are both 0; secondly, when the output of the thermal power unit is the rated minimum value, the upper boundary of the action space is the minimum value between the current value plus the climbing limit value and the rated maximum output. If the number of startup steps required for maintenance is , the lower boundary of the action space is 0, otherwise the lower boundary of the action space is the rated minimum output of the unit; thirdly, when the output of the thermal power unit is greater than the rated minimum value, the lower boundary of the action space is the maximum value of the difference between the current value and the climbing value and the rated minimum output, and the upper boundary of the action space is the minimum value of the sum of the current value and the climbing value and the rated maximum output.
[0017] The lower boundary of the green power group action space is 0, and the upper boundary of the green power group action space is the predicted maximum output.
[0018] Considering the uncertainty of green power generation and load forecast, the present invention aims to create a scheduling plan with a certain degree of exploratory nature to avoid the learning of the generator network falling into the local optimum, so the action space set The size is designed to be determined by the following probability density function: ; In the formula, express The probability density function of and express The mean and standard deviation of .
[0019] In order to improve the scheduling strategy generated by the generator network and narrow the gap between the generated strategy and the expert strategy, this paper introduces the objective function of the proximal policy optimization (PPO) algorithm. Based on this, the loss function of the fully connected neural network model is established as follows: ; ; In the formula, represents the expected value function, represents the state-action pairs generated by the safety constraint unit combination generator network model, Indicated in Choose an action when following the current policy and the old policy The probability ratio of represents the input state space set, Represents the discriminator network for The probability of the output; represents the pinch-off function, As parameters, For the current strategy, is the old strategy. The advantage function in the PPO algorithm objective function is the probability value output by the discriminator replaced.
[0020] S3, establish a safety constraint unit combination discriminator network model, where: The safety constraint unit combination discriminator network model is a fully connected neural network model, which includes an input layer, a hidden layer, an activation function and an output layer.
[0021] The input state-action pair of the input layer of the fully connected neural network model , the output probability size of the output layer of the fully connected neural network model The loss function of the fully connected neural network model is: ; Where: represents the expected value function, represents the state-action pairs generated by the safety constraint unit combination generator network model, Represents the state-action pairs generated by the expert system database.
[0022] S4, training the safety constraint unit combination generator network model and the safety constraint unit combination discriminator network model, the training steps include: S41, initialize the weight parameter θ of the safety constraint unit combination generator network model, initialize the weight parameter of the safety constraint unit combination discriminator network model , initialize the total number of training scenes K, counter k=1; S42, sample state-action pairs from the expert system database ; S43, fix the generator network parameters, use the gradient descent method, and perform reverse gradient propagation on the discriminator based on the loss function of the safety constraint unit combination discriminator network model; S44, fix the discriminator network parameters, use the gradient descent method, and perform reverse gradient propagation on the generator based on the loss function of the safety constraint unit combination generator network model; S45, k=k+1, determine whether k is greater than or equal to K, if yes, end; if not, return to S42.
[0023] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0024] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.
Claims
1. A unit combination generative adversarial imitation learning method considering the participation of green power units, characterized in that: The following steps are involved: S1, establish a safety constraint unit combination expert system; S2, establish a safety constraint unit commitment generator network model; S3, establishing a safety constraint unit combination discriminator network model; S4, conducts training of the safety-constrained unit combination generative adversarial imitation learning network model.
2. According to claim 1, a unit combination generation adversarial imitation learning method considering the participation of green power units is characterized in that: In S1, the safety constraint unit commitment expert system consists of two parts: one is a historical unit commitment operation database, and the other is a database generated by a model-driven method based on perfect prediction information; The data in the historical unit combination operation database is derived from the unit combination operation records of the actual power grid, covering the actual output of all units, the start and stop status of thermal power units, the maximum output of green power units, section flow and node voltage; The data in the database generated by the model-driven method based on perfect prediction information comes from solving the following optimization model; Objective function: , Constraint 1: , Constraint 2: , Constraint three: , In the formula, f is the optimization target, m and M represent thermal power unit m and unit M respectively. represents the operating cost associated with thermal power unit m, n and N represent green power unit n and unit N respectively, represents the operating cost associated with green power unit n; , , and As parameters, Indicates the output of thermal power units. Indicates the start and stop status of the thermal power unit. represents the output of the green power unit; represents the physical operation constraints related to thermal power unit m; represents the physical operating constraints associated with green generator set n; The optimization model is solved by a method based on power transfer distribution factors, and the solving steps are as follows: Step 1, establish a basic model of unit combination mixed integer programming, and solve it based on a commercial solver; Step 2, perform a safety check based on power flow calculation, and go to Step 4 if the check passes, otherwise go to Step 3; Step 3, construct a power flow constraint condition that violates the safety requirements based on the power transfer distribution factor and add it to the unit combination model of Step 1, and go to Step 1; Step 4, end.
3. According to claim 1, a unit combination generation adversarial imitation learning method considering the participation of green power units is characterized in that: In S2, the safety constraint unit combination generator network model is a fully connected neural network model, including an input layer, a hidden layer, an activation function and an output layer; Among them, the input of the input layer of the fully connected neural network model is: ； In the formula, represents the input state space set, Indicates that the green generator set 1: N is The output size at the time, Indicates that the green generator set 1: N is The maximum predicted output at the time, Indicates that thermal power unit 1:M is The output size at the time, Indicates that thermal power unit 1:M is The start and stop status at all times, and They represent the number of shutdown and startup steps required for maintenance of thermal power unit 1:M, Represents the power system The load size at any time, express The node voltage at the moment, express The current of the line at the moment; The output of the output layer of the fully connected neural network model is: ； In the formula, represents the set of output action spaces, For green generator set 1:N The output size at the time, Thermal power unit 1: M in The output size at the time; The action space of thermal power units has the following constraints: First, when the output of thermal power unit m , if the number of downtime steps required for maintenance , the upper boundary of the action space is the rated minimum output of the unit, and the lower boundary is 0, otherwise the upper and lower boundaries are both 0; secondly, when the output of the thermal power unit is the rated minimum value, the upper boundary of the action space is the minimum value between the current value plus the climbing limit value and the rated maximum output. If the number of startup steps required for maintenance is , the lower boundary of the action space is 0, otherwise the lower boundary of the action space is the rated minimum output of the unit; thirdly, when the output of the thermal power unit is greater than the rated minimum value, the lower boundary of the action space is the maximum value of the difference between the current value and the climbing value and the rated minimum output, and the upper boundary of the action space is the minimum value of the sum of the current value and the climbing value and the rated maximum output; The lower boundary of the green power unit action space is 0, and the upper boundary of the green power unit action space is the predicted maximum output; Action Space Collection The size is designed to be determined by the following probability density function: ； In the formula, express The probability density function of and express The mean and standard deviation of The loss function of the fully connected neural network model is established as follows: ； ； In the formula, represents the expected value function, represents the state-action pairs generated by the safety constraint unit combination generator network model, Indicates that Choose an action when following the current policy and the old policy The probability ratio of represents the input state space set, Represents the discriminator network for The probability of the output; represents the pinch-off function, As parameters, For the current strategy, For the old strategy.
4. According to claim 1, a unit combination generation adversarial imitation learning method considering the participation of green power units is characterized in that: The safety constraint unit combination discriminator network model is a fully connected neural network model, including an input layer, a hidden layer, an activation function and an output layer; The input state-action pair of the input layer of the fully connected neural network model is , the output probability size of the output layer of the fully connected neural network model is , the loss function of the fully connected neural network model is: ； Where: represents the expected value function, represents the state-action pairs generated by the safety constraint unit combination generator network model, Represents the state-action pairs generated by the expert system database.
5. According to claim 1, a unit combination generation adversarial imitation learning method considering the participation of green power units is characterized in that: In S4, the safety constraint unit combination generation adversarial imitation learning network model training is performed, and the steps are: S41, initialize the network parameters θ of the safety constraint unit combination generator, and initialize the network parameters of the safety constraint unit combination discriminator , initialize the total number of training scenes K, counter k=1; S42, sample state-action pairs from the expert system database ; S43, fix the generator network parameters, use the gradient descent method, and perform reverse gradient propagation on the discriminator based on the loss function of the safety constraint unit combination discriminator network model; S44, fix the discriminator network parameters, use the gradient descent method, and perform reverse gradient propagation on the generator based on the loss function of the safety constraint unit combination generator network model; S45, k=k+1, determine whether k is greater than or equal to K, if yes, end; if not, return to S42.