Power system unit commitment optimization method and system based on multi-agent reinforcement learning
Through multi-intelligent reinforcement learning model and clustering method, the problem of unit combination optimization of power system is solved, and high-quality scheduling solutions are quickly obtained, the solution efficiency and prediction stability are improved, and the low efficiency and insufficient safety in the existing technology are solved. The problem.
Patent Information
- Application Number
- CN202410947824.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-26
AI Technical Summary
The existing technology is difficult to obtain high-quality power system unit scheduling solutions in a short time, and deep learning technology cannot guarantee the reliability and safety of predictions, lacks a clear model understanding and interpretation mechanism, and the solution efficiency of traditional solvers is low.
The power system unit combination optimization method based on multi-intelligent reinforcement learning is adopted. By training the multi-intelligent reinforcement learning model, the unit state change value and prediction value are obtained, and the power system unit combination optimization model is optimized to solve the power system unit combination optimization model. The clustering method divides traditional units into multiple categories, uses new energy units as a single category, makes decisions to quickly obtain scheduling plans, and performs safety verification to improve the stability and safety of prediction.
It realizes rapid and efficient acquisition of the power system unit combination optimization solution, improves the problem solving efficiency and optimization ability, enhances the stability and safety of prediction, and ensures the efficient and stable operation of the power system.
Smart Images

Figure CN119026790A_ABST
Abstract
Description
Technical Field
[0001] This case involves the field of power system optimization and dispatching, and in particular, involves a method and system for optimizing power system unit combination based on multi-agent reinforcement learning. Background Art
[0002] As a key infrastructure for the operation of modern society, the stable operation of the power system is crucial to ensuring various social activities. In the power system, the unit combination problem is particularly critical. It involves how to reasonably configure and dispatch various generating units to ensure the efficient and stable operation of the system under various loads and operating conditions. Its essence is a mixed integer programming problem. However, the traditional mixed integer programming method to directly solve the unit combination problem is difficult to obtain a high-quality unit scheduling solution in a short time.
[0003] With the application of ultra-large-scale integrated circuit computers, computer computing power continues to increase. Collecting, organizing and analyzing historical data has become an important research method. The rapid development of artificial intelligence technology has also opened up new paths in the field of power system optimization and dispatching.
[0004] Although current artificial intelligence technology has made some progress in the field of power system dispatching, it still faces multiple challenges when dealing with complex areas such as power system unit combinations. First, existing deep learning technology cannot guarantee the reliability and safety of the predicted start and stop plans, and lacks the design of safety verification strategies. Secondly, current artificial intelligence technology lacks a clear model understanding and interpretation mechanism, which makes it difficult for decision makers to deeply understand the working mechanism within the network, thereby weakening the credibility of dispatching decisions. Third, due to the large-scale access of new energy units to the power grid, the variables of unit combination decisions have increased rapidly, and the efficiency of traditional solvers has been greatly reduced. Summary of the invention
[0005] The purpose of this case is to propose a method and system for optimizing the unit combination of a power system based on multi-agent reinforcement learning, which is used to solve the problem that it is difficult for existing technologies to obtain high-quality unit scheduling solutions in a short time.
[0006] To achieve the above-mentioned purpose, this case adopts a method for optimizing the unit combination of a power system based on multi-agent reinforcement learning, including the following steps: obtaining the unit state corresponding to each agent at the current moment, and taking the power system state at the current moment as the auxiliary state, inputting the trained multi-agent reinforcement learning model, and obtaining the initial solution of the power system unit combination optimization model from the output unit state change value and unit state prediction value at the next moment, and then using the initial solution to solve the power system unit combination optimization model for the optimal solution; wherein, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model, and each agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and classifies traditional units by clustering, and units of the same classification have the same state.
[0007] In the above technical solution, the training of the multi-agent reinforcement learning model includes the following steps: the state of the new energy unit at time t is recorded as S r,t , the state of the traditional unit at time t is recorded as S g,t , the power system state at time t is recorded as S e,t The power system state at time t includes the actual total active power of the load at time t and the system node characteristics, wherein the system node characteristics are the node loads of the system nodes from time t to time t+k, where k is a set value; the resource abandonment rate of the new energy unit at time t for the next time is taken as the new energy unit state change value A r,t The predicted value of the output change of the traditional unit at the next moment, the predicted value of the upward spin reserve rate change, and the predicted value of the downward spin reserve rate change at time t are recorded as the traditional unit state change value A g,t ; During training, the first agent is based on the state S of the new energy unit r,t , in the power system state S e,t With the help of r,t , get the new energy unit status S r,t+1 , power system status S e,t+1 ; During training, the second agent is based on the traditional unit state S g,t , in the power system state S e,t With the help of g,t , get the traditional unit status S g,t+1 , power system status S e,t+1 , and in the new energy unit state S r,t+1 , Traditional unit status S g,t+1 , power system status S e,t+1 The state change value A of the traditional unit is g,t Perform safety check and obtain updated traditional unit status S′ g,t+1 ; Among them, for the power system state S e,t+1, new energy unit state change value A r,t , New energy unit status S r,t+1 , Traditional unit state change value A g,t , Traditional unit status S g,t+1 Maximize the joint reward of the first agent and the second agent.
[0008] In the above technical solution, in one implementation of clustering and classifying traditional units, the steps include: obtaining traditional unit data, the characteristics of each unit data include the minimum output of the traditional unit, the maximum output of the traditional unit, the coefficients of the power generation cost function of the unit, the minimum shutdown time of the unit, and the minimum startup time of the unit; randomly selecting K units as initial cluster centers, and using the Euclidean distance formula to calculate the distance from each unit to the K cluster centers in the feature space, so as to assign each unit to the class corresponding to the cluster center with the smallest distance; for each cluster of the K clusters, recalculate its cluster center position: μ k is the kth cluster center, C k is the kth category.
[0009] In the above technical solution, one implementation method of the security verification is as follows:
[0010]
[0011] Where: is the predicted value of the output change of unit g at time t for the next moment, T t g is the continuous on or off time of unit g up to time t, is the minimum startup time of unit g, ΔP g,t is the change in output of unit g between time t and the next time, is the maximum value of the downward ramp of unit g power, a is the scaling factor, is the maximum value of the unit g power climbing upward, is the minimum shutdown time of unit g, is the predicted value of the change in the spin-up reserve rate of all traditional units at time t for the next moment, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of all traditional units at time t to the next moment, is the downspin reserve rate of the traditional unit g at time t.
[0012] In the above technical solution, in one calculation method of the power imbalance penalty amount, the steps include: calculating the total output forecast value of all units at time t+1 Calculate the downspin reserve rate of the traditional unit at time t+1 Calculate the spin-up reserve rate of the traditional unit at time t+1 If P t+1 >D t+1 and Then: ΔPP t+1 =(P t+1 -D t+1 )×P f1 If P t+1 >D t+1 and but: If P t+1 <D t+1 and Then: ΔPP t+1 =-(P t+1 -D t+1 )×P f1 If P t+1 <D t+1 and but: In the above formula: D t+1 is the actual total active power of the load in period t+1, P r,t+1 is the active power of the new energy unit r in the period t+1, P g,t+1 is the active power of the traditional unit g in the period t+1, N g is the total number of traditional units, is the downspin reserve rate of the traditional unit g at time t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of the traditional unit g at time t to the next moment, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t to the next moment, P f1 is the first penalty coefficient of the reserve part, P f2 is the second penalty coefficient for the portion exceeding the reserve amount.
[0013] In the above technical solution, one method of calculating the line power flow penalty includes: if PTDF(P t+1 -D t+1 )>P l,max , then: load =1000×PTDF(P t+1 -D t+1 )-P l,max ; If PTDF(P t+1 -D t+1 )<P l,min , then: load =P l,min -1000×PTDF(P t+1 -Dt+1 ) ; Calculate the power flow penalty of line l at time t+1 Where: PTDF is the power transfer distribution factor, which represents the power flow change caused by the power injection change at a certain node in the system on a specific transmission line, P t+1 is the total output of all units at time t+1, D t+1 is the actual total active power of the load in period t+1, P l,max is the maximum flow through line l, P l,min is the minimum flow through line l, Υ l,load is the violation of line l, ρ l is the penalty coefficient for line l violation, N bus is the total number of lines.
[0014] In one implementation of the above technical solution, the second agent updates the current state using the following state transfer function:
[0015]
[0016] Where: is the predicted value of the output change of the traditional unit g at time t for the next moment, T t g is the continuous on or off time of the traditional unit g at time t, is the minimum output of the traditional unit g, P g,t is the active power of the traditional unit g in period t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t for the next moment, is the downspin reserve rate of the traditional unit g at time t, It is the predicted value of the change of the downward spin reserve rate of the traditional unit g at time t to the next moment.
[0017] In one implementation of the above technical solution, the common reward of the first agent and the second agent is calculated using the following function:
[0018]
[0019] Where: α is the penalty coefficient corresponding to the sum of operating cost and start-stop cost, f g (P g,t+1 ) is the operating cost function of the traditional unit at time t+1, is the startup cost of the traditional unit g, u g,t+1 is the startup signal of the traditional unit g at time t+1, is the downtime cost of the traditional unit g, v g,t+1is the shutdown signal of the traditional unit g at time t+1, ω is the penalty coefficient corresponding to the spinning reserve term, c r is the unit price of the spin-up reserve of the traditional unit g, is the spin-up reserve rate of the traditional unit g at time t+1, is the downspin reserve rate of the traditional unit g at time t+1, c d is the unit price of the conventional unit g's reserve capacity, and the penalty coefficient corresponding to the β-breaking cost, is the power flow penalty of line l at time t+1, γ is the penalty coefficient corresponding to the power flow penalty of the line, APP t+1 is the power imbalance penalty at time t+1, λ is the penalty coefficient corresponding to the resource abandonment penalty item of the new energy unit, η w is the proportion of new energy units output in all units output, The total penalty value of the abandoned resource amount of the new energy unit r at time t+1, G is the set of traditional units, R is the set of new energy units, L is the set of lines, D t+1 It is the actual total active power of the load in the period t+1.
[0020] In one implementation of the above technical solution, the power system unit combination optimization model is to minimize the following cost function while satisfying system power balance constraints, system reserve constraints, network maximum transmission capacity constraints, traditional unit operation constraints, and new energy unit operation constraints:
[0021]
[0022] Where: P g,t is the active power of the traditional unit g in period t, u g,t is the startup signal of the traditional unit g at time t, v g,t is the shutdown signal of the traditional unit g at time t, is the resource abandonment rate of the new energy unit r at time t, is the spin-up reserve rate of the traditional unit g at time t, is the downspin reserve rate of the traditional unit g at time t.
[0023] In one implementation of the above technical solution, a method for obtaining an initial solution includes the following steps: obtaining a state change value of each new energy unit as an initial solution of a corresponding parameter in a power system unit combination optimization model; obtaining the output of each traditional unit and the upper and lower rotation reserve rate in the state value of each traditional unit as an initial solution of the corresponding parameter in the power system unit combination optimization model; obtaining the output change of each traditional unit after safety correction, obtaining the continuous disconnection time in the state value of each traditional unit after executing the safety correction action, and determining the startup signal and the shutdown signal in the power system unit combination optimization model according to the following rules The initial solution: if the continuous on-off time is greater than 0, the initial solution of the power-on signal is 1, and the initial solution of the power-off signal is 0; if the continuous on-off time is less than 0, the initial solution of the power-off signal is 1, and the initial solution of the power-on signal is 0; if the continuous on-off time is 0, and the output change prediction value is -1, it indicates that the power state is changed from the on state to the off state. At this time, the initial solution of the power-off signal is 1, and the initial solution of the power-on signal is 0; if the continuous on-off time is 0, and the output change prediction value is 1, it indicates that the power state is changed from the off state to the on state. At this time, the initial solution of the power-on signal is 1, and the initial solution of the power-off signal is 0.
[0024] In order to achieve the above technical objectives, this case also proposes a computer-readable storage medium, which stores a computer program that can be loaded by a processor and execute any of the above methods.
[0025] In order to achieve the above technical objectives, this case also proposes a power system unit combination optimization system based on multi-agent reinforcement learning corresponding to the method, and the system includes a power system unit combination optimization model and a multi-agent reinforcement learning model. The system is used to obtain the unit state corresponding to each agent at the current moment, and use the power system state at the current moment as an auxiliary state, input the trained multi-agent reinforcement learning model, obtain the initial solution of the power system unit combination optimization model from the output unit state change value and unit state prediction value at the next moment, and then use the initial solution to solve the power system unit combination optimization model for the optimal solution; wherein, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model, and each agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and traditional units by clustering, and the units of the same classification have the same state.
[0026] The beneficial technical effects of this case are: according to the power system unit combination optimization problem to be solved, the traditional units are divided into multiple categories through clustering methods, and the new energy units are regarded as a separate category. Each category corresponds to an intelligent agent, and multi-agent reinforcement learning is used for decision-making to quickly and efficiently obtain a feasible scheduling plan, so that the final optimization goal can be obtained by using traditional solvers, which improves the problem solving efficiency and optimization ability. Furthermore, the actions of the intelligent agent are safely checked and corrected to improve the stability and safety of artificial intelligence in predicting the start and stop of the power system and the output forecast. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0028] Figure 1 , one Schematic diagram of the structure of the power system unit combination optimization model based on multi-agent reinforcement learning assisted solution in this implementation method.
[0029] Figure 2 , one Schematic diagram of the initial solution obtained based on multi-agent reinforcement learning in one implementation.
[0030] Figure 3 , one Schematic diagram of the optimal solution obtained based on multi-agent reinforcement learning in one implementation. DETAILED DESCRIPTION
[0031] The meaning of the parameters in this case is shown in Table 1. If the parameter subscript changes from time t to time t+1, the change in the parameter meaning only involves the change in the value at that time.
[0032] Table 1
[0033]
[0034]
[0035]
[0036]
[0037]
[0038] In view of the problem that it is difficult for the existing technology to obtain high-quality unit dispatching solutions in a short time, this case provides an acceleration method for optimizing the combination of power system units based on multi-agent reinforcement learning. According to this method, the corresponding device or system can be realized. In the method, the unit state corresponding to each agent at the current moment is obtained, and the power system state at the current moment is used as an auxiliary state, and the trained multi-agent reinforcement learning model is input. The initial solution of the power system unit combination optimization model is obtained from the output unit state change value and unit state prediction value at the next moment, and then the initial solution is used to solve the power system unit combination optimization model for the optimal solution; wherein, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model, and each agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and traditional units by clustering classification. The units of the same classification have the same state. The optimal solution is used to optimize the dispatching of the power system. The method improves the problem solving efficiency and optimization capability.
[0039] The following will be combined with the accompanying drawings to clearly and completely describe how to implement the power system unit combination optimization model based on multi-agent reinforcement learning to accelerate the solution. Obviously, the described implementation method is only a part of the implementation method of this case, not all of the implementation methods. Based on the implementation methods in this case, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0040] 1. Mixed integer programming model for power system unit commitment
[0041] The power system in this case contains traditional units and new energy units. In the following example, the traditional units are thermal power units, and the new energy units include solar power units and wind power units.
[0042] According to the units and working status, the cost function C of power system operation cost and new energy cost reduction is constructed. t , see formula (1).
[0043]
[0044] In formula (1), For running costs, Cut costs for new energy.
[0045] In formula (1), u g,t In one embodiment, if unit g is in the off state at time t-1 and is turned on at time t, then u g,t =1, otherwise u g,t =0.
[0046] In formula (1), vg,t In one embodiment, if the unit g is in the on state at time t-1 and is turned off at time t, then v g,t =1, otherwise v g,t =0.
[0047] By making the above cost function C t Get the minimum value and determine P g,t 、u g,t 、v g,t , The optimal dispatching scheme of the unit combination of traditional units and new energy units is obtained by taking the value of . According to formula (1), the solution of each unknown number is a two-dimensional matrix about the unit and the cycle.
[0048] The above cost function C t When the minimum value is obtained, the power system can still work normally, and the normal operation of the power system needs to meet the system power balance constraint, system reserve constraint, network maximum transmission capacity constraint and unit operation constraint. Among them, the unit operation constraint is composed of traditional unit operation constraint and new energy unit operation constraint.
[0049] In formula (1), the cost function f of the unit is g (·), in some embodiments, is a convex quadratic function. In order to improve the solution efficiency, it is set as a linear function. For example, for a traditional unit g, the cost function f g (·) Specifically:
[0050] f g (x) = a g x 2 +b g x+c g (2)
[0051] (1) Traditional unit operation constraints
[0052] The unit operation constraints specifically include unit operation status logic constraints, output limit constraints, minimum start and stop time constraints, and maximum climbing rate constraints.
[0053] (1.1) Logical constraints on unit operation status
[0054] The unit operating state logical constraint represents the operating state x of the generator unit g in the system. g,t With its boot status u g,t , shutdown state v g,t For the relationship between them, see (3)-(4):
[0055] u g,t +v g,t ≤1, g∈G, t∈T (3)
[0056] v g,t -v g,t-1 =u g,t -v g,t , g∈G, t∈T (4)
[0057] If the generator set g starts at time t, then v g,t-1 =0,x g,t =1, then u g,t =1,v g,t = 0. If the generator set g is shut down at time t, then x g,t-1 =1,v g,t =0, then u g,t =0, v g,t =1.
[0058] (1.2) Output Limitation Constraints
[0059] The output limit constraint requires that the traditional unit operates between the corresponding upper and lower limits of the output and cannot operate outside the output range, which can be expressed as formula (5):
[0060]
[0061] The maximum output limit depends on the rated capacity of the unit design and manufacturing. The rated capacity is the maximum capacity that can be continuously output during the entire normal service life. The minimum output limit, also known as the technical minimum output, refers to the minimum output that the unit must produce under the technical conditions so that the boiler can burn stably and the generator can continue to operate stably.
[0062] (1.3) Minimum start and stop time constraints
[0063] Since the start and stop of thermal power units are realized by the start and stop of boilers and steam turbines, frequent start and stop of boilers will cause damage to the boiler pipe joints, and frequent start and stop of steam turbines will cause the shaft to bend, which will greatly reduce the life of thermal power units. In order to reduce parts wear and mechanical fatigue effects, thermal power units cannot be started and stopped continuously in a short period of time, so a minimum start and stop constraint is set.
[0064] The minimum start-stop time constraint requires that the unit should not be in a continuous on / off state for less than a certain time limit. In one implementation, its modeling is shown in equations (6) and (7):
[0065]
[0066] In equations (6) and (7), k is a time variable.
[0067] (1.4) Maximum climbing rate constraint
[0068] The maximum ramp rate constraint means that when the unit is running continuously, the output change between two consecutive adjacent periods cannot exceed a certain threshold. The modeling is as follows:
[0069] ΔP g,t =P g,t+1 -P g,t
[0070]
[0071] Generally, the climbing speed of thermal power units is 10% to 20% of rated power per hour, while the climbing speed of gas and oil units is faster, reaching 50% to 100% of rated power per hour.
[0072] (2) Operation constraints of new energy units
[0073]
[0074] Formulas (9) and (10) represent the output limits of new energy generating units.
[0075] (3) System power balance constraints
[0076] The power balance of the power system refers to the balance between power supply and demand, which requires that the total power generation and total load of the power system should be balanced at all times, which can be expressed as:
[0077]
[0078] (4) System backup constraints
[0079] The spinning reserve usage of thermal power units refers to the reserve capacity reserved to balance the load fluctuation of the power grid and the load forecast error. Usually, the spinning reserve is the sum of the difference between the current output of the synchronous units in the system and their maximum available output, which can be expressed as:
[0080]
[0081] If unit g is turned on, then x g,t =1, otherwise x g,t =0.
[0082] In order to balance the instability of the output of new energy units, the constraints of upward and downward spin standby are as follows:
[0083]
[0084] (5) Maximum network transmission capacity constraints
[0085] The power grid is a power transmission network, and its physical boundary is the maximum transmission capacity boundary of the line. The impact on the consumption of new energy is mainly due to the insufficient maximum transmission capacity and line congestion. The maximum transmission capacity boundary of the network is usually modeled by DC power flow, as shown below. The maximum transmission capacity boundary of the line means that the power flowing through the line cannot exceed a certain threshold, and is usually modeled by DC power flow as follows:
[0086] P l,min ≤P l,t ≤P l,max , l∈L, t∈T (16)
[0087] As can be seen from the above, the model composed of the cost function C and its constraints is used as the power system unit combination optimization model in this case. This model is a mixed integer linear optimization (MILP) problem, and it is difficult to obtain a high-quality unit dispatch solution in a short time. If an initial feasible solution can be obtained, it will help speed up the solution of the problem. Therefore, this case proposes to obtain an initial feasible solution of the model based on multi-agent reinforcement learning.
[0088] Next, we will introduce how to use multi-agent reinforcement learning to obtain the initial feasible solution of the power system unit combination optimization model.
[0089] (II) Traditional unit clustering
[0090] According to the scale of power generation, the new energy units are classified into a separate category. For traditional units, the traditional units of the system are classified using a clustering method based on the power system lines and unit parameters. In one implementation, a clustering method is used to classify them according to the scale cost of their power generation. It should be noted that the new energy units can also be clustered and classified as needed.
[0091] The traditional units of the system are classified using the clustering method. The steps include:
[0092] S21. Acquire traditional unit data, where the characteristics of each unit data include the minimum output of the traditional unit, the maximum output of the traditional unit, various coefficients of the power generation cost function of the unit, the minimum shutdown time of the unit, and the minimum startup time of the unit.
[0093] Exemplary, traditional unit data m is the total number of traditional unit data.
[0094] S22, for the characteristic data of the traditional unit X g Perform preprocessing. Preprocessing includes removing outliers, standardizing and normalizing.
[0095] S23. Based on the preprocessed feature data, a clustering method is used to divide the unit into a predetermined number of clusters to ensure dimensionless feature effects.
[0096] The number of units with similar characteristics is identified by the elbow method, thereby obtaining the optimal unit classification result. In one implementation, the clustering method uses KsMeans, and k is 3.
[0097] In KsMeans, K units are randomly selected as the initial cluster centers, and the Euclidean distance formula is used to calculate the distance from each unit to the K cluster centers in the feature space, and the units are assigned to the class corresponding to the cluster center with the smallest distance. For each cluster, the position of its cluster center is recalculated, and the distance between the unit and its cluster center in the feature space is iterated to achieve the minimum goal:
[0098]
[0099] 3. Markov Decision Process
[0100] According to the above classification, an agent is created for each category, in which the new energy unit is divided into an agent separately. The agent corresponding to the new energy unit is used as the first agent, and the agent corresponding to the traditional unit is used as the second agent. The units of the same classification have the same status, that is, the same type of units execute the start-stop strategy or output strategy corresponding to the same agent.
[0101] The state space of the agent is composed of the unit state and the power system state. The power system state is composed of the actual total active power of the load and the system node characteristics. The system node characteristics include the node load situation at the future time, where the future time period is selected as the minimum start and stop time of all units. This feature is used as the node state variable, and its value is a prediction based on the corresponding historical data. The unit characteristics are related to the corresponding unit type.
[0102] For a traditional unit, its unit status may include the unit's continuous start and stop time, the unit's output at the next moment, the unit's upward spin reserve rate, and the downward spin reserve rate. In one embodiment, the state of the traditional unit at time t is recorded as S g,t , The state space of the second agent at time t is expressed as follows:
[0103]
[0104] For new energy units, such as wind energy and / or solar energy, their state space may include the actual output of the new energy unit after abandoning resources. In one embodiment, the state of the new energy unit at time t is denoted as S r,t , S r,t = {P r,t}, the state space of the first agent is represented as follows:
[0105]
[0106] The above system node characteristics are the node loads of the system nodes from time t to time t+k, where k is a set value. The actual total active power of the load and the system node characteristics are updated with time. Adding the actual total active power and system node characteristics of the load at time t to the state space plays an auxiliary role in improving the accuracy of the unit characteristics at time t+1 and the action prediction at time t.
[0107] The state change value from time t to time t+1 is taken as the action. For the traditional unit, the state change value A of the traditional unit at time t g,t Including the predicted value of the output change of the traditional unit at the next moment, the predicted value of the upward spin reserve rate change, and the predicted value of the downward spin reserve rate change at time t, the action space is:
[0108]
[0109] In one implementation, the predicted value of the change in the spin-up reserve rate of all conventional units at time t for the next time is The predicted value of the change in the downspin reserve rate of all traditional units at time t for the next moment All of them are between [-0.02, 0.03]. The predicted output change of all traditional units at time t for the next moment is Between [-1, 1]. It is to integrate the start-stop decision with the unit output decision to achieve the unit output change amount that continuously processes the discrete problem. The part greater than 0.8 and less than -0.8 is set as the opening and closing decision, and the remaining values are regarded as the coefficients for output relaxation adjustment of each unit based on the output of the previous moment.
[0110] For the new energy unit, the state change value A of the new energy unit at time t r,t There is only one action in the action space, namely: the predicted value of the resource abandonment rate of the new energy unit r at time t for the next moment It is between [0, 1].
[0111] 4. Multi-agent reinforcement learning
[0112] First, a multi-agent reinforcement learning model is established to solve the above-mentioned power system unit combination optimization model, and a corresponding number of decision-making agents are constructed. Each agent corresponds to a unit classification in the power system. As mentioned above, the unit classification classifies new energy units as one category and traditional units by clustering. Units of the same classification have the same status.
[0113] Design independent Actot and Critic networks for each agent. Initialize network parameters and define training hyperparameters such as optimizer and learning rate. The optimizer is used to update network parameters. Initialize the experience replay pool, which is a buffer used to store experience samples of the agent's interaction with the environment. Initialize the environment for each round, and use the current strategy and state S at each time step. t Select Action A t , the agent gets a new state S from the environment t+1 and reward R t , will (S t , A t , R t , S t+1 ) quadruple is stored in the experience replay pool. When the experience replay pool reaches the upper limit, a batch of samples are randomly selected, the Q value is calculated using the target network, the Critic network is updated using the minimization loss function, and the Actor network is updated using policy gradient descent. Then, the target network parameters are soft-updated based on the above two networks until the final round is reached.
[0114] In one implementation, the Actor network consists of three parts: the input is divided into two parts of the network, the first part consists of EGAT and LSTM, and the second part consists of a three-layer fully connected neural network. The output features of the two are combined and input into the third part of the network consisting of three layers of fully connected layers, that is, three layers of fully connected layers to obtain the final result. For the Critic network, its structural setting is basically consistent with the current Actor network, with the purpose of sharing parameters, reducing the calculation amount of the model, and saving computing resources.
[0115] In one implementation, the experience replay pool uses a queue to store information such as status, action, reward, next status, and completion flag.
[0116] Next, according to the parameters of the unit and power system, an environment for interaction with the intelligent group is constructed. In the environment, the action safety check of the intelligent agent's decision is performed to ensure the safety of the strategy, and a suitable reward function and state transfer function are designed.
[0117] Specifically, for each scheduling period t, all decision variables are corrected to a safe action space range according to the unit constraints. In one embodiment, the action verification is to correct the action decision variables according to the minimum start and stop time and climbing constraints of the unit, and the maximum and minimum output. In one embodiment, according to the unit constraints, each unit action Perform the following safety checks:
[0118]
[0119] An exemplary value of the scaling factor a is 0.9. When it is -1, it means that the traditional unit g changes from the on state to the off state at time t. At this time, the off signal v g,t =1, power-on signal u g,t =0; when action When it is 1, it means that the traditional unit g changes from the shutdown state to the startup state at time t. At this time, the startup signal u g,t =1, shutdown signal v g,t =0.
[0120] The following safety checks are performed on the predicted values of the change in the unit's upward and downward spin reserve rates:
[0121]
[0122] In formulas (21)-(23), when the conditions on the right are met, the variables on the left are updated. For example, in formula (22), if If established, The value of is updated to
[0123] The action after action verification is a safe action, which solves the problem of lack of reliability and safety of artificial intelligence, and can ensure that the unit meets the minimum breaking time constraint, climbing constraint, maximum and minimum output constraint, and standby constraint, that is:
[0124]
[0125] In the environment, the unit output balance constraint and line flow constraint are added as penalty items to the reward function, which also includes the unit's operating cost and start-up and shutdown cost at a certain moment. Considering the introduction of new energy wind turbines and photovoltaic units, in order to meet load demand and power supply, the power system usually adopts spinning reserve, wind power restriction and other measures to cope with the volatility and uncertainty caused by new energy units.
[0126] Therefore, the reward function includes wind power, photovoltaic power generation abandonment penalty items, and spinning reserve penalty items. In one embodiment, the reward function in the environment consists of seven parts: the sum of the operating cost and the start-stop cost of the scheduling period t, the spinning reserve penalty, the line flow penalty, the power imbalance penalty, and the abandoned resource penalty of the new energy unit.
[0127] In this case, all agents share the same reward function, which is denoted by r t , then:
[0128]
[0129] In formula (28), the proportion of the output of new energy units in all units is
[0130] In one embodiment, the power imbalance penalty ΔPP t+1 Adopt the segmented calculation rule. Let the total output forecast value of all units at time t+1 be The spin-up reserve rate of traditional units at time t+1 The spin-up reserve rate of traditional units at time t+1 Then the power imbalance penalty ΔPP t+1 The segment calculation rules are as follows:
[0131] (4.1.1) If P t+1 >D t+1 and but:
[0132] ΔPP t+1 =(P t+1 -D t+1 )×P f1 (29)
[0133] (4.1.2) If P t+1 >D t+1 and but:
[0134]
[0135] (4.1.3) If P t+1 <D t+1 and but:
[0136] ΔPP t+1 =-(P t+1 -D t+1 )×P f1 (31)
[0137] (4.1.4) If P t+1 <D t+1 and but:
[0138]
[0139]
[0140] In one embodiment, if the power imbalance penalty ΔPP t+1 If the reserve amount is not exceeded, the second penalty coefficient P for the part exceeding the reserve amount will be f2 Set to 6000, its value is the first penalty coefficient P f1 twice as much.
[0141] In one embodiment, the line flow penalty of line l at time t+1 is is the line power flow constraint, which is calculated as follows:
[0142]
[0143] In one embodiment, the line power flow constraint The corresponding penalty coefficient ρ l The value is 0.9.
[0144] In one embodiment, the violation amount Y of line 1 l,load The segmented calculation rules are as follows:
[0145] (4.2.1) If PTDF(P t+1 -D t+1 )>P l,max ,but:
[0146] Υ load =1000×PTDF(P t+1 -D t+1 )-P l,max (34)
[0147] (4.2.2) If PTDF(P t+1 -D t+1 )<P l,min ,but:
[0148] Υ load =P l,min -1000×PTDF(P t+1 -D t+1 ) (35)
[0149] The power transfer distribution factor PTDF in the above-mentioned represents the power flow change caused by the power injection change of a certain node in the system on a specific transmission line. After processing the predicted action, the environment converts the current state according to the transfer function to obtain the state at the next moment. Among them, the total active power of the traditional unit at the next moment is given, and the state transfer function of the continuous start-stop time, the unit output at the next moment, the unit spin-up reserve rate, and the unit spin-down reserve rate is shown in the following formula.
[0150] (4.3.1) Continuous start and stop time
[0151]
[0152] (4.3.2) Unit output at the next moment
[0153]
[0154] (4.3.3) Unit spin-up reserve rate
[0155]
[0156] (4.3.3) Unit downspin rate
[0157]
[0158] For new energy units, the resource abandonment rate of new energy units at the next moment is is the predicted value of the agent, which can be used to obtain the next state of the new energy unit. The agent is trained by interacting with the environment. During training, the action predicted by the agent is input into the environment, and the environment calculates the reward value based on the action of the agent and feeds it back to the agent. Based on the received reward value and the updated environment state, the agent updates the network parameters through the reinforcement learning algorithm until the agent strategy reaches the optimal value. The Adam optimizer is used to update its network parameters. The above interaction cycle is repeated many times, and the agent optimizes its strategy in continuous interaction with the environment.
[0159] In one embodiment, the multi-agent reinforcement learning model training process includes: during training, the first agent is based on the state S of the new energy unit r,t , in the power system state S e,t With the help of r,t , get the new energy unit status S r,t+1 , power system status S e,t+1 During training, the second agent is based on the traditional unit state S g,t , in the power system state S e,t With the help of g,t , get the traditional unit status S g,t+1 , power system status S e,t+1 , and in the new energy unit state S r,t+1 , Traditional unit status S g,t+1 , power system status S e,t+1 The state change value A of the traditional unit is g,t Perform safety check and obtain updated traditional unit status S′ g,t+1 Among them, for the power system state S e,t+1 , new energy unit state change value A r,t , New energy unit status S r,t+1 , Traditional unit state change value A g,t , Traditional unit status S g,t+1 Maximize the joint reward of the first agent and the second agent.
[0160] In one embodiment, in training, the power system state data preferably uses historical real data. Therefore, the power system state data at time t+1 obtained by each intelligent agent at time t is historical real data. In this case, the power system state data is used to assist the action prediction of each intelligent agent, which can improve the accuracy of the prediction.
[0161] At this point, the implementation process of the complete multi-agent reinforcement learning model is as follows: first initialize the network parameters of all agents, and for each agent i, initialize the Actor current network π i 、critic current network Q i 、Actor target network π′ i , critic target network Q′ i , initialize the experience replay pool. For each episode, initialize the environment state S, and for each agent i in each time step according to the current strategy π i and S, choose action a i Then execute the actions of all agents, input the actions into the environment to obtain the reward R and the new state, and convert this group (S, a i , R, S′) are stored in the experience replay pool. After completing all steps, for each agent i, data is randomly collected from the experience replay pool. For each set of collected data, the target network is used to calculate the Q value, and the critic network Q i , update the actor network π i , and finally soft-update the target network parameters:
[0162] π′ i ←τπ i +(1-τ)π′ i (40)
[0163] Q′ i ←τQ i +(1-τ)Q′ i (41)
[0164] When updating the network, the Adam optimizer is used to calculate the gradient of the loss to the model parameters, and the model parameters are updated according to the gradient information to minimize the loss function.
[0165] Finally, the trained deep reinforcement learning model is used to solve the unit combination problem of introducing new energy units. The output unit start and stop and output strategy are input into the solver as the initial feasible solution to the unit combination problem to obtain the final optimal solution. Specifically, deploy an existing solver, such as Gurobi, and build a unit combination solution model in the solver according to the problem objective function and constraints in step 1. Input the data into the trained deep reinforcement model, and input the solution obtained by deep reinforcement learning as the initial solution of hot start into the solver for iteration. Since the deep reinforcement learning model will quickly obtain a feasible solution close to the optimal solution, the iteration process in the solver will be accelerated and the solution time will be shortened.
[0166] In one implementation, when applying the trained multi-agent reinforcement learning model, the unit state corresponding to each agent at the current moment is obtained, and the power system state at the current moment is used as an auxiliary state, and the trained multi-agent reinforcement learning model is input, and the initial solution of the power system unit combination optimization model is obtained from the output unit state change value and unit state prediction value at the next moment, and then the initial solution is used to solve the power system unit combination optimization model for the optimal solution. The optimal solution is used to optimize the dispatch of the power system.
[0167] In one implementation, the initial solution acquisition step includes:
[0168] Obtaining the state change value of each renewable energy unit as the initial solution of the corresponding parameters in the power system unit combination optimization model;
[0169] The output of traditional units and the up-down rotation reserve rate in the state values of each traditional unit are obtained as the initial solutions of the corresponding parameters in the power system unit combination optimization model;
[0170] The output change of each traditional unit after safety correction is obtained, and the continuous disconnection time in the state value of each traditional unit after executing the safety correction action is obtained. The initial solution of the start-up signal and the shutdown signal in the power system unit combination optimization model is determined according to the following rules:
[0171] If the continuous on-off time is greater than 0, the initial solution of the power-on signal is 1, and the initial solution of the power-off signal is 0;
[0172] If the continuous breaking time is less than 0, the initial solution of the shutdown signal is 1, and the initial solution of the startup signal is 0;
[0173] If the continuous breaking time is 0 and the output change prediction value is -1, it means that the power supply has changed from the on state to the off state. At this time, the initial solution of the off signal is 1, and the initial solution of the on signal is 0.
[0174] If the continuous breaking time is 0 and the output change prediction value is 1, it means that the power supply has changed from the shutdown state to the startup state. At this time, the initial solution of the startup signal is 1 and the initial solution of the shutdown signal is 0.
[0175] The terms "first" and "second" mentioned above are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.
[0176] (V) Example Analysis
[0177] In order to verify the effectiveness of the optimization acceleration method of power system unit combination based on multi-agent reinforcement learning, a simulation environment was built based on the improved IEEE-118 node system and the IEEE-30 node system to test its performance. The improved IEEE-118 node includes 5 wind turbines, 6 photovoltaic generators and 45 thermal power units, and the improved IEEE-30 node includes 1 wind turbine and 10 thermal power generators. The scheduling period of the simulation environment is 1 day, and the scheduling interval is 1 hour, which includes 24 scheduling periods.
[0178] Figure 1 The overall model structure of the optimization acceleration method of power system unit combination based on multi-agent reinforcement learning is demonstrated.
[0179] Figure 2 The output start and stop results of the IEEE-30 example using the reinforcement learning model are shown.
[0180] Figure 3 The final result after using the reinforcement learning result as the initial solution through the optimizer is shown.
[0181] Table 1 shows the specific quantitative analysis results of the above example. It can be seen from the table that when the number of units increases to 56, the RL algorithm solves the problem 197 times faster than the traditional solver. Substituting this as the initial solution into the traditional solver greatly shortens the solution time, thereby improving the efficiency of the solution. From the perspective of the cost used, the RL model also performs well. Although its target cost is slightly higher than that of the traditional method, the gap is not obvious and is still within an acceptable range.
[0182] Table 1
[0183]
[0184] Through the description of the above implementation methods, the technical personnel in the relevant field can clearly understand that according to the disclosed method, the corresponding system can be realized. Exemplarily, the system includes a power system unit combination optimization model and a multi-agent reinforcement learning model. The system is used to obtain the unit state corresponding to each agent at the current moment, and use the power system state at the current moment as an auxiliary state, input the trained multi-agent reinforcement learning model, and obtain the initial solution of the power system unit combination optimization model from the output unit state change value and unit state prediction value at the next moment, and then use the initial solution to solve the power system unit combination optimization model for the optimal solution; wherein, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model, and each agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and traditional units by clustering classification, and the units of the same classification have the same state.
[0185] Through the description of the above implementation modes, those skilled in the art can clearly understand that the method or system of the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.
[0186] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.
Claims
1. A method for optimizing power system unit commitment based on multi-agent reinforcement learning. It is characterized in that The method comprises the following steps: Obtain the unit state corresponding to each agent at the current moment, and use the current power system state as the auxiliary state, input the trained multi-agent reinforcement learning model, obtain the initial solution of the power system unit combination optimization model from the output unit state change value and unit state prediction value at the next moment, and then use the initial solution to solve the optimal solution of the power system unit combination optimization model; Among them, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model. Each intelligent agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and traditional units by clustering. Units of the same classification have the same status.
2. The method according to claim 1, It is characterized in that The training of a multi-agent reinforcement learning model consists of the following steps: The state of the new energy unit at time t is recorded as S r,t , the state of the traditional unit at time t is recorded as S g,t , the power system state at time t is recorded as S e,t , the power system state at time t includes the actual total active power of the load at time t and the system node characteristics, the system node characteristics are the node loads of the system nodes from time t to time t+k, k is a set value; The resource abandonment rate of the new energy unit at time t to the next time is taken as the state change value A of the new energy unit r,t The predicted value of the output change of the traditional unit at the next moment, the predicted value of the upward spin reserve rate change, and the predicted value of the downward spin reserve rate change at time t are recorded as the traditional unit state change value A g,t ; During training, the first agent is based on the state S of the new energy unit r,t , in the power system state S e,t With the help of r,t , get the new energy unit status S r,t+1 , power system status S e,t+1 ; During training, the second agent is based on the traditional unit state S g,t , in the power system state S e,t With the help of g,t , get the traditional unit status S g,t+1 , power system status S e,t+1 , and in the new energy unit state S r,t+1 , Traditional unit status S g,t+1 , power system status S e,t+1 The state change value A of the traditional unit is g,t Perform safety check and obtain updated traditional unit status S′ g,t+1 ; Among them, for the power system state S e,t+1 , new energy unit state change value A r,t , New energy unit status S r,t+1 、Conventional unit state change value A g,t , Traditional unit status S g,t+1 Maximize the joint reward of the first agent and the second agent.
3. The method according to claim 1, It is characterized in that The traditional units are clustered and classified, and the steps include: Obtaining traditional unit data, the characteristics of each unit data include the minimum output of the traditional unit, the maximum output of the traditional unit, various coefficients of the power generation cost function of the unit, the minimum shutdown time of the unit, and the minimum startup time of the unit; K units are randomly selected as the initial cluster centers, and the Euclidean distance formula is used to calculate the distance from each unit to the K cluster centers in the feature space, so that each unit is assigned to the class corresponding to the cluster center with the smallest distance; For each of the K clusters, recalculate the cluster center position: μ k is the kth cluster center, C k is the kth category.
4. The method according to claim 2, It is characterized in that The safety verification includes: Where: is the predicted value of the output change of unit g at time t for the next moment, T t g is the continuous on or off time of unit g up to time t, is the minimum startup time of unit g, ΔP g,t is the change in output of unit g between time t and the next time, is the maximum value of the downward ramp of unit g power, a is the scaling factor, is the maximum value of the unit g power climbing upward, is the minimum shutdown time of unit g, is the predicted value of the change in the spin-up reserve rate of all traditional units at time t for the next moment, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of all traditional units at time t to the next moment, is the downspin reserve rate of the traditional unit g at time t.
5. The method according to claim 2, It is characterized in that The calculation steps of the power imbalance penalty include: Calculate the total output forecast value of all units at time t+1 Calculate the downspin reserve rate of the traditional unit at time t+1 Calculate the spin-up reserve rate of the traditional unit at time t+1 If P t+1 >D t+1 and but: ΔPP t+1 =(P t+1 -D t+1 )×P f1 If P t+1 >D t+1 and but: If P t+1 <D t+1 and but: ΔPP t+1 =-(P t+1 -D t+1 )×P f1 If P t+1 <D t+1 and but: In the above formula: D t+1 is the actual total active power of the load in period t+1, P r,t+1 is the active power of the new energy unit r in the period t+1, P g,t+1 is the active power of the traditional unit g in the period t+1, N g is the total number of traditional units, is the downspin reserve rate of the traditional unit g at time t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of the traditional unit g at time t to the next moment, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t to the next moment, P f1 is the first penalty coefficient of the reserve part, P f2 is the second penalty coefficient for the portion exceeding the reserve amount.
6. The method according to claim 2, It is characterized in that The calculation of line flow penalty includes: If PTDF(P t+1 -D t+1 )>P l,max ,but: ϒ load =1000×PTDF(P t+1 -D t+1 )-P l,max ; If PTDF(P t+1 -D t+1 )<P l,min ,but: ϒ load =P l,min -1000×PTDF(P t+1 -D t+1 ); Calculate the power flow penalty of line l at time t+1 Where: PTDF is the power transfer distribution factor, which represents the power flow change caused by the power injection change at a certain node in the system on a specific transmission line, P t+1 is the total output of all units at time t+1, D t+1 is the actual total active power of the load in period t+1, P l,max is the maximum flow through line l, P l,min is the minimum flow through line l, Υ l,load is the violation of line l, ρ l is the penalty coefficient for line l violation, N bus is the total number of lines.
7. The method according to claim 2, It is characterized in that The second agent updates the current state using the following state transfer function: Where: is the predicted value of the output change of the traditional unit g at time t for the next moment, T t g is the continuous on or off time of the traditional unit g at time t, is the minimum output of the traditional unit g, P g,t is the active power of the traditional unit g in period t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t for the next moment, is the downspin reserve rate of the traditional unit g at time t, It is the predicted value of the change of the downward spin reserve rate of the traditional unit g at time t to the next moment.
8. The method according to claim 2, It is characterized in that The common reward of the first agent and the second agent is calculated using the following function: Where: α is the penalty coefficient corresponding to the sum of operating cost and start-stop cost, f g (P g,t+1 ) is the operating cost function of the traditional unit at time t+1, is the startup cost of the traditional unit g, u g,t+1 is the startup signal of the traditional unit g at time t+1, is the downtime cost of the traditional unit g, v g,t+1 is the shutdown signal of the traditional unit g at time t+1, ω is the penalty coefficient corresponding to the spinning reserve term, c r is the unit price of the spin-up reserve of the traditional unit g, is the spin-up reserve rate of the traditional unit g at time t+1, is the downspin reserve rate of the traditional unit g at time t+1, c d is the unit price of the conventional unit g's reserve capacity, and the penalty coefficient corresponding to the β-breaking cost, is the power flow penalty of the line at time t+1, γ is the penalty coefficient corresponding to the power flow penalty of the line, ΔPP t+1 is the power imbalance penalty at time t+1, λ is the penalty coefficient corresponding to the resource abandonment penalty item of the new energy unit, η w is the proportion of new energy units output in all units output, The total penalty value of the abandoned resource amount of the new energy unit r at time t+1, G is the set of traditional units, R is the set of new energy units, L is the set of lines, D t+1 It is the actual total active power of the load in the period t+1.
9. The method according to claim 1, It is characterized in that The power system unit combination optimization model is to minimize the following cost function while satisfying the system power balance constraint, system reserve constraint, network maximum transmission capacity constraint, traditional unit operation constraint, and new energy unit operation constraint: Where: P g,t is the active power of the traditional unit g in period t, u g,t is the startup signal of the traditional unit g at time t, v g,t is the shutdown signal of the traditional unit g at time t, is the resource abandonment rate of the new energy unit r at time t, is the spin-up reserve rate of the traditional unit g at time t, is the downspin reserve rate of the traditional unit g at time t.
10. The method according to claim 1, It is characterized in that The initial solution acquisition steps include: Obtaining the state change value of each renewable energy unit as the initial solution of the corresponding parameters in the power system unit combination optimization model; The output of traditional units and the up-down rotation reserve rate in the state values of each traditional unit are obtained as the initial solutions of the corresponding parameters in the power system unit combination optimization model; The output change of each traditional unit after safety correction is obtained, and the continuous disconnection time in the state value of each traditional unit after executing the safety correction action is obtained. The initial solution of the start-up signal and the shutdown signal in the power system unit combination optimization model is determined according to the following rules: If the continuous on-off time is greater than 0, the initial solution of the power-on signal is 1, and the initial solution of the power-off signal is 0; If the continuous breaking time is less than 0, the initial solution of the shutdown signal is 1, and the initial solution of the startup signal is 0; If the continuous breaking time is 0 and the output change prediction value is -1, it means that the power supply is switched from the on state to the off state. At this time, the initial solution of the off signal is 1, and the initial solution of the on signal is 0. If the continuous breaking time is 0 and the output change prediction value is 1, it means that the power supply has changed from the shutdown state to the startup state. At this time, the initial solution of the startup signal is 1 and the initial solution of the shutdown signal is 0.
11. A computer-readable storage medium, Features: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 10.
12. A power system unit combination optimization system based on multi-agent reinforcement learning, Features: The system is used to obtain the unit state corresponding to each agent at the current moment, and use the current power system state as an auxiliary state, input the trained multi-agent reinforcement learning model, obtain the initial solution of the power system unit combination optimization model from the output unit state change value and unit state prediction value at the next moment, and then use the initial solution to solve the power system unit combination optimization model for the optimal solution; Among them, the multi-agent reinforcement learning model is constructed based on the power system unit combination optimization model. Each intelligent agent corresponds to a unit classification in the power system. The unit classification classifies new energy units as one category and traditional units by clustering. Units of the same classification have the same status.
13. The system according to claim 12, It is characterized in that Multi-agent reinforcement learning model training includes: The state of the new energy unit at time t is recorded as S r,t , the state of the traditional unit at time t is recorded as S g,t , the power system state at time t is recorded as S e,t , the power system state at time t includes the actual total active power of the load at time t and the system node characteristics, the system node characteristics are the node loads of the system nodes from time t to time t+k, k is a set value; The resource abandonment rate of the new energy unit at time t to the next time is taken as the state change value A of the new energy unit r,t The predicted value of the output change of the traditional unit at the next moment, the predicted value of the upward spin reserve rate change, and the predicted value of the downward spin reserve rate change at time t are recorded as the traditional unit state change value A g,t ; During training, the first agent is based on the state S of the new energy unit r,t , in the power system state S e,t With the help of r,t , get the new energy unit status S r,t+1 , power system status S e,t+1 ; During training, the second agent is based on the traditional unit state S g,t , in the power system state S e,t With the help of g,t , get the traditional unit status S g,t+1 , power system status S e,t+1 , and in the new energy unit state S r,t+1 , Traditional unit status S g,t+1 , power system status S e,t+1 The state change value A of the traditional unit is g,t Perform safety check and obtain updated traditional unit status S′ g,t+1 ; Among them, for the power system state S e,t+1 , new energy unit state change value A r,t , New energy unit status S r,t+1 、Conventional unit state change value A g,t , Traditional unit status S g,t+1 Maximize the joint reward of the first agent and the second agent.
14. The system according to claim 12, Features: The system uses a clustering module to cluster and determine the classification of traditional units; The clustering module is configured to first obtain the traditional unit data, the characteristics of each unit data include the minimum output of the traditional unit, the maximum output of the traditional unit, the coefficients of the power generation cost function of the unit, the minimum shutdown time of the unit, and the minimum startup time of the unit; then randomly select K units as the initial cluster centers, and use the Euclidean distance formula to calculate the distance from each unit to the K cluster centers in the feature space, so as to assign each unit to the class corresponding to the cluster center with the smallest distance; finally, for each cluster of the K clusters, recalculate its cluster center position: μ k is the kth cluster center, C k is the kth category.
15. The system according to claim 13, It is characterized in that The safety verification includes: Where: is the predicted value of the output change of unit g at time t for the next moment, T t g is the continuous on or off time of unit g up to time t, is the minimum startup time of unit g, ΔP g,t is the change in output of unit g between time t and the next time, is the maximum value of the downward ramp of unit g power, a is the scaling factor, is the maximum value of the unit g power climbing upward, is the minimum shutdown time of unit g, is the predicted value of the change in the spin-up reserve rate of all traditional units at time t for the next moment, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of all traditional units at time t to the next moment, is the downspin reserve rate of the traditional unit g at time t.
16. The system according to claim 13, It is characterized in that The calculation steps of the power imbalance penalty include: Calculate the total output forecast value of all units at time t+1 Calculate the downspin reserve rate of the traditional unit at time t+1 Calculate the spin-up reserve rate of the traditional unit at time t+1 If P t+1 >D t+1 and but: ΔPP t+1 =(P t+1 -D t+1 )×P f1 If P t+1 >D t+1 and but: If P t+1 <D t+1 and but: ΔPP t+1 =-(P t+1 -D t+1 )×P f1 If P t+1 <D t+1 and but: In the above formula: D t+1 is the actual total active power of the load in period t+1, P r,t+1 is the active power of the new energy unit r in the period t+1, P g,t+1 is the active power of the traditional unit g in the period t+1, N g is the total number of traditional units, is the downspin reserve rate of the traditional unit g at time t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the downspin reserve rate of the traditional unit g at time t to the next moment, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t to the next moment, P f1 is the first penalty coefficient of the reserve part, P f2 is the second penalty coefficient for the portion exceeding the reserve amount.
17. The system according to claim 13, It is characterized in that The calculation of line power flow penalty includes: If PTDF(P t+1 -D t+1 )>P l,max ,but: ϒ load =1000×PTDF(P t+1 -D t+1 )-P l,max ; If PTDF(P t+1 -D t+1 )<P l,min ,but: ϒ load =P l,min -1000×PTDF(P t+1 -D t+1 ); Calculate the power flow penalty of line l at time t+1 Where: PTDF is the power transfer distribution factor, which represents the power flow change caused by the power injection change at a certain node in the system on a specific transmission line, P t+1 is the total output of all units at time t+1, D t+1 is the actual total active power of the load in period t+1, P l,max is the maximum flow through line l, P l,min is the minimum flow through line l, Υ l,load is the violation of line l, ρ l is the penalty coefficient for line l violation, N bus is the total number of lines.
18. The system according to claim 13, It is characterized in that The second agent updates the current state using the following state transfer function: Where: is the predicted value of the output change of the traditional unit g at time t for the next moment, T t g is the continuous on or off time of the traditional unit g at time t, is the minimum output of the traditional unit g, P g,t is the active power of the traditional unit g in period t, is the spin-up reserve rate of the traditional unit g at time t, is the predicted value of the change in the spin-up reserve rate of the traditional unit g at time t for the next moment, is the downspin reserve rate of the traditional unit g at time t, It is the predicted value of the change of the downward spin reserve rate of the traditional unit g at time t to the next moment.
19. The system according to claim 13, It is characterized in that The common reward of the first agent and the second agent is calculated using the following function: Where: α is the penalty coefficient corresponding to the sum of operating cost and start-stop cost, f g (P g,t+1 ) is the operating cost function of the traditional unit at time t+1, is the startup cost of the traditional unit g, u g,t+1 is the startup signal of the traditional unit g at time t+1, is the downtime cost of the traditional unit g, v g,t+1 is the shutdown signal of the traditional unit g at time t+1, ω is the penalty coefficient corresponding to the spinning reserve term, c r is the unit price of the spin-up reserve of the traditional unit g, is the spin-up reserve rate of the traditional unit g at time t+1, is the downspin reserve rate of the traditional unit g at time t+1, c d is the unit price of the conventional unit g's reserve capacity, and the penalty coefficient corresponding to the β-breaking cost, is the power flow penalty of line l at time t+1, γ is the penalty coefficient corresponding to the power flow penalty of the line, ΔPP t+1 is the power imbalance penalty at time t+1, λ is the penalty coefficient corresponding to the resource abandonment penalty item of the new energy unit, η w is the proportion of new energy units output in all units output, The total penalty value of the abandoned resource amount of the new energy unit r at time t+1, G is the set of traditional units, R is the set of new energy units, L is the set of lines, D t+1 It is the actual total active power of the load in the period t+1.
20. The system according to claim 12, It is characterized in that The power system unit combination optimization model is to minimize the following cost function while satisfying the system power balance constraint, system reserve constraint, network maximum transmission capacity constraint, traditional unit operation constraint, and new energy unit operation constraint: Where: P g,t is the active power of the traditional unit g in period t, u g,t is the startup signal of the traditional unit g at time t, v g,t is the shutdown signal of the traditional unit g at time t, is the resource abandonment rate of the new energy unit r at time t, is the spin-up reserve rate of the traditional unit g at time t, is the downspin reserve rate of the traditional unit g at time t.
21. The system according to claim 12, It is characterized in that The initial solution acquisition steps include: Obtaining the state change value of each renewable energy unit as the initial solution of the corresponding parameters in the power system unit combination optimization model; The output of traditional units and the up-down rotation reserve rate in the state values of each traditional unit are obtained as the initial solutions of the corresponding parameters in the power system unit combination optimization model; The output change of each traditional unit after safety correction is obtained, and the continuous disconnection time in the state value of each traditional unit after executing the safety correction action is obtained. The initial solution of the start-up signal and the shutdown signal in the power system unit combination optimization model is determined according to the following rules: If the continuous on-off time is greater than 0, the initial solution of the power-on signal is 1, and the initial solution of the power-off signal is 0; If the continuous breaking time is less than 0, the initial solution of the shutdown signal is 1, and the initial solution of the startup signal is 0; If the continuous breaking time is 0 and the output change prediction value is -1, it means that the power supply has changed from the on state to the off state. At this time, the initial solution of the off signal is 1, and the initial solution of the on signal is 0. If the continuous breaking time is 0 and the output change prediction value is 1, it means that the power supply has changed from the shutdown state to the startup state. At this time, the initial solution of the startup signal is 1 and the initial solution of the shutdown signal is 0.
Citation Information
Cited By
Machine learning enhanced unit commitment problem optimization solving method and system and storage medium
CN120597126A
Generator set steady state regulation and control method and system based on load change response
CN121863433A
