Unit commitment decision-making method based on Monte Carlo tree search

Abstract

A unit commitment decision-making method based on Monte Carlo tree search comprises the steps that 1, Monte Carlo tree node information suitable for unit commitment is defined, and the node information comprises the node action state, the node directional link and the node value of a thermal power unit; 2, on the basis of obtaining the node information in the step 1, according to a recursion principle of a tree structure, generating a Monte Carlo tree suitable for unit commitment; and 3, searching and selecting an optimal node according to an upper limit confidence interval strategy, and solving the unit commitment decision model based on Monte Carlo tree search. In order to thoroughly get rid of the technical problem that an existing unit commitment decision-making method depends on sample data, the unit commitment decision-making method based on Monte Carlo tree search is provided, and the unit commitment decision-making method based on Monte Carlo tree search aims at providing certain reference for research of a zero-data-driven unit commitment decision-making theory.

Description

Technical Field

[0001] The present invention relates to the field of power system and automation technology, and in particular to a unit combination decision method based on Monte Carlo tree search in the case of lack of historical dispatching data. Background Art

[0002] The power generation dispatching plan formulated by the power market operation and management department based on the short-term load forecast, power supply capacity, safety constraints, etc. of the power grid aims to meet the electricity consumption of the whole society as much as possible, realize more reasonable allocation of energy resources, and greatly improve utilization efficiency. As an important theoretical basis for day-ahead power market decision-making and power generation plan compilation, the unit combination problem is to minimize the total operating cost within the dispatching cycle. On the premise of meeting the unit constraints and system constraints, the independent system operator or the power grid dispatching department determines the start and stop status and output plan of the unit in advance. Reasonable consideration of the safety constraint unit combination decision-making plan can not only ensure the stable operation of the system, but also greatly improve the economic benefits of the system operation. According to data from the Argonne National Laboratory in the United States, if the optimality gap of the mixed integer programming solver within 30 minutes is reduced from 0.1% to 0.05%, more than 200 million US dollars in costs may be saved. Therefore, the unit combination problem has long been widely concerned by theoretical researchers and application decision makers in the field of electrical engineering. With safety and abundance as the basic premise, studying the unit combination decision-making theory with high adaptability and high precision, ensuring energy and power security, and promoting the coordination of power supply and demand have important theoretical and engineering significance for the development and reform of my country's power industry.

[0003] Traditional unit combination decision-making based on physical model drive, because the model construction and algorithm proposal are based on the mechanism research of unit combination problem, through the two steps of optimizing model and solving algorithm, a mapping relationship with clear physical process and clear mathematical logic between input parameters and output decisions is established. Therefore, once the model and algorithm are determined, they cannot be modified or self-learned. For different input quantities, as long as the model and algorithm remain unchanged, their solution efficiency and quality are the same, and will not improve with the accumulation of historical data. Therefore, in the face of emerging theoretical problems and challenges, this method cannot meet the needs of the rapid development of power systems.

[0004] In fact, once the scheduling method is used in practice, it will often accumulate a large amount of structured historical data. In the long run, the unit combination decision-making also has a certain degree of repeatability. The historical decision-making plans and experience accumulated in the past are also instructive for future unit combination decisions. At the same time, the actual operation decision data is generally the product of a combination of model calculation and manual correction. Therefore, in theory, it can be said that it takes into account the various influencing factors and constraints faced at the time.

[0005] The idea of ​​unit combination decision-making based on data-driven can, on the one hand, greatly simplify the process and complexity of modeling and solving, fully take into account the input variables that cannot be modeled in traditional methods, and respond to various emerging theoretical problems and challenges through self-learning; on the other hand, it can also use its own generalization ability to enable the same set of neural networks to automatically switch between different data samples and different decision-making goals, thereby giving the unit combination decision-making process an ability similar to "immune memory".

[0006] However, with the booming development of artificial intelligence technology, this data-driven research approach also faces many challenges:

[0007] (1) Relying on huge amounts of data, high-quality data labels, and powerful computing resources. From a data-driven perspective, among the four methods of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning, the decision-making method with the most theoretical significance and practical value is still the unit combination based on supervised learning. Although for the modern power market, its operating departments generally accumulate a large amount of historical decision-making data, it is impossible to effectively guarantee whether these historical data are the optimal solutions under the circumstances at that time. In addition, the sample structure used for model training usually includes load and unit start-stop state variable sequences. Considering the high-dimensional nature of these sequences, the computing power requirements for computers are relatively high. Therefore, unit combination based on supervised learning is an effective idea when dealing with unit combination problems under high-quality samples, but in terms of solving model training time, it still faces a series of theoretical challenges that need to be broken through.

[0008] (2) When faced with data-scarce unit combination problems (such as drought, heavy precipitation, high temperature, low temperature, tropical cyclone, rainstorm / snow, strong wind, lightning and other extreme weather), its historical data itself is difficult to obtain, let alone guarantee the optimality of the response strategy in such scenarios. Unit combination based on unsupervised learning, although not dependent on sample data, cannot be directly used in the decision-making process; unit combination based on reinforcement learning, which is also independent of data, can gain decision-making experience through continuous trial and error and learning, but it takes more time to accumulate experience and has limited generalization ability. In order to make better use of existing data, unit combination based on semi-supervised learning has emerged, which uses both labeled samples and unlabeled samples for decision learning, effectively reducing resources and costs, and improving the accuracy and generalization ability of the model, but it does not fundamentally solve the dependence on sample data. From the existing results and research, there is basically no small sample unit combination decision-making method that perfectly combines the advantages of supervised learning and unsupervised learning data-driven methods, which can not only ensure the accuracy of the model but also get rid of the dependence on data to improve the generalization ability of the model.

[0009] In view of this, the present invention introduces Monte Carlo tree search into the field of unit combination decision-making, aiming to use heuristic tree search algorithm to solve the unit combination decision-making problem with huge space and complex constraints. Given the unit parameters and load requirements, by simulating randomness and exploring potential decision paths. First, the present invention explains the difference between unit combination decision-making and the Go algorithm; on this basis, a Monte Carlo tree model of unit combination decision-making composed of the expansion of unit start and stop action state nodes is established; finally, the upper limit confidence interval is used to evaluate the value of the node's action state to find the optimal unit combination decision search path, which can effectively solve the unit combination decision problem in the case of data scarcity. Summary of the invention

[0010] The purpose of the present invention is to solve the technical problems of the traditional physical model-driven unit combination decision method, such as the complex modeling and solution process, the difficulty in modifying the model algorithm, the long research cycle, and the insufficient applicability, flexibility and scalability; at the same time, get rid of the dependence of data-driven supervised learning and semi-supervised learning unit combination decision methods on sample data, overcome the limitations of the generalization ability of the unit combination decision method based on reinforcement learning, and provide a unit combination decision method based on Monte Carlo tree search for data-scarce situations. This method can both ensure the accuracy of the model and get rid of the dependence on data to improve the generalization ability of the model.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0012] A unit commitment decision method based on Monte Carlo tree search comprises the following steps:

[0013] Step 1: Define the Monte Carlo tree node information applicable to unit combination, including: node action status, node directional link and node value of thermal power units;

[0014] Step 2: Based on the node information obtained in step 1, a Monte Carlo tree suitable for unit commitment is generated according to the recursive principle of the tree structure;

[0015] Step 3: According to the upper confidence interval strategy, search and select the optimal node to solve the unit commitment decision model based on Monte Carlo tree search.

[0016] In step 1, the Monte Carlo tree node information applicable to the unit combination is defined. The node information mainly consists of three parts: the node action state of the thermal power unit, the node directional link and the node value, which are specifically expressed as follows:

[0017] 1. Define node action state N t (s t ,a i,t ):

[0018] In the search node action state N t (s t ,a i,t ), two default unit start and stop action rules are set:

[0019] 1) At each time t, each unit selects the start and stop action a by default i,t The probability of a is equal, where a i,t Indicates the start and stop action of unit i at time t. If a i,t =0, the unit is considered shut down; if a i,t =1, regarded as the unit is turned on; s t Refers to the load state at time t;

[0020] 2) If the unit is turned on, the output upper limit (maximum output power) of the unit will be included in the total output power of the node by default;

[0021] Under the guidance of the above two default unit start-up and shutdown action rules, the Monte Carlo tree applicable to unit combination can be used according to the scheduling period T and load state s t The order of (t=1,2,…,T) gives the feasible solutions for the start and stop actions of the unit at time t; therefore, the number of feasible solutions at time t will no longer be 2 N This reduces the number of nodes in each layer of the Monte Carlo tree applicable to unit combination from the breadth level.

[0022] 2. Define node-directed link π t =Pr(a i,t+1 |s t ):

[0023] In the Monte Carlo tree applicable to unit combination, it points to the next layer of unit start and stop action a i,t+1 Directed links to child nodesπ t Indicates the operation that leads to the action state of the subsequent node; and the action state N of the next layer of child nodes t+1 (s t+1 ,a i,t+1 ) In addition to meeting the default unit start-stop action rule constraints, it is also necessary to meet the minimum continuous start-stop time constraints of the unit, which reduces the number of branches of the Monte Carlo tree applicable to the unit combination from the depth level; among them, s t+1 Refers to the load status at time t+1.

[0024] 3. Define Node Value

[0025] For feasible node action states N t (s t ,a i,t ) is evaluated depending on the child nodes The unit start and stop action ai,t Node value generated by running the simulation At time t child node Total operating cost of thermal power units The reciprocal of is taken as the value function of the node Specifically, if the total operating cost of the thermal power unit at this node is The smaller it is, the greater the corresponding value.

[0026] Based on the node value function of the Monte Carlo tree that is applicable to the unit combination, the total operating cost model of the thermal power unit is formulated as follows:

[0027] Objective function:

[0028] The unit combination problem refers to seeking the optimal generator start-stop and output plan under the premise of satisfying the physical characteristics constraints of the unit and the network security constraints, so that the total operating cost of the system is minimized within a certain scheduling period (usually the next day). For the start-stop plan at time t, U Git Output scheme is P Git Child nodes of The specific objective function of the thermal power unit i is:

[0029]

[0030] Where: F G is the total operating cost of the unit; T is a scheduling cycle, generally 24 hours; N is the total number of thermal power units; U Git , U Gi(t-1) are the start and stop states of unit i in time periods t and t-1, 0 means shutdown and 1 means startup; P Git is the active power output of unit i in period t; the start-up and shutdown cost S(τ Git ) and fuel cost R Git (P Git ) are:

[0031]

[0032] R Git (P Git )=a i (P Git ) 2 +b i P Git +c i (3)

[0033] Where: α i is the startup and maintenance cost of the steam turbine of unit i; i is the startup cost of the boiler of unit i under cooling environment; τ Gitis the continuous shutdown time of unit i at time t; ω i is the time constant of the cooling rate of unit i; a i 、b i 、c i are the coal consumption coefficients of unit i.

[0034] Constraints:

[0035] 1) Unit output constraint

[0036]

[0037] Where: and are the lower and upper limits of the active output of unit i respectively.

[0038] 2) Minimum continuous start-stop time constraints

[0039]

[0040] Where: and are the continuous start-up time and shutdown time of unit i in period t respectively; and are the minimum continuous start and stop time of unit i respectively.

[0041] 3) Maximum start and stop times constraint

[0042]

[0043] Where: n i is the maximum allowed number of starts and stops of unit i within the decision cycle.

[0044] 4) System power balance constraint

[0045]

[0046] Where: D t is the load demand of the system in period t.

[0047] In step 2, the start-stop action variables of each thermal power unit in the system are controlled by specific parameters (such as load, continuous start-stop time, start-stop times, etc.). When generating a Monte Carlo tree suitable for unit combination according to the generation order of the scheduling cycle and the combination of the start-stop action variables of the unit, the following two recursive principles of the tree structure must be followed:

[0048] 1) Each node in the Monte Carlo tree applicable to unit commitment represents a feasible combination of unit start-stop action variables under the current load state;

[0049] 2) The generation order of scheduling cycles corresponding to different levels in the Monte Carlo tree applicable to unit combination.

[0050] In step 3, the topological structure of the Monte Carlo tree applicable to unit combination grows asymmetrically, and the growth order is also unpredictable. When selecting the optimal node according to the upper confidence interval strategy in the Monte Carlo tree applicable to unit combination, the following steps are specifically included:

[0051] 1) The exploration mechanism of unsearched nodes in the tree ensures that the generated Monte Carlo tree suitable for unit combination will not miss the optimal unit start-stop action strategy;

[0052] 2) The utilization mechanism of the searched nodes in the tree prevents the generated Monte Carlo tree applicable to unit commitment from being too large, improves the search efficiency of the Monte Carlo tree applicable to unit commitment, and ensures the reliability and economy of the decision-making scheme;

[0053] 3) Upper confidence interval strategy for exploration and utilization of compromise nodes within the tree

[0054] From the root node v of the tree u First, for each non-leaf node's child node The evaluation value at a time t is calculated according to the Monte Carlo tree selection strategy applicable to unit combination, namely the upper confidence interval algorithm, and a child node is selected for the next selection according to the upper confidence interval value until a leaf node is reached.

[0055] Then the upper confidence interval strategy of the Monte Carlo tree applicable to unit commitment can be expressed as:

[0056]

[0057] Where: is the upper confidence interval value of the node; For child nodes The number of visits; N(v u ) is the root node v u Number of visits; U Git , U Gi(t-1) are the start and stop states of unit i at time t and t-1, 0 means shutdown and 1 means startup; P Git is the active power output of unit i in period t; S(τ Git ) is the start-up and shutdown cost of unit i at time t; R Git (P Git ) is the fuel cost of unit i at time t; the upper confidence interval strategy formula consists of two parts, the first half of which is For Node The average value of represents the "utilization mechanism" of the node; the second half of the formula Represents the "exploration mechanism" of the node; parameter c is the compromise coefficient between "utilization" and "exploration", a small c emphasizes utilization, and a large c emphasizes exploration.

[0058] In step 3, the optimal node is selected according to the upper confidence interval strategy within the tree, and the unit commitment decision model based on Monte Carlo tree search is solved, including the following steps:

[0059] 1) Starting from the root node, search for the currently feasible unit start-stop plan subnode according to the scheduling cycle order using the default unit start-stop action rules;

[0060] 2) The upper confidence interval value of the sub-node of the computer group start-stop action state, search and select the appropriate node for simulation, there are the following three situations:

[0061] 2-1) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are infinite, a node is randomly selected for simulation;

[0062] 2-2) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not all infinite, then an infinite node is randomly selected to expand the simulation;

[0063] 2-3) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not infinite, then randomly select the node with the larger upper confidence interval value to expand the simulation.

[0064] 3) Based on the current load demand, the Gurobi solver is called to calculate the output power and operating cost of the node unit;

[0065] 4) Backtracking update: backtrack to each parent node along the search path, and update the value, number of visits, and upper confidence interval value of the parent node on the path;

[0066] 5) Determine whether the termination state has been reached. If so, execute step 6; if not, return to step 2;

[0067] 6) Stop the calculation and output the search path composed of the unit start-stop action state sub-nodes with larger upper confidence interval values, which represents the unit combination decision plan within the entire scheduling cycle.

[0068] Compared with the prior art, the present invention has the following technical effects:

[0069] 1) The present invention can effectively compress the number of unit start-stop action states that need to be searched in each scheduling cycle, and effectively avoid the occurrence of the "dimensionality disaster" by pruning the Monte Carlo tree suitable for the unit combination in breadth and depth.

[0070] 2) The unit combination decision scheme obtained by the present invention is 43.73% faster than the unit combination decision scheme obtained by mixed integer programming in terms of solution time, and the two methods have the same operating cost. This method significantly improves the decision-making efficiency of the model while ensuring the economy and reliability of the unit combination decision scheme.

[0071] 3) The upper confidence interval strategy of compromise node exploration and utilization in the present invention is used to select the start and stop action nodes of the unit, which is in line with the common dispatching strategy in power system dispatching, that is, in the low load stage, the Monte Carlo tree search tends to start those units with high start and stop costs but low fuel costs; because the fuel costs of these units are relatively low during operation, and they can meet low load demands without frequent start and stop, thereby reducing the expenditure of start and stop costs. In the peak load stage, the Monte Carlo tree search will select units with low start and stop costs but high fuel costs to bear the load; although the fuel costs of these units are high during operation, it is an economically reasonable choice to start them because they need to meet high load demands, because they usually have higher power generation capacity and faster response speed.

[0072] 4) Compared with the unit combination decision method based on supervised learning that requires massive historical data, the present invention gets rid of the dependence on historical samples and does not require a historical unit decision method. In the case of only information such as load and unit characteristic parameters, the method can give a correct unit combination decision plan and has generalization. Compared with the unit combination decision method based on unsupervised learning that does not require historical scheduling data, the present invention does not need to spend more time on trial and error to accumulate experience, and improves the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0074] Figure 1 This is a framework diagram for unit commitment decision-making based on Monte Carlo tree search;

[0075] Figure 2 To solve the flow chart;

[0076] Figure 3 It is the wiring diagram of IEEE-6 node system;

[0077] Figure 4 This is the unit start and stop plan of method 2;

[0078] Figure 5 This is the unit output plan of method 2;

[0079] Figure 6 This is the effect of random simulation times on node value. DETAILED DESCRIPTION

[0080] A unit commitment decision method based on Monte Carlo tree search comprises the following steps:

[0081] Step 1: Define the Monte Carlo tree node information applicable to unit combination, including: node action status, node directional link and node value of thermal power units;

[0082] Step 2: Based on the node information obtained in step 1, a Monte Carlo tree suitable for unit commitment is generated according to the recursive principle of the tree structure;

[0083] Step 3: According to the upper confidence interval strategy, search and select the optimal node to solve the unit commitment decision model based on Monte Carlo tree search.

[0084] In step 1, the Monte Carlo tree node information applicable to the unit combination is defined. The node information mainly consists of three parts: the node action state of the thermal power unit, the node directional link and the node value, which are specifically expressed as follows:

[0085] 1. Define node action state N t (s t ,a i,t ):

[0086] In the search node action state N t (s t ,a i,t ), two default unit start and stop action rules are set:

[0087] 1) At each time t, each unit selects the start and stop action a by default i,t The probability of a is equal, where a i,t Indicates the start and stop action of unit i at time t. If a i,t =0, the unit is considered shut down; if a i,t =1, regarded as the unit is turned on; s t Refers to the load state at time t;

[0088] 2) If the unit is turned on, the output upper limit (maximum output power) of the unit will be included in the total output power of the node by default;

[0089] Under the guidance of the above two default unit start-up and shutdown action rules, the Monte Carlo tree applicable to unit combination can be used according to the scheduling period T and load state s t The order of (t=1,2,…,T) gives the feasible solutions for the start and stop actions of the unit at time t; therefore, the number of feasible solutions at time t will no longer be 2 N The number of nodes in each layer of the Monte Carlo tree applicable to unit combination is reduced from the breadth level;

[0090] 2. Define node-directed link πt =Pr(a i,t+1 |s t ):

[0091] In the Monte Carlo tree applicable to unit combination, it points to the next layer of unit start and stop action a i,t+1 Directed links to child nodesπ t Indicates the operation that leads to the action state of the subsequent node; and the action state N of the next layer of child nodes t+1 (s t+1 ,a i,t+1 ) In addition to meeting the default unit start-stop action rule constraints, it is also necessary to meet the minimum continuous start-stop time constraints of the unit, which reduces the number of branches of the Monte Carlo tree applicable to the unit combination from the depth level; among them, s t+1 Refers to the load state at time t+1;

[0092] 3. Define Node Value

[0093] For feasible node action states N t (s t ,a i,t ) is evaluated depending on the child nodes The unit start and stop action a i,t Node value generated by running the simulation At time t child node Total operating cost of thermal power units The reciprocal of is taken as the value function of the node Specifically, if the total operating cost of the thermal power unit at this node is The smaller it is, the greater the corresponding value.

[0094] Based on the node value function of the Monte Carlo tree that is applicable to the unit combination, the total operating cost model of the thermal power unit is formulated as follows:

[0095] Objective function:

[0096] For the start-stop scheme at time t, U Git Output scheme is P Git Child nodes of The specific objective function of the thermal power unit i is:

[0097]

[0098] Where: F G is the total operating cost of the unit; T is a scheduling cycle, generally 24 hours; N is the total number of thermal power units; U Git , U Gi(t-1) are the start and stop states of unit i in time periods t and t-1, 0 means shutdown and 1 means startup; PGit is the active power output of unit i in period t; the start-up and shutdown cost S(τ Git ) and fuel cost R Git (P Git ) are:

[0099]

[0100] R Git (P Git )=a i (P Git ) 2 +b i P Git +c i (3)

[0101] Where: α i is the startup and maintenance cost of the steam turbine of unit i; i is the startup cost of the boiler of unit i under cooling environment; τ Git is the continuous shutdown time of unit i at time t; ω i is the time constant of the cooling rate of unit i; a i 、b i 、c i are the coal consumption coefficients of unit i;

[0102] Constraints:

[0103] 1) Unit output constraint

[0104]

[0105] Where: and are the lower and upper limits of active output of unit i respectively;

[0106] 2) Minimum continuous start-stop time constraints

[0107]

[0108] Where: and are the continuous start-up time and shutdown time of unit i in period t respectively; and are the minimum continuous start and stop time of unit i respectively;

[0109] 3) Maximum start and stop times constraint

[0110]

[0111] Where: n i is the maximum allowed number of starts and stops of unit i within the decision cycle;

[0112] 4) System power balance constraint

[0113]

[0114] Where: D t is the load demand of the system in period t.

[0115] In step 2, when generating a Monte Carlo tree suitable for unit combination according to the generation order of the scheduling cycle and the combination of the start and stop action variables of the unit, the following two recursive principles of the tree structure must be followed:

[0116] 1) Each node in the Monte Carlo tree applicable to unit commitment represents a feasible combination of unit start-stop action variables under the current load state;

[0117] 2) The generation order of scheduling cycles corresponding to different levels in the Monte Carlo tree applicable to unit combination.

[0118] In step 3, the topological structure of the Monte Carlo tree applicable to unit combination grows asymmetrically, and the growth order is also unpredictable. When selecting the optimal node according to the upper confidence interval strategy in the Monte Carlo tree applicable to unit combination, the following steps are specifically included:

[0119] 1) The exploration mechanism of unsearched nodes in the tree ensures that the generated Monte Carlo tree suitable for unit combination will not miss the optimal unit start-stop action strategy;

[0120] 2) The utilization mechanism of the searched nodes in the tree prevents the generated Monte Carlo tree applicable to unit commitment from being too large, improves the search efficiency of the Monte Carlo tree applicable to unit commitment, and ensures the reliability and economy of the decision-making scheme;

[0121] 3) Upper confidence interval strategy for exploration and utilization of compromise nodes within the tree

[0122] From the root node v of the tree u First, for each non-leaf node's child node The evaluation value at a time t is calculated according to the Monte Carlo tree selection strategy applicable to unit commitment, namely the upper confidence interval algorithm, and a child node is selected for the next step according to the upper confidence interval value until a leaf node is reached;

[0123] Then the upper confidence interval strategy of the Monte Carlo tree applicable to unit commitment can be expressed as:

[0124]

[0125] Where: is the upper confidence interval value of the node; For child nodes The number of visits; N(v u ) is the root node v u Number of visits; U Git , U Gi(t-1) are the start and stop states of unit i at time t and t-1, 0 means shutdown and 1 means startup; P Git is the active power output of unit i in period t; S(τ Git ) is the start-up and shutdown cost of unit i at time t; R Git (P Git ) is the fuel cost of unit i at time t; the upper confidence interval strategy formula consists of two parts, the first half of which is For Node The average value of represents the "utilization mechanism" of the node; the second half of the formula Represents the "exploration mechanism" of the node; parameter c is the compromise coefficient between "utilization" and "exploration", a small c emphasizes utilization, and a large c emphasizes exploration.

[0126] A common dispatching strategy in power system dispatching is: according to the changes in load demand, the operator will dynamically select generators with different cost characteristics to meet the power demand in order to minimize the total cost of the power system. In the low load phase, system operators tend to start those units with higher start-up and shutdown costs but lower fuel costs; because the fuel costs of these units are relatively low during operation, and they can meet low load demands without frequent start-up and shutdown, thereby reducing the expenditure of start-up and shutdown costs. In the peak load phase, system operators will choose units with lower start-up and shutdown costs but higher fuel costs to bear the load; although these units have higher fuel costs during operation, starting them is an economically reasonable choice due to the need to meet high load demands, because they usually have higher power generation capacity and faster response speed.

[0127] Since different units have different cost characteristic parameters, their corresponding start-stop costs and fuel costs are also quite different. Therefore, when taking into account the upper confidence interval value generated by the start-stop action state of the unit and using it as the Monte Carlo tree search strategy for unit combination decision-making, each node considers both utilization and exploration to achieve a balance of units with different cost characteristics to meet load requirements and minimize operating costs.

[0128] In step 3, the optimal node is selected according to the upper confidence interval strategy within the tree, and the unit commitment decision model based on Monte Carlo tree search is solved, including the following steps:

[0129] 1) Starting from the root node, search for the currently feasible unit start-stop plan subnode according to the scheduling cycle order using the default unit start-stop action rules;

[0130] 2) The upper confidence interval value of the sub-node of the computer group start-stop action state, search and select the appropriate node for simulation, there are the following three situations:

[0131] 2-1) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are infinite, a node is randomly selected for simulation;

[0132] 2-2) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not all infinite, then an infinite node is randomly selected to expand the simulation;

[0133] 2-3) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not infinite, then randomly select nodes with larger upper confidence interval values ​​to expand the simulation;

[0134] 3) Based on the current load demand, the Gurobi solver is called to calculate the output power and operating cost of the node unit;

[0135] 4) Backtracking update: backtrack to each parent node along the search path, and update the value, number of visits, and upper confidence interval value of the parent node on the path;

[0136] 5) Determine whether the termination state has been reached. If so, execute step 6; if not, return to step 2;

[0137] 6) Stop the calculation and output the search path composed of the unit start-stop action state sub-nodes with larger upper confidence interval values, which represents the unit combination decision plan within the entire scheduling cycle.

[0138] Example:

[0139] The present invention uses the IEEE-6 node system for testing, such as Figure 3 As shown in Figure 1, the system consists of three generator sets, five transmission lines and two tap-switched transformers. The characteristic parameters of the generators are shown in Table 1, and the hourly load distribution within 24 hours is shown in Table 2.

[0140] Table 1 Characteristic parameters of generator

[0141]

[0142] Table 2 24h power load data

[0143]

[0144] The unit downtime cost is negligible and is assumed to be zero in this case. To discuss the efficiency of the proposed method in detail, the following two cases are considered:

[0145] Method 1: Unit Commitment Decision Method Based on Mixed Integer Programming

[0146] Method 2: Unit Commitment Decision Method Based on Monte Carlo Tree Search, i.e., the Method of the Present Invention

[0147] The computer hardware environment used in the experiment is an IBM-PC 64-bit compatible machine with 16.0GB memory, a GenIntel(R) Core(TM) i7-9700 CPU, and a main frequency of 3.0GHz. The software environment is MATLAB R2022b and Python 3.8.16, the program compiler is PyCharm 2020, and the solver Gurobi 10.0.1 (academic version) is used for calculation.

[0148] For the unit combination decision problem, as mentioned above, for the situation of 3 units for 24 hours, the number of unit start and stop action states of the optimized scheduling can reach 4722366482869645213696, which is obviously a "dimensionality disaster" for the traversal process. To meet this challenge, the present invention uses constraints such as the system power balance constraint of the current layer, the minimum continuous start and stop time constraint of the units between layers, and the maximum start and stop number constraint to prune the Monte Carlo tree of unit combination decision in breadth and depth. The number of unit start and stop action states in the first 12 scheduling cycles of the Monte Carlo tree constructed by this method is shown in Table 3.

[0149] Table 3 Number of states per layer of the tree

[0150]

[0151] As shown in Table 3, the number of feasible solutions for the start-stop action state of the unit is relatively limited in the initial stage; however, with the increase in the number of search layers, the number of states of the start-stop action of the unit in the 9th layer alone has exceeded 10,000, and the 12th layer has even reached more than 320,000. For such a large order of magnitude, even using a brute force search algorithm, computer equipment requires huge computing resources. Therefore, in the search process, it is an effective way to solve the problem of large-scale search space by adopting a reasonable strategy to evaluate the value of the current state and select more valuable nodes for search.

[0152] As the first attempt to apply Monte Carlo tree search to the field of unit combination, the method of the present invention does not consider any network security constraints when searching for feasible solutions to the node action states of unit combination decisions along the depth direction. However, after outputting the final unit combination decision plan, it can be verified whether the plan meets the corresponding network security constraints; if it is found that some constraints are not met, the search will be repeated to generate a final decision plan that meets the constraints. The unit start-stop action states and output conditions obtained by method 2 are as follows: Figure 4 and Figure 5shown.

[0153] Depend on Figure 5 It can be seen that the unit output plan formulated by the unit commitment decision method based on Monte Carlo tree search can meet the 24-hour load demand of the system.

[0154] To further verify the correctness of the method of the present invention, it is compared with the unit commitment decision method based on mixed integer programming in method 1. Considering the grid structure of the 6-node system, the unit commitment decision scheme obtained by method 1 is shown in Table 4.

[0155] Table 4 Unit output plan of method 1

[0156]

[0157] according to Figure 5 The results in Table 4 show that the start-stop action states and output powers of the units obtained by the two methods are consistent. Starting from 11:00, the output power of the G1 unit can no longer meet the load demand. At this time, the G2 unit or the G3 unit needs to be started to meet the power balance constraint. For the G2 unit and the G3 unit, on the one hand, the startup cost of the G2 unit is 249.22$, and the startup cost of the G3 unit is 0.0$; on the other hand, if the G2 unit and the G3 unit output the same unit of power, the fuel cost per unit power of the G2 unit is 202.6171$ / MW, and the fuel cost of the G3 unit is 193.3043$ / MW. Relatively speaking, it is more economical to choose to start the G3 unit at this time.

[0158] At 13:00, due to the limited maximum output power of the G3 unit, the G2 unit had to be started to cope with the upcoming peak load period. At 20:00, the load gradually entered the valley period. Since the shutdown cost of the unit was negligible, it was more economical to shut down the G2 unit with a higher unit power fuel cost and keep the G3 unit.

[0159] The total system operating cost and solution time of the two methods are shown in Table 5.

[0160] Table 5 Total running cost and solution time of Method 1 and Method 2

[0161]

[0162] As can be seen from Table 5, the total system operating cost obtained by the two methods is the same, because the unit output conditions obtained by the two methods are the same. However, the solution time of Method 2 is less than that of Method 1. The main reason is that Method 2 specifies the unit start-stop plan for the first hour when searching for the unit start-stop action state node, which reduces the number of states in the search space from the root. From the perspective of system abundance, the system load demand is low during 1:00-12:00 hours, and only starting the G1 unit can fully meet the load demand, so there is no need to frequently start and stop the G2 and G3 units. From the perspective of system economy, the remaining three feasible unit start-stop action state nodes in the first hour involve the start of the G2 or G3 units. At this time, it is reasonable to abandon the subsequent search of the remaining three feasible start-stop action state nodes without increasing the additional startup cost, while meeting the load demand.

[0163] In order to further analyze the impact of the number of random simulations in each layer scheduling cycle on the value function of the node action state selected by the upper confidence interval strategy, the number of simulations is set from 1 to 20 when executing the selection and expansion steps. The change of the value of each level node with the number of simulations is shown in Figure 6 shown.

[0164] Depend on Figure 6 It can be seen that as the number of random simulations per layer increases, the value of the nodes in each layer gradually tends to stabilize. Specifically, starting from the 16th random simulation, the value function of the action state of each node in each layer reaches a convergence state. The main reason for the above phenomenon is that in the selection and simulation stage of the Monte Carlo tree search, only one node can be randomly selected for simulation, and usually, the most valuable node usually has a smaller value Q. Since the simulation process has a certain degree of randomness, only by simulating multiple times when searching at each layer can we hope to effectively search for the most valuable node on the basis of balancing the average value and the unexplored state.

[0165] In summary, the unit commitment decision method based on Monte Carlo tree search proposed in the present invention is correct and effective in solving the unit combination problem with a small number of units.

Claims

1. A unit commitment decision method based on Monte Carlo tree search, characterized in that: The following steps are involved: Step 1: Define the Monte Carlo tree node information applicable to unit combination, including: node action status, node directional link and node value of thermal power units; Step 2: Based on the node information obtained in step 1, a Monte Carlo tree suitable for unit commitment is generated according to the recursive principle of the tree structure; Step 3: According to the upper confidence interval strategy, search and select the optimal node to solve the unit commitment decision model based on Monte Carlo tree search.

2. The method according to claim 1, characterized in that In step 1, the Monte Carlo tree node information applicable to the unit combination is defined. The node information mainly consists of three parts: the node action state of the thermal power unit, the node directional link and the node value, which are specifically expressed as follows:

1. Define node action state N t (s t ,a i,t ): In the search node action state N t (s t ,a i,t ), two default unit start and stop action rules are set: 1) At each time t, each unit selects the start and stop action a by default i,t The probability of a is equal, where a i,t Indicates the start and stop action of unit i at time t. If a i,t =0, the unit is considered shut down; if a i,t =1, regarded as the unit is turned on; s t Refers to the load state at time t; 2) If the unit is turned on, the output upper limit of the unit will be included in the total output power of the node by default; Under the guidance of the above two default unit start-up and shutdown action rules, the Monte Carlo tree applicable to unit combination can be used according to the scheduling period T and load state s t The order of gives the feasible solutions for the start and stop actions of the unit at time t; therefore, the number of feasible solutions at time t will no longer be 2 N The number of nodes in each layer of the Monte Carlo tree applicable to unit combination is reduced from the breadth level; 2. Define node-directed link π t =Pr(a i,t+1 |s t ): In the Monte Carlo tree applicable to unit combination, it points to the next layer of unit start and stop action a i,t+1 Directed links to child nodesπ t Indicates the operation that leads to the action state of the subsequent node; and the action state N of the next layer of child nodes t+1 (s t+1 ,a i,t+1 ) In addition to meeting the default unit start-stop action rule constraints, it is also necessary to meet the minimum continuous start-stop time constraints of the unit, which reduces the number of branches of the Monte Carlo tree applicable to the unit combination from the depth level; among them, s t+1 Refers to the load state at time t+1; 3. Define Node Value For feasible node action states N t (s t ,a i,t ) is evaluated depending on the child nodes The unit start and stop action a i,t Node value generated by running the simulation At time t child node Total operating cost of thermal power units The reciprocal of is taken as the value function of the node Specifically, if the total operating cost of the thermal power unit at this node is The smaller it is, the greater the corresponding value.

3. The method according to claim 2, characterized in that Based on the node value function of the Monte Carlo tree that is applicable to the unit combination, the total operating cost model of the thermal power unit is formulated as follows: Objective function: For the start-stop scheme at time t, U Git The output scheme is P Git Child nodes of The specific objective function of the thermal power unit i is: Where: F G is the total operating cost of the unit; T is a scheduling cycle, generally 24 hours; N is the total number of thermal power units; U Git , U Gi(t-1) are the start and stop states of unit i in time periods t and t-1, 0 means shutdown and 1 means startup; P Git is the active power output of unit i in period t; the start-up and shutdown cost S(τ Git ) and fuel cost R Git (P Git ) are: Where: α i is the startup and maintenance cost of the steam turbine of unit i; i is the startup cost of the boiler of unit i under cooling environment; τ Git is the continuous shutdown time of unit i at time t; ω i is the time constant of the cooling rate of unit i; a i 、b i 、c i are the coal consumption coefficients of unit i; Constraints: 1) Unit output constraint Where: are the lower and upper limits of active output of unit i respectively; 2) Minimum continuous start-stop time constraints Where: and are the continuous start-up time and shutdown time of unit i in period t respectively; and are the minimum continuous start and stop time of unit i respectively; 3) Maximum start and stop times constraint Where: n i is the maximum allowed number of starts and stops of unit i within the decision cycle; 4) System power balance constraint Where: D t is the load demand of the system in period t.

4. The method according to claim 1, characterized in that In step 2, when generating a Monte Carlo tree suitable for unit combination according to the generation order of the scheduling cycle and the combination of the start and stop action variables of the unit, the following two recursive principles of the tree structure must be followed: 1) Each node in the Monte Carlo tree applicable to unit commitment represents a feasible combination of unit start-stop action variables under the current load state; 2) The generation order of scheduling cycles corresponding to different levels in the Monte Carlo tree applicable to unit combination.

5. The method according to claim 1, characterized in that In step 3, the topological structure of the Monte Carlo tree applicable to unit combination grows asymmetrically, and the growth order is also unpredictable. When selecting the optimal node according to the upper confidence interval strategy in the Monte Carlo tree applicable to unit combination, the following steps are specifically included: 1) The exploration mechanism of unsearched nodes in the tree ensures that the generated Monte Carlo tree suitable for unit combination will not miss the optimal unit start-stop action strategy; 2) The utilization mechanism of the searched nodes in the tree prevents the generated Monte Carlo tree applicable to unit commitment from being too large, improves the search efficiency of the Monte Carlo tree applicable to unit commitment, and ensures the reliability and economy of the decision-making scheme; 3) Upper confidence interval strategy for exploration and utilization of compromise nodes within the tree From the root node v of the tree u First, for each non-leaf node's child node The evaluation value at a time t is calculated according to the Monte Carlo tree selection strategy applicable to unit commitment, namely the upper confidence interval algorithm, and a child node is selected for the next step according to the upper confidence interval value until a leaf node is reached; Then the upper confidence interval strategy of the Monte Carlo tree applicable to unit commitment can be expressed as: Where: is the upper confidence interval value of the node; For child nodes The number of visits; N(v u ) is the number of visits to the root node vu; U Git , U Gi(t-1) are the start and stop states of unit i at time t and t-1, 0 means shutdown and 1 means startup; P Git is the active power output of unit i in period t; S(τ Git ) is the start-up and shutdown cost of unit i at time t; R Git (P Git ) is the fuel cost of unit i at time t; the upper confidence interval strategy formula consists of two parts, the first half of which is For Node The average value of represents the "utilization mechanism" of the node; the second half of the formula Represents the "exploration mechanism" of the node; parameter c is the trade-off coefficient between "utilization" and "exploration", a small c emphasizes utilization, and a large c emphasizes exploration.

6. The method according to claim 1, characterized in that In step 3, the optimal node is selected according to the upper confidence interval strategy within the tree, and the unit commitment decision model based on Monte Carlo tree search is solved, including the following steps: 1) Starting from the root node, search for the currently feasible unit start-stop plan subnode according to the scheduling cycle order using the default unit start-stop action rules; 2) The upper confidence interval value of the sub-node of the computer group start-stop action state, search and select the appropriate node for simulation, there are the following three situations: 2-1) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are infinite, a node is randomly selected for simulation; 2-2) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not all infinite, then an infinite node is randomly selected to expand the simulation; 2-3) If the upper confidence interval values ​​of all feasible unit start-stop action state sub-nodes are not infinite, then randomly select nodes with larger upper confidence interval values ​​to expand the simulation; 3) Based on the current load demand, the Gurobi solver is called to calculate the output power and operating cost of the node unit; 4) Backtracking update: backtrack to each parent node along the search path, and update the value, number of visits, and upper confidence interval value of the parent node on the path; 5) Determine whether the termination state has been reached. If so, execute step 6; if not, return to step 2; 6) Stop the calculation and output the search path composed of the unit start-stop action state sub-nodes with larger upper confidence interval values, which represents the unit combination decision plan within the entire scheduling cycle.

Images