Monthly unit combination method based on flexible action device-evaluator combined logic Benders decomposition

Abstract

The invention discloses a monthly unit commitment method based on flexible action device-evaluator combined logic Benders decomposition. The monthly unit commitment method comprises the following steps: 1) establishing a monthly security constraint unit commitment model; 2) establishing time-varying wind power prediction error probability distribution; 3) decomposing the M-SCUC model into a unit commitment decision model and a relaxation optimal power flow model; 4) establishing an environment and key elements of a unit commitment decision main problem M1; establishing an objective function and a constraint condition of the unit commitment decision sub-problem M2; 5) generating a unit commitment decision action of a unit commitment decision main problem M1 based on a flexible action device-evaluator deep reinforcement learning baseline algorithm; 6) solving a unit commitment decision sub-problem M2, substituting an output result into a unit commitment decision main problem M1, and solving the unit commitment decision main problem M1 by using an LBD-SAC algorithm; and 7) repeating the steps 5)-6) until the reward function meets the convergence requirement, and outputting a final unit commitment decision. Reliability and flexibility of system operation are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of long-term unit combination dispatching of power systems based on reinforcement learning, and in particular to a monthly unit combination method based on a flexible actor-evaluator combined with logical Benders decomposition. Background Art

[0002] The high penetration rate of wind power has brought great challenges to the operation of the power system. Monthly security constrained unit commitment (M-SCUC) considers the uncertainty of wind power from a longer-term perspective, and then makes unit commitment decisions to ensure the reliability and flexibility of system operation.

[0003] M-SCUC faces two challenges: a) The complexity of uncertainty increases. As the scheduling cycle lengthens, the prediction error will significantly increase in time series; b) The model scale expands. The dimensions of various variables and actual operating constraints increase significantly. These NP-hard characteristics of randomness, non-convexity and strong uncertainty bring great challenges to the solution of M-SCUC. Existing research works aim to address these challenges by simplifying models and algorithm design.

[0004] In terms of model simplification, existing studies have proposed a power flow-based unit clustering method to reduce the decision variables for medium-term scheduling. Other studies have developed redundant constraint identification and elimination techniques to simplify long-term and large-scale SCUC models. However, the effectiveness of these compact models under complex uncertainty conditions, as well as the trade-off between compactness and tightness, have not been fully verified.

[0005] Task decomposition is an effective method for algorithm design to solve uncertain M-SCUC problems. Some studies have used Lagrangian relaxation to decompose stochastic long-term SCUC into multiple tractable short-term SCUC subproblems. However, this method has problems such as multiplier oscillation and high cost of sub-gradient calculation. Other studies have introduced an advanced stochastic variant of nested Benders decomposition (BD), called stochastic dual dynamic integer programming (SDDiP), to solve multi-stage uncertain scheduling problems. However, since SDDiP requires a long solution time and a long scheduling cycle, this hinders its further application in M-SCUC.

[0006] Deep reinforcement learning (DRL) is an effective artificial intelligence algorithm for sequential decision-making problems, which shifts the heavy computational burden to the offline training process. DRL can ensure the accuracy of the model without any compact approximation. DRL has been widely used in short-term SCUC problems. It has shown great potential in solving complex uncertain M-SCUC problems efficiently and accurately. However, given the characteristics of M-SCUC, DRL still faces the following challenges.

[0007] 1) The curse of dimensionality in state and action spaces

[0008] As the scale of the SCUC problem increases, the state and action spaces grow exponentially, which greatly increases the spatiotemporal complexity of DRL, leading to a decrease in convergence efficiency and optimization performance, especially on long time scales.

[0009] 2) Constraint Satisfaction

[0010] For M-SCUC, it is crucial to ensure the satisfaction of constraints in DRL. It turns out that it is a difficult task to incorporate only the penalty function for violating constraints into the reward function. Some safe DRL (SDRL) algorithms use the safety layer to correct the actions of the agent. However, the learning of the agent is easily disturbed by the safety layer, resulting in poor convergence and local optimality. Some pioneering works have established SDRL algorithms based on constrained Markov decision processes (CMDP), such as CPO, IPO, and C-SAC. This type of algorithm has shown great potential for solving constrained optimization problems in power systems. However, the high-dimensional state and action space and the long scheduling cycle reduce the efficiency of safety surveys. Therefore, DRL still has a key bottleneck of safety when dealing with the constraints of the M-SCUC problem.

[0011] 3) Complex and uncertain environment simulation

[0012] Forecast accuracy decreases as the forecast time increases. The common practice of adopting independent and identical wind forecast error distributions affects the accuracy of mid- and long-term look-ahead scheduling. Therefore, it has become a challenge to apply a fixed state transition probability function in the agent-environment interaction to M-SCUC. Accurately modeling forecast error distributions with time correlation and incorporating them into the environment is a difficult task. Summary of the invention

[0013] The object of the present invention is to provide a monthly unit commitment method based on flexible actor-judge combined with logical Benders decomposition, comprising the following steps:

[0014] 1) Establish a monthly safety constraint unit combination model considering wind power uncertainty.

[0015] 2) Based on the conditional generative adversarial network, the time-varying wind power prediction error probability distribution is established.

[0016] 3) The logical Benders decomposition method is used to decouple discrete variables and continuous variables, and the M-SCUC model is decomposed into a unit commitment decision model characterized by a constrained Markov decision process framework and a relaxed optimal power flow model.

[0017] 4) Based on the unit combination decision model, the environment, state space, action space, state transfer function, cost function and reward function of the unit combination decision main problem M1 are established.

[0018] Based on the relaxed optimal power flow model, the objective function and constraints of the unit commitment decision subproblem M2 are established.

[0019] 5) Taking the time-varying wind power forecast error probability distribution as the uncertainty of the environment, the unit combination decision action of the unit combination decision main problem M1 is generated based on the flexible actor-judge deep reinforcement learning baseline algorithm, and the unit combination decision action is input as the boundary condition to the unit combination decision sub-problem M2.

[0020] 6) Based on the unit combination decision action of the unit combination decision main problem M1, solve the unit combination decision subproblem M2, and regard the relaxation level output by the unit combination decision subproblem M2 as the survey cost and substitute it into the unit combination decision main problem M1, and use the LBD-SAC algorithm to solve the unit combination decision main problem M1.

[0021] 7) Repeat steps 5) to 6) until the reward function meets the convergence requirements and output the final unit combination decision.

[0022] Furthermore, the monthly safety constraint unit combination model considering wind power uncertainty is as follows:

[0023]

[0024] Where t represents the time. T represents the dispatch period. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. A represents the expected comprehensive dispatch cost. t , D t , H t and M t Both represent coefficient matrices. t is the coefficient vector. x 、N y 、N z 、N ζThey represent the dimensions of the unit commitment decision variable vector x, the unit output power vector y, other state variables z, and the wind power prediction error vector ζ. M0 represents the monthly safety constraint unit commitment model.

[0025] Furthermore, the step of establishing a time-varying wind power prediction error probability distribution based on a conditional generative adversarial network includes:

[0026] 2.1) Obtain wind power forecast data P d and historical measured data H d .

[0027] 2.2) Based on wind power forecast data P d , using the generator network to generate the simulated wind power data sequence G d .

[0028] 2.3) The simulated wind power data sequence G d With wind power forecast data P d Make stitching.

[0029] The wind power forecast data P d and historical measured data H d Make stitching.

[0030] 2.4) Input the spliced ​​data into the discriminator network to fit the time-varying wind power prediction error distribution.

[0031] 2.5) Based on the fitted time-varying wind power forecast error distribution, establish the time-varying wind power forecast error probability distribution.

[0032] Furthermore, the unit commitment decision model represented by the constrained Markov decision process framework is as follows:

[0033]

[0034] In the formula, t represents the time. is the scheduling period. x represents the unit commitment decision variable vector. represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. t (·) represents the startup cost and shutdown cost of the unit. Indicates the expected operating cost. Represents the coefficient matrix of the unit commitment decision model characterized by the constrained Markov decision process framework. is the coefficient vector of the unit commitment decision model characterized by the constrained Markov decision process framework. N x Represents the dimension of the unit commitment decision variable vector x. M1 represents the unit commitment decision problem of unit start-stop state decision.

[0035] Further, the relaxed optimal power flow model is as follows:

[0036]

[0037]

[0038] Where t represents the time. T represents the scheduling period. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s represents the slack variable vector. Represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. Represents the probability distribution of time-varying wind power forecast error. represents the expected operating cost. t (·) represents the operating cost within a certain sampling prediction error. Both represent the coefficient matrices of the relaxed optimal power flow model. is the coefficient vector of the relaxed optimal power flow model. N y 、N z 、N ζ 、N s They represent the dimensions of the unit output power vector y, other state variables z, wind power prediction error vector ζ, and slack variable vector s respectively. M2 represents the relaxed optimal power flow model under the on / off state of the fixed unit.

[0039] Furthermore, the state space of the main unit commitment decision problem M1 is as follows:

[0040]

[0041] Where t represents the time. W, I, and G represent the set of wind turbine generators, nodes, and units, respectively. Represents the state space. t Represents the state at time t. They represent the vector of predicted wind power output and the vector of predicted wind power load at time t respectively. Indicates the on / off status of the unit at time t-1. Indicates the duration of the unit status at time t-1.

[0042] Among them, the first-order difference form of wind power output prediction in adjacent time periods is As shown below:

[0043]

[0044] In the formula, represents the vector of predicted wind power output at time t-1. Δt represents the time difference.

[0045] The action space of the main unit commitment decision problem M1 is as follows:

[0046]

[0047] In the formula, represents the action space. t Indicates the action at time t. N is the number of units. Indicates the on / off status of the unit at time t.

[0048] The state transfer function of the unit commitment decision main problem M1 is as follows:

[0049]

[0050] In the formula, Indicates the duration of the unit state at time t.

[0051] The cost function of the main unit commitment decision problem M1 is as follows:

[0052]

[0053] In the formula, c π,ρ represents the cost function. ζ represents the wind power prediction error vector. π represents the action strategy. They represent the constraint limit of the unit commitment decision model represented by the constrained Markov decision process framework and the relaxation degree of the relaxed optimal power flow model. g is the unit number. i is the node number. UT g , DT g They respectively represent the minimum startup time and minimum shutdown time of the g-th unit. They represent the on / off status of the g-th unit at time t and t-1 respectively. is the state duration of the g-th unit at time t-1. (·)+ represents max{·,0}. is the slack variable. represents the time-varying wind power forecast error probability distribution. ρ represents the forecast error probability.

[0054] The reward function of the main unit commitment decision problem M1 is as follows:

[0055]

[0056] In the formula, r(s t ,a t ) represents the reward function. It represents the on / off state of the unit at time t obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. Represents the startup cost and shutdown cost of the unit. Indicates the expected operating cost. represents the penalty function for violating the constraint. Indicates the power of the gth unit. g 、b g 、c g 、h g d g They represent the constant cost coefficient, first-order cost coefficient, second-order cost coefficient, startup cost, and shutdown cost of the g-th unit respectively. v , κ δ All represent cost coefficients. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. represents the probability distribution of time-varying wind power forecast error. t (·) represents the operating cost within a certain sampling prediction error.

[0057] The environment of the unit commitment decision main problem M1 integrates the reward and cost functions involving the unit commitment decision model represented by the constrained Markov decision process framework and the relaxed optimal power flow model, the probability distribution of the time-varying wind power prediction error, and the state transfer function.

[0058] Furthermore, the objective function of the unit commitment decision sub-problem M2 is as follows:

[0059]

[0060] In the formula, t represents the time. g is the unit number. G represents the set of units. g 、b g 、c g They respectively represent the constant cost coefficient, linear cost coefficient and quadratic cost coefficient of the g-th unit. Represents the power of the g-th unit at time t. κ δ represents the cost coefficient. i is the node number. I represents the set of nodes. is the slack variable.

[0061] The constraints of the unit combination decision sub-problem M2 include node power balance constraints, unit and wind power output power constraints, branch power flow constraints, start / shutdown decision constraints, and unit ramp constraints.

[0062] The node power balance constraint is as follows:

[0063]

[0064] In the formula, G iRepresents the set of wind turbines at node i. w represents the wind turbine generator set number. i Represents the set of wind turbines at node i. represents the actual grid-connected power generation of the w-th wind power station. j is the node number. ij represents the branch from node i to node j. i Represents the branch set of node i. The vector representing the predicted wind power load at node i at time t. Represents the power flow of branch ij.

[0065] The output power constraints of the units and wind power are as follows:

[0066]

[0067] In the formula, represents the prediction error sample obtained by the w-th wind turbine from the environment. Represents the predicted output power of the w-th wind turbine. It represents the on / off state of the g-th unit at time t obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. P g They respectively represent the upper and lower bounds of the output power of the g-th unit.

[0068] The branch power flow constraints are as follows:

[0069]

[0070] In the formula, K i-j,i I represents the power distribution coefficient of branch ij and node i. i Represents the set of branches whose first segment node is i. They represent the upper and lower bounds of the power flow of branch ij respectively. Represents the active load power of node i.

[0071] The startup / shutdown decision constraints are as follows:

[0072]

[0073] In the formula, v g 、z g They respectively represent the startup and shutdown decisions of the g-th unit. It represents the on / off state of the g-th unit at time t-1 obtained by the unit commitment decision model characterized by the constrained Markov decision process framework.

[0074] The unit climbing constraints are as follows:

[0075]

[0076] In the formula, Indicates the power of the gth unit at time t-1. RU g , RD g ,SU g , SD g They respectively represent the ramp-up rate, ramp-down rate, startup rate and shutdown rate of the g-th unit.

[0077] Further, the step of solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm includes:

[0078] 6.1) Introduce entropy constraints and cost constraints as follows:

[0079]

[0080] Where θ represents the neural network parameters of the action strategy π, represents the entropy threshold. t represents the time. T is the scheduling period. π represents the action strategy. s t Indicates the state at time t. t Indicates the action at time t. t =(s t ,a t ,…,s T ,a T ) represents the decision trajectory of the probability distribution of time-varying wind power forecast error starting from time t. r(s t ,a t ) represents the reward function. Q c (s t ,a t ) represents compliance with strategy π and trajectory χ t The expected value of the cumulative discounted costs incurred. represents the discounted cost threshold, where d→0. ζ represents the wind power forecast error vector. ρ| t represents the probability distribution of time-varying wind power forecast error. π,ρ is the cost function. γ is the discount rate.

[0081] 6.2) Construct the LBD-SAC solution module and initialize the number of iterations k = 1.

[0082] The LBD-SAC solution module includes a CGAN module, two reward value judgment network modules, two cost value judgment network modules, and an action network module.

[0083] The CGAN module is used to generate a dataset with time-varying uncertainty distribution.

[0084] Two reward value judgment network modules are used to evaluate the soft Q value of the reward.

[0085] A cost-value judgment network module is used to evaluate the soft Q-value of the constraint crossing cost of the unit commitment decision model represented by the constrained Markov decision process framework.

[0086] A cost-value judgment network module is used to calculate the relaxation level of the relaxed optimal power flow model.

[0087] The action network makes unit combination decisions based on the current system operating status.

[0088] 6.3) Based on the distance-based 2-Wasserstein loss function, the target network parameters of the reward value judgment network module and the cost value judgment network module are calculated.

[0089] The target network parameters of the reward value judgment network module and the cost value judgment network module are as follows:

[0090]

[0091] In the formula, are the soft Q functions of reward and cost respectively. J c (μ) are soft Q functions Approximate value of . Both are Soft-Bellman operators. They represent the target network parameters of the reward value judgment network module and the cost value judgment network module respectively. is the set of unit commitment decisions.

[0092] 6.4) Based on the APDO method, a Lagrangian function is established to transform the entropy constraint and cost constraint maximization problem into a dual optimization problem, and the loss function of the action network module is derived.

[0093] 6.5) Determine whether the number of iterations k is equal to the preset threshold. If not, store the Lagrange multiplier in this iteration into the experience pool, set k = k + 1, update the action strategy and Lagrange multiplier, and return to step 6.3). If yes, sample and select a Lagrange multiplier from the experience pool to adjust the Lagrange multiplier of the last iteration, and enter step 6.6).

[0094] 6.6) Output the optimal solution for unit combination decision.

[0095] Further, the Lagrangian function is as follows:

[0096]

[0097] Where α and λ are Lagrange multipliers related to entropy value and safety survey cost. is the Lagrangian function. θ For action strategy. Represents the entropy threshold. is the cost function.

[0098] The dual optimization problem is as follows:

[0099]

[0100] In the formula, is the optimal primal-dual solution.

[0101] The loss function of the action network module is as follows:

[0102]

[0103] In the formula, J π (θ) is the loss function of the action network module.

[0104] Furthermore, the calculation formula of the update action strategy and Lagrange multiplier is as follows:

[0105]

[0106]

[0107] In the formula, θ (k+1) ,θ (k) They represent the neural network parameters in the k+1th and kth iterations respectively. ω is the learning rate. is the gradient function. (k) , (k) is the Lagrange multiplier in the kth iteration process. (k+1) , (k+1) is the Lagrange multiplier in the k+1th iteration process. is the action strategy in the k+1th iteration.

[0108] The technical effect of the present invention is unquestionable. Aiming at the demand for safe and flexible operation of the power system, the present invention proposes an M-SCUC method based on LBD-SAC to achieve efficient and accurate solution of the medium- and long-term operation scheduling model of the power system and ensure the reliability and flexibility of the system operation.

[0109] The LBD-based DRL training mechanism proposed in this invention combines the ideas of task decomposition and optimization assistance, which can reduce the processing scale of the algorithm and improve constraint satisfaction and convergence. In order to cope with the complex M-SCUC solution problem, a DRL training idea is provided that can ensure scalability, constraint satisfaction and effective handling of uncertainty.

[0110] The LBD-SAC-based M-SCUC method proposed in the present invention combines the APDO-based constraint policy search method with the SAC baseline architecture on the basis of the LBD-based training mechanism. The algorithm adds offline policy training Lagrange multipliers involving security survey costs and policy entropy in the dual update process, driving the intelligent agent to perform efficient maximum entropy adaptive learning on the M-SCUC problem. The present invention also provides a proof of the policy convergence of the proposed LBD-SAC algorithm, mathematically proves the feasibility of the algorithm, and provides a theoretical basis for the application of the algorithm in the M-SCUC reliability problem.

[0111] This invention technically proposes a DRL algorithm for LBD-SAC for the first time, which is used to deal with the difficulties in solving the M-SCUC model due to its large scale, non-convexity, and complex uncertainty, while overcoming the key technical bottlenecks of the existing DRL in terms of low convergence and constraint satisfaction. The algorithm uses the idea of ​​LBD to establish a training mechanism assisted by task decomposition and optimization, thereby improving the efficiency of the intelligent agent's safety survey. At the same time, the algorithm incorporates CGAN into the environment to simulate the time-varying prediction error distribution of the CMDP environment. In addition, the algorithm also integrates the APDO security policy search method to effectively explore the optimal feasible strategy for LBD-SAC. It provides a safe, feasible, and efficient solution for M-SCUC.

[0112] The LBD-SAC method proposed in the present invention is suitable for solving uncertainty optimization models of power systems with larger scales, longer time scales, and complex models. It can accurately simulate the uncertainty of the actual system operating environment and achieve efficient decision-making. The algorithm proposed in the present invention greatly improves the safety and feasibility of DRL applications in the field of power systems, and makes certain contributions to promoting the intelligent development of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] Figure 1 This is the principle architecture diagram of CGAN;

[0114] Figure 2 This is the diagram of the proposed LBD-based training mechanism;

[0115] Figure 3 The architecture diagram of the proposed M-SCUC solution method based on LBD-SAC;

[0116] Figure 4 It is the unit commitment decision made by LBD-SAC under the IEEE-118 test system;

[0117] Figure 5 This is a comparison chart of the training performance of each DRL algorithm; Figure 5 (a) is a comparison chart of reward training performance; Figure 5 (b) is a cost training performance comparison chart;

[0118] Figure 6 is the net load scenario set of the up / down regulation curves and obtained by each algorithm;

[0119] Figure 7 It is the frequency and distribution of over-limit of insufficient energy supply capacity in random scenarios;

[0120] Figure 8 It is the unit commitment decision made by LBD-SAC under the IEEE-300 test system;

[0121] Fig. 9 It is the pseudo code of LBD-SAC. DETAILED DESCRIPTION

[0122] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0123] Embodiment 1:

[0124] See also Figures 1 to 9 A monthly unit commitment method based on flexible actor-judge combined with logical Benders decomposition includes the following steps:

[0125] 1) Establish a monthly safety constraint unit combination model considering wind power uncertainty.

[0126] 2) Based on the conditional generative adversarial network, the time-varying wind power prediction error probability distribution is established.

[0127] 3) The logical Benders decomposition method is used to decouple discrete variables and continuous variables, and the M-SCUC model is decomposed into a unit commitment decision model characterized by a constrained Markov decision process framework and a relaxed optimal power flow model.

[0128] 4) Based on the unit combination decision model, the environment, state space, action space, state transfer function, cost function and reward function of the unit combination decision main problem M1 are established.

[0129] Based on the relaxed optimal power flow model, the objective function and constraints of the unit commitment decision subproblem M2 are established.

[0130] 5) Taking the time-varying wind power forecast error probability distribution as the uncertainty of the environment, the unit combination decision action of the unit combination decision main problem M1 is generated based on the flexible actor-judge deep reinforcement learning baseline algorithm, and the unit combination decision action is input as the boundary condition to the unit combination decision sub-problem M2.

[0131] 6) Based on the unit combination decision action of the unit combination decision main problem M1, solve the unit combination decision subproblem M2, and regard the relaxation level output by the unit combination decision subproblem M2 as the survey cost and substitute it into the unit combination decision main problem M1, and use the LBD-SAC algorithm to solve the unit combination decision main problem M1.

[0132] 7) Repeat steps 5) to 6) until the reward function meets the convergence requirements and output the final unit combination decision.

[0133] Embodiment 2:

[0134] A monthly unit combination method based on flexible actor-evaluator combined with logical Benders decomposition, the main technical content of which is shown in Example 1. Further, the monthly safety constraint unit combination model considering wind power uncertainty is as follows:

[0135]

[0136] Where t represents the time. T represents the dispatch period. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. A represents the expected comprehensive dispatch cost. t , D t , H t and M t Both represent coefficient matrices. t is the coefficient vector. x 、N y 、N z 、N ζ They represent the dimensions of the unit commitment decision variable vector x, the unit output power vector y, other state variables z, and the wind power prediction error vector ζ. M0 represents the monthly safety constraint unit commitment model.

[0137] Embodiment 3:

[0138] A monthly unit combination method based on a flexible actor-evaluator combined with a logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 2. Further, the step of establishing a time-varying wind power prediction error probability distribution based on a conditional generative adversarial network includes:

[0139] 2.1) Obtain wind power forecast data P d and historical measured data H d .

[0140] 2.2) Based on wind power forecast data P d , using the generator network to generate the simulated wind power data sequence G d .

[0141] 2.3) The simulated wind power data sequence G d With wind power forecast data P d Make stitching.

[0142] The wind power forecast data P d and historical measured data H d Make stitching.

[0143] 2.4) Input the spliced ​​data into the discriminator network to fit the time-varying wind power prediction error distribution.

[0144] 2.5) Based on the fitted time-varying wind power forecast error distribution, establish the time-varying wind power forecast error probability distribution.

[0145] Embodiment 4:

[0146] A monthly unit commitment method based on a flexible actor-evaluator combined with a logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 3. Further, the unit commitment decision model characterized by the constrained Markov decision process framework is as follows:

[0147]

[0148] Where t represents the time. T is the scheduling period. x represents the unit commitment decision variable vector. Represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. Represents the startup cost and shutdown cost of the unit. Indicates the expected operating cost. Represents the coefficient matrix of the unit commitment decision model characterized by the constrained Markov decision process framework. is the coefficient vector of the unit commitment decision model characterized by the constrained Markov decision process framework. N x Represents the dimension of the unit commitment decision variable vector x. M1 represents the unit commitment decision problem of unit start-stop state decision.

[0149] Embodiment 5:

[0150] A monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 4. Further, the relaxed optimal power flow model is as follows:

[0151]

[0152] Where t represents the time. T represents the scheduling period. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s represents the slack variable vector. Represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. Represents the probability distribution of time-varying wind power forecast error. represents the expected operating cost. t (·) represents the operating cost within a certain sampling prediction error. Both represent the coefficient matrices of the relaxed optimal power flow model. is the coefficient vector of the relaxed optimal power flow model. N y 、N z 、N ζ 、N s They represent the dimensions of the unit output power vector y, other state variables z, wind power prediction error vector ζ, and slack variable vector s respectively. M2 represents the relaxed optimal power flow model under the on / off state of the fixed unit.

[0153] Embodiment 6:

[0154] A monthly unit commitment method based on a flexible actor-evaluator combined with a logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 5. Further, the state space of the unit commitment decision main problem M1 is as follows:

[0155]

[0156] Where t represents the time. W, I, and G represent the set of wind turbine generators, nodes, and units, respectively. Represents the state space. t Represents the state at time t. They represent the vector of predicted wind power output and the vector of predicted wind power load at time t respectively. Indicates the on / off status of the unit at time t-1. Indicates the duration of the unit status at time t-1.

[0157] Among them, the first-order difference form of wind power output prediction in adjacent time periods is As shown below:

[0158]

[0159] In the formula, represents the vector of predicted wind power output at time t-1. Δt represents the time difference.

[0160] The action space of the main unit commitment decision problem M1 is as follows:

[0161]

[0162] In the formula, represents the action space. t Indicates the action at time t. N is the number of units. Indicates the on / off status of the unit at time t.

[0163] The state transfer function of the unit commitment decision main problem M1 is as follows:

[0164]

[0165] In the formula, Indicates the duration of the unit state at time t.

[0166] The cost function of the main unit commitment decision problem M1 is as follows:

[0167]

[0168] In the formula, c π,ρ represents the cost function. ζ represents the wind power prediction error vector. π represents the action strategy. They represent the constraint limit of the unit commitment decision model represented by the constrained Markov decision process framework and the relaxation degree of the relaxed optimal power flow model. g is the unit number. i is the node number. UT g , DT g They respectively represent the minimum startup time and minimum shutdown time of the g-th unit. They represent the on / off status of the g-th unit at time t and t-1 respectively. is the state duration of the g-th unit at time t-1. (·)+ represents max{·,0}. is the slack variable. represents the time-varying wind power forecast error probability distribution. ρ represents the forecast error probability.

[0169] The reward function of the main unit commitment decision problem M1 is as follows:

[0170]

[0171] In the formula, r(s t ,a t ) represents the reward function. represents the on / off state of the unit at time t obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. t (·) represents the startup cost and shutdown cost of the unit. Indicates the expected operating cost. represents the penalty function for violating the constraint. Indicates the power of the gth unit. g , b g 、c g 、h g ,d g They represent the constant cost coefficient, first-order cost coefficient, second-order cost coefficient, startup cost, and shutdown cost of the g-th unit respectively. v , κ δ All represent cost coefficients. x represents the unit combination decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. represents the probability distribution of time-varying wind power forecast error. t (·) represents the operating cost within a certain sampling prediction error.

[0172] The environment of the unit commitment decision main problem M1 integrates the reward and cost functions involving the unit commitment decision model represented by the constrained Markov decision process framework and the relaxed optimal power flow model, the probability distribution of the time-varying wind power prediction error, and the state transfer function.

[0173] Embodiment 7:

[0174] A monthly unit commitment method based on a flexible actor-evaluator combined with logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 6. Further, the objective function of the unit commitment decision sub-problem M2 is as follows:

[0175]

[0176] In the formula, t represents the time. g is the unit number. G represents the set of units. g , b g 、c g They respectively represent the constant cost coefficient, linear cost coefficient and quadratic cost coefficient of the g-th unit. Represents the power of the g-th unit at time t. κ δ represents the cost coefficient. i is the node number. I represents the set of nodes. is the slack variable.

[0177] The constraints of the unit combination decision sub-problem M2 include node power balance constraints, unit and wind power output power constraints, branch power flow constraints, start / shutdown decision constraints, and unit ramp constraints.

[0178] The node power balance constraint is as follows:

[0179]

[0180] In the formula, G i Represents the set of wind turbines at node i. w represents the wind turbine generator set number. i Represents the set of wind turbines at node i. represents the actual grid-connected power generation of the w-th wind power station. j is the node number. ij represents the branch from node i to node j. i Represents the branch set of node i. The vector representing the predicted wind power load at node i at time t. Represents the power flow of branch ij.

[0181] The output power constraints of the units and wind power are as follows:

[0182]

[0183] In the formula, represents the prediction error sample obtained by the w-th wind turbine from the environment. Represents the predicted output power of the w-th wind turbine. It represents the on / off state of the g-th unit at time t obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. P g They respectively represent the upper and lower bounds of the output power of the g-th unit.

[0184] The branch power flow constraints are as follows:

[0185]

[0186] In the formula, K i-j,i I represents the power distribution coefficient of branch ij and node i. i Represents the set of branches whose first segment node is i. They represent the upper and lower bounds of the power flow of branch ij respectively. Represents the active load power of node i.

[0187] The startup / shutdown decision constraints are as follows:

[0188]

[0189] In the formula, v g 、z g They respectively represent the startup and shutdown decisions of the g-th unit. It represents the on / off state of the g-th unit at time t-1 obtained by the unit commitment decision model characterized by the constrained Markov decision process framework.

[0190] The unit climbing constraints are as follows:

[0191]

[0192] In the formula, Indicates the power of the gth unit at time t-1. RU g , RD g ,SU g , SD g They respectively represent the ramp-up rate, ramp-down rate, startup rate and shutdown rate of the g-th unit.

[0193] Embodiment 8:

[0194] A monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition, the main technical content of which is shown in any one of embodiments 1 to 7. Further, the step of solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm includes:

[0195] 6.1) Introduce entropy constraints and cost constraints as follows:

[0196]

[0197] Where θ represents the neural network parameters of the action strategy π, represents the entropy threshold. t represents the time. T is the scheduling period. π represents the action strategy. s t Indicates the state at time t. t Indicates the action at time t. t =(s t ,a t ,…,s T ,a T ) represents the decision trajectory of the probability distribution of time-varying wind power forecast error starting from time t. r(s t ,a t ) represents the reward function. Q c (s t ,a t ) represents the expected value of the cumulative discounted cost of following strategy π and trajectory χt. represents the discounted cost threshold, where d→0. ζ represents the wind power forecast error vector. ρ| t represents the probability distribution of time-varying wind power forecast error. π,ρ is the cost function. γ is the discount rate.

[0198] 6.2) Construct the LBD-SAC solution module and initialize the number of iterations k = 1.

[0199] The LBD-SAC solution module includes a CGAN module, two reward value judgment network modules, two cost value judgment network modules, and an action network module.

[0200] The CGAN module is used to generate a dataset with time-varying uncertainty distribution.

[0201] Two reward value judgment network modules are used to evaluate the soft Q value of the reward.

[0202] A cost-value judgment network module is used to evaluate the soft Q-value of the constraint crossing cost of the unit commitment decision model represented by the constrained Markov decision process framework.

[0203] A cost-value judgment network module is used to calculate the relaxation level of the relaxed optimal power flow model.

[0204] The action network makes unit combination decisions based on the current system operating status.

[0205] 6.3) Based on the distance-based 2-Wasserstein loss function, the target network parameters of the reward value judgment network module and the cost value judgment network module are calculated.

[0206] The target network parameters of the reward value judgment network module and the cost value judgment network module are as follows:

[0207]

[0208] In the formula, are the soft Q functions of reward and cost respectively. J c (μ) are soft Q functions Approximate value of . Both are Soft-Bellman operators. They represent the target network parameters of the reward value judgment network module and the cost value judgment network module respectively. is the set of unit commitment decisions.

[0209] 6.4) Based on the APDO method, a Lagrangian function is established to transform the entropy constraint and cost constraint maximization problem into a dual optimization problem, and the loss function of the action network module is derived.

[0210] 6.5) Determine whether the number of iterations k is equal to the preset threshold. If not, store the Lagrange multiplier in this iteration into the experience pool, set k = k + 1, update the action strategy and Lagrange multiplier, and return to step 6.3). If yes, sample and select a Lagrange multiplier from the experience pool to adjust the Lagrange multiplier of the last iteration, and enter step 6.6).

[0211] 6.6) Output the optimal solution for unit combination decision.

[0212] Embodiment 9:

[0213] A monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 8, and further, the Lagrangian function is as follows:

[0214]

[0215] Where α and λ are Lagrange multipliers related to entropy value and safety survey cost. is the Lagrangian function. θ For action strategy. Represents the entropy threshold. is the cost function.

[0216] The dual optimization problem is as follows:

[0217]

[0218] In the formula, is the optimal primal-dual solution.

[0219] The loss function of the action network module is as follows:

[0220]

[0221] In the formula, J π (θ) is the loss function of the action network module.

[0222] Embodiment 10:

[0223] A monthly unit commitment method based on a flexible actor-evaluator combined with a logical Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 9. Further, the calculation formula of the update action strategy and the Lagrange multiplier is as follows:

[0224]

[0225] In the formula, θ (k+1) ,θ (k) Respectively represent the neural network parameters in the k+1th and kth iterations. ω is the learning rate. ▽ is the gradient function. α (k) , (k) is the Lagrange multiplier in the kth iteration process. (k+1) , (k+1) is the Lagrange multiplier in the k+1th iteration process. is the action strategy in the k+1th iteration.

[0226] Embodiment 11:

[0227] See also Figures 1 to 9A monthly unit commitment method based on flexible actor-judge combined with logical Benders decomposition is proposed. The main steps are as follows:

[0228] 1) Establish a monthly safety constraint unit combination model considering wind power uncertainty. The monthly safety constraint unit combination model can be regarded as minimizing the expected comprehensive dispatch cost under the physical constraints of the system's overall and individual adjustable resources and probabilistic resources. This paper calls it the original problem (M0), and its compact form is as follows:

[0229]

[0230] In the formula, the superscript t represents the time; T is the dispatch period; x represents the unit combination decision variable vector, which involves the start and stop status of the unit; ζ represents the wind power prediction error vector, which obeys the time-varying conditional probability distribution ρ| t ; y represents the unit output power vector; z represents other state variables, including node voltage and branch power flow; A represents the expected comprehensive dispatching cost; t , D t , H t and M t represents the coefficient matrix; et is the coefficient vector; N x 、N y 、N z 、N ζ denotes the dimension of the respective variable.

[0231] 2) Establish a time-varying wind power forecast error probability distribution model based on CGAN. The idea of ​​modeling the time-varying wind power forecast error probability distribution based on CGAN can be found in Figure 1 .exist Figure 1 In the generator network (Generator, G), based on the predicted data P d and historical measured data H d Capture the time correlation and distribution characteristics of wind power forecast errors. The simulated wind power data series G d In the prediction data P d The discriminator network (Discriminator, G) evaluates the input data from H d The training goal of G is to maximize the probability of D's evaluation error, while the goal of D is to minimize this probability. Through adversarial interactive training between G and D, G is able to accurately fit the time-varying wind power forecast error distribution. The trained generator G* can accurately simulate the uncertain distribution that changes over time in the M-SCUC problem.

[0232] 3) A DRL training mechanism based on LBD is proposed. In view of the challenges brought by the complexity of the M-SCUC model (M0) considering wind power uncertainty mentioned in 1) to the deployment and application of DRL, the present invention develops a training mechanism based on LBD. The LBD method decomposes the task into decision-related main problems and sub-problems. Compared with the traditional BD method, it liberates the form of the main problem and sub-problems. The sub-problems tighten the solution space of the main problem through iterative logical deduction, and finally converge to the feasible domain of the original problem. In view of the task decomposition idea and logical reasoning strategy of LBD, the DRL training mechanism based on LBD proposed in the present invention is as follows Figure 2 shown.

[0233] According to the specific structure of M0, it is decomposed into a main problem (M1) and a sub-problem (M2) by variable type and model function classification. M1 represents the unit combination decision problem of unit start-stop state decision, and the compact form of the model is shown in formula (2). M2 represents the relaxed optimal power flow model under the fixed unit on / off state, and the compact form of the model is shown in formula (3).

[0234]

[0235] in, represents the fixed unit commitment decision obtained by M1; represents the startup cost and shutdown cost of the unit; s is the slack variable vector, which represents the degree of relaxation of M1 and serves as a feasible Benders cut signal; represents the expected operating cost; ft() represents the operating cost within a certain sampling prediction error.

[0236] In the proposed LBD-based training mechanism, the constraint limit of M1 and the relaxation degree of M2 are regarded as the safe exploration cost of the agent in the decision space. The training goal is to limit these costs to approach the feasible domain of M0. and the operating costs of M2 It is regarded as a negative reward. Then, through logical deduction, a tightening safe exploration space is provided for the agent of M1. The iterative interaction between M1 and M2 at each time step t trains the agent to explore the optimal feasible solution within the entire scheduling period T. The LBD-based training mechanism effectively reduces the dimensions of the action and state space through task decomposition, thereby improving scalability and convergence. By optimizing auxiliary training, the satisfaction of operation constraints is significantly improved.

[0237] 4) Establish the CMDP model of the main problem M1 of unit commitment decision. The CMDP model of M1 is defined as a tuple in represents the state space; represents the action space; is the state transfer function; r represents Choose ) received; c represents the reward obtained in s t Select a t The obtained safety survey cost; d is the cost threshold; γ (γ∈[0,1]) represents the discount rate. The environment and key elements involved in the CMDP model of M1 are as follows:

[0238] 4.1) Environment: The environment integrates the reward and cost functions involving M1 and M2, the time-varying wind power error distribution fitted from CGAN, and the state transition function. The agent makes unit commitment decisions based on the action strategy and the perceived operating state. When calculating the reward and cost, the wind power forecast error in CGAN is sampled using Monte Carlo simulation. It then transfers to the next state according to the state transition function.

[0239] 4.2) State space: The definition of state space is shown in formula (4), which includes the source load forecast information and trend, the start and stop status of the unit at the previous moment and the duration of the state.

[0240]

[0241] In the formula, the subscripts W, I, and G represent the set of wind turbines, nodes, and units, respectively; A vector representing the predicted wind power output and load; Indicates the on / off status of the unit; Indicates the duration of the unit status; It represents the first-order difference form of the predicted wind power output in adjacent time periods. This difference form is introduced to consider the prediction trend, as follows:

[0242]

[0243] 4.3) Action Space: The action space involves the decision variables x in M1, which make unit combination decisions to determine the on / off status of the units. An action is defined as a binary vector of N units as follows:

[0244]

[0245] 4.4) State transfer function: For exogenous states that are not affected by actions, their probability transfer characteristics can be learned directly from the dataset. In this paper, in order to solve the problem of scarce historical data, CGAN generates such a dataset with time-varying uncertainty distribution. As for the endogenous state that can respond to action feedback, including the duration of the unit state, it is as follows:

[0246]

[0247] 4.5) Cost function: The cost function maps the state and unit combination decision to the safety survey cost, as defined in equations (8)-(9). The cost function provides a basis for the agent to tighten the solution space to the feasible domain of M0, thereby ensuring the safety of operation.

[0248]

[0249] Where: π represents the action strategy; and They represent the constraint violation of M1 and the relaxation degree obtained by M2 respectively; (·)+ represents max{·,0}; UT g , DT g They represent the minimum startup time and minimum shutdown time of the g-th unit respectively. is a slack variable, which represents the slack power of the i-th node.

[0250] 4.6) Reward function: The reward function is composed of the negative value of the objective function of M1 and the penalty function for violating the constraints. The reward function encourages the agent to explore the optimal solution, as defined in equations (10)-(11).

[0251]

[0252] In the formula, represents the penalty function for violating the constraint; represents the power of the g-th unit; a g 、b g 、c g 、h g d g They represent the constant cost coefficient, first-order cost coefficient, second-order cost coefficient, startup cost and shutdown cost of the g-th unit respectively; κ v , κ δ denote the cost coefficients, which are incorporated into the reward function to improve the efficiency of constraint learning.

[0253] The CMDP model takes the cost threshold as the cost constraint. As defined in equation (12), the goal of the training strategy is to maximize the cumulative discounted reward while satisfying the cost constraint. This lays the foundation for the model to explore the optimal and feasible unit combination decision-making.

[0254]

[0255] In the formula, χ t =(s t ,a t ,…,s T ,a T ) represents the decision trajectory starting from time t; Q c (st ,a t ) represents the trajectory χ that follows the policy π and the prediction error distribution ρ|t t the expected value of the cumulative discounted costs incurred; represents the discounted cost threshold, where d→0, indicating that the operation constraints need to be met.

[0256] 5) Establish a sub-problem model (M2) based on the relaxed optimal power flow. M2 is the expected relaxation level and operating cost of multiple sampling scenarios in the computing environment. It evaluates the feasibility and optimality of the unit combination decision made by M1 under uncertain conditions. The relaxed optimal power flow model of a certain sampling scenario is defined in equations (14)-(23). Among them, the f involved t The objective function is expressed as equation (14). The node power balance constraint is defined as equation (15). Constraints (16)-(17) impose limits on the output power of the unit and wind power, respectively. Constraints (18)-(19) constrain the branch power flow. Equations (20)-(21) map the on / off state of the unit to the start / shutdown decision. The unit ramp limit is established by equations (22)-(23).

[0257]

[0258] In the formula, the symbols “-” and “_” are the upper and lower bounds of each variable respectively; the subscript ij represents the branch from node i to j; B represents the branch set; K i-j,i represents the power flow distribution coefficient of branch ij and node i; represents the actual grid-connected power generation of the w-th wind power station; represents the predicted output power of the w-th wind turbine; represents the prediction error sample obtained from the environment; v g 、z g Respectively represent the startup and shutdown decisions of the g-th unit; RU g , RD g ,SU g , SD g They represent the ramp-up, ramp-down, startup and shutdown rates of the g-th unit respectively.

[0259] 6) Establish the M-SCUC solution method based on the LBD-SAC algorithm

[0260] LBD-SAC uses the SAC baseline DRL framework to parameterize the CMDP. To encourage the agent to fully explore the solution space of M1, LBD-SAC introduces entropy constraints and cost constraints, as shown in (24). Secondly, the APDO-based constraint strategy search method is regarded as a logical deduction process of the LBD training mechanism, which is used to drive the CMDP parameterized by the neural network to achieve maximum entropy adaptive learning. LBD-SAC ensures the feasibility of the decision and achieves the global optimality of M-SCUC.

[0261]

[0262] Where θ represents the neural network parameters of the action strategy π, Represents the entropy threshold.

[0263] See the above process for Figure 3 , which describes the complete LBD-SAC algorithm. It consists of five core modules: CGAN, reward value critic networks (RV-CNs), cost value critic networks (CV-CNs), action network, and Lagrange multipliers. Detailed descriptions of these modules are provided below. Fig. 9 The pseudo code of the proposed LBD-SAC is provided.

[0264] 6.1) Reward Value Evaluation Network and Cost Value Evaluation Network, RV-CNs and CV-CNs evaluate the soft Q values ​​of rewards and costs respectively based on the current operating state and the unit combination decision made by the action strategy. RV-CNs and CV-CNs are parameterized by φ and μ respectively, and the mapping is constructed:

[0265] For the action strategy π, we define the Soft-Bellman operator and Rewards and cost The soft Q function is as follows:

[0266]

[0267] To approximate these soft Q functions, the distance-based 2-Wasserstein loss function is given in (27) and (28) for RV-CN and CV-CN, respectively, which aims to minimize the Soft-Bellman residual.

[0268]

[0269] In the formula, It is an experience replay pool, which is used to ensure full utilization of exploration experience and avoid heavy training process; Represent the corresponding target network parameters respectively. The introduction of the target network is to improve the stability of training. The target network performs soft updates as the exponential moving average of the evaluation network. Figure 3 The Double RV-CNs shown conservatively estimate the Q-value of the reward, which is believed to reduce overestimation during policy improvement. The two CV-CNs are used to estimate the soft Q-value of the M1 constraint violation cost and the relaxation level of the M2 calculation, respectively.

[0270] 6.2) Based on the action network and Lagrange multipliers of APDO, the action network is an action strategy that makes unit commitment decisions according to the current system operation status. The action network involves an action strategy π with parameter θ, which constructs a mapping: In order to find the optimal strategy and improve the efficiency of safety survey, this patent introduces the APDO method. By establishing the Lagrangian function (see (29)), the constraint maximization problem is transformed into a dual optimization problem. The iteration between the original strategy and the Lagrangian multiplier of the Lagrangian function is the logical reasoning process in the LBD training mechanism. In addition, the Lagrangian multiplier trained by the offline strategy is incorporated into the dual update process, which improves the convergence performance of the action network and the Lagrangian multiplier.

[0271]

[0272] Where the Lagrange multipliers related to entropy and safety survey cost are parameterized by α and λ, respectively.

[0273] Then, the CMDP formula (24) is transformed into an unconstrained optimization problem as shown in (30).

[0274]

[0275] In the formula, is the optimal primal-dual solution.

[0276] In addition, the loss function of the action network is derived from formula (29), as shown in formula (31).

[0277]

[0278] The primal-dual update process for the kth iteration is as follows:

[0279] For strategy π θ The original update of , fix the Lagrange multiplier α (k) , (k) , and perform policy gradient ascent with a learning rate ω, as shown in (32).

[0280]

[0281] For the dual update of the Lagrange multipliers, fix And perform dual gradient ascent as shown in (33)-(34).

[0282]

[0283] After a certain number of iterations, the Lagrange multiplier α trained using the offline strategy off , off To α (k+1) , (k+1) Perform dual adjustment, that is, α off →α (k+1) ,λ off →λ (k+1) .

[0284] 7) Establish the convergence proof of the M-SCUC problem solving strategy based on the proposed LBD-SAC algorithm as follows:

[0285] To prove the convergence of the LBD-SAC algorithm, we first prove the boundedness of the slack variables in claim 6, that is, The proof is as follows:

[0286] Will Decomposition into positive slack variables and negative slack variables Right now in and Then equation (14) and equation (15) can be equivalently transformed into:

[0287]

[0288] Further deducing The value range is and The value range is This proves is bounded, indicating that the safety survey cost c(s t ,a t ) is bounded.

[0289] Secondly, the action strategy π in LBD-SAC θ The optimal convergence of is equivalent to the optimal convergence of the soft Q value, that is, for any The following inequality holds.

[0290]

[0291] To prove equation (37), let set up is the optimal solution of the formula (where n>k), which has the following definition:

[0292]

[0293] because Therefore, existence From this we can deduce:

[0294]

[0295] Where N c Indicates c π,ρ The dimension of .

[0296] Restate inequality (39) as (40).

[0297]

[0298] In the formula, and are the soft state value functions of reward and cost, respectively, as shown in (41) and (42).

[0299]

[0300] According to the soft Bellman equation, it can be further deduced that:

[0301]

[0302] It can be seen from (43) that the sequence shows a monotonically increasing trend. This indicates that each new action strategy improves the performance. In M-SCUC, due to the finite dimensionality of the action space, the reward r(s t ,a t ) and entropy are bounded, and the cost c(s t ,a t ) is proved to be bounded. It can be deduced that It is bounded on π∈Π, and the sequence converges to a certain action strategy Given that In logic, it is known Therefore, the soft Q value of the final converged strategy is better than the soft Q value of any other strategy in the range of Π, which confirms that Therefore, the strategy convergence of the proposed LBD-SAC algorithm is proved.

[0303] Embodiment 12:

[0304] See also Figures 1 to 9, a monthly unit commitment method based on flexible actor-judge combined with logical Benders decomposition, this embodiment is based on the improved IEEE-118 test system, and two wind farms are connected to busbars 9 and 51 respectively. The hourly load data comes from the measured data in a certain area of ​​China. The wind power generation data comes from the public Belgian wind farm data. The force data is linearly scaled to between 0% and 40% of the total unit capacity. This embodiment uses Gurobi, Pytorch and Tensorflow in Python, and is executed on an Intel Core i9-10900K CPU and an NVIDIA GeForce RTX 3090 GPU.

[0305] 1) Establish an M-SCUC model that takes into account wind power uncertainty. The M-SCUC model can be regarded as minimizing the expected comprehensive dispatch cost under the physical constraints of the system's overall and individual adjustable resources and probabilistic resources. This paper calls it the original problem (M0), and its compact form is as follows:

[0306]

[0307] In the formula, the superscript t represents the time; T is the dispatch period; x represents the unit combination decision variable vector, which involves the start and stop status of the unit; ζ represents the wind power prediction error vector, which obeys the time-varying conditional probability distribution ρ| t ; y represents the unit output power vector; z represents other state variables, including node voltage and branch power flow; A represents the expected comprehensive dispatching cost; t , D t , H t and M t represents the coefficient matrix; et is the coefficient vector; N x 、N y 、N z 、N ζ denotes the dimension of the respective variable.

[0308] 2) Establish a time-varying wind power forecast error probability distribution model based on CGAN. The idea of ​​modeling the time-varying wind power forecast error probability distribution based on CGAN can be found in Figure 1 .exist Figure 1 In the generator network (Generator, G), based on the predicted data P d and historical measured data H d Capture the time correlation and distribution characteristics of wind power forecast errors. The simulated wind power data series G d In the prediction data P d The discriminator network (Discriminator, G) evaluates the input data from H dThe training goal of G is to maximize the probability of D's evaluation error, while the goal of D is to minimize this probability. Through adversarial interactive training between G and D, G is able to accurately fit the time-varying wind power forecast error distribution. The trained generator G* can accurately simulate the uncertain distribution that changes over time in the M-SCUC problem.

[0309] 3) Execute the proposed LBD training mechanism. The LBD method decomposes the task into decision-related main problems and sub-problems. Compared with the traditional BD method, it liberates the form of the main problem and sub-problems. The sub-problems tighten the solution space of the main problem through iterative logical deduction, and finally converge to the feasible domain of the original problem. In view of the task decomposition idea and logical reasoning strategy of LBD, the DRL training mechanism based on LBD proposed in this invention is as follows Figure 2 As shown. According to the specific structure of M0, it is classified by variable type and model function and decomposed into a main problem (M1) and a sub-problem (M2). Among them, M1 represents the unit combination decision problem of unit start-stop state decision, and the compact form of the model is shown in formula (2). M2 represents the relaxed optimal power flow model under the fixed unit on / off state, and the compact form of the model is shown in formula (3).

[0310]

[0311] in, represents the fixed unit commitment decision obtained by M1; represents the startup cost and shutdown cost of the unit; s is the slack variable vector, which represents the degree of relaxation of M1 and serves as a feasible Benders cut signal; represents the expected operating cost; ft() represents the operating cost within a certain sampling prediction error.

[0312] In the proposed LBD-based training mechanism, the constraint limit of M1 and the relaxation degree of M2 are regarded as the safe exploration cost of the agent in the decision space. The training goal is to limit these costs to approach the feasible domain of M0. and the operating costs of M2 It is regarded as a negative reward. Then, through logical deduction, a tightening safe exploration space is provided for the agent of M1. The iterative interaction between M1 and M2 at each time step t trains the agent to explore the optimal feasible solution within the entire scheduling period T. The LBD-based training mechanism effectively reduces the dimensions of the action and state space through task decomposition, thereby improving scalability and convergence. By optimizing auxiliary training, the satisfaction of operation constraints is significantly improved.

[0313] 4) Establish the CMDP model of the main problem M1 of unit commitment decision. The CMDP model of M1 is defined as a tuple in represents the state space; represents the action space; is the state transfer function; r represents Choose The reward obtained; c means in s t Select a t The obtained safety survey cost; d is the cost threshold; γ (γ∈[0,1]) represents the discount rate. The environment and key elements involved in the CMDP model of M1 are as follows:

[0314] 4.1) Environment: The environment integrates the reward and cost functions involving M1 and M2, the time-varying wind power error distribution fitted from CGAN, and the state transition function. The agent makes unit commitment decisions based on the action strategy and the perceived operating state. When calculating the reward and cost, the wind power forecast error in CGAN is sampled using Monte Carlo simulation. It then transfers to the next state according to the state transition function.

[0315] 4.2) State space: The definition of state space is shown in formula (4), which includes the source load forecast information and trend, the start and stop status of the unit at the previous moment and the duration of the state.

[0316]

[0317] In the formula, the subscripts W, I, and G represent the set of wind turbines, nodes, and units, respectively; A vector representing the predicted wind power output and load; Indicates the on / off status of the unit; Indicates the duration of the unit status; It represents the first-order difference form of the predicted wind power output in adjacent time periods. This difference form is introduced to consider the prediction trend. It is as follows:

[0318]

[0319] 4.3) Action Space: The action space involves the decision variables x in M1, which make unit combination decisions to determine the on / off status of the units. An action is defined as a binary vector of N units as follows:

[0320]

[0321] 4.4) State transfer function: For exogenous states that are not affected by actions, their probability transfer characteristics can be learned directly from the dataset. In this paper, in order to solve the problem of scarce historical data, CGAN generates such a dataset with time-varying uncertainty distribution. As for the endogenous state that can respond to action feedback, including the duration of the unit state, it is as follows:

[0322]

[0323] 4.5) Cost function: The cost function maps the state and unit combination decision to the safety survey cost, as defined in equations (8)-(9). The cost function provides a basis for the agent to tighten the solution space to the feasible domain of M0, thereby ensuring the safety of operation.

[0324]

[0325] Where: π represents the action strategy; and They represent the constraint violation of M1 and the relaxation degree obtained by M2 respectively; (·)+ represents max{·,0}; UT g , DT g They represent the minimum startup time and minimum shutdown time of the g-th unit respectively. is a slack variable, which represents the slack power of the i-th node.

[0326] 4.6) Reward function: The reward function is composed of the negative value of the objective function of M1 and the penalty function for violating the constraints. The reward function encourages the agent to explore the optimal solution, as defined in equations (10)-(11).

[0327]

[0328] In the formula, represents the penalty function for violating the constraint; represents the power of the g-th unit; a g 、b g 、c g 、h g d g They represent the constant cost coefficient, first-order cost coefficient, second-order cost coefficient, startup cost and shutdown cost of the g-th unit respectively; κ v , κ δ denote the cost coefficients, which are incorporated into the reward function to improve the efficiency of constraint learning.

[0329] The CMDP model takes the cost threshold as the cost constraint. As defined in equation (12), the goal of the training strategy is to maximize the cumulative discounted reward while satisfying the cost constraint. This lays the foundation for the model to explore the optimal and feasible unit combination decision-making.

[0330]

[0331] In the formula, χ t =(s t ,a t ,…,s T ,aT ) represents the decision trajectory starting from time t; Q c (s t ,a t ) represents the trajectory χ that follows the policy π and the prediction error distribution ρ|t t the expected value of the cumulative discounted costs incurred; represents the discounted cost threshold, where d→0, indicating that the operation constraints need to be met.

[0332] 5) Establish a sub-problem model based on relaxed optimal power flow (M2)

[0333] M2 is the expected relaxation level and operating cost of multiple sampling scenarios in the computing environment. It evaluates the feasibility and optimality of the unit combination decision made by M1 under uncertainty. The relaxed optimal power flow model for a certain sampling scenario is defined in equations (14)-(23). Among them, the objective function involving ft is expressed by equation (14). The node power balance constraint is defined as equation (15). Constraints (16)-(17) impose restrictions on the output power of the unit and wind power respectively. Constraints (18)-(19) constrain the branch power flow. Equations (20)-(21) map the on / off state of the unit to a start / shutdown decision. The unit ramp limit is established by equations (22)-(23).

[0334]

[0335] In the formula, the symbols “-” and “_” are the upper and lower bounds of each variable respectively; the subscript ij represents the branch from node i to j; B represents the branch set; K i-j,i represents the power flow distribution coefficient of branch ij and node i; represents the actual grid-connected power generation of the w-th wind power station; represents the predicted output power of the w-th wind turbine; represents the prediction error sample obtained from the environment; v g 、z g Respectively represent the startup and shutdown decisions of the g-th unit; RU g , RD g ,SU g , SD g They represent the ramp-up, ramp-down, startup and shutdown rates of the g-th unit respectively.

[0336] 6) Establish the M-SCUC solution method based on the LBD-SAC algorithm

[0337] LBD-SAC uses the SAC baseline DRL framework to parameterize the CMDP. To encourage the agent to fully explore the solution space of M1, LBD-SAC introduces entropy constraints and cost constraints, as shown in (24). Secondly, the APDO-based constraint strategy search method is regarded as a logical deduction process of the LBD training mechanism, which is used to drive the CMDP parameterized by the neural network to achieve maximum entropy adaptive learning. LBD-SAC ensures the feasibility of the decision and achieves the global optimality of M-SCUC.

[0338]

[0339] Where θ represents the neural network parameters of the action strategy π, Represents the entropy threshold.

[0340] See the above process for Figure 3 , which describes the complete LBD-SAC algorithm. It consists of five core modules: CGAN, reward value critic networks (RV-CNs), cost value critic networks (CV-CNs), action network, and Lagrange multipliers. Detailed descriptions of these modules are provided below.

[0341] 6.1) Reward Value Evaluation Network and Cost Value Evaluation Network, RV-CNs and CV-CNs evaluate the soft Q values ​​of rewards and costs respectively based on the current operating state and the unit combination decision made by the action strategy. RV-CNs and CV-CNs are parameterized by φ and μ respectively, and the mapping is constructed:

[0342] For the action strategy π, we define the Soft-Bellman operator and Rewards and cost The soft Q function is as follows:

[0343]

[0344] To approximate these soft Q functions, the distance-based 2-Wasserstein loss function is given in (27) and (28) for RV-CN and CV-CN, respectively, which aims to minimize the Soft-Bellman residual.

[0345]

[0346] In the formula, It is an experience replay pool, which is used to ensure full utilization of exploration experience and avoid heavy training process; They represent the corresponding target network parameters respectively. The introduction of the target network is to improve the stability of training. The target network performs soft updates as the exponential moving average of the judgment network.

[0347] exist Figure 3 The Double RV-CNs shown conservatively estimate the Q-value of the reward, which is believed to reduce overestimation during policy improvement. The two CV-CNs are used to estimate the soft Q-value of the M1 constraint violation cost and the relaxation level of the M2 calculation, respectively.

[0348] 6.2) Based on the action network and Lagrange multipliers of APDO, the action network is an action strategy that makes unit commitment decisions according to the current system operation status. The action network involves an action strategy π with parameter θ, which constructs a mapping: In order to find the optimal strategy and improve the efficiency of safety survey, this patent introduces the APDO method. By establishing the Lagrangian function (see (29)), the constraint maximization problem is transformed into a dual optimization problem. The iteration between the original strategy and the Lagrangian multiplier of the Lagrangian function is the logical reasoning process in the LBD training mechanism. In addition, the Lagrangian multiplier trained by the offline strategy is incorporated into the dual update process, which improves the convergence performance of the action network and the Lagrangian multiplier.

[0349]

[0350] Where the Lagrange multipliers related to entropy and safety survey cost are parameterized by α and λ, respectively.

[0351] Then, the CMDP formula (24) is transformed into an unconstrained optimization problem as shown in (30).

[0352]

[0353] In the formula, is the optimal primal-dual solution.

[0354] In addition, the loss function of the action network is derived from formula (29), as shown in formula (31).

[0355]

[0356] The primal-dual update process for the kth iteration is as follows: For strategy π θ The original update of , fix the Lagrange multiplier α (k) , (k) , and perform policy gradient ascent with a learning rate ω, as shown in (32).

[0357]

[0358] For the dual update of the Lagrange multiplier, fix π θ (k+1) And perform dual gradient ascent as shown below.

[0359]

[0360]

[0361] After a certain number of iterations, the Lagrange multiplier α trained using the offline strategy off , off To α (k+1) , (k+1) Perform dual adjustment, that is, α off →α (k+1) ,λ off →λ (k+1) .

[0362] 7) Verification of the effectiveness of the method proposed in the present invention

[0363] The following six methods are established to demonstrate the performance of the LBD-SAC algorithm proposed in the present invention in terms of M-SCUC solution training performance, economy, decision quality and solution efficiency. M1: The LBD-SAC algorithm proposed in the present invention, the network architecture and parameters of LBD-SAC are shown in Table 1, and the network architecture of CGAN in its environment is shown in Table 2-3. The unit combination decision made by LBD-SAC can be found in Figure 4 .

[0364] Table 1 LBD-SAC neural network architecture and parameters of the IEEE-118 test system

[0365]

[0366] Table 2 Structure and parameters of CGAN generator network

[0367]

[0368] Table 3 Structure and parameters of CGAN discriminator network

[0369]

[0370] 7.1) Training performance analysis, the rewards and total safety survey costs of Algorithms 1-3 change with the number of training iterations. Figure 5 . Figure 5(a) shows that the reward of LBD-SAC stabilizes quickly after a brief oscillation and converges in 4041 cycles. Compared with C-SAC and SAC, LBD-SAC has higher convergence efficiency and higher rewards. Figure 5 In (b), the cost of LBD-SAC is significantly reduced and close to zero. This indicates that there is no constraint violation, indicating that the agent's exploration space is close to the feasible domain of M0. These advantages come from the proposed LBD training mechanism based on task decomposition and optimization assistance and the accelerated constraint policy search method APDO. These improvements significantly improve the safe exploration capability and convergence of DRL for solving large-scale, long-period M-SCUC problems.

[0371] 7.2) Economic Analysis,This section compares and analyzes the operating costs of algorithms 1-6 to evaluate their economic efficiency.,Specifications are shown in Table 4.

[0372] Table 4 Running costs of various algorithms

[0373]

[0374] As can be seen from Table 1, M1 exhibits lower operating costs in solving the M-SCUC problem compared to M2-M3. This difference is related to the limited safety survey capabilities of C-SAC and SAC, which hinder the making of reasonable unit combination decisions and lead to a large number of out-of-limit penalties. In addition, the cost of M1 is slightly increased by 1.0% compared to M4. This increase is attributed to the more comprehensive perception of the uncertainty distribution by LBD-SAC, which leads to the startup of more units to deal with uncertainties, which is proven to enhance safety in the following subsections. Due to the more conservative RO of M5, its operating cost is significantly higher than that of M1 by 9.2%. However, the economic benefits of the LBD-SAC proposed in M1 are not as good as the DA in M6 that does not consider uncertainty.

[0375] 7.3) Analysis of unit commitment decision quality. In this section, 1500 random wind power scenarios generated by CGAN are used to evaluate the system regulation capability a posteriori to evaluate the unit commitment decision quality of all algorithms. The up / down regulation curves of the system are determined by the maximum / minimum load capacity based on the established unit commitment decision. The up / down regulation curves and net load scenario sets obtained by each method are shown in Figure 6 In addition, Figure 7 The frequency and distribution of under-capacity crossings are shown for random scenarios. This details situations where the over-capacity or under-capacity crossings occur, depicting and Here, and denote the maximum and minimum load capacities in the tth cycle, respectively. In addition, in order to quantify the risk of the unit commitment decision of each algorithm, two evaluation indicators are defined in (35). Where (35a) represents the probability of insufficient power regulation capacity, and θ P It indicates that it reflects the probability of load shedding or wind curtailment. (35b) indicates the expectation of load shedding and wind curtailment. The evaluation indicators of each method are listed in Table 2.

[0376]

[0377]

[0378] Where Ns is the number of random scenes; They respectively represent the occurrence of load shedding and wind curtailment in the tth period in the sth scenario. If so, the value is 1, otherwise, the value is 0.

[0379] Table 5 Risk metrics for each algorithm

[0380]

[0381] Depend on Figure 6 It can be seen that the width of the net load confidence interval varies with time, indicating a time-varying uncertainty distribution. Compared with M2-M6, the up-down adjustment curves generated by M1 most completely envelop the 100% confidence interval of the net load. Figure 7 From the frequency and distribution of limit violations described in , and the risk indicators described in Table 5 , it can be seen that M1 has the lowest risk of power supply and demand imbalance. Figure 6 The regulation curves obtained by M2-M3 show frequent start-stop and large power imbalance of the units, which indicates that the unit combination decision is significantly unreasonable. This further verifies the advantages of the proposed LBD-SAC in improving the satisfaction of DRL constraints. Compared with M1, M4 is less sensitive to uncertainty. Figure 6 As shown in Figure 2, M4 exhibits insufficient reserves during low net load demand periods (e.g., during 360-420 hours) and peak load periods (e.g., during 690-720 hours). Figure 7 The lack of regulation capability is more obvious in M4, where the probability of power mismatch is twice as high as that in M1. The main reason for this difference is that the SDDIP in M4 relies on a limited set of scenarios to represent uncertainty. In contrast, LBD-SAC has been trained through billions of simulations of interactions with uncertain environments. This accumulated experience has significantly enhanced the ability of LBD-SAC to effectively perceive and handle uncertainty. In M5, the conservative unit commitment decisions made by the RO ensure strong upward regulation capability, which is reflected in the fact that there are almost no load shedding events, such as Figure 7However, RO failed to fully coordinate the ability to adjust up and down, resulting in a higher risk of imbalance in power supply and demand than M1. This highlights the advantages of LBD-SAC in balancing power supply reliability and wind power absorption capacity. In addition, the data analysis in M6 only focuses on forecast information, so that the adjustable capacity of the decision-making results only covers 60% of the net load confidence interval. This will lead to a load shedding and wind and solar abandonment probability of up to 2.2087%, and the expected gap will reach 2811.7MW.

[0382] In summary, the LBD-SAC algorithm achieves the optimal unit commitment decision quality with the lowest operation risk in the M-SCUC problem due to its superior uncertainty perception and decision-making ability. LBD-SAC achieves a good balance between robustness and economic benefits, ensuring the economic and safe operation of the power system.

[0383] 7.4) Computational Efficiency Analysis,This section aims to demonstrate the superior computational performance of the,proposed method by providing the solution time of M1 and M4-M6.,The results are shown in Table 6.

[0384] Table 6 Solution time of each algorithm

[0385]

[0386] The results in Table 6 show that M4-M6 takes significantly longer to solve, requiring days to solve the complex M-SCUC problem. In stark contrast, M1 achieves a solution in seconds, with an efficiency improvement of more than 99.9986% (69,000 times) relative to M4-M6. This significant performance leap is attributed to LBD-SAC, which makes unit combination decisions through forward propagation calculations in the action network. This replaces the modeling and iterative optimization process used in optimization-based methods, greatly improving solution efficiency.

[0387] Embodiment 13:

[0388] See also Figures 1 to 9 , a monthly unit combination method based on flexible actor-evaluator combined with logical Benders decomposition. The main technical content is shown in Example 12. Furthermore, this embodiment is based on the improved IEEE-300 test system, and the five wind farms are connected to busbars 3, 31, 109, 190 and 268 respectively. The hourly load data comes from the measured data in a certain area of ​​​​China. The wind power generation data comes from the public Belgian wind farm data. The force data is linearly scaled to between 0%-40% of the total capacity of the unit. This embodiment uses Gurobi, Pytorch and Tensorflow in Python, and is executed on Intel Core i9-10900K CPU and NVIDIA GeForce RTX 3090GPU. See the unit combination decision results for details. Figure 8 The neural network architecture and parameters of LBD-SAC are shown in Table 7.

[0389] Table 7 LBD-SAC neural network architecture and parameters of the IEEE-300 test system

[0390]

[0391] Table 8 Solution results of various algorithms based on IEEE-300 test system

[0392]

[0393] From the convergence points in Table 8, it can be seen that M1 has the best convergence efficiency compared with M2-M3. M1 shows the lowest risk among the risk indicators. This shows that for M-SCUC problems of different scales, the decisions made by LBD-SAC are still safer than other methods. In addition, the analysis of the solution time shows that as the scale of the system increases, the challenges in computational time cost and computing power of the optimization methods in M4-M6 become more obvious. The proposed LBD-SAC method effectively alleviates the computational burden brought by the M-SCUC problem with NP-hard. As the scale of the system increases, its advantage in solution efficiency becomes more obvious, which is more than 99.9994% higher than that of M4-M6.

Claims

1. A monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition, characterized in that: The following steps are involved: 1) Establish a monthly safety constraint unit combination model considering wind power uncertainty; 2) Based on the conditional generative adversarial network, the probability distribution of time-varying wind power forecasting error is established; 3) The logical Benders decomposition method is used to decouple discrete variables and continuous variables, and the M-SCUC model is decomposed into a unit commitment decision model characterized by a constrained Markov decision process framework and a relaxed optimal power flow model. 4) Based on the unit combination decision model, the environment, state space, action space, state transfer function, cost function and reward function of the unit combination decision main problem M1 are established; Based on the relaxed optimal power flow model, the objective function and constraints of the unit commitment decision subproblem M2 are established. 5) Taking the probability distribution of time-varying wind power forecast error as the uncertainty of the environment, the unit combination decision action of the unit combination decision main problem M1 is generated based on the flexible actor-judge deep reinforcement learning baseline algorithm, and the unit combination decision action is input as the boundary condition to the unit combination decision sub-problem M2; 6) Based on the unit combination decision action of the unit combination decision main problem M1, solve the unit combination decision subproblem M2, and substitute the relaxation level output by the unit combination decision subproblem M2 into the unit combination decision main problem M1 as the survey cost, and solve the unit combination decision main problem M1 using the LBD-SAC algorithm; 7) Repeat steps 5) to 6) until the reward function meets the convergence requirements and output the final unit combination decision.

2. According to claim 1, a monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition is characterized in that: The monthly safety constraint unit combination model considering wind power uncertainty is as follows: In the formula, t represents the time; T represents the dispatch period; x represents the unit combination decision variable vector; ζ represents the wind power prediction error vector; y represents the unit output power vector; z represents other state variables; f0 t (·) represents the expected comprehensive dispatching cost; A t , D t , H t and M t Both represent coefficient matrices; e t is the coefficient vector; N x 、N y 、N z 、N ζ They respectively represent the dimensions of the unit commitment decision variable vector x, the unit output power vector y, other state variables z, and the wind power prediction error vector ζ; M0 represents the monthly safety constraint unit commitment model.

3. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1 is characterized in that: The step of establishing a time-varying wind power prediction error probability distribution based on a conditional generative adversarial network comprises: 2.1) Obtain wind power forecast data P d and historical measured data H d ; 2.2) Based on wind power forecast data P d , using the generator network to generate the simulated wind power data sequence G d ; 2.3) The simulated wind power data sequence G d With wind power forecast data P d To splice; The wind power forecast data P d and historical measured data H d To splice; 2.4) Input the spliced ​​data into the discriminator network to fit the time-varying wind power prediction error distribution; 2.5) Based on the fitted time-varying wind power forecast error distribution, establish the time-varying wind power forecast error probability distribution.

4. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1 is characterized in that: The unit commitment decision model represented by the constrained Markov decision process framework is as follows: In the formula, t represents the time; T is the scheduling period; x represents the unit commitment decision variable vector; represents the fixed unit commitment decision obtained by the unit commitment decision model represented by the constrained Markov decision process framework; f1 t (·) represents the startup cost and shutdown cost of the unit; represents the expected operating cost; represents the coefficient matrix of the unit commitment decision model characterized by the constrained Markov decision process framework; is the coefficient vector of the unit commitment decision model represented by the constrained Markov decision process framework; N x Represents the dimension of the unit combination decision variable vector x; M1 represents the unit combination decision problem of unit start-stop status decision.

5. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1 is characterized in that: The relaxed optimal power flow model is as follows: In the formula, t represents the time; T represents the scheduling period; x represents the unit combination decision variable vector; ζ represents the wind power prediction error vector; y represents the unit output power vector; z represents other state variables; s represents the slack variable vector; represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework; represents the probability distribution of time-varying wind power forecast error; represents the expected operating cost; f t (·) represents the operating cost within a certain sampling prediction error; All represent the coefficient matrix of the relaxed optimal power flow model; is the coefficient vector of the relaxed optimal power flow model; N y 、N z 、N ζ 、N s They respectively represent the dimensions of the unit output power vector y, other state variables z, wind power prediction error vector ζ, and relaxation variable vector s; M2 represents the relaxed optimal power flow model under the fixed unit on / off state.

6. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1, characterized in that: The state space of the main unit commitment decision problem M1 is as follows: In the formula, t represents the time; W, I, and G represent the set of wind turbine generator sets, nodes, and units, respectively; represents the state space; s t represents the state at time t; They represent the vector of predicted wind power output and the vector of predicted wind power load at time t respectively; Indicates the on / off status of the unit at time t-1; Indicates the duration of the unit state at time t-1; Among them, the first-order difference form of wind power output prediction in adjacent time periods is As shown below: In the formula, represents the vector of predicted wind power output at time t-1; Δt represents the time difference; The action space of the main unit commitment decision problem M1 is as follows: In the formula, represents the action space; a t Indicates the action at time t; N is the number of units; Indicates the on / off status of the unit at time t; The state transfer function of the unit commitment decision main problem M1 is as follows: In the formula, Indicates the duration of the unit state at time t; The cost function of the main unit commitment decision problem M1 is as follows: In the formula, c π,ρ represents the cost function; ζ represents the wind power prediction error vector; π represents the action strategy; They represent the constraint exceeding limit of the unit commitment decision model represented by the constrained Markov decision process framework and the relaxation degree of the relaxed optimal power flow model; g is the unit number; i is the node number; UT g , DT g They represent the minimum start-up time and minimum shutdown time of the g-th unit respectively; Respectively represent the on / off status of the g-th unit at time t and t-1; is the state duration of the g-th unit at time t-1; (·)+ represents max{·,0}; is the slack variable; represents the probability distribution of time-varying wind power forecast error; ρ represents the forecast error probability; The reward function of the main unit commitment decision problem M1 is as follows: In the formula, r(s t ,a t ) represents the reward function; represents the on / off state of the unit at time t obtained by the unit commitment decision model represented by the constrained Markov decision process framework; f1 t (·) represents the startup cost and shutdown cost of the unit; represents the expected operating cost; represents the penalty function for violating the constraint; represents the power of the g-th unit; a g 、b g 、c g 、h g d g They represent the constant cost coefficient, first-order cost coefficient, second-order cost coefficient, startup cost, and shutdown cost of the g-th unit respectively; κ v , κ δ All represent cost coefficients; x represents the unit combination decision variable vector; ζ represents the wind power prediction error vector; y represents the unit output power vector; z represents other state variables; s is the slack variable vector; represents the probability distribution of time-varying wind power forecast error; f t (·) represents the operating cost within a certain sampling prediction error; The environment of the unit commitment decision main problem M1 integrates the reward and cost functions involving the unit commitment decision model represented by the constrained Markov decision process framework and the relaxed optimal power flow model, the probability distribution of the time-varying wind power prediction error, and the state transfer function.

7. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1, characterized in that: The objective function of the unit commitment decision sub-problem M2 is as follows: In the formula, t represents the time; g is the unit number; G represents the set of units; a g 、b g 、c g They represent the constant cost coefficient, the first-order cost coefficient, and the second-order cost coefficient of the g-th unit respectively; represents the power of the g-th unit at time t; κ δ represents the cost coefficient; i is the node number; I represents the set of nodes; is the slack variable; The constraints of the unit combination decision sub-problem M2 include node power balance constraints, unit and wind power output power constraints, branch power flow constraints, start / shutdown decision constraints, and unit ramp constraints; The node power balance constraint is as follows: In the formula, G i represents the set of wind turbines at node i; w represents the wind turbine generator set number; W i represents the set of wind turbines at node i; represents the actual grid-connected power generation of the w-th wind power station; j is the node number; ij represents the branch from node i to node j; B i represents the branch set of node i; The vector representing the predicted wind power load at node i at time t; represents the power flow of branch ij; The output power constraints of the units and wind power are as follows: In the formula, represents the prediction error sample obtained by the w-th wind turbine from the environment; represents the predicted output power of the w-th wind turbine; represents the on / off state of the g-th unit at time t obtained by the unit commitment decision model characterized by the constrained Markov decision process framework; P g They represent the upper and lower bounds of the output power of the g-th unit respectively; The branch power flow constraints are as follows: In the formula, K i-j,i I represents the power distribution coefficient of branch ij and node i; i represents the set of branches whose first segment node is i; They represent the upper and lower bounds of the power flow of branch ij respectively; represents the active load power of node i; The startup / shutdown decision constraints are as follows: In the formula, v g 、z g They represent the startup and shutdown decisions of the g-th unit respectively; represents the on / off state of the g-th unit at time t-1 obtained by the unit commitment decision model characterized by the constrained Markov decision process framework; The unit climbing constraints are as follows: In the formula, represents the power of the gth unit at time t-1; RU g , RD g ,SU g , SD g They respectively represent the ramp-up rate, ramp-down rate, startup rate and shutdown rate of the g-th unit.

8. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 1, characterized in that: The steps of solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm include: 6.1) Introduce entropy constraints and cost constraints as follows: Where θ represents the neural network parameters of the action strategy π, represents the entropy threshold; t represents the time; T represents the scheduling period; π represents the action strategy; s t represents the state at time t; a t represents the action at time t; t =(s t ,a t ,…,s T ,a T ) represents the decision trajectory of the probability distribution of time-varying wind power forecast error starting from time t; r(s t ,a t ) represents the reward function; Q c (s t ,a t ) represents compliance with strategy π and trajectory χ t the expected value of the cumulative discounted costs incurred; represents the discounted cost threshold, where d→0; ζ represents the wind power prediction error vector; ρ| t represents the probability distribution of time-varying wind power forecast error; c π,ρ is the cost function; γ is the discount rate. 6.2) Construct the LBD-SAC solution module and initialize the number of iterations k = 1; The LBD-SAC solution module includes a CGAN module, two reward value judgment network modules, two cost value judgment network modules, and an action network module; The CGAN module is used to generate a data set with a time-varying uncertainty distribution; Two reward value judgment network modules are used to evaluate the soft Q value of the reward; A cost-value judgment network module is used to evaluate the soft Q value of the constraint crossing cost of the unit commitment decision model represented by the constrained Markov decision process framework; A cost-value judgment network module is used to calculate the relaxation level of the relaxed optimal power flow model; The action network makes unit commitment decisions based on the current system operating status; 6.3) Calculate the target network parameters of the reward value judgment network module and the cost value judgment network module based on the distance-based 2-Wasserstein loss function; The target network parameters of the reward value judgment network module and the cost value judgment network module are as follows: In the formula, are the soft Q functions of reward and cost respectively; J c (μ) are soft Q functions The approximate value of All are Soft-Bellman operators; They represent the target network parameters of the reward value judgment network module and the cost value judgment network module respectively; is the set of unit commitment decisions; 6.4) Based on the APDO method, the Lagrangian function is established to transform the entropy constraint and cost constraint maximization problem into a dual optimization problem, and the loss function of the action network module is derived; 6.5) Determine whether the number of iterations k is equal to the preset threshold. If not, store the Lagrange multiplier in this iteration into the experience pool, set k=k+1, update the action strategy and Lagrange multiplier, and return to step 6.3); if yes, sample and select a Lagrange multiplier from the experience pool to adjust the Lagrange multiplier of the last iteration, and enter step 6.6); 6.6) Output the optimal solution for unit combination decision.

9. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 8, characterized in that: The Lagrangian function is as follows: Where α and λ are Lagrange multipliers related to entropy value and safety survey cost; is the Lagrangian function; π θ For action strategy; represents the entropy threshold; is the cost function; The dual optimization problem is as follows: In the formula, is the optimal primal-dual solution; The loss function of the action network module is as follows: In the formula, J π (θ) is the loss function of the action network module.

10. The monthly unit commitment method based on flexible actor-evaluator combined with logical Benders decomposition according to claim 9, characterized in that: The calculation formulas for the update action strategy and Lagrange multipliers are as follows: In the formula, θ (k+1) ,θ (k) They represent the neural network parameters in the k+1th and kth iterations respectively; ω is the learning rate; is the gradient function; α (k) , (k) is the Lagrange multiplier in the kth iteration process; α (k+1) , (k+1) is the Lagrange multiplier in the k+1th iteration process; is the action strategy in the k+1th iteration.

Images