A monthly unit commitment method based on flexible actuators-judger combined logic benders decomposition

Abstract

A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, comprising the following steps: 1) establishing a monthly security constrained unit commitment model; 2) establishing a time-varying wind power prediction error probability distribution; 3) decomposing the M-SCUC model into a unit commitment decision model and a relaxed optimal power flow model; 4) establishing the environment and key elements of the unit commitment decision main problem M1; establishing the objective function and constraint condition of the unit commitment decision sub-problem M2; 5) generating the unit commitment decision action of the unit commitment decision main problem M1 based on the flexible actor-critic deep reinforcement learning baseline algorithm; 6) solving the unit commitment decision sub-problem M2, and substituting the output result into the unit commitment decision main problem M1, and solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm; 7) repeating steps 5) to 6) until the reward function meets the convergence requirement, and outputting the final unit commitment decision. The present application guarantees the reliability and flexibility of system operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of long-term unit commitment scheduling in power systems using reinforcement learning, and particularly relates to a monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition. BACKGROUND

[0002] High penetration of wind power brings great challenges to power system operation. The monthly security constrained unit commitment (M-SCUC) considers the uncertainty of wind power from a longer perspective, and then makes decisions on unit commitment to ensure the reliability and flexibility of system operation.

[0003] M-SCUC faces two challenges: a) the complexity of uncertainty increases. With the extension of the scheduling period, the prediction error will be significantly expanded in time sequence; b) the model scale expands. The increase in the dimension of various variables and actual operation constraints significantly increases. These NP-hard features of randomness, non-convexity and strong uncertainty bring great challenges to the solution of M-SCUC. Existing research works aim to cope with these challenges by simplifying the model and algorithm design.

[0004] In terms of model simplification, existing research proposes a unit clustering method based on power flow to reduce the decision variables of medium-term scheduling. There are also studies on the identification and elimination of redundant constraints to simplify the long-term and large-scale SCUC model. However, under complex uncertainty conditions, the effectiveness of these compact models, as well as the trade-off between compactness and tightness, has not been fully verified.

[0005] In terms of algorithm design for solving uncertain M-SCUC problems, task decomposition is an effective method. There are studies that use Lagrangian relaxation to decompose stochastic long-term SCUC into multiple short-term SCUC sub-problems that can be handled. However, this method has problems such as multiplier oscillation and high cost of sub-gradient calculation. There are also studies that introduce a high-level stochastic variant of nested Benders decomposition (BD), called stochastic dual dynamic integer programming (SDDiP), for solving multi-stage uncertain scheduling problems. However, due to the long solution time and long scheduling period required by SDDiP, it hinders its further application in M-SCUC.

[0006] Deep reinforcement learning (DRL) is an effective artificial intelligence algorithm for time-series decision problems, which transfers the heavy computational burden to the offline training process. DRL can ensure the accuracy of the model without any compactness approximation. DRL has been widely applied to short-term SCUC problems. It has shown great potential in efficiently and accurately solving complex uncertain M-SCUC problems. However, given the characteristics of M-SCUC, DRL still faces the following challenges.

[0007] 1) Dimensionality disaster of state and action space

[0008] With the expansion of the scale of SCUC problems, the state and action space grows exponentially, which greatly increases the space-time complexity of DRL, leading to a decline in convergence efficiency and optimization performance, especially on a long time scale.

[0009] 2) Constraint satisfaction

[0010] For M-SCUC, it is crucial to ensure the satisfaction of constraints in DRL. It has been proven to be a difficult task to incorporate a penalty function for violating constraints into the reward function. Some safe DRL (SDRL) algorithms use a safety layer to correct the actions of the agent. However, the learning of the agent is easily disturbed by the safety layer, leading to poor convergence and local optimum. Some pioneering work establishes SDRL algorithms based on constrained Markov decision processes (CMDP), such as CPO, IPO, and C-SAC. Such algorithms have shown great potential in solving constrained optimization problems in power systems. However, high-dimensional state and action space and longer scheduling periods reduce the efficiency of safety exploration. Therefore, there is still a key bottleneck of safety in DRL when dealing with the constraints of M-SCUC problems.

[0011] 3) Complex uncertain environment simulation

[0012] The prediction accuracy decreases as the prediction time increases. The common practice of using independent and identical wind power prediction error distributions can affect the accuracy of medium and long-term forward scheduling. Therefore, it has become a challenge to apply the fixed state transition probability function in the interaction between the agent and the environment to M-SCUC. It is a difficult task to accurately simulate the time-dependent prediction error distribution and integrate it into the environment. SUMMARY

[0013] The purpose of the present application is to provide a monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, comprising the following steps:

[0014] 1) Establish a monthly security constrained unit commitment model considering wind power uncertainty.

[0015] 2) Based on conditional generative adversarial networks, the time-varying wind power prediction error probability distribution is established.

[0016] 3) The logical Benders decomposition method is used to decouple discrete variables and continuous variables, and the M-SCUC model is decomposed into a unit commitment decision model represented in the constraint Markov decision process framework and a relaxed optimal power flow model.

[0017] 4) Based on the unit commitment decision model, the environment, state space, action space, state transition function, cost function and reward function of the unit commitment decision main problem M1 are established.

[0018] Based on the relaxed optimal power flow model, the objective function and constraint condition of the unit commitment decision sub-problem M2 are established.

[0019] 5) The time-varying wind power prediction error probability distribution is used as the uncertainty of the environment, and the flexible actor-critic deep reinforcement learning baseline algorithm is used to generate the unit commitment decision action of the unit commitment decision main problem M1, and the unit commitment decision action is input as the boundary condition into the unit commitment decision sub-problem M2.

[0020] 6) Based on the unit commitment decision action of the unit commitment decision main problem M1, the unit commitment decision sub-problem M2 is solved, and the relaxed level output by the unit commitment decision sub-problem M2 is regarded as the exploration cost and substituted into the unit commitment decision main problem M1, and the LBD-SAC algorithm is used to solve the unit commitment decision main problem M1.

[0021] 7) Repeat steps 5) to 6) until the reward function meets the convergence requirement, and output the final unit commitment decision.

[0022] Further, the monthly security-constrained unit commitment model considering wind power uncertainty is as follows:

[0023]

[0024] In the formula, t represents the time. T is the scheduling period. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. represents the expected comprehensive scheduling cost. A t , D t , H t and M t all represent coefficient matrices. e t is a coefficient vector. N x , N y , N z , N ζM0 represents the dimension of the unit commitment decision variable vector x, the unit output power vector y, the other state variable z, and the wind power prediction error vector ζ. M0 represents the monthly security-constrained unit commitment model.

[0025] Further, the step of establishing the time-varying wind power prediction error probability distribution based on the conditional generative adversarial network comprises:

[0026] 2.1) Obtain wind power prediction data P d and historical measured data H d .

[0027] 2.2) Based on the wind power prediction data P d , use the generator network to generate the simulated wind power data sequence G d .

[0028] 2.3) Concatenate the simulated wind power data sequence G d and the wind power prediction data P d .

[0029] Concatenate the wind power prediction data P d and the historical measured data H d .

[0030] 2.4) Input the concatenated data into the discriminator network to fit the time-varying wind power prediction error distribution.

[0031] 2.5) Based on the fitted time-varying wind power prediction error distribution, establish the time-varying wind power prediction error probability distribution.

[0032] Further, the unit commitment decision model represented in the constrained Markov decision process framework is as follows:

[0033]

[0034] In the formula, t represents the time. T is the scheduling period. x represents the unit commitment decision variable vector. f1 represents the fixed unit commitment decision obtained by the unit commitment decision model represented in the constrained Markov decision process framework. t (·) represents the start-up cost and shutdown cost of the unit. represents the expected operation cost. represents the coefficient matrix of the unit commitment decision model represented in the constrained Markov decision process framework. N is the coefficient vector of the unit commitment decision model represented in the constrained Markov decision process framework. x M1 represents the unit commitment decision problem of the unit start-stop state decision. M1 represents the dimension of the unit commitment decision variable vector x. M1 represents the unit commitment decision problem of the unit start-stop state decision.

[0035] Further, the relaxed optimal power flow model is shown as follows:

[0036]

[0037]

[0038] where t represents time. T is the dispatch period. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. represents the fixed unit commitment decision obtained from the unit commitment decision model characterized in the constrained Markov decision process framework. represents the time-varying wind power prediction error probability distribution. represents the expected operation cost. f t (·) represents the operation cost within a certain sampling prediction error. all represent the coefficient matrix of the relaxed optimal power flow model. is the coefficient vector of the relaxed optimal power flow model. N y , N z , N ζ , N s respectively represent the dimension of the unit output power vector y, the other state variable z, the wind power prediction error vector ζ, and the slack variable vector s. M2 represents the relaxed optimal power flow model under the fixed unit on / off state.

[0039] Further, the state space of the unit commitment decision main problem M1 is shown as follows:

[0040]

[0041] where t represents time. W, I, G respectively represent the set of wind turbine generators, nodes, and units. represents the state space. s t represents the state at time t. respectively represent the vector of predicted wind power output and the vector of predicted wind power load at time t. represents the on / off state of the unit at time t-1. represents the unit state duration at time t-1.

[0042] where the first-order difference form of the predicted wind power output in the adjacent period is shown as follows:

[0043]

[0044] where, represents the vector of predicted wind power output at time t-1. Δt represents the time difference.

[0045] The action space of the unit commitment decision main problem M1 is as follows:

[0046]

[0047] In the formula, indicates the action space.a t indicates the action at time t. N is the number of units. indicates the on / off state of the unit at time t.

[0048] The state transition function of the unit commitment decision main problem M1 is as follows:

[0049]

[0050] In the formula, indicates the unit state duration at time t.

[0051] The cost function of the unit commitment decision main problem M1 is as follows:

[0052]

[0053] In the formula, c π,ρ indicates the cost function. ζ indicates the wind power prediction error vector. π indicates the action strategy. respectively indicate the constraint overrun quantity of the unit commitment decision model characterized in the constraint Markov decision process framework and the relaxation degree of the relaxed optimal power flow model. g is the unit number. i is the node number. UT g , DT g respectively indicate the minimum start-up time and the minimum shutdown time of the gth unit. respectively indicate the on / off state of the gth unit at time t and time t-1. is the state duration of the gth unit at time t-1. (·)+indicates max{·,0}. is the relaxation variable. indicates the time-varying wind power prediction error probability distribution. ρ indicates the prediction error probability.

[0054] The reward function of the unit commitment decision main problem M1 is as follows:

[0055]

[0056] In the formula, r(s t ,a t ) indicates the reward function. indicates the on / off state of the unit at time t obtained by the unit commitment decision model characterized in the constraint Markov decision process framework. f1 t(·) represents the start-up cost and shut-down cost of the unit. represents the expected operation cost. represents the penalty function of violating the constraints. represents the power of the gth unit. g , b g , c g , h g , d g represents the constant term cost coefficient, the linear term cost coefficient, the quadratic term cost coefficient, the start-up cost, the shut-down cost of the gth unit, respectively. v , κ δ all represent the cost coefficient. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. represents the time-varying wind power prediction error probability distribution. t (·) represents the operation cost within a certain sampling prediction error.

[0057] The environment set of the unit commitment decision main problem M1 integrates the reward and cost functions of the unit commitment decision model characterized in the constraint Markov decision process framework and the relaxed optimal power flow model, the time-varying wind power prediction error probability distribution, and the state transition function.

[0058] Further, the objective function of the unit commitment decision sub-problem M2 is as follows:

[0059]

[0060] In the formula, t represents the time. g is the unit number. G represents the set of units. g , b g , c g represent the constant term cost coefficient, the linear term cost coefficient, the quadratic term cost coefficient of the gth unit, respectively. represents the power of the gth unit at t time. κ δ represents the cost coefficient. i is the node number. I represents the set of nodes. is the slack variable.

[0061] The constraint conditions of the unit commitment decision sub-problem M2 include the node power balance constraint, the output power constraint of the unit and the wind power, the branch power flow constraint, the start-up / shut-down decision constraint, and the unit ramping constraint.

[0062] The node power balance constraint is as follows:

[0063]

[0064] In the formula, G iSet of units at node i. w represents the wind turbine set index. W i Set of wind turbines at node i. Actual on-grid power of the wth wind power station. j represents the node index. i-j represents the branch from node i to node j. B i Set of branches of node i. Vector of predicted wind power load at node i at time t. Power flow of branch i-j.

[0065] The output power constraints of the units and wind power are shown as follows:

[0066]

[0067] In the formula, Predicted error sample obtained by the wth wind turbine from the environment. Predicted output power of the wth wind turbine. On / Off state of the gth unit at time t obtained by the unit combination decision model characterized in the constrained Markov decision process framework. Upper and lower bounds of the output power of the gth unit, respectively.

[0068] The branch power flow constraints are shown as follows:

[0069]

[0070] In the formula, K i-j,i Power flow distribution coefficient of branch i-j and node i. I i Set of branches with the first section node being i. Upper and lower bounds of the power flow of branch i-j, respectively. Active load power of node i.

[0071] The start / stop decision constraints are shown as follows:

[0072]

[0073] In the formula, v g , z g Start and stop decisions of the gth unit, respectively. On / Off state of the gth unit at time t-1 obtained by the unit combination decision model characterized in the constrained Markov decision process framework.

[0074] The unit ramping constraints are shown as follows:

[0075]

[0076] wherein, Pgt-1denotes the power of the gth unit at t-1. g , RD g , SU g , SD g denote the up ramp rate, down ramp rate, start-up rate, shut-down rate of the gth unit, respectively.

[0077] Further, the step of solving the main problem M1 of unit commitment decision by using the LBD-SAC algorithm comprises:

[0078] 6.1) Introduce entropy value constraint and cost constraint as follows:

[0079]

[0080] wherein, θ denotes the neural network parameter of action policy π, denotes the entropy threshold value. t denotes the time. T denotes the scheduling period. π denotes the action policy. s t denotes the state at t. a t denotes the action at t. χ t = (s t , a t , …, s T , a T ) denotes the decision trajectory of the time-varying wind power prediction error probability distribution starting from time t. r(s t , a t ) denotes the reward function. Q c (s t , a t ) denotes the expected value of the cumulative discounted cost generated by following the policy π and the trajectory χ t . denotes the discounted cost threshold value, wherein d→0. ζ denotes the wind power prediction error vector. ρ t denotes the time-varying wind power prediction error probability distribution. c π,ρ is the cost function. γ denotes the discount rate.

[0081] 6.2) Construct the LBD-SAC solving module and initialize the iteration number k = 1.

[0082] The LBD-SAC solving module comprises a CGAN module, two reward value judgment network modules, two cost value judgment network modules, and an action network module.

[0083] The CGAN module is used to generate a data set with time-varying uncertainty distribution.

[0084] The two reward value judgment network modules are used to evaluate the soft Q value of the reward.

[0085] A cost value critic network module is used to evaluate the soft Q-value of the constraint violation cost of the unit commitment decision model represented in the constrained Markov decision process framework.

[0086] A cost value critic network module is used to calculate the relaxation level of the relaxed optimal power flow model.

[0087] The action network formulates the unit commitment decision according to the current operating state of the system.

[0088] 6.3) The distance-based 2-Wasserstein loss function is used to calculate the target network parameters of the reward value critic network module and the cost value critic network module.

[0089] The target network parameters of the reward value critic network module and the cost value critic network module are as follows:

[0090]

[0091] wherein, are the soft Q-functions of the reward and the cost, respectively. J c are the approximate values of the soft Q-functions , respectively. are the Soft-Bellman operators. and represent the target network parameters of the reward value critic network module and the cost value critic network module, respectively. is the set of unit commitment decisions.

[0092] 6.4) The Lagrangian function is established based on the APDO method, the maximum entropy value constraint and cost constraint problem is converted into a dual optimization problem, and the loss function of the action network module is derived.

[0093] 6.5) Determine whether the iteration number k is equal to the preset threshold value, if not, store the Lagrange multiplier in this iteration in the experience pool, let k=k+1, update the action policy and the Lagrange multiplier, and return to step 6.3). If yes, sample a Lagrange multiplier from the experience pool to adjust the Lagrange multiplier of the last iteration, and go to step 6.6).

[0094] 6.6) Output the optimal solution of the unit commitment decision.

[0095] Further, the Lagrangian function is as follows:

[0096]

[0097] wherein, α and λ are the Lagrange multipliers related to the entropy value and the safety survey cost. is the Lagrangian function. πθ is an action policy. represents an entropy threshold. is a cost function.

[0098] The dual optimization problem is as follows:

[0099]

[0100] wherein, is an optimal primal-dual solution.

[0101] The loss function of the action network module is as follows:

[0102]

[0103] wherein, J π is a loss function of the action network module.

[0104] Further, the calculation formula of the updated action policy and the Lagrange multiplier is as follows:

[0105]

[0106]

[0107] wherein, θ (k+1) , θ (k) respectively represent the neural network parameters in the k+1, k iteration process. ω is a learning rate. is a gradient function. α (k) , λ (k) is a Lagrange multiplier in the k iteration process. α (k+1) , λ (k+1) is a Lagrange multiplier in the k+1 iteration process. is an action policy in the k+1 iteration process.

[0108] The technical effect of the present application is self-evident. The present application proposes an M-SCUC method based on LBD-SAC to realize efficient and accurate solution of long-term operation scheduling model in a power system and guarantee the reliability and flexibility of system operation, aiming at the demand for safe and flexible operation of the power system.

[0109] The DRL training mechanism based on LBD proposed in the present application combines the ideas of task decomposition and optimization assistance, can reduce the processing scale of the algorithm, and improve the constraint satisfaction and convergence. To cope with the complex M-SCUC solving problem, the present application provides a DRL training idea that can ensure scalability, constraint satisfaction and effective processing of uncertainty.

[0110] The M-SCUC method based on LBD-SAC proposed in the application combines the constraint strategy search method based on APDO with the SAC baseline architecture on the basis of the LBD-based training mechanism. The algorithm adds the offline policy training Lagrange multiplier related to the security survey cost and the policy entropy in the dual update process, and drives the agent to perform efficient maximum entropy adaptive learning on the M-SCUC problem. The application also provides the policy convergence proof of the proposed LBD-SAC algorithm, and proves the feasibility of the algorithm mathematically, which provides a theoretical basis for the application of the algorithm in the M-SCUC reliable problem.

[0111] The application first proposes a LBD-SAC DRL algorithm in the technical field, which is used to solve the problems caused by the characteristics of the M-SCUC model such as large scale, non-convex, complex uncertainty, and overcomes the key technical bottlenecks of low convergence and constraint satisfaction of the existing DRL. The algorithm uses the idea of LBD to establish a task decomposition and optimization auxiliary training mechanism, which improves the security survey efficiency of the agent. At the same time, the algorithm incorporates CGAN in the environment to simulate the time-varying prediction error distribution of the CMDP environment. In addition, the algorithm integrates the security policy search method of APDO to effectively explore the optimal feasible policy of LBD-SAC. The application provides a safe, feasible and efficient solution for M-SCUC.

[0112] The LBD-SAC method proposed in the application is suitable for solving the uncertainty optimization model of a large-scale, long-time scale and complex model power system. It can accurately simulate the uncertainty of the actual system operation environment and realize efficient decision-making. The algorithm proposed in the application greatly improves the safety and feasibility of the application of DRL in the field of power systems, and makes certain contribution to the intelligent development of power systems. BRIEF DESCRIPTION OF DRAWINGS

[0113] Figure 1 is the principle architecture diagram of CGAN;

[0114] Figure 2 is the LBD-based training mechanism diagram proposed in the application;

[0115] Figure 3 is the architecture diagram of the M-SCUC solving method based on LBD-SAC proposed in the application;

[0116] Figure 4 is the unit commitment decision made by LBD-SAC under the IEEE-118 test system;

[0117] Figure 5 is the training performance comparison diagram of each DRL algorithm; Figure 5 (a) is the reward training performance comparison diagram; Figure 5 (b) is the cost training performance comparison diagram.

[0118] Figure 6 is the set of up / down regulation curves and the net load scenarios obtained by each algorithm;

[0119] Figure 7 is the out-of-limit frequency and distribution of insufficient power supply ability under the random scenario;

[0120] Figure 8 is the unit commitment decision made by LBD-SAC under IEEE-300 test system;

[0121] Figure 9 is the pseudo code of LBD-SAC. DETAILED DESCRIPTION

[0122] The application will be further described below in conjunction with the embodiments, but should not be understood as limiting the above-mentioned subject matter of the application to the following embodiments. According to ordinary technical knowledge and conventional means in the art, various substitutions and modifications can be made without departing from the above-mentioned technical idea of the application, and all should be included in the protection scope of the application.

[0123] Embodiment 1:

[0124] Referring to Figures 1 to 9 A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition includes the following steps:

[0125] 1) Establish a monthly security-constrained unit commitment model considering wind power uncertainty.

[0126] 2) Based on conditional generative adversarial network, establish the time-varying wind power prediction error probability distribution.

[0127] 3) Use the logic Benders decomposition method to decouple discrete variables and continuous variables, and decompose the M-SCUC model into a unit commitment decision model represented in the constraint Markov decision process framework and a relaxed optimal power flow model.

[0128] 4) Based on the unit commitment decision model, establish the environment, state space, action space, state transition function, cost function and reward function of the unit commitment decision main problem M1.

[0129] Based on the relaxed optimal power flow model, establish the objective function and constraint condition of the unit commitment decision sub-problem M2.

[0130] 5) Take the time-varying wind power prediction error probability distribution as the uncertainty of the environment, generate the unit commitment decision action of the unit commitment decision main problem M1 based on the flexible actor-critic deep reinforcement learning baseline algorithm, and input the unit commitment decision action as the boundary condition to the unit commitment decision sub-problem M2.

[0131] 6) Based on the unit commitment decision-making main problem M1, the unit commitment decision-making action is determined, the unit commitment decision-making sub-problem M2 is solved, and the relaxation level output by the unit commitment decision-making sub-problem M2 is taken as the survey cost and substituted into the unit commitment decision-making main problem M1, and the LBD-SAC algorithm is used to solve the unit commitment decision-making main problem M1.

[0132] 7) Repeat steps 5) to 6) until the reward function meets the convergence requirement, and output the final unit commitment decision.

[0133] Embodiment 2:

[0134] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of which is shown in Embodiment 1, further, the monthly security constrained unit commitment model considering wind power uncertainty is as follows:

[0135]

[0136] In the formula, t represents the time. T is the scheduling period. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. represents the expected comprehensive scheduling cost. A t , D t , H t and M t all represent coefficient matrices. e t is a coefficient vector. N x , N y , N z , N ζ respectively represent the dimension of the unit commitment decision variable vector x, the unit output power vector y, the other state variable z, and the wind power prediction error vector ζ. M0 represents the monthly security constrained unit commitment model.

[0137] Embodiment 3:

[0138] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of which is shown in any one of Embodiments 1 to 2, further, the step of establishing the time-varying wind power prediction error probability distribution based on the conditional generative adversarial network comprises:

[0139] 2.1) Obtain wind power prediction data P d and historical measured data H d .

[0140] 2.2) Based on the wind power prediction data P d , a simulated wind power data sequence Gd .

[0141] 2.3) Concatenate the simulated wind power data sequence G d with the wind power forecast data P d .

[0142] Concatenate the wind power forecast data P d and the historical measured data H d .

[0143] 2.4) Input the concatenated data into the discriminator network to fit the time-varying wind power forecast error distribution.

[0144] 2.5) Based on the fitted time-varying wind power forecast error distribution, establish the time-varying wind power prediction error probability distribution.

[0145] Embodiment 4:

[0146] A monthly unit commitment method based on flexible actor-discriminator combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 3, further, the unit commitment decision model characterized by the constrained Markov decision process framework is as follows:

[0147]

[0148] In the formula, t represents the time. T is the scheduling period. x represents the unit commitment decision variable vector. represents the fixed unit commitment decision obtained by the unit commitment decision model characterized by the constrained Markov decision process framework. represents the start-up cost and shutdown cost of the unit. represents the expected operation cost. represents the coefficient matrix of the unit commitment decision model characterized by the constrained Markov decision process framework. is the coefficient vector of the unit commitment decision model characterized by the constrained Markov decision process framework. N x represents the dimension of the unit commitment decision variable vector x. M1 represents the unit commitment decision problem of the unit start-stop state decision.

[0149] Embodiment 5:

[0150] A monthly unit commitment method based on flexible actor-discriminator combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 4, further, the relaxed optimal power flow model is as follows:

[0151]

[0152] where t represents time. T is the dispatch period. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. represents the fixed unit commitment decision obtained from the unit commitment decision model characterized in the constrained Markov decision process framework. represents the time-varying wind power prediction error probability distribution. represents the expected operation cost. f t (·) represents the operation cost within a certain sampling prediction error. all represent the coefficient matrix of the relaxed optimal power flow model. is the coefficient vector of the relaxed optimal power flow model. N y , N z , N ζ , N s respectively represent the dimension of the unit output power vector y, the other state variable z, the wind power prediction error vector ζ, and the slack variable vector s. M2 represents the relaxed optimal power flow model under the fixed unit on / off state.

[0153] Embodiment 6:

[0154] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 5, further, the state space of the unit commitment decision main problem M1 is as follows:

[0155]

[0156] where t represents time. W, I, G respectively represent the set of wind turbine generators, nodes, and units. represents the state space. s t represents the state at time t. respectively represent the vector of predicted wind power output at time t, and the vector of predicted wind power load. represents the on / off state of the unit at time t-1. represents the unit state duration at time t-1.

[0157] where the first-order difference form of the predicted wind power output in the adjacent period is as follows:

[0158]

[0159] where, represents the vector of predicted wind power output at time t-1. Δt represents the time difference.

[0160] The action space of the unit commitment decision main problem M1 is as follows:

[0161]

[0162] wherein, denotes the action space.a t denotes the action at time t. N is the number of units. denotes the on / off state of the unit at time t.

[0163] The state transition function of the unit commitment decision main problem M1 is shown as follows:

[0164]

[0165] wherein, denotes the state duration of the unit at time t.

[0166] The cost function of the unit commitment decision main problem M1 is shown as follows:

[0167]

[0168] wherein, c π,ρ denotes the cost function. ζ denotes the wind power prediction error vector. π denotes the action policy. denote the constraint violation of the unit commitment decision model characterized in the framework of constrained Markov decision process and the relaxation degree of the relaxed optimal power flow model, respectively. g is the unit number. i is the node number. UT g , DT g denote the minimum start-up time and the minimum shut-down time of the gth unit, respectively. denote the on / off state of the gth unit at time t and t-1, respectively. is the state duration of the gth unit at time t-1. (·)+denotes max{·,0}. is the relaxation variable. denotes the time-varying wind power prediction error probability distribution. ρ denotes the prediction error probability.

[0169] The reward function of the unit commitment decision main problem M1 is shown as follows:

[0170]

[0171] wherein, r(s t ,a t ) denotes the reward function. denotes the on / off state of the unit at time t obtained by the unit commitment decision model characterized in the framework of constrained Markov decision process.f1 t denotes the start-up cost and the shut-down cost of the unit. denotes the expected operation cost. a penalty function representing violation of constraints. Pgt represents the power of the gth unit. g , b g , c g , h g , d g represent the constant term cost coefficient, the linear term cost coefficient, the quadratic term cost coefficient, the start-up cost, the shut-down cost of the gth unit respectively. v , κ δ all represent cost coefficients. x represents the unit commitment decision variable vector. ζ represents the wind power prediction error vector. y represents the unit output power vector. z represents other state variables. s is the slack variable vector. f represents the time-varying wind power prediction error probability distribution. t (·) represents the operation cost within a certain sampling prediction error.

[0172] The environment set of the unit commitment decision main problem M1 integrates the reward and cost functions of the unit commitment decision model characterized in the constraint Markov decision process framework and the relaxed optimal power flow model, the time-varying wind power prediction error probability distribution, and the state transition function.

[0173] Embodiment 7:

[0174] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of which is any one of embodiments 1 to 6, further, the objective function of the unit commitment decision sub-problem M2 is as follows:

[0175]

[0176] In the formula, t represents the time. g is the unit number. G represents the set of units. g , b g , c g represent the constant term cost coefficient, the linear term cost coefficient, the quadratic term cost coefficient of the gth unit respectively. Pgt represents the power of the gth unit at time t. δ represent the cost coefficients. i is the node number. I represents the set of nodes. s is the slack variable.

[0177] The constraint conditions of the unit commitment decision sub-problem M2 include the node power balance constraint, the output power constraint of the unit and the wind power, the branch power flow constraint, the start-up / shut-down decision constraint, and the unit ramping constraint.

[0178] The node power balance constraint is as follows:

[0179]

[0180] where G i denotes the set of wind turbine generators at node i. w denotes the wind turbine generator index. W i denotes the set of wind turbine generators at node i. denotes the actual grid-connected power of the wth wind farm. j denotes the node index. i-j denotes the branch from node i to node j. B i denotes the set of branches at node i. denotes the predicted wind power load vector at node i at time t. denotes the power flow of branch i-j.

[0181] The output power constraints of the wind turbines and wind farms are shown as follows:

[0182]

[0183] where denotes the prediction error sample obtained from the environment by the wth wind turbine generator. denotes the predicted output power of the wth wind turbine generator. denotes the on / off state of the gth wind turbine at time t obtained by the unit commitment decision model characterized in the constrained Markov decision process framework. P g denote the upper and lower bounds of the output power of the gth wind turbine, respectively.

[0184] The power flow constraints of the branches are shown as follows:

[0185]

[0186] where K i-j,i denotes the power flow distribution coefficient of branch i-j and node i. I i denotes the set of branches with the first node i. denote the upper and lower bounds of the power flow of branch i-j, respectively. denotes the active power load at node i.

[0187] The start-up / shut-down decision constraints are shown as follows:

[0188]

[0189] where v g , z g denote the start-up and shut-down decisions of the gth wind turbine, respectively. denotes the on / off state of the gth wind turbine at time t-1 obtained by the unit commitment decision model characterized in the constrained Markov decision process framework.

[0190] The unit climbing constraint is as follows:

[0191]

[0192] In the formula, RU represents the power of the gth unit at t-1. g , RD g , SU g , SD g respectively represent the upper climbing rate, the lower climbing rate, the startup rate, and the shutdown rate of the gth unit.

[0193] Embodiment 8:

[0194] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 7, further, the step of solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm comprises:

[0195] 6.1) Introduce entropy value constraints and cost constraints as follows:

[0196]

[0197] In the formula, θ represents the neural network parameters of the action policy π, represents the entropy threshold. t represents the time. T is the scheduling period. π represents the action policy. s t represents the state at t. a t represents the action at t. χ t = (s t , a t , …, s T , a T ) represents the decision trajectory starting from time t. r(s t , a t ) represents the reward function. Q c (s t , a t ) represents the expected value of the cumulative discounted cost generated by following the policy π and the trajectory χ t . represents the discounted cost threshold, where d→0. ζ represents the wind power prediction error vector. ρ| t represents the time-varying wind power prediction error probability distribution. c π,ρ is the cost function. γ represents the discount rate.

[0198] 6.2) Build an LBD-SAC solving module and initialize the iteration number k=1.

[0199] The LBD-SAC solving module comprises a CGAN module, two reward value evaluation network modules, two cost value evaluation network modules, and an action network module.

[0200] The CGAN module is configured to generate a data set with time-varying uncertainty distribution.

[0201] The two reward value evaluation network modules are configured to evaluate soft Q values of rewards.

[0202] One of the cost value evaluation network modules is configured to evaluate soft Q values of constraint overrun costs of a unit commitment decision model represented in a constrained Markov decision process framework.

[0203] One of the cost value evaluation network modules is configured to calculate a relaxation level of a relaxed optimal power flow model.

[0204] The action network formulates a unit commitment decision according to a current operation state of a system.

[0205] 6.3) A distance-based 2-Wasserstein loss function is used to calculate target network parameters of the reward value evaluation network module and the cost value evaluation network module.

[0206] The target network parameters of the reward value evaluation network module and the cost value evaluation network module are as follows:

[0207]

[0208] In the formula, are soft Q functions of rewards and costs, respectively. J c are approximate values of soft Q functions , respectively. are Soft-Bellman operators. and represent target network parameters of the reward value evaluation network module and the cost value evaluation network module, respectively. is a set of unit commitment decisions.

[0209] 6.4) A Lagrangian function is established based on the APDO method, a maximum entropy value constraint and a cost constraint problem are converted into a dual optimization problem, and a loss function of the action network module is derived.

[0210] 6.5) It is determined whether the iteration number k is equal to a preset threshold value. If not, the Lagrangian multiplier in this iteration is stored in an experience pool, k is set to k+1, the action policy and the Lagrangian multiplier are updated, and step 6.3) is returned. If yes, a Lagrangian multiplier is selected from the experience pool to adjust the Lagrangian multiplier in the last iteration, and step 6.6) is entered.

[0211] 6.6) output the optimal solution of the unit commitment decision.

[0212] Embodiment 9:

[0213] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 8, further, the Lagrangian function is as follows:

[0214]

[0215] In the formula, alpha, lambda are the Lagrange multipliers related to the entropy value and the safety survey cost. The Lagrangian function is pi. θ The action policy is pi. The entropy threshold is h. The cost function is c.

[0216] The dual optimization problem is as follows:

[0217]

[0218] In the formula, The optimal primal-dual solution is (pi, alpha, lambda).

[0219] The loss function of the action network module is as follows:

[0220]

[0221] In the formula, J π The loss function of the action network module is J(theta).

[0222] Embodiment 10:

[0223] A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content of any one of embodiments 1 to 9, further, the calculation formula of the updated action policy and the Lagrange multiplier is as follows:

[0224]

[0225] In the formula, theta (k+1) , theta (k) respectively represent the neural network parameters in the k+1, k iteration process. Omega is the learning rate. The gradient function is. Alpha (k) , lambda (k) are the Lagrange multipliers in the k iteration process. Alpha (k+1) , lambda (k+1) are the Lagrange multipliers in the k+1 iteration process. The action policy in the k+1 iteration process is pi.

[0226] Embodiment 11

[0227] Referring to Figures 1 to 9 A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main steps are as follows:

[0228] 1) A monthly security constrained unit commitment model considering wind power uncertainty is established, and the monthly security constrained unit commitment model can be regarded as minimizing the expected comprehensive scheduling cost under the physical constraint conditions of the overall system and individual adjustable resources and probabilistic resources. The present application refers to it as the original problem (M0), and its compact form is as follows:

[0229]

[0230] In the formula, the superscript t represents the time; T is the scheduling period; x represents the unit commitment decision variable vector, which relates to the start and stop state of the unit; ζ represents the wind power prediction error vector, which is subject to the time-varying conditional probability distribution ρ t ; y represents the unit output power vector; z represents other state variables, including node voltage and branch power flow; represents the expected comprehensive scheduling cost; A t , D t , H t and M t represent the coefficient matrix; et is the coefficient vector; N x , N y , N z , N ζ represent the dimensions of the respective variables.

[0231] 2) A time-varying wind power prediction error probability distribution model based on CGAN is established, and the modeling idea of the time-varying wind power prediction error probability distribution based on CGAN is described in Figure 1 . In Figure 1 , the generator network (Generator, G) captures the time correlation and distribution characteristics of wind power prediction error based on the prediction data P d and the historical measured data H d . The simulated wind power data sequence G d is generated by G under the condition of the prediction data P d . The discriminator network (Discriminator, G) evaluates the probability that the input data comes from the H d distribution. The training goal of G is to maximize the evaluation error probability of D, while the goal of D is to minimize this probability. Through the antagonistic interaction training between G and D, G can accurately fit the time-varying wind power prediction error distribution. The completed generator G* can accurately simulate the time-varying uncertainty distribution in the M-SCUC problem.

[0232] 3) Propose a LBD-based DRL training mechanism, given the complexity of the M-SCUC model (M0) considering wind power uncertainty in 1) poses challenges for DRL deployment applications, the invention develops a LBD-based training mechanism. The LBD method decomposes the task into a main problem and sub-problems related to decision-making. Compared with the traditional BD method, it liberates the form of the main problem and sub-problems. The sub-problems tighten the solution space of the main problem through iterative logical deduction, and finally converge to the feasible region of the original problem. Given the task decomposition idea and logical reasoning strategy of LBD, the LBD-based DRL training mechanism proposed in the invention is as shown in Figure 2 .

[0233] According to the specific structure of M0, it is classified into a main problem (M1) and a sub-problem (M2) according to variable type and model function. M1 represents the unit commitment decision-making problem of unit start-stop state decision, and the compact form of the model is shown in formula (2). M2 represents the relaxed optimal power flow model under the fixed unit on / off state, and the compact form of the model is shown in formula (3).

[0234]

[0235] wherein, represents the fixed unit commitment decision obtained by M1; represents the start-up cost and shutdown cost of the unit; s is the relaxation variable vector, which represents the relaxation degree of M1 as a feasible Benders cut signal; represents the expected operation cost; f t () represents the operation cost within a certain sampling prediction error.

[0236] In the proposed LBD-based training mechanism, the constraint overrun of M1 and the relaxation degree of M2 are regarded as the safety exploration cost of the agent in the decision space. The training purpose is to limit these costs to approximate the feasible region of M0. The start-stop cost of M1 and the operation cost of M2 are regarded as negative rewards. Then through logical deduction, a constantly tightening safety exploration space is provided for the agent of M1. The iterative interaction between M1 and M2 at each time step t trains the agent to explore the optimal feasible solution within the entire scheduling period T. The LBD-based training mechanism effectively reduces the dimensionality of the action and state space through task decomposition, thereby improving scalability and convergence. Through auxiliary training optimization, the satisfaction of operation constraints is significantly improved.

[0237] 4) Establish the CMDP model of the unit commitment decision-making main problem M1, and the CMDP model of M1 is defined as the tuple c,d,γ},wherein represents the state space; represents the action space; is the state transition function; r represents the reward obtained by selecting at s ; c represents the safety exploration cost obtained by selecting a t at s t ; d is the cost threshold; γ (γ ∈ [0, 1]) represents the discount rate. The CMDP model of M1 involves the following environment and key elements:

[0238] 4.1) Environment: The environment integrates the reward and cost functions involving M1 and M2, the time-varying wind power error distribution fitted from CGAN, and the state transition function. The agent makes unit commitment decisions according to the action policy and the perceived operating state. In calculating the reward and cost, Monte Carlo simulation is used to sample the wind power prediction error in CGAN. Then it is transferred to the next state according to the state transition function.

[0239] 4.2) State space: The definition of the state space is shown in equation (4), which contains the source load prediction information and trends, the start-stop state of the unit at the previous time, and the state duration.

[0240]

[0241] In the formula, the subscripts W, I, and G represent wind power generators, nodes, and sets of units, respectively; represents the predicted wind power output and load vector; represents the on / off state of the unit; represents the unit state duration; represents the first-order difference form of the predicted wind power output in the adjacent period, which is introduced to consider the prediction trend, as follows:

[0242]

[0243] 4.3) Action space: The action space involves the decision variable x in M1, which makes unit commitment decisions to determine the start / stop state of the unit. An action is defined as a binary vector of N units, as follows:

[0244]

[0245] 4.4) State transition function: For exogenous states that are not affected by actions, their probability transition characteristics can be learned directly from the data set. In this paper, in order to solve the problem of scarce historical data, CGAN generates such a data set with time-varying uncertainty distribution. As for endogenous states that can respond to action feedback, including the duration of the unit state, as follows:

[0246]

[0247] 4.5) Cost function: The cost function maps the state and unit commitment decisions to the security survey cost, as defined in equations (8)-(9). The cost function provides the basis for the agent to tighten the solution space to the feasible region of M0, thus ensuring the security of the operation.

[0248]

[0249] where π denotes the action policy; and denote the constraint violation amount of M1 and the relaxation degree derived from M2, respectively; (·)+ denotes max{·,0}; UT g , DT g denote the minimum on-time and minimum off-time of the gth unit, respectively. is the relaxation variable, denoting the relaxed power of the ith node.

[0250] 4.6) Reward function: The reward function is composed of the negative value of the objective function of M1 and the penalty function of violating constraints. The reward function encourages the agent to explore the optimal solution, as defined in equations (10)-(11).

[0251]

[0252] where, denotes the penalty function of violating constraints; denotes the power of the gth unit; a g , b g , c g , h g , d g denote the constant term cost coefficient, the linear term cost coefficient, the quadratic term cost coefficient, the startup cost, and the shutdown cost of the gth unit, respectively; κ v , κ δ denote the cost coefficients, which are incorporated into the reward function to improve the efficiency of constraint learning.

[0253] The CMDP model sets meeting the cost threshold as the cost constraint condition. As defined in equation (12), the goal of training the policy is to maximize the cumulative discounted reward while meeting the cost constraint. This lays the foundation for the agent to explore the optimal feasible unit commitment decision.

[0254]

[0255] where χ t = (s t , a t , …, s T , a T) represents the decision trajectory starting from time t; Q c (s t ,a t ) represents the trajectory χ t The expected value of the cumulative discounted cost generated; represents the discounted cost threshold, where d→0, represents the need to meet the operating constraints.

[0256] 5) Establish a sub-problem model (M2) based on relaxed optimal power flow. M2 is to calculate the expected relaxed level and operating cost of multiple sampling scenarios in the environment. It evaluates the feasibility and optimality of the unit commitment decision made by M1 under uncertainty. The relaxed optimal power flow model of a certain sampling scenario is defined in equations (14)-(23). Where the objective function involving f t is represented by equation (14). The node power balance constraint condition is defined as equation (15). Constraints (16)-(17) impose limits on the output power of units and wind power, respectively. Constraint conditions (18)-(19) constrain the branch power flow. Equations (20)-(21) map the on / off state of the unit to the start-up / shut-down decision. The unit ramping limit is established by equations (22)-(23).

[0257]

[0258] In the formula, the symbols "-" and "_" are the upper and lower bounds of each variable, respectively; the subscript i-j represents the branch from node i to j; B represents the branch set; K i-j,i represents the power flow distribution coefficient of branch i-j and node i; represents the actual grid-connected power generation of the wth wind power station; represents the predicted output power of the wth wind turbine; represents the prediction error sample obtained from the environment; v g , z g represent the start-up and shut-down decisions of the gth unit, respectively; RU g , RD g , SU g , SD g represent the upper ramping, lower ramping, start-up and shut-down probabilities of the gth unit, respectively.

[0259] 6) Establish M-SCUC solving method based on LBD-SAC algorithm

[0260] LBD-SAC uses the SAC baseline DRL framework to parameterize CMDP. To encourage the agent to fully explore the solution space of M1, LBD-SAC introduces an entropy constraint and a cost constraint, as shown in (24). Second, the constraint policy search method based on APDO is regarded as the logical deduction process of the LBD training mechanism, which is used to drive the neural network parameterized CMDP to achieve maximum entropy adaptive learning. LBD-SAC ensures the feasibility of decision-making while achieving the global optimality of M-SCUC.

[0261]

[0262] where θ represents the neural network parameters of the action policy π, denotes the entropy threshold.

[0263] The above process is described in Figure 3 , which describes the complete LBD-SAC algorithm. It consists of five core modules: CGAN, reward value critic networks (RV-CNs), cost value critic networks (CV-CNs), action networks, and Lagrange multipliers. Detailed descriptions of these modules are provided below. In addition, the Figure 9 Pseudocode of the proposed LBD-SAC is provided in the Appendix.

[0264] 6.1) Reward value critic networks and cost value critic networks, RV-CNs and CV-CNs, respectively evaluate the soft Q-values of rewards and costs based on the current operating state and the unit commitment decision made by the action policy. RV-CNs and CV-CNs are parameterized by φ and μ, respectively, and construct the mappings:

[0265] For the action policy π, the soft Q-functions of the reward and the cost are defined with Soft-Bellman operators and respectively, as follows:

[0266]

[0267] To approximate these soft Q-functions, distance-based 2-Wasserstein loss functions are used for RV-CN and CV-CN, respectively, as shown in (27) and (28), which aim to minimize the Soft-Bellman residual.

[0268]

[0269] where To experience the playback pool, to ensure the full use of exploration experience, to avoid heavy training process; respectively represent the corresponding target network parameters. The introduction of the target network aims to improve the stability of training. The target network performs soft update as the exponential moving average of the evaluation network. In Figure 3 The Double RV-CNs shown in Fig. 4 can conservatively estimate the Q value of the reward, which is believed to reduce the overestimation in the policy improvement process. Two CV-CNs are used to evaluate the soft Q value of the M1 constraint overrun cost and the relaxation level calculated by M2, respectively.

[0270] 6.2) Based on the action network of APDO and the Lagrange multiplier, the action network acts as an action policy to make unit commitment decisions according to the current operating state of the system. The action network involves an action policy π with parameters θ, which constructs a mapping: In order to find the optimal strategy and improve the efficiency of safety exploration, the APDO method is introduced in this patent. By establishing the Lagrange function (see (29)), the constraint maximization problem is converted into a dual optimization problem. The iteration between the original strategy and the Lagrange multiplier of the Lagrange function is the logical reasoning process in the LBD training mechanism. In addition, the Lagrange multiplier of the offline policy training is included in the dual update process, which improves the convergence performance of the action network and the Lagrange multiplier.

[0271]

[0272] In the formula, the Lagrange multipliers related to the entropy value and the safety exploration cost are parameterized by α and λ parameters, respectively.

[0273] Then, the CMDP formula (24) is converted into an unconstrained optimization problem, as shown in (30).

[0274]

[0275] In the formula, is the optimal original dual solution.

[0276] In addition, the loss function of the action network derived from formula (29) is shown in formula (31).

[0277]

[0278] The original dual update process of the kth iteration is as follows:

[0279] For the original update of the policy π θ , fix the Lagrange multipliers α (k) , λ (k) , and perform policy gradient ascent with learning rate ω, as shown in (32).

[0280]

[0281] For the dual update of Lagrange multipliers, fixed And perform dual gradient ascent, as shown in (33)-(34).

[0282]

[0283] After a certain number of iterations, the Lagrange multipliers α trained using an offline strategy off , λ off For α (k+1) , λ (k+1) Perform dual adjustment, i.e., α off →α (k+1) , λ off →λ (k+1) .

[0284] 7) The convergence proof of the M-SCUC problem-solving strategy based on the proposed LBD-SAC algorithm is as follows:

[0285] To prove the convergence of the LBD-SAC algorithm, we first prove the boundedness of the slack variables in weight requirement 6, i.e. The proof is as follows:

[0286] Will Decomposed into positive slack variables and negative slack variables Right now in and Equation (14) and Equation (15) can be equivalently transformed into:

[0287]

[0288] Further derivation The range of values ​​is and The range of values ​​is This proves It is bounded, indicating that the safety survey cost c(s) t ,a t () is bounded.

[0289] Secondly, the action policy π in LBD-SAC θ The optimal convergence of is equivalent to the optimal convergence of the soft Q value, that is, for any The following inequalities hold true.

[0290]

[0291] To prove equation (37), let... Let be the optimal solution of the form (where n > k), i.e., has the following definition:

[0292]

[0293] Since there exists a certain relationship From which we can deduce that:

[0294]

[0295] where N c represents the dimension of c π,ρ .

[0296] Inequality (39) is re-expressed as (40).

[0297]

[0298] where and are the soft state value functions of the reward and cost, respectively, as shown in (41) and (42).

[0299]

[0300] According to the soft Bellman equation, we can further deduce that:

[0301]

[0302] From (43), we can see that the sequence presents a monotonically increasing trend. This indicates that each new action policy improves performance. In M-SCUC, due to the finite dimensional nature of the action space, the reward r(s t ,a t ) and the entropy are bounded, and the cost c(s t ,a t ) is proven to be bounded. We can deduce that is bounded on π ∈ Π, and the sequence converges to a certain action policy Given it is logically known Therefore, the soft Q value of the final converged strategy is better than that of any other strategy within the range of Π, which confirms the optimality of . Thus, the policy convergence of the proposed LBD-SAC algorithm is proven.

[0303] Embodiment 12:

[0304] Referring to Figures 1 to 9A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the embodiment is based on the improved IEEE-118 test system, two wind farms are connected to bus 9 and 51 respectively. The hourly load data is derived from the measured data of a certain area in China. The wind power data comes from the public Belgian wind farm data. The linear scaling of the force data is between 0%-40% of the total capacity of the unit. This embodiment uses Gurobi, Pytorch and Tensorflow in Python, and is executed on Intel Core i9-10900K CPU and NVIDIA GeForce RTX 3090 GPU.

[0305] 1) Establish the M-SCUC model considering wind power uncertainty, the M-SCUC model can be regarded as minimizing the expected comprehensive scheduling cost under the physical constraint conditions of the overall system and individual adjustable resources and probabilistic resources. The invention calls it the original problem (M0), and its compact form is as follows:

[0306]

[0307] In the formula, the superscript t represents the time; T is the scheduling period; x represents the unit commitment decision variable vector, which involves the start-stop state of the unit; ζ represents the wind power prediction error vector, which is subject to the time-varying conditional probability distribution ρ t ; y represents the unit output power vector; z represents other state variables, including node voltage and branch power flow; E represents the expected comprehensive scheduling cost; A t , D t , H t and M t represent the coefficient matrix; et is the coefficient vector; N x , N y , N z , N ζ represent the dimensions of the respective variables.

[0308] 2) Establish the time-varying wind power prediction error probability distribution model based on CGAN, see Figure 1 for the modeling idea of the time-varying wind power prediction error probability distribution based on CGAN. Figure 1 In the , the generator network (Generator, G) captures the time correlation and distribution characteristics of wind power prediction error based on the prediction data P d and historical measured data H d . The simulated wind power data sequence G d is generated by G under the condition of prediction data P d . The discriminator network (Discriminator, G) evaluates the input data from H dThe probability of distribution. The training target of G is to maximize the evaluation error probability of D, and the target of D is to minimize the probability. Through the adversarial interaction training between G and D, G can accurately fit the time-varying wind power prediction error distribution. The completed generator G* can accurately simulate the time-varying uncertainty distribution in the M-SCUC problem.

[0309] 3) The proposed LBD training mechanism is executed, and the LBD method decomposes the task into a main problem and a sub-problem related to decision-making. Compared with the traditional BD method, it liberates the form of the main problem and the sub-problem. The sub-problem tightens the solution space of the main problem through iterative logical deduction, and finally converges to the feasible region of the original problem. In view of the task decomposition idea and logical reasoning strategy of LBD, the LBD-based DRL training mechanism proposed in the application is as shown in the figure. Figure 2 According to the specific structure of M0, it is classified according to variable types and model functions, and is decomposed into a main problem (M1) and a sub-problem (M2). M1 represents the unit commitment decision problem of unit start-stop state decision, and the compact form of the model is as shown in formula (2). M2 represents the relaxed optimal power flow model under the fixed unit on / off state, and the compact form of the model is as shown in formula (3).

[0310]

[0311] Wherein, M1 represents the fixed unit commitment decision obtained by M1; The start-up cost and shutdown cost of the unit; s is a relaxation variable vector, representing the relaxation degree of M1, as a feasible Benders cut signal; The expected operation cost; f t () represents the operation cost within a certain sampling prediction error.

[0312] In the proposed LBD-based training mechanism, the constraint overrun of M1 and the relaxation degree of M2 are regarded as the safety exploration cost of the agent in the decision space. The training purpose is to limit these costs to approximate the feasible region of M0. The start-stop cost of M1 and the operation cost of M2 are regarded as negative rewards. Then through logical deduction, a constantly tightening safety exploration space is provided for the agent of M1. The iterative interaction training between M1 and M2 at each time step t enables the agent to explore the optimal feasible solution within the entire scheduling period T. The LBD-based training mechanism effectively reduces the dimensions of the action and state space through task decomposition, thereby improving scalability and convergence. Through auxiliary training optimization, the satisfaction of operation constraints is significantly improved.

[0313] 4) The CMDP model of the unit commitment decision main problem M1 is established, and the CMDP model of M1 is defined as a tuple where denotes the state space; denotes the action space; is the state transition function; r denotes the reward obtained by choosing at s ; c denotes the safety inspection cost obtained by choosing a t at s t ; d is the cost threshold; γ (γ ∈ [0, 1]) denotes the discount rate. The CMDP model of M1 involves the environment and key elements as follows:

[0314] 4.1) Environment: The environment integrates the reward and cost functions involving M1 and M2, the time-varying wind power error distribution fitted from CGAN, and the state transition function. The agent makes unit commitment decisions according to the action policy and the perceived operating state. In the calculation of the reward and cost, Monte Carlo simulation is used to sample the wind power prediction error in CGAN. Then it is transferred to the next state according to the state transition function.

[0315] 4.2) State space: The definition of the state space is shown in equation (4), which contains the source load prediction information and trends, the start-stop state of the unit at the previous time, and the state duration.

[0316]

[0317] In the equation, the subscripts W, I, and G represent wind power generators, nodes, and the set of units, respectively; denotes the vector of predicted wind power output and load; denotes the on / off state of the unit; denotes the unit state duration; denotes the first-order difference form of the predicted wind power output in the adjacent period, which is introduced to consider the prediction trend. As follows:

[0318]

[0319] 4.3) Action space: The action space involves the decision variable x in M1, which makes unit commitment decisions to determine the start / stop state of the unit. An action is defined as a binary vector of N units, as follows:

[0320]

[0321] 4.4) State transition function: For exogenous states that are not affected by actions, their probability transition characteristics can be learned directly from the data set. In this paper, in order to solve the problem of scarce historical data, CGAN generates such a data set with time-varying uncertainty distribution. As for endogenous states that can respond to action feedback, including the duration of the unit state, as follows:

[0322]

[0323] 4.5) Cost function: The cost function maps the state and unit commitment decisions to the security survey cost, as defined in Eqs. (8)-(9). The cost function provides the basis for the agent to tighten the solution space to the feasible region of M0, thus ensuring the security of the operation.

[0324]

[0325] where π denotes the action policy; and denote the constraint violation of M1 and the relaxation level of M2, respectively; (·)+denotes max{·,0}; UT g , DT g denote the minimum up-time and minimum down-time of the gth unit, respectively. is the relaxation variable, which denotes the relaxed power of the ith node.

[0326] 4.6) Reward function: The reward function is composed of the negative value of the objective function of M1 and the penalty function of violating constraints. The reward function encourages the agent to explore the optimal solution, as defined in Eqs. (10)-(11).

[0327]

[0328] where denotes the penalty function of violating constraints; denotes the power of the gth unit; a g , b g , c g , h g , d g denote the constant cost coefficient, the linear cost coefficient, the quadratic cost coefficient, the startup cost, and the shutdown cost of the gth unit, respectively; κ v , κ δ denote the cost coefficients, which are incorporated into the reward function to improve the efficiency of constraint learning.

[0329] The CMDP model takes satisfying the cost threshold as the cost constraint condition. As defined in Eq. (12), the goal of training the policy is to maximize the cumulative discounted reward while satisfying the cost constraint. This lays the foundation for the agent to explore the optimal feasible unit commitment decision.

[0330]

[0331] where χ t = (s t , a t , …, sT , a T denotes the decision trajectory from time t; Q c (s t , a t denotes the trajectory χ t generated by the policy π and the prediction error distribution ρ|t; E denotes the discounted cost threshold, where d→0, indicates the operational constraints need to be satisfied.

[0332] 5) Establish the sub-problem model (M2) based on relaxed optimal power flow

[0333] M2 is to calculate the expected relaxed level and operational cost of multiple sampling scenarios in the environment. It evaluates the feasibility and optimality of the unit commitment decision made by M1 under uncertainty. The relaxed optimal power flow model of a certain sampling scenario is defined in equations (14)-(23). Where, the objective function involving f t is represented by equation (14). The node power balance constraint condition is defined as equation (15). Constraints (16)-(17) impose limits on the output power of units and wind power, respectively. Constraints (18)-(19) constrain the branch power flow. Equations (20)-(21) map the on / off state of the unit to the start-up / shut-down decision. The unit ramping limit is established by equations (22)-(23).

[0334]

[0335] In the formula, the symbols "-" and "_" are the upper and lower limits of each variable, respectively; the subscript i-j represents the branch from node i to j; B represents the branch set; K i-j,i denotes the power flow distribution coefficient of branch i-j and node i; denotes the actual grid-connected power generation of the wth wind power station; denotes the predicted output power of the wth wind turbine; denotes the prediction error sample obtained from the environment; v g , z g denote the start-up and shut-down decisions of the gth unit; RU g , RD g , SU g , SD g denote the upper ramping, lower ramping, start-up and shut-down probabilities of the gth unit.

[0336] 6) Establish the M-SCUC solving method based on LBD-SAC algorithm

[0337] LBD-SAC uses the SAC baseline DRL framework to parameterize CMDP. To encourage the agent to fully explore the solution space of M1, LBD-SAC introduces entropy constraints and cost constraints, as shown in (24). Second, the constraint policy search method based on APDO is regarded as the logical deduction process of the LBD training mechanism, which is used to drive the neural network parameterized CMDP to achieve maximum entropy adaptive learning. LBD-SAC ensures the feasibility of decision-making while achieving the global optimality of M-SCUC.

[0338]

[0339] where θ represents the neural network parameters of the action policy π, denotes the entropy threshold.

[0340] The above process is described in Figure 3 , which describes the complete LBD-SAC algorithm. It consists of five core modules: CGAN, reward value critic networks (RV-CNs), cost value critic networks (CV-CNs), action networks, and Lagrange multipliers. Detailed descriptions of these modules are provided below.

[0341] 6.1) Reward value critic networks and cost value critic networks, RV-CNs and CV-CNs, respectively evaluate the soft Q-values of rewards and costs based on the current operating state and the unit commitment decision made by the action policy. RV-CNs and CV-CNs are parameterized by φ and μ, respectively, and construct the mappings:

[0342] For the action policy π, the soft Q-functions of the reward and the cost are defined with Soft-Bellman operators and , respectively, as follows:

[0343]

[0344] To approximate these soft Q-functions, the distance-based 2-Wasserstein loss function is used to give RV-CN and CV-CN in (27) and (28), respectively, which aims to minimize the Soft-Bellman residual.

[0345]

[0346] where For experience replay pool, to ensure the full use of exploration experience, avoid heavy training process; respectively represent the corresponding target network parameters. The introduction of the target network aims to improve the stability of training. The target network performs soft update as the exponential moving average of the judge network.

[0347] In Figure 3 The Double RV-CNs can conservatively estimate the Q value of the reward, which is believed to reduce overestimation in the policy improvement process. Two CV-CNs are used to evaluate the soft Q value of M1 constraint overrun cost and the relaxation level calculated by M2, respectively.

[0348] 6.2) Action network based on APDO and Lagrange multiplier, the action network as an action policy, makes unit commitment decision according to the current system operating state. The action network involves the action policy π with parameter θ, which constructs a mapping: In order to find the optimal strategy and improve the safety survey efficiency, the APDO method is introduced in this patent. By establishing the Lagrange function (29), the constraint maximization problem is converted into a dual optimization problem. The iteration between the original strategy and the Lagrange multiplier of the Lagrange function is the logical reasoning process in the LBD training mechanism. In addition, the Lagrange multiplier of the offline policy training is included in the dual update process, which improves the convergence performance of the action network and the Lagrange multiplier.

[0349]

[0350] In the formula, the Lagrange multipliers related to the entropy value and the safety survey cost are parameterized by α and λ parameters respectively.

[0351] Then, the CMDP formula (24) is converted into an unconstrained optimization problem, as shown in (30).

[0352]

[0353] In the formula, is the optimal original dual solution.

[0354] In addition, the loss function of the action network is derived from formula (29), as shown in formula (31).

[0355]

[0356] The original dual update process of the kth iteration is as follows: for the policy π θ The original update, fix the Lagrange multipliers α (k) , λ (k) , and perform policy gradient ascent with learning rate ω, as shown in (32).

[0357]

[0358] For the dual update of the Lagrange multiplier, fix and perform dual gradient ascent as follows.

[0359]

[0360] After a certain number of iterations, the Lagrange multiplier α off , λ off is trained offline (k+1) , λ (k+1) Dual adjustment is performed on α off → α (k+1) , λ off → λ (k+1) .

[0361] 7) Verification of the effectiveness of the method proposed in the application

[0362] The following six methods are established to show the performance of the LBD-SAC algorithm proposed in the application in terms of training performance, economy, decision quality and solving efficiency in M-SCUC. M1: the LBD-SAC algorithm proposed in the application, the network architecture and parameters of LBD-SAC are shown in Table 1, and the network architecture of CGAN in the environment is shown in Tables 2-3. The unit commitment decision made by LBD-SAC is shown in Figure 4 .

[0363] Table 1 LBD-SAC neural network architecture and parameters of IEEE-118 test system

[0364]

[0365] Table 2 Structure and parameters of CGAN generator network

[0366]

[0367] Table 3 Structure and parameters of CGAN discriminator network

[0368]

[0369] 7.1) Training performance analysis, the reward and total safety survey cost of algorithms 1-3 with the change of training iteration number are shown in Figure 5 . Figure 5 (a) shows the rapid stabilization of the reward of LBD-SAC after a short oscillation, and converges at 4041 cycles. Compared with C-SAC and SAC, LBD-SAC has higher convergence efficiency and higher reward. In Figure 5In (b), the cost of LBD-SAC is significantly reduced, close to zero. This indicates that there is no constraint violation, indicating that the exploration space of the agent is close to the feasible region of M0. These advantages come from the proposed LBD training mechanism based on task decomposition and optimization assistance and the accelerated constraint policy search method APDO. These improvements significantly improve the safe exploration ability and convergence of DRL in solving large-scale, long-period M-SCUC problems.

[0370] 7.2) Economic analysis, this part compares the operation cost of algorithms 1-6 to evaluate their economic efficiency, see table 4 for details.

[0371] Table 4 Operation cost of various algorithms

[0372]

[0373] As shown in Table 1, compared with M2-M3, M1 shows lower operation cost in solving M-SCUC problem. This difference is related to the safe exploration ability of C-SAC and SAC, which hinders the formulation of reasonable unit commitment decision and leads to a large number of constraint violations. In addition, the cost of M1 increases by 1.0% compared with M4. This growth is attributed to the more comprehensive perception of LBD-SAC to uncertainty distribution, which leads to the start of more units to deal with uncertainty, which is proved to enhance safety in the following subsection. Since the RO of M5 is conservative, its operation cost is significantly higher than that of M1 by 9.2%. However, the economic benefit of the proposed LBD-SAC in M1 is not as good as DA in M6, which does not consider uncertainty.

[0374] 7.3) Analysis of unit commitment decision quality, in this section, the system regulation ability is evaluated a posteriori using the 1500 random wind scenarios generated by CGAN to evaluate the quality of unit commitment decision of all algorithms. The up / down regulation curve of the system is determined by the maximum / minimum load capacity based on the given unit commitment decision. The up / down regulation curve and the net load scenario set obtained by each method are shown in Figure 6 In addition, Figure 7 shows the frequency and distribution of energy supply shortage under random scenarios. This describes the situation of exceeding the up-regulation capacity or falling below the down-regulation capacity, depicting and distribution. Here, and represent the maximum and minimum load capacity of the tth period, respectively. In addition, in order to quantify the risk of unit commitment decision of each algorithm, two evaluation indexes are defined. In equation (33), the probability of insufficient power regulation capacity is represented by , which reflects the probability of load shedding or wind curtailment. (34) represents the expectation of load shedding and wind curtailment. The evaluation indexes of each method are listed in Table 2.

[0375]

[0376] where Ns is the number of scenarios; denotes the occurrence of load shedding and wind curtailment in the st scenario at the tth time period, and 0 otherwise.

[0377] Risk metrics of each algorithm

[0378]

[0379] From Figure 6 It can be seen that the width of the net load confidence interval varies over time, representing the time-varying uncertainty distribution. Compared with M2-M6, the up and down regulation curves generated by M1 most completely envelope the 100% confidence interval of the net load. From Figure 7 the exceedance frequency and distribution described in Table 5 and the risk metrics described in Table 5, it can be seen that M1 has the lowest risk of power supply and demand imbalance. In addition, as shown in Figure 6 the regulation curves obtained by M2-M3 show frequent start-stop of units and large power imbalance, which indicates that the unit commitment decision is significantly unreasonable. This further verifies the advantage of the proposed LBD-SAC in improving the constraint satisfaction of DRL. Compared with M1, M4 has lower sensitivity to uncertainty. As shown in Figure 6 M4 shows insufficient reserves during periods of low net load demand (e.g., 360-420) and high peak load periods (e.g., 690-720). Therefore, M4 shows more obvious regulation capacity shortage in Figure 7 the probability of power mismatch is twice as high as M1. The main reason for this difference is that SDDIP in M4 relies on a limited set of scenarios to represent uncertainty. In contrast, LBD-SAC has been trained through billions of interactions with uncertain environments. This accumulated experience significantly enhances the ability of LBD-SAC to effectively perceive and handle uncertainty. In M5, the conservative unit commitment decision made by RO ensures strong up-regulation capacity, which is reflected in almost no load shedding events, as shown in Figure 7 However, RO fails to adequately coordinate the ability of up-regulation and down-regulation, resulting in a higher risk of power supply and demand imbalance than M1. This highlights the advantage of LBD-SAC in balancing power supply reliability and wind power accommodation capacity. In addition, the data analysis in M6 only focuses on the predicted information, making the adjustable capacity of the decision result only cover 60% of the net load confidence interval. This will result in a high possibility of load shedding and wind curtailment of 2.2087%, and the expected gap will reach 2811.7 MW.

[0380] In summary, the LBD-SAC algorithm achieves the optimal unit commitment decision quality with the lowest operational risk in the M-SCUC problem due to its superior uncertainty perception and decision-making ability. The LBD-SAC achieves a good balance between robustness and economic benefits, ensuring the economic and safe operation of the power system.

[0381] 7.4) Computational efficiency analysis, this section aims to demonstrate the superior computational performance of the proposed method by providing the solution time of M1 and M4-M6, the results are shown in Table 6.

[0382] Table 6 Solution time of each algorithm

[0383]

[0384] The results of Table 6 show that the solution time of M4-M6 is significantly longer, it takes several days to solve complex M-SCUC problems. In sharp contrast, M1 achieves second-level solution, with an efficiency improvement of more than 99.9986% (69,000 times) relative to M4-M6. This significant performance leap is attributed to LBD-SAC, which formulates unit commitment decisions through forward propagation calculations in the action network. This replaces the modeling and iterative optimization process used in optimization-based methods, significantly improving solution efficiency.

[0385] Example 13:

[0386] Reference Figures 1 to 9 A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, the main technical content is seen in Example 12, further, this embodiment is based on an improved IEEE-300 test system, five wind farms are connected to busbars 3, 31, 109, 190 and 268 respectively. The hourly load data is derived from the measured data of a certain region in China. The wind power data comes from the public Belgian wind farm data. The linear scaling of the force data is between 0% and 40% of the total capacity of the unit. This embodiment uses Gurobi, Pytorch and Tensorflow in Python, and is executed on an Intel Core i9-10900K CPU and a NVIDIA GeForce RTX 3090 GPU. The unit commitment decision results are seen in Figure 8 . The neural network architecture and parameters of LBD-SAC are seen in Table 7.

[0387] Table 7 LBD-SAC neural network architecture and parameters of IEEE-300 test system

[0388]

[0389] Table 8 Solution results of each algorithm based on IEEE-300 test system

[0390]

[0391] From the convergence points of Table 8, it can be seen that M1 has the best convergence efficiency compared with M2-M3. M1 shows the lowest risk in the risk indicators. This indicates that for M-SCUC problems of different scales, the decisions made by LBD-SAC are still safer than other methods. In addition, the analysis of the solving time shows that as the system scale expands, the computational time cost and the challenge in computing capacity of the optimization methods in M4-M6 become more obvious. The proposed LBD-SAC method effectively reduces the computational burden of the M-SCUC problem with NP-hard. As the system scale expands, its advantage in solving efficiency is more obvious, which is more than 99.9994% higher than M4-M6.

Claims

1. A monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition, characterized in that, The method comprises the following steps: 1) establishing a monthly security-constrained unit commitment model considering wind power uncertainty, i.e., an M-SCUC model; 2) establishing a time-varying wind power prediction error probability distribution based on a conditional generative adversarial network; 3) decoupling discrete variables and continuous variables by using a logic Benders decomposition method, and decomposing the M-SCUC model into a unit commitment decision model represented in a constraint Markov decision process framework and a relaxed optimal power flow model; the unit commitment decision model represented in the constraint Markov decision process framework is as follows: where t represents time; T is a dispatch cycle; x represents a unit commitment decision variable vector; represents a fixed unit commitment decision obtained by a unit commitment decision model characterized in a constrained Markov decision process framework; f1 t (·) represents the start-up cost and shutdown cost of the unit; represents the expected operation cost; represents a coefficient matrix of a unit commitment decision model characterized in a constrained Markov decision process framework; is a coefficient vector of a unit commitment decision model characterized in a constrained Markov decision process framework; N x represents the dimension of the unit commitment decision variable vector x; M1 represents a unit commitment decision problem of the unit start-stop state decision; the relaxed optimal power flow model is as follows: where t denotes time; T is the dispatch period; x denotes the unit commitment decision variable vector; ζ denotes the wind power prediction error vector; y denotes the unit output power vector; z denotes other state variables; s is the slack variable vector; denotes the fixed unit commitment decision obtained from the unit commitment decision model characterized in the constrained Markov decision process framework; denotes the time-varying wind power prediction error probability distribution; denotes the expected operation cost; f t (·) denotes the operation cost within a certain sampling prediction error; denotes the coefficient matrix of the relaxed optimal power flow model; M2 is the coefficient vector of the relaxed optimal power flow model; N y , N z , N ζ , N s denote the dimensions of the unit output power vector y, the other state variable z, the wind power prediction error vector ζ, and the slack variable vector s, respectively; M2 represents the relaxed optimal power flow model under the fixed unit on / off state; 4) based on the unit commitment decision model, establishing an environment, a state space, an action space, a state transition function, a cost function and a reward function of a unit commitment decision main problem M1; based on the relaxed optimal power flow model, establishing an objective function and a constraint condition of a unit commitment decision sub-problem M2; 5) taking the time-varying wind power prediction error probability distribution as the uncertainty of the environment, generating a unit commitment decision action of the unit commitment decision main problem M1 based on a flexible actor-critic deep reinforcement learning baseline algorithm, and inputting the unit commitment decision action as a boundary condition into the unit commitment decision sub-problem M2; 6) based on the unit commitment decision action of the unit commitment decision main problem M1, solving the unit commitment decision sub-problem M2, and taking a relaxation level output by the unit commitment decision sub-problem M2 as an exploration cost to be substituted into the unit commitment decision main problem M1, and solving the unit commitment decision main problem M1 by using an LBD-SAC algorithm; the step of solving the unit commitment decision main problem M1 by using the LBD-SAC algorithm comprises: 6.1) introducing an entropy constraint and a cost constraint, as follows: where θ represents the neural network parameters of the action policy π, denotes an entropy threshold; t represents a time instant; T is a scheduling period; π represents an action policy; s t denotes a state at time t; a t denotes an action at time t; χ t = (s t , a t ,..., s T , a T ) denotes a decision trajectory starting from time instant t of the time-varying wind power prediction error probability distribution; r(s t , a t ) denotes a reward function; Q c (s t , a t ) denotes an expected value of the cumulative discounted cost generated by following the policy π and the trajectory χ t ; denotes a discounted cost threshold, where d→0; ζ represents a wind power prediction error vector; p t denotes a time-varying wind power prediction error probability distribution; c π,ρ is a cost function; γ represents a discount rate; 6.2) constructing an LBD-SAC solving module and initializing an iteration number k = 1; the LBD-SAC solving module comprises a CGAN module, two reward value evaluation network modules, two cost value evaluation network modules and an action network module; the CGAN module is used for generating a data set with a time-varying uncertainty distribution; the two reward value evaluation network modules are used for evaluating soft Q values of rewards; one cost value evaluation network module is used for evaluating a soft Q value of a constraint overrun cost of the unit commitment decision model represented in the constraint Markov decision process framework; one cost value evaluation network module is used for calculating a relaxation level of the relaxed optimal power flow model; the action network module formulates a unit commitment decision according to a current system operating state; 6.3) based on a distance-based 2-Wasserstein loss function, calculating target network parameters of the reward value evaluation network modules and the cost value evaluation network modules; the target network parameters of the reward value evaluation network modules and the cost value evaluation network modules are as follows: In the formula, respectively are soft Q functions of rewards and costs; J c (μ) respectively are approximate values of soft Q functions of rewards and costs; All are Soft-Bellman operators; respectively represent target network parameters of the reward value evaluation network module and the cost value evaluation network module; and φ and μ respectively represent network parameters of the reward value evaluation network module and the cost value evaluation network module; is a unit commitment decision set; 6.4) based on an APDO method, establishing a Lagrange function, converting an entropy constraint and a cost constraint maximization problem into a dual optimization problem, and deducing a loss function of the action network module; the Lagrange function is as follows: In the formula, a and λ are Lagrange multipliers related to the entropy value and the cost of safe exploration. is a Lagrangian function; π θ is an action policy; represents an entropy threshold value; is a cost function; the dual optimization problem is as follows: In the formula, is the optimal primal-dual solution; the loss function of the action network module is as follows: In the formula, J π (θ) is the loss function of the action network module; 6.5) judging whether the iteration number k is equal to the preset threshold value, if not, storing the Lagrange multiplier in this iteration into the experience pool, setting k=k+1, updating the action policy and the Lagrange multiplier, and returning to step 6.3); if yes, sampling a Lagrange multiplier from the experience pool to adjust the Lagrange multiplier of the last iteration, and entering step 6.6); The calculation formula of the updated action policy and the Lagrange multiplier is as follows: where (·)+ represents max{·, 0}; θ (k+1) , θ (k) represent the neural network parameters in the k+1th, kth iteration process, respectively; ω is a learning rate; is a gradient function; α (k) , λ (k) is a Lagrange multiplier in the kth iteration process; α (k+1) , λ (k+1) is a Lagrange multiplier in the k+1th iteration process; is an action policy in the k+1th iteration process; 6.6) outputting the optimal solution of the unit commitment decision; 7) repeating steps 5) to 6) until the reward function meets the convergence requirement, and outputting the final unit commitment decision.

2. The monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition according to claim 1, characterized in that, The monthly security-constrained unit commitment model considering wind power uncertainty is as follows: where t represents time; T is the dispatch cycle; x represents the unit commitment decision variable vector; ζ represents the wind power prediction error vector; y represents the unit output power vector; z represents other state variables; represents the expected comprehensive dispatch cost; A t , D t , H t and M t all represent coefficient matrices; e t is a coefficient vector; N x , N y , N z , N ζ respectively represent the dimension of the unit commitment decision variable vector x, the unit output power vector y, other state variables z, and the wind power prediction error vector ζ; M0 represents the monthly security constrained unit commitment model.

3. The monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition according to claim 1, characterized in that, The steps of establishing the time-varying wind power prediction error probability distribution based on the conditional generative adversarial network include: 2.1) Obtain wind power forecast data P d and historical measured data H d ; 2.2) based on wind power forecast data P d , generating simulated wind power data sequences G with a generator network d ; 2.3) concatenating the simulated wind power data sequence G d with the wind power prediction data P d ; The wind power prediction data P d and the historical measured data H d are spliced; 2.4) inputting the spliced data into the discriminator network to fit the time-varying wind power prediction error distribution; 2.5) based on the fitted time-varying wind power prediction error distribution, establishing the time-varying wind power prediction error probability distribution.

4. The monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition of claim 1, wherein, The state space of the unit commitment decision main problem M1 is as follows: In the formula, t represents time; W, I, G respectively represent a wind turbine generator set, a node, and a set of units; represents a state space; s t represents the state at t time; respectively represent a vector of predicted wind power output and a vector of predicted wind power load at t time; represents the on / off state of the unit at t-1 time; represents the unit state duration at t-1 time; wherein the first order difference form of the wind power output in adjacent time periods is predicted As shown below: In the formula, represents the vector of the predicted wind power output at time t-1; Δt represents the time difference; The action space of the unit commitment decision main problem M1 is as follows: In the formula, denotes the action space; a t denotes the action at time t; N is the number of units; denotes the on / off state of the unit at time t; The state transition function of the unit commitment decision main problem M1 is as follows: In the formula, denotes the duration of the unit state at time t; The cost function of the unit commitment decision main problem M1 is as follows: where c π,ρ denotes the cost function; ζ denotes the wind power prediction error vector; π denotes the action policy; denote the constraint violation of the unit commitment decision model and the relaxation degree of the relaxed optimal power flow model, respectively, which are characterized in the constrained Markov decision process framework; g is the unit number; i is the node number; UT g , DT g denote the minimum up time and the minimum down time of the gth unit, respectively; denote the on / off state of the gth unit at time t and t-1, respectively; is the state duration of the gth unit at time t-1; (·)+denotes max{·,0}; is the relaxation variable; denotes the time-varying wind power prediction error probability distribution; p denotes the prediction error probability; The reward function of the unit commitment decision main problem M1 is as follows: where r(s t , a t ) represents the reward function; represents the on / off state of the unit at time t obtained from the unit commitment decision model characterized in the constrained Markov decision process framework; f1 t (·) represents the start-up cost and shut-down cost of the unit; represents the expected operation cost; represents the penalty function for violating the constraints; represents the power of the gth unit; a g , b g , c g , h g , d g represent the constant term cost coefficient, the first-order term cost coefficient, the second-order term cost coefficient, the start-up cost, and the shut-down cost of the gth unit, respectively; κ v , κ δ both represent the cost coefficient; x represents the unit commitment decision variable vector; ζ represents the wind power prediction error vector; y represents the unit output power vector; z represents other state variables; s is the slack variable vector; represents the time-varying wind power prediction error probability distribution; f t (·) represents the operation cost within a certain sampling prediction error; The environment set of the unit commitment decision main problem M1 integrates the reward and cost functions of the unit commitment decision model characterized in the constrained Markov decision process framework and the relaxed optimal power flow model, the time-varying wind power prediction error probability distribution, and the state transition function.

5. The monthly unit commitment method based on flexible actor-critic combined logic Benders decomposition of claim 1, wherein, The objective function of the unit commitment decision sub-problem M2 is as follows: In the formula, t represents time; g is the unit number; G represents the set of units; a g , b g , c g respectively represent the constant term cost coefficient, the first term cost coefficient, and the second term cost coefficient of the gth unit; Pgt represents the power of the gth unit at time t; κ δ represents the cost coefficient; i is the node number; I represents the set of nodes; is the slack variable; The constraint conditions of the unit commitment decision sub-problem M2 include the node power balance constraint, the output power constraint of the unit and wind power, the branch power flow power constraint, the start-up / shut-down decision constraint, and the unit ramping constraint; The node power balance constraint is as follows: where G i represents the set of wind turbine generators at node i; w represents the wind turbine generator serial number; W i represents the set of wind turbine generators at node i; represents the actual grid-connected power of the wth wind power station; j is the node serial number; i-j represents the branch from node i to node j; B i represents the set of branches at node i; represents the vector of predicted wind power loads at node i at time t; represents the power flow of branch i-j; The output power constraint of the unit and wind power is as follows: In the formula, represents the wind power prediction error sample obtained by the wth wind turbine from the environment; represents the predicted output power of the wth wind turbine; represents the on / off state of the gth unit at time t obtained by the unit combination decision model characterized in a constrained Markov decision process framework; P g respectively represent the upper and lower bounds of the output power of the gth unit. The branch power flow power constraint is as follows: where K i-j,i denotes the power flow distribution factor of branch i-j and node i; I i denotes the branch set with the first node i; denotes the upper and lower bounds of the power flow of branch i-j, respectively; denotes the active load power of node i; The start-up / shut-down decision constraint is as follows: where v g , z g represent the start-up and shut-down decision of the gth unit, respectively; denotes the on / off state of the gth unit at time t-1 obtained from the unit commitment decision model characterized in the framework of constrained Markov decision process. The unit ramping constraint is as follows: In the formula, Pgt-1 represents the power of the gth unit at t-1; RU g , RD g , SU g , SD g Pgt-1 represents the power of the gth unit at t-1; RU

Images