Discrete amplitude and phase controllable phase shifter control method based on security reinforcement learning and hierarchical rolling optimization

Abstract

This invention discloses a discrete amplitude and phase controllable phase shifter control method based on secure reinforcement learning and hierarchical rolling optimization, belonging to the fields of power system operation control and renewable energy consumption technology. The method includes: acquiring grid state and source-load prediction data; calculating the node flexibility requirements and line allocation for each time period in the rolling time domain; constructing the input state vector of a pre-trained strategy network; enabling the strategy network to generate candidate phase shifter adjustment actions for the current first time period in the rolling time domain under action mask constraints; solving the source-grid variables and future tap positions using the hierarchical rolling optimization model based on these candidate phase shifter adjustment actions; retaining the solution if a solution exists, and discarding and correcting the candidate phase shifter adjustment action if no solution exists, to determine the final phase shifter action; and executing the first time period control command after AC power flow verification. This invention, through the synergy of the strategy network and AC physical constraints, can improve the renewable energy consumption and line transmission adaptability.

Description

Technical Field

[0001] This invention belongs to the field of power system operation control and new energy consumption technology, specifically involving a discrete amplitude and phase controllable phase shifter control method based on security reinforcement learning and hierarchical rolling optimization. Background Technology

[0002] As the scale of wind and solar power integration continues to increase, the daily operating status of the power system constantly changes with variations in renewable energy output and load. Spatially, there are often spatial differences between renewable energy-rich areas and load centers. Even if the total power generation capacity of the system can meet load demand, some transmission lines or key sections may still approach their transmission limits due to uneven power flow distribution, thus restricting the grid-connected power of wind and solar power. The renewable energy absorption capacity depends not only on the adjustability of the power source side but also on whether the transmission network can reserve sufficient transmission capacity for future power fluctuations.

[0003] During intraday operation, the control center can obtain the actual operating status for the current period and continuously update wind power, solar power, and load forecasts for several future periods. If only eliminating over-limits or reducing operating costs based on the current operating point, the resulting solution may cause some lines to approach their rated capacity in subsequent forecast periods. When the net load changes significantly in the future, unit ramp-up, reserve deployment, and adjustments to renewable energy grid connection will further alter network flow, leading to insufficient renewable energy transmission capacity. Therefore, it is necessary to consider both the current actual status and future rolling forecasts when formulating current control commands.

[0004] Amplitude and phase controllable phase shifters can redistribute active and reactive power flows by changing the line amplitude ratio and phase shift angle, and affect node voltage and line apparent power. This device uses a finite number of discrete shift points, and its actual operation is also limited by factors such as the maximum number of shift steps in adjacent time periods, the total number of switching operations, and the shortest hold time. Phase shifter shift points are discrete control variables on the grid side, but cannot constitute a complete operating scheme on their own; their function needs to be coordinated with the active and reactive power output of conventional units, upward and downward reserve capacity, wind and solar grid-connected power, and future time period operating plans.

[0005] Some existing methods evaluate line safety margins after scheduling by the difference between line load rate or line rated capacity and planned power flow, without first quantifying the node flexibility requirements caused by net load changes and prediction errors, nor specifying which lines will bear these requirements after network allocation. Some power flow control methods mainly optimize phase shifter or other FACTS device parameters, without incorporating unit output, reserve capacity, renewable energy grid connection, and reactive power and voltage variables into the same source-grid operation model; if reserve capacity is only set with upper and lower bounds and costs without establishing a coverage relationship with reserve demands caused by prediction errors, the optimization results will render reserve arrangements ineffective. Other methods directly weight and sum new energy consumption, line margin, and operating costs, and the results are easily affected by weight selection. Although reinforcement learning can learn discrete scheduling patterns, if a complete control scheme is directly provided by the policy network, it is difficult to strictly guarantee power balance, reserve coverage, line capacity, voltage, and phase shifter operation constraints.

[0006] Glossary

[0007] Safe reinforcement learning (SRL) algorithms allow agents to learn strategies through interaction with the environment, maximizing long-term accumulated rewards to ensure system performance while satisfying certain safety constraints in their behavior. Summary of the Invention

[0008] The purpose of this invention is to provide a discrete amplitude and phase controllable phase shifter control method based on secure reinforcement learning and hierarchical rolling optimization, which solves the problem of inconsistency between the learning strategy output and the physical feasible region of the power grid, realizes the synergy between the learning strategy and AC physical constraints, and improves the adaptability of new energy consumption and line transmission.

[0009] The technical solution adopted in this invention is as follows.

[0010] On one hand, the present invention provides a discrete amplitude and phase controllable phase shifter control method based on secure reinforcement learning and hierarchical rolling optimization, including performing the following operations within the rolling optimization cycle corresponding to each rolling moment:

[0011] Acquire basic power grid parameter data, power grid operation status data, and source-load prediction data;

[0012] Based on the acquired data, the node flexibility requirements and corresponding line flexibility transmission margins for each time period in the rolling time domain are calculated using a pre-built AC power flow operation model.

[0013] Based on the power grid operation status data, source load prediction data, node flexibility requirements and node flexibility requirement line allocation, a strategy network input state vector is constructed, which includes the current operation status sub-vector, prediction status sub-vector, node flexibility requirement vector, node flexibility requirement line allocation vector, and current phase shifter status sub-vector. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the rolling time domain based on the input state vector under the constraints of a preset action mask.

[0014] Based on the candidate phase shifter adjustment actions in the current first time period, the source-grid variables and future time period phase shifter adjustment actions are solved using a pre-constructed hierarchical rolling optimization model. The source-grid variables include conventional unit output, reserve capacity, renewable energy grid-connected power, node voltage, line power flow, and future time period phase shifter settings.

[0015] If a solution exists, the final action of the phase shifter in the current first time period is determined based on the candidate phase shifter adjustment action.

[0016] If there is no solution, discard the candidate phase shifter adjustment action and solve the hierarchical rolling optimization model again to obtain the phase shifter adjustment action in each time period in the rolling time domain in the optimal solution, and then calculate the final execution action of the phase shifter in the current first time period.

[0017] The AC power flow is verified based on the optimal solution of the hierarchical rolling optimization model corresponding to the final action of the phase shifter. If the AC power flow verification passes, the corresponding control command is issued.

[0018] Optionally, the basic power grid parameter data includes at least power grid topology, line parameters, and conventional unit parameter data; the power grid operation status data includes at least the operation parameters of power grid nodes, branches, conventional units, wind farms, photovoltaic power plants, and phase- and amplitude-controllable phase shifters, such as load, available wind power, available photovoltaic power, unit output, node voltage, line power flow, and current phase shifter position; the source-load forecast data includes at least wind power forecast, photovoltaic forecast, and load forecast data for multiple future time periods.

[0019] Optionally, the rolling time domain corresponding to each rolling moment includes multiple consecutive time periods, represented as follows:

[0020] ,

[0021] in, Indicates from the rolling moment The initial rolling time domain; Indicates the number of time periods included in the rolling time domain; This represents the set of future time periods excluding the current first time period;

[0022] Within any rolling optimization cycle, the data to be acquired uses the current measured values ​​for the current first time period and the predicted values ​​for the corresponding time period for future time periods.

[0023] In this invention, during each rolling optimization cycle, the method decides on the operating plan for the next H time periods starting from the current moment, but actually only executes the decision result of the current first time period. After entering the next time period, the rolling time domain is updated, and the future first time period in the rolling time domain of the previous rolling optimization cycle is transformed into the new current first time period. The acquired data is updated along with the real-time values ​​and the updated predicted values, thereby realizing the rolling optimization solution.

[0024] Optionally, based on the acquired data, the present invention uses a pre-built AC power flow operation model to calculate the node flexibility requirements and their corresponding line flexibility transmission margins for each time period, including:

[0025] For each time period, the node flexibility requirement is calculated based on the net load change with the adjacent future time period and the prediction error of load and new energy.

[0026] The power transmission distribution factor is calculated based on the basic parameters of the power grid, and the node flexibility requirements are mapped to the line allocation of each line based on the power transmission distribution factor. The basic parameters of the power grid used to calculate the power transmission distribution factor mainly involve network topology, line parameters and reference nodes. The specific calculation method refers to the existing technology.

[0027] The line flexibility transmission margin is determined based on the line allocation amount, line active power, and line active transmission power upper limit required for node flexibility.

[0028] The mathematical models used in the above calculations are all pre-built. In some embodiments, node flexibility requirements are one of the constraints of the AC power flow operation model.

[0029] Optionally, the hierarchical rolling optimization model is a three-layer rolling optimization model, where the source network optimization variables are the same for each layer, and the model is solved sequentially according to the optimization objective; wherein:

[0030] The first-level optimization objective of the three-layer rolling optimization model is to minimize the amount of wind and solar power curtailment, and the objective function is expressed as follows:

[0031] ,

[0032] In the formula, Indicates the rolling time The optimization variables include the active and reactive power output of conventional units, the upward and downward reserve capacity, the active and reactive power of wind power / solar power connected to the grid and the curtailed wind / solar power, the amplitude and phase angle of the phase shifter and the switching status. Represents the mixed-integer feasible region defined by the aforementioned AC power flow operation model; This represents the minimum amount of wind and solar power curtailed after the first layer of optimization. Represents the rolling time within the rolling time domain The amount of wind and solar power curtailed is expressed as:

[0033] ,

[0034] In the formula, Indicates from the rolling moment The initial rolling time domain, Indicates the duration of a time period within the rolling time domain; Indicates a collection of wind farms. Indicates a collection of photovoltaic power plants. Indicates wind farm During the period wind curtailment power, Indicates photovoltaic power station During the period The abandoned power; in the first-level optimization solution, node voltage, phase angle, active and reactive power and apparent power of the line and the transmission margin of line flexibility are operating states or auxiliary variables determined by the model constraints.

[0035] The second-level optimization objective of the three-layer rolling optimization model is to maximize the line flexibility transmission margin in the rolling time domain within the tolerance range of the minimum wind and solar power curtailment obtained from the first-level optimization. Its objective function is expressed as:

[0036] ,

[0037] In the formula, This represents the maximum line flexibility transmission margin after the second layer optimization. This represents the minimum value of the critical path flexibility propagation margin within the rolling time domain. This indicates the allowable tolerance for abandoned power; the second layer optimizes the transmission margin of the line flexibility within the tolerance range of the first layer's optimal value, avoiding the infeasibility of the next layer caused by non-convex AC model and discrete variable solution errors;

[0038] The objective of the third layer of the three-layer rolling optimization model is to minimize operation and scheduling costs within the tolerance range of the minimum wind and solar power curtailment obtained from the first layer optimization and the margin tolerance range of the maximum line flexibility transmission margin obtained from the second layer optimization. Its objective function is expressed as:

[0039] ,

[0040] In the formula, This represents the minimum operating and adjustment cost after the third layer of optimization; This indicates the allowable line flexibility transmission margin tolerance. For the rolling moment The operating and file adjustment costs are expressed as:

[0041] ,

[0042] in:

[0043] ,

[0044] ,

[0045] ,

[0046] In the formula, Indicates the rolling time The unit operating cost; Indicates conventional units During the period Those who have made meritorious contributions; , and These represent conventional generating units. The coefficients of the quadratic term, the linear term, and the constant term of the quadratic function of operating cost; Indicates the rolling time The cost of spare capacity; and These represent the upward reserve unit cost and the downward reserve unit cost, respectively. , These represent conventional generating units. During the period Upward reserve capacity, downward reserve capacity; Indicates the rolling time Phase shifter adjustment cost; This represents a set of discrete amplitude and phase controllable phase shifters. and These represent phase shifters. Unit adjustment cost for amplitude and phase angle settings; , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period The phase angle settings of the phase shifter.

[0047] Optionally, the construction of the AC power flow operation model includes: constructing a discrete amplitude and phase controllable phase shifter model, which uses discrete variables to define the phase shifter gear combination relationship, the amplitude ratio, phase shift angle, and equivalent admittance corresponding to each gear, and establishing constraints on gear changes and switching times, as well as constraints on the shortest hold time; wherein:

[0048] The phase shifter range combination relationship is expressed as follows:

[0049] ,

[0050] In the formula, This represents a set of discrete amplitude and phase controllable phase shifters; Indicates the phase shifter number; and These represent the amplitude range set and the phase angle range set, respectively. and These represent the corresponding gear number; Indicates time period Do you want to select a gear combination? , Indicates a choice. Indicates no selection; and These represent the amplitude range number and the phase angle range number, respectively.

[0051] The amplitude ratio, phase shift angle, and equivalent admittance corresponding to the gear position are expressed as follows:

[0052] ,

[0053] In the formula, and These represent phase shifters. During the period The amplitude ratio and phase shift angle; and These represent phase shifters. Corresponding amplitude range Amplitude ratio and corresponding phase angle settings The phase shift angle; and These represent phase shifters. During the period The equivalent conductance and susceptance; and These represent gear combinations. The conductance and susceptance include the equivalent impedance of the line and phase shifter;

[0054] The constraints on gear changes and switching frequency are expressed as follows:

[0055] ,

[0056] In the formula, , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period Phase angle settings of the phase shifter; and These represent phase shifters. The maximum number of adjustment steps for adjacent time periods of amplitude and phase; For phase shifter The upper bound constant of the gear change; For phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain; Indicates the rolling time The remaining number of allowed switching attempts;

[0057] The minimum hold time constraint is expressed as follows:

[0058] ,

[0059] In the formula, Indicates phase shifter The shortest duration of holding; Indicates phase shifter At the rolling moment The number of consecutive periods that the current gear has been maintained; This indicates the summation period number. The minimum hold time constraint formula simultaneously limits the switching interval within the rolling time domain and the remaining hold time from the previous rolling moment to the current time domain.

[0060] Optionally, the construction of the AC power flow operation model further includes:

[0061] The phase shifter amplitude ratio, phase shift angle, and equivalent admittance are mapped into the corresponding AC branch through the installation line and expressed as follows:

[0062] ,

[0063] , Indicates phase shifter Installation wiring; This represents the set of lines on which phase shifters are installed; They represent the lines respectively. During the period Amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance; These represent the installation of phase shifters. The route during the time period The amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance of the line without a phase shifter during the time period. The amplitude ratio, phase shift angle, conductance, and susceptance adopt fixed line parameters. ;

[0064] Substituting the phase shifter complex ratio, line series admittance, and charging susceptance into the AC branch model, active, reactive, and apparent power constraints at both ends of the line are established, where:

[0065] The product coefficient of line phase angle difference and voltage considering the complex ratio of phase shifter is expressed as:

[0066] ,

[0067] In the formula, , They represent the lines respectively. The first and last nodes; Indicates the time period The electrical phase angle difference of the line after phase shift is taken into account; , The lines are respectively Both ends in time period Electrical phase angle; Indicates the line During the period The voltage product coefficient, which is the ratio of the voltage amplitude product to the amplitude transformation ratio, They represent the lines respectively. Both ends in time period voltage amplitude, Indicates the line During the period The amplitude ratio of the phase shifter;

[0068] The active and reactive power flow constraints at both ends of the line are represented as follows:

[0069] ,

[0070] In the formula, , , , They represent the lines respectively. The two ends of the time period Active power and reactive power flowing into the line; Indicates the line Total charging current;

[0071] The apparent power constraint at both ends of the line is expressed as:

[0072] ,

[0073] In the formula, Indicates the line of End of time period Apparent power; Indicates the line of End of time period Apparent power; Indicates the line The rated apparent power may differ at both ends when there are line losses, so capacity verification and corresponding constraints need to be set separately.

[0074] Furthermore, the construction of the AC power flow operation model also includes: establishing unit output constraints, reserve capacity constraints, unit ramping constraints, wind / solar grid-connected power and wind / solar curtailment constraints, node active and reactive power balance constraints, node voltage and line phase angle difference constraints, node flexibility demand model, and line flexibility transmission margin constraints, wherein:

[0075] The unit output constraints and reserve capacity constraints are expressed as follows:

[0076] ,

[0077] In the formula, This represents a collection of conventional generating units. , These represent conventional generating units. During the period The contribution of effort that yields results and the contribution that yields no results; and These represent conventional generating units. Lower and upper limits of effort and contribution; and These represent conventional generating units. The lower and upper limits of reactive power output; , These represent conventional generating units. During the period Upward reserve capacity, downward reserve capacity; , These represent conventional generating units. Uphill climbing ability per unit time, downhill climbing ability per unit time; Indicates the duration of a time period within the rolling time domain; This represents the set of time periods in the rolling time domain, excluding the last time period. , Representing the rolling time respectively corresponding time period Upward and downward reserve demand; reserve capacity is used to cover additional power imbalances caused by forecast error correction, avoiding the same net load change being repeatedly included in planned output and reserve demand.

[0078] The unit ramping constraint is expressed as follows:

[0079] ,

[0080] This constraint is used to limit the changes in active power output of the unit between adjacent time periods;

[0081] The grid-connected power of wind power and photovoltaic power and the constraints on wind curtailment and solar curtailment are expressed as follows:

[0082] ,

[0083] In the formula, Indicates wind farm During the period The actual grid-connected active power; Indicates wind farm During the period The wind curtailment power; Indicates photovoltaic power station During the period The actual grid-connected active power; Indicates photovoltaic power station During the period The power of abandoned light; Indicates wind farm During the period Unproductive efforts; Indicates photovoltaic power station During the period Unproductive efforts; Indicates wind farm The rated apparent capacity of the converter; Indicates photovoltaic power station The rated apparent capacity of the converter;

[0084] The active and reactive power balance constraint of the node is expressed as follows:

[0085] ,

[0086] In the formula, For a set of nodes, , and Representing nodes respectively A collection of conventional generating units, wind farms, and photovoltaic power stations; Represents nodes A set of connected lines; , Representing nodes respectively During the period Inner Scrolling Time The active and reactive loads; combined with the aforementioned active and reactive power flow constraints at both ends of the line, the active and reactive power at the nodes. , Take the injection power at the corresponding beginning or end of the line where the node is located;

[0087] The node voltage and line phase angle difference constraints are expressed as follows:

[0088] ,

[0089] In the formula, and Representing nodes respectively voltage amplitude The lower and upper limits; Indicates reference node The voltage phase angle; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively.

[0090] The node net load and the node flexibility requirement model are expressed as follows:

[0091] ,

[0092] In the formula, Represents a node At the rolling moment corresponding time period The projected net load; Represents a node At the rolling moment corresponding time period The need for flexibility; Indicates wind farm At the rolling moment corresponding time period Available active power; Indicates photovoltaic power station At the rolling moment corresponding time period Available active power; and Representing nodes respectively During the period The load forecasting error correction amount and the new energy forecasting error correction amount; Indicates the rolling time corresponding time period The system's net prediction error;

[0093] The aforementioned load forecasting error correction and new energy forecasting error correction are determined based on the corresponding historical forecasting residuals. The historical forecasting residuals are statistically analyzed according to the forecast lead time. Positive and negative quantiles are used to form the forecasting error correction in the corresponding direction, so that different future periods adopt a correction range that matches their forecasting accuracy. A positive value corresponds to upward flexibility requirements, and a negative value corresponds to downward flexibility requirements. In the aforementioned reserve capacity constraints, upward and downward reserve requirements are determined based on the load and new energy prediction error correction amounts. In the node net load and the node flexibility requirement model formula, the prediction error correction amounts of each node are summarized into the system net prediction error, and its positive and negative parts are taken to form the upward and downward reserve requirements, respectively.

[0094] The line flexibility transmission margin constraint is expressed as follows:

[0095] ,

[0096] in, Indicates the time of the roll corresponding time period The route The required line allocation for node flexibility is expressed as:

[0097] ,

[0098] In the formula, Indicates the line At the rolling moment corresponding time period Internal line flexibility transmission margin; Indicates the line According to the first node Point to the end node The positive reference direction in the time period The active power is taken from the active power at the beginning of the line. , >0 indicates that active power is transmitted along the reference positive direction. <0 indicates that active power is transmitted in the opposite direction; Indicates the line The upper limit of active transmission power; Represents the set of critical paths. For the set of all routes, when evaluating all routes, take... ; This represents the minimum value of the critical path flexibility propagation margin within the rolling time domain. Indicates the time of the roll corresponding time period Under the network structure and line parameters, nodes Changes in unit active power injection on the line The power transmission distribution factor of active power flow; when and When they are in the same direction, the line allocation uses the remaining transmission capacity in the existing power flow direction; when they are in opposite directions, the existing power flow first releases space for the reverse transmission. This indicates that the line's transmission capacity in the corresponding direction is insufficient to meet the node flexibility requirements.

[0099] Optionally, the policy network includes a shared feature extraction layer, amplitude action branches and phase angle action branches corresponding to each phase shifter, and a state value branch. The shared feature extraction layer is used to receive the input state vector and extract state features. The input state vector includes the normalized power grid prediction state, current operating state, node flexibility requirements, line allocation, and phase shifter cross-time domain state, etc. Each action branch is used to output the corresponding gear increment probability based on the state features under a preset action mask constraint, and the state value branch is used to output the current state value.

[0100] The policy network is trained offline, and the training method includes:

[0101] Based on basic power grid parameter data, historical power grid operation status data, and concurrent source-load forecast data, multiple sets of rolling time-domain data samples are constructed.

[0102] Based on the rolling time-domain data sample, at each rolling moment, the hierarchical rolling optimization model is solved, and the optimal solution and its corresponding optimization target value are used as a reference benchmark.

[0103] Based on the rolling time domain data samples, a strategy network input state vector is constructed corresponding to each rolling moment. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the current rolling time domain based on the input state vector under the constraints of a preset action mask, and obtains the corresponding state value through the state value branch.

[0104] The hierarchical rolling optimization model is solved based on the candidate phase shifter adjustment actions of the current first time period. The immediate reward is calculated based on the feasibility of the model solution and the comparison results between the optimization target value obtained when the model is feasible and the reference benchmark. The advantage estimate for policy update is obtained based on the immediate reward and state value corresponding to multiple sets of rolling time domain data samples.

[0105] Based on the aforementioned advantage estimation, the policy network is updated using a proximal policy optimization algorithm until the set iteration termination condition is met.

[0106] Optionally, the input state vector of the policy network is represented as:

[0107] ,

[0108] In the formula, Indicates the rolling time The input state vector; Indicates the rolling time The corresponding rolling time domain consists of a predicted state sub-vector composed of load, wind power, and photovoltaic forecast data. Representing the rolling time respectively The corresponding scrolling time in the scrolling time domain To the rolling moment The active load vector, reactive load vector, available active power vector of wind farm, and available active power vector of photovoltaic power station are used, where the measured values ​​are used for the current first time period and the predicted values ​​are used for future time periods. , Representing the rolling time respectively The corresponding node flexibility requirement vector and node flexibility requirement line allocation vector, both of which include the rolling time. The corresponding rolling time domain is used to obtain node flexibility demand data and node flexibility demand line allocation data based on the difference in net load between adjacent time periods and the prediction error correction amount. Among them, the node flexibility demand is calculated based on the net load change between adjacent time periods and the prediction error correction amount, and the node flexibility demand line allocation amount is obtained by mapping the node flexibility demand to each line based on the power transmission distribution factor. Indicates the rolling time Current measured node voltage Phase angle Active power output of the unit and line apparent power The resulting sub-vectors of operating states; , These represent the amplitude range vector and phase angle range vector of each phase shifter at the previous rolling moment, respectively. Indicates the rolling time Vector of remaining allowable switching times for each phase shifter Indicates the rolling time Each phase shifter has continuously maintained the current gear's time period vector, and the three of them constitute the current phase shifter state sub-vector;

[0109] The magnitude action branch and phase action branch of the policy network are based on the rolling time. The input state vector is the decision output of the current first time period action vector. Represented as:

[0110] ,

[0111] In the formula, These represent the phase shifters for the current first time period. The amplitude increment and phase angle increment; Indicates the time of the roll The set of joint selectable actions is filtered according to a preset action mask constraint, wherein the preset action mask constraint defines gear boundaries, gear shifting steps, remaining switching times, and cross-time domain hold-up status.

[0112] The policy probability after filtering by action mask constraints is expressed as:

[0113] ,

[0114] In the formula, Indicates the policy network parameters; Indicates the action branch of any value. Or phase angle action branch ; The policy network represents the input state vector. After making decisions under the constraints of the action mask, the action vector is output. The joint strategy probability is obtained by multiplying the magnitude branch and phase angle branch probabilities of each phase shifter. and These are the corresponding amplitude action branch output and phase action branch output, respectively; Indicates phase shifter Corresponding amplitude action branch or phase action branch Based on the input state vector Output amplitude or phase motion under motion mask constraints. The probability of the strategy; Indicates phase shifter Corresponding amplitude action branch or phase action branch The original strategy probability; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The corresponding set of branch actions, Use it as the variable for the branch action; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The action mask obtained by performing action mask constraint judgment on the action* is obtained by judging each candidate action of the corresponding branch according to action mask constraints such as gear boundary, maximum number of gear shifting steps, remaining number of switching times and minimum holding time. If the action meets the constraints, the action mask is 1, otherwise it is 0. This is used to limit each action branch to retain at least one non-shifting action, so that the normalized denominator is always greater than 0.

[0115] The instant rewards at each rolling moment are represented as follows:

[0116] ,

[0117] In the formula, Indicates the rolling time Instant rewards; Indicates the time of the roll Fixed candidate phase shifter gear adjustment action Then, the feasible region is jointly defined by the constraints of the AC power flow operation model and the hierarchical rolling optimization model. Indicates fixed candidate actions The post-layered rolling optimization model is not feasible; , and These represent the optimization target values ​​obtained from solving the hierarchical rolling optimization model for candidate phase shifter adjustment actions based on policy network decision-making, and are relative to the normalized power abandonment difference, line flexibility transmission margin difference, and cost difference of the reference benchmark, all of which are limited to between 0 and 1. , , They represent action-based The rolling time obtained by solving the layered rolling model The amount of wind and solar power curtailed, the transmission margin of line flexibility, and the operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between wind and solar power curtailment. This represents the penalty constant when a candidate action causes a degradation in the amount of wind and solar power curtailed in the first layer. This represents the penalty constant when the transmission flexibility margin of the second-level optimization objective deteriorates, assuming no over-tolerance degradation occurs in the first-level optimization objective. Let represent the penalty constant when a candidate action makes the hierarchical rolling optimization model infeasible, and satisfy . , and ;

[0118] Before evaluating the real-time rewards at each rolling moment, this invention first determines whether the hierarchical optimization model corresponding to the fixed candidate action is feasible, i.e., whether there is an optimal solution. Then, it evaluates the abandoned power, the transmission margin of line flexibility, and the operating cost in turn. If any layer optimization objective is not feasible, the solution of the hierarchical optimization model is terminated, so that the improvement of the later layer cannot compensate for the degradation of the previous layer, while improving the efficiency of policy network training.

[0119] The advantage estimate for policy updating, obtained based on the immediate reward and state value corresponding to multiple sets of rolling time-domain data samples, is expressed by the following formula:

[0120] ,

[0121] In the formula, Indicates the rolling time The corresponding advantage estimate is used for policy updates; Indicates the corresponding rolling time The temporal difference residuals of the training rounds; Indicates the length of the advantage estimate; Indicates the discount factor; Indicates the advantage estimate attenuation coefficient; The state value branch parameter is... State value function;

[0122] The policy network is updated using a near-end policy optimization algorithm, wherein the near-end policy optimization objective is expressed as:

[0123] ,

[0124] In the formula, This indicates the policy network parameters that need to be updated. The policy network parameters are The corresponding objective function for optimizing the near-end policy, This indicates taking the expected value; Represents the joint vector actions of the new and old policies. The probability ratio; This represents the policy network parameters before the update; This is a truncation function. This represents the probability ratio cutoff coefficient. Represents the joint masking policy entropy. This represents the policy entropy coefficient.

[0125] Optionally, the AC power flow verification based on the optimal solution of the hierarchical rolling optimization model corresponding to the final execution action of the phase shifter includes determining whether the following AC power flow verification conditions are met:

[0126] ,

[0127] In the formula, and Representing nodes respectively During the period The active power balance residual and the reactive power balance residual; , These represent the maximum allowable active power balance residual and the maximum reactive power balance residual, respectively. They represent the lines respectively. of End and End of time period Apparent power Indicates the line Rated apparent power; and Representing nodes respectively During the period voltage amplitude The lower and upper limits; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively; These represent phase shifters. During the period Amplitude and phase settings; and These represent the amplitude range set and the phase angle range set, respectively. Indicates phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain, Indicates the number of time periods included in the rolling time domain; Indicates the rolling time The remaining number of allowed switching attempts; Indicates phase shifter The shortest duration of holding time.

[0128] The formula for the above AC power flow verification conditions can be used to verify the node power balance, line capacity at both ends, node voltage, line electrical phase angle difference, phase shifter position, remaining switching times, and minimum hold time for each time period in the rolling time domain.

[0129] In some possible embodiments, the phase shifter position is only part of the current first-period source-grid control command; at the same time, commands for the active and reactive power output of conventional units, the upward and downward reserve capacity, and the active and reactive power grid connection of wind power and photovoltaic power are also issued simultaneously. Node voltage and line power flow are the operating states formed after executing the above control commands and are not issued as independent commands.

[0130] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the discrete amplitude and phase controllable phase shifter control method based on secure reinforcement learning and hierarchical rolling optimization provided in the first aspect.

[0131] Beneficial effects

[0132] Compared with the prior art, the present invention has the following advantages and advancements:

[0133] (1) The current actual operating status and future rolling forecast are used together to form the optimization input, so that the current control command obtained from the decision can take into account the current operating conditions, the new energy transmission demand in the subsequent forecast period, and the adjustment demand caused by the forecast error.

[0134] (2) Incorporate the active and reactive power output of conventional units, the upward and downward reserve capacity, the active and reactive power grid connection of new energy sources, the curtailment of wind and solar power, and the discrete positions of phase shifters into the same source-grid optimization model, and use the reserve coverage constraint to ensure that the cost of reserve capacity corresponds to the actual prediction error requirements, so as to avoid only optimizing the phase shifter positions or setting reserve variables that have no practical effect.

[0135] (3) Calculate the node flexibility requirements based on the net load changes and historical prediction errors in adjacent prediction periods, obtain the line allocation amount of the node flexibility requirements using the power transmission distribution factor, and construct the line flexibility transmission margin according to the relationship between the line allocation amount and the existing power flow direction, so that the index can reflect the remaining capacity of the line after completing the source load fluctuation transmission task.

[0136] (4) A complete AC branch model is used to describe the combined effect of phase shifter amplitude ratio, phase shift angle and equivalent admittance on active power, reactive power and voltage of the line. At the same time, discrete taps, tap adjustment steps, remaining switching times and cross-rolling time domain holding time constraints are set to make the obtained scheme conform to the actual operating characteristics of the device.

[0137] (5) The safety reinforcement learning strategy only generates candidate shifting actions for the current first time period phase shifter. The complete hierarchical rolling optimization model is responsible for the unit, new energy, standby and future phase shifter schemes. Action mask, optimization feasibility correction, standby coverage, AC power flow and device constraint verification jointly ensure the feasibility of the final execution command.

[0138] (6) Solve the three-layer model in the order of new energy consumption, line flexibility transmission margin and operating cost to avoid the weight sensitivity caused by directly weighting different dimension targets, and make the optimization results and control priority relationship of each layer clear and interpretable.

[0139] (7) The training reference benchmark generation, candidate action evaluation and online infeasibility correction all adopt the same hierarchical rolling optimization model with complete communication power flow constraints, so that the policy learning objectives, online action evaluation and actual execution constraints are consistent, and the unexecutable adjustment actions caused by the inconsistency between the training model and the running model are reduced. Attached Figure Description

[0140] Figure 1 The diagram shows the implementation path of the discrete amplitude and phase controllable phase shifter control method based on security reinforcement learning and hierarchical rolling optimization in an embodiment of the present invention.

[0141] Figure 2 The diagram shown is a schematic representation of the current actual state and the future rolling prediction update process in an embodiment of the present invention.

[0142] Figure 3 The diagram shown is a schematic of the offline security reinforcement learning training and online source-network collaborative rolling control architecture in an embodiment of the present invention. Detailed Implementation

[0143] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details.

[0144] The core inventive concept of this invention is as follows: Under the conditions of continuous updates to new energy and load forecasts, discrete phase shifters with limited switching times and minimum hold time, discrete phase shifters with controllable amplitude and phase are used as a means of grid-side power flow control. This coordinates the output and reserve capacity of conventional units, the grid connection of active and reactive power from new energy sources, and the discrete phase shifter positions. Based on the current actual operating status, the flexibility requirements of future rolling forecast calculation nodes, and the transmission margin of line flexibility, the new energy consumption and line flexibility transmission margin are incorporated into the rolling decision reference for the phase shifter adjustment actions. The invention also uses secure reinforcement learning to generate candidate phase shifter adjustment actions for the current first time period. Then, a mixed integer hierarchical rolling optimization model with complete AC power flow constraints is used to verify the candidate adjustment actions. When the candidate adjustment actions are not feasible, an executable correction scheme is obtained, ensuring that the adjustment actions generated by reinforcement learning are consistent with AC power flow, reserve coverage, and source-grid coordinated scheduling. Finally, after the AC power flow verification is passed, a complete source-grid control scheme is determined.

[0145] Example 1

[0146] This embodiment introduces a control method for a discrete amplitude and phase controllable phase shifter based on secure reinforcement learning and hierarchical rolling optimization, referencing... Figure 3 The process and method of online intraday rolling control on the right side of the middle section include performing the following operations within the rolling optimization cycle corresponding to each rolling moment:

[0147] Acquire basic power grid parameter data, power grid operation status data, and source-load prediction data;

[0148] Based on the acquired data, the node flexibility requirements and corresponding line flexibility transmission margins for each time period in the rolling time domain are calculated using a pre-built AC power flow operation model.

[0149] Based on the power grid operation status data, source load prediction data, node flexibility requirements and node flexibility requirement line allocation, a strategy network input state vector is constructed, which includes the current operation status sub-vector, prediction status sub-vector, node flexibility requirement vector, node flexibility requirement line allocation vector, and current phase shifter status sub-vector. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the rolling time domain based on the input state vector under the constraints of a preset action mask.

[0150] Based on the candidate phase shifter adjustment actions in the current first time period, the source-grid variables and future time period phase shifter adjustment actions are solved using a pre-constructed hierarchical rolling optimization model. The source-grid variables include conventional unit output, reserve capacity, renewable energy grid-connected power, node voltage, line power flow, and future time period phase shifter settings.

[0151] If a solution exists, the final action of the phase shifter in the current first time period is determined based on the candidate phase shifter adjustment action.

[0152] If there is no solution, discard the candidate phase shifter adjustment action and solve the hierarchical rolling optimization model again to obtain the phase shifter adjustment action in each time period in the rolling time domain in the optimal solution, and then calculate the final execution action of the phase shifter in the current first time period.

[0153] The AC power flow is verified based on the optimal solution of the hierarchical rolling optimization model corresponding to the final action of the phase shifter. If the AC power flow verification passes, the corresponding control command is issued.

[0154] Implementation path reference for this embodiment Figure 1 In summary, this invention first establishes an AC power flow operation model and a hierarchical rolling optimization model containing discrete amplitude and phase controllable phase shifters. Based on the current actual state and future rolling predictions to form node flexibility requirements, the physical benchmark is formed by optimizing the results of the complete hierarchical rolling optimization model when the action is not fixed during the offline training of the security reinforcement learning policy network. During the online rolling optimization operation phase, only the candidate shifting actions of the phase shifter in the first time period within the rolling time domain are obtained based on the output of the policy network. Other source network variables and future shift positions are still optimized by the complete hierarchical rolling optimization model. Furthermore, when the candidate shifting actions output by the policy network are not feasible, the action correction is achieved by the complete hierarchical rolling optimization model, and control commands are issued after the AC power flow verification is passed.

[0155] The following sections will describe in detail the contents involved in the implementation of this embodiment.

[0156] I. Construction of the Scrolling Time Domain for Scrolling Optimization:

[0157] In both the offline training phase and the online operation phase, this embodiment requires acquiring and updating the rolling time domain and its corresponding parameter data for each rolling optimization cycle.

[0158] As the basis for power grid operation control decisions, the data required in this embodiment includes basic power grid parameter data, power grid operation status data, and source-load forecast data. The basic parameter data includes at least power grid topology, line parameters, and conventional unit parameter data. The power grid operation status data includes at least the operation parameters of power grid nodes, branches, conventional units, wind farms, photovoltaic power plants, and phase- and amplitude-controllable phase shifters, such as load, available wind power, available photovoltaic power, unit output, node voltage, line power flow, and current phase shifter position. The source-load forecast data includes at least wind power forecast, photovoltaic forecast, and load forecast data for multiple future time periods.

[0159] The method for obtaining and updating the rolling data required for multiple daily rolling optimization cycles is as follows:

[0160] S11, at any rolling moment It acquires the operating parameters of grid nodes, branches, conventional units, wind farms, photovoltaic power stations and amplitude and phase controllable phase shifters, and reads the current actual operating status and the latest updated predicted values;

[0161] S12, with the current scrolling time Establish a rolling time domain of length H starting from the beginning; the rolling time domain for each rolling optimization cycle is represented as follows:

[0162] ,

[0163] in, Indicates from the rolling moment The initial rolling time domain; Indicates the number of time periods included in the rolling time domain; This represents the set of future time periods excluding the current first time period;

[0164] In some possible embodiments, the length of each time period within the rolling time domain can be 15 minutes, and the length H of the rolling time domain can be 8 time periods.

[0165] S13 sets the load, wind power availability, and photovoltaic power availability data for the current first time period within the rolling time domain to measured values, and sets the data for the remaining future time periods to predicted values;

[0166] The unified assignment relationship between the current measured value and the future predicted value satisfies equation (2):

[0167] ,

[0168] In the formula, Represents a node The rolling moment of the first time period within the rolling time domain. The active load is taken as the nodal measured active load at the corresponding time. ; Represents a node The rolling moment of the first time period within the rolling time domain. The reactive load is determined by the measured reactive load at the node at the corresponding time. ; Indicates wind farm The rolling moment of the first time period within the rolling time domain. The available active power is taken as the measured available active power of the wind farm at the corresponding time. ; Indicates photovoltaic power station The rolling moment of the first time period within the rolling time domain. The available active power is the measured available active power of the photovoltaic power station at the corresponding time. ;

[0169] Represents a node Future prediction period within the rolling time domain The rolling moment The active load is the predicted value of the nodal active load at the corresponding time. ; Represents a node Future prediction period within the rolling time domain The rolling moment The reactive load is the predicted reactive load value of the node at the corresponding time. ; Indicates wind farm Future prediction period within the rolling time domain The rolling moment The available active power is the predicted value of the available active power of the wind farm at the corresponding time. ; Indicates photovoltaic power station Future prediction period within the rolling time domain The rolling moment The available active power is the predicted value of the future available active power of the photovoltaic power station at the corresponding time. .

[0170] S14, at the start of the next rolling optimization cycle, read the new actual state and update the future prediction values, shifting the rolling time domain forward by one time period, such as... Figure 2 As shown.

[0171] In this invention, during each rolling optimization cycle, the method decides on the operating plan for the next H time periods starting from the current moment, but actually only executes the decision result of the current first time period. After entering the next time period, the rolling time domain is updated, and the future first time period in the rolling time domain of the previous rolling optimization cycle is transformed into the new current first time period. The acquired data is updated along with the real-time values ​​and the updated predicted values, thereby realizing the rolling optimization solution.

[0172] II. Construction of the AC power flow operation model:

[0173] This embodiment uses discrete variables to describe the amplitude and phase angle ranges of the phase shifter, establishing a unique range combination selection, the number of range adjustment steps in adjacent time periods, the remaining switching count within a specified statistical period, and the minimum hold time constraint across the rolling time domain. The amplitude ratio, phase angle, and equivalent admittance corresponding to each range are mapped to the installed line, constructing an AC power flow operation model that considers the complex ratio, line loss, and apparent power capacity at both ends. The modeling process specifically includes:

[0174] S21, set an amplitude range set and a phase angle range set for each phase shifter, and use 0-1 variables to select a unique range combination for each time period;

[0175] S22, determine the amplitude ratio, phase shift angle and equivalent admittance of the selected range according to the device range parameter table of each phase shifter, and write the device parameters into the corresponding AC branch through the installation line mapping.

[0176] S23, set the number of adjustment steps for adjacent time periods, the remaining number of switching times, and the minimum hold time constraint across the rolling time domain;

[0177] S24. Substitute the complex ratio of the phase shifter, the series admittance of the line, and the charging susceptance into the AC branch model to establish active, reactive, and apparent power constraints at both ends of the line.

[0178] S25 establishes the coverage of active and reactive power output of conventional units, upward and downward reserve capacity and their demand, grid connection of active and reactive power of new energy sources, wind and solar curtailment, node power balance, voltage, unit ramping, node flexibility requirements and line flexibility transmission margin constraints.

[0179] 2.1 Constructing a discrete amplitude and phase controllable phase shifter model

[0180] In this embodiment, the aforementioned device range parameter table is pre-established for each phase shifter. The amplitude range determines the amplitude ratio, and the phase angle range determines the phase shift angle. If the equivalent impedance of the device changes with the range, the range parameter table simultaneously provides the corresponding equivalent conductance and susceptance. Each phase shifter is only allowed to select one amplitude-phase angle combination in each time period, and the device parameters of the phase shifter are written into the corresponding AC branch through the installation line mapping.

[0181] The discrete amplitude and phase controllable phase shifter model uses discrete variables to define the phase shifter's gear combination relationship, as well as the amplitude ratio, phase shift angle, and equivalent admittance corresponding to each gear. It also establishes constraints on gear changes and switching frequency, and a minimum hold time constraint.

[0182] The phase shifter range combination relationship is expressed as follows:

[0183] ,

[0184] In the formula, This represents a set of discrete amplitude and phase controllable phase shifters; Indicates the phase shifter number; and These represent the amplitude range set and the phase angle range set, respectively. and These represent the corresponding gear number; Indicates time period Do you want to select a gear combination? If selected, the value is 1; otherwise, it is 0. and These represent the amplitude range number and the phase angle range number, respectively.

[0185] The amplitude ratio, phase shift angle, and equivalent admittance corresponding to the gear position are expressed as follows:

[0186] ,

[0187] In the formula, and These represent phase shifters. During the period The amplitude ratio and phase shift angle; and These represent phase shifters. Corresponding amplitude range Amplitude ratio and corresponding phase angle settings The phase shift angle; and These represent phase shifters. The route during the time period The conductance and susceptance include the equivalent impedance of the line and phase shifter; and These represent gear combinations. The conductance and susceptance include the equivalent impedance of the line and phase shifter;

[0188] The constraints on gear changes and switching frequency are expressed as follows: ,

[0189] In the formula, , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period Phase angle settings of the phase shifter; and These represent phase shifters. The maximum number of adjustment steps for adjacent time periods of amplitude and phase; For phase shifter The upper bound constant of the gear change; For phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain; Indicates the rolling time The remaining allowed number of switching attempts will be deducted accordingly after the first time period action is executed;

[0190] The minimum hold time constraint is expressed as follows: ,

[0191] In the formula, Indicates phase shifter The shortest duration of holding; Indicates phase shifter At the rolling moment The number of consecutive periods that the current gear has been maintained; The summation period number is indicated. Equation (6) simultaneously limits the switching interval within the rolling time domain and the remaining hold time from the previous rolling moment to the current time domain, preventing the loss of the hold requirements that were not completed in the previous switching after the rolling time domain is re-established.

[0192] In some possible embodiments, the amplitude ratio settings of the phase shifter can be set to 0.95, 0.975, 1.00, 1.025, and 1.05, and the phase angle settings can be set to -10°, -5°, 0°, 5°, and 10°. The amplitude and phase angle settings in adjacent time periods are constrained to change by a maximum of one level, and at least two time periods are maintained after each adjustment. The maximum number of switching operations within the statistical period is given according to the device manufacturing parameters and operating procedures. The above values ​​are used to illustrate the discrete control process and do not limit other setting constraint methods that meet the device's technical requirements.

[0193] 2.2 The phase shifter amplitude ratio, phase shift angle, and equivalent admittance are mapped into the corresponding AC branch through the installation line, and expressed as:

[0194] ,

[0195] , Indicates phase shifter Installation wiring; This represents the set of lines on which phase shifters are installed; They represent the lines respectively. During the period Amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance; These represent the installation of phase shifters. The route during the time period The amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance of the line without a phase shifter during the time period. The amplitude ratio, phase shift angle, conductance, and susceptance adopt fixed line parameters. ;

[0196] Substituting the phase shifter complex ratio, line series admittance, and charging susceptance into the AC branch model, active, reactive, and apparent power constraints at both ends of the line are established, where:

[0197] The product coefficient of line phase angle difference and voltage considering the complex ratio of phase shifter is expressed as:

[0198] ,

[0199] In the formula, , They represent the lines respectively. The first and last nodes; Indicates the time period The electrical phase angle difference of the line after phase shift is taken into account; , The lines are respectively Both ends in time period Electrical phase angle; Indicates the line During the period The voltage product coefficient, which is the ratio of the voltage amplitude product to the amplitude transformation ratio, They represent the lines respectively. Both ends in time period voltage amplitude, Indicates the line During the period The amplitude ratio of the phase shifter;

[0200] The active and reactive power flow constraints at both ends of the line are represented as follows:

[0201] ,

[0202] In the formula, , , , They represent the lines respectively. The two ends of the time period Active power and reactive power flowing into the line; Indicates the line Total charging current;

[0203] The apparent power constraint at both ends of the line is expressed as:

[0204] ,

[0205] In the formula, Indicates the line of End of time period Apparent power; Indicates the line of End of time period Apparent power; Indicates the line The rated apparent power may differ at both ends when there are line losses, so capacity verification and corresponding constraints need to be set separately.

[0206] 2.3 Establish constraints on unit output, reserve capacity, unit ramp-up, wind / solar grid-connected power and curtailment, node active and reactive power balance, node voltage and line phase angle difference, node flexibility demand model, and line flexibility transmission margin constraints, among which:

[0207] The unit output constraints and reserve capacity constraints are expressed as follows:

[0208] ,

[0209] In the formula, This represents a collection of conventional generating units. , These represent conventional generating units. During the period The contribution of effort that yields results and the contribution that yields no results; and These represent conventional generating units. Lower and upper limits of effort and contribution; and These represent conventional generating units. The lower and upper limits of reactive power output; , These represent conventional generating units. During the period Upward reserve capacity, downward reserve capacity; , These represent conventional generating units. Uphill climbing ability per unit time, downhill climbing ability per unit time; Indicates the duration of a time period within the rolling time domain; This represents the set of time periods in the rolling time domain, excluding the last time period. , Representing the rolling time respectively corresponding time period The upward and downward reserve requirements are determined by equation (15); the reserve capacity is used to cover the additional power imbalance caused by the prediction error correction, and to avoid repeatedly including the same net load change in the planned output and reserve requirements.

[0210] The unit ramping constraint is expressed as follows:

[0211] (11),

[0212] This constraint is used to limit the changes in active power output of the unit between adjacent time periods;

[0213] The grid-connected power of wind power and photovoltaic power and the constraints on wind curtailment and solar curtailment are expressed as follows:

[0214] ,

[0215] In the formula, Indicates wind farm During the period The actual grid-connected active power; Indicates wind farm During the period The wind curtailment power; Indicates photovoltaic power station During the period The actual grid-connected active power; Indicates photovoltaic power station During the period The power of abandoned light; Indicates wind farm During the period Unproductive efforts; Indicates photovoltaic power station During the period Unproductive efforts; Indicates wind farm The rated apparent capacity of the converter; Indicates photovoltaic power station The rated apparent capacity of the converter;

[0216] The active and reactive power balance constraint of the node is expressed as follows:

[0217] ,

[0218] In the formula, For a set of nodes, , and Representing nodes respectively A collection of conventional generating units, wind farms, and photovoltaic power stations; Represents nodes A set of connected lines; , Representing nodes respectively During the period Inner Scrolling Time The active and reactive loads; combined with the aforementioned active and reactive power flow constraints at both ends of the line, the active and reactive power at the nodes. , The injection power is taken from the corresponding start or end of the line where the node is located.

[0219] The node voltage and line phase angle difference constraints are expressed as follows:

[0220] (14)

[0221] In the formula, and Representing nodes respectively voltage amplitude The lower and upper limits; Indicates reference node The voltage phase angle; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively.

[0222] The node net load and the node flexibility requirement model are expressed as follows:

[0223] ,

[0224] In the formula, Represents a node At the rolling moment corresponding time period The projected net load; Represents a node At the rolling moment corresponding time period The need for flexibility; Indicates wind farm At the rolling moment corresponding time period Available active power; Indicates photovoltaic power station At the rolling moment corresponding time period Available active power; and Representing nodes respectively During the period The load forecasting error correction amount and the new energy forecasting error correction amount; Indicates the rolling time corresponding time period The system's net prediction error.

[0225] The aforementioned load forecast error correction and new energy forecast error correction are determined based on the corresponding historical forecast residuals. The historical forecast residuals are statistically analyzed according to the forecast lead time. The positive and negative quantiles are used to form the forecast error correction in the corresponding direction, so that different future periods adopt a correction range that matches their forecast accuracy.

[0226] In some possible implementations, historical forecast residuals are statistically analyzed at 15 min, 30 min, 45 min, and other forecast lead times. In an example with a 95% confidence level, the 2.5% and 97.5% quantiles of the residuals for each group can be used to form downward and upward correction boundaries, respectively. For wind and solar power available power, the positive and negative residual corrections should also satisfy the restrictions that the available power is non-negative and does not exceed the corresponding installed capacity.

[0227] In the aforementioned reserve capacity constraints, the upward and downward reserve requirements are determined based on the load and new energy prediction error correction amounts. Equation (15) summarizes the prediction error correction amounts of each node into the system net prediction error, and takes its positive and negative parts to form the upward and downward reserve requirements respectively: When the value is positive, it corresponds to the upward flexibility requirement; when the value is negative, it corresponds to the downward flexibility requirement. The spatial distribution of node error is still retained in the flexibility requirements of each node and is used for the line allocation calculation in the following formula (17); the system summary value is only used to constrain the total amount of reserve capacity.

[0228] In addition, Equation (15) quantifies the demand for regulation capacity caused by source load fluctuations at the node level, providing input for subsequent analysis of the power changes borne by each line when the demand is transmitted through the transmission network.

[0229] The line itself does not generate or consume flexibility, but it provides a transmission channel for balancing the supply and demand of flexibility between nodes. Line flexibility transmission margin refers to the active power transmission space that can still be utilized along the corresponding transmission direction after the line has been allocated to meet the flexibility needs of the nodes. This indicator depends simultaneously on the node flexibility requirements, network allocation relationships, the current active power flow of the line, and its transmission power limit.

[0230] The line flexibility transmission margin constraint is expressed as follows:

[0231] ,

[0232] in, Indicates the time of the roll corresponding time period line The required line allocation for node flexibility is expressed as:

[0233] ,

[0234] Equation (17) allocates the flexibility requirements of each node to the line according to their impact on the active power flow of the line, resulting in... Preserve direction information. In the formula, Indicates the line At the rolling moment corresponding time period Line flexibility and transmission margin; Indicates the line According to the first node Point to the end node The positive reference direction in the time period The active power is taken from the active power at the beginning of the line in equation (8). , >0 indicates that active power is transmitted along the reference positive direction. <0 indicates that active power is transmitted in the opposite direction; Indicates the line The upper limit of active transmission power; For all routes, Indicates the rolling time The critical path set is selected by choosing paths whose load rate exceeds a preset threshold during the current or predicted future period. When evaluating all paths, the critical path set is selected from these paths. ; This represents the minimum value of the critical path flexibility propagation margin within the rolling time domain. Indicates the time period Inner Scrolling Time Under the network structure and line parameters, nodes Changes in unit active power injection on the line The power transmission distribution factor of active power flow. When and When they are in the same direction, the line allocation uses the remaining transmission capacity in the existing power flow direction; when they are in opposite directions, the allocation first offsets the existing power flow and releases the reverse transmission space. Take the minimum path flexibility transmission margin of the critical path set in the rolling time domain.

[0235] Line flexibility transmission margin allows for negative values. This indicates that the transmission capacity of the line in the corresponding direction is insufficient to meet the flexibility requirements of the node. This negative value directly reflects the degree of insufficiency and serves as the target to be improved in the second layer of optimization during the subsequent hierarchical rolling optimization process. No additional variable is set to replace this deficiency.

[0236] III. Construction of the hierarchical rolling optimization model:

[0237] The hierarchical rolling optimization model used in this embodiment is a three-layer rolling optimization model. The source-grid optimization variables corresponding to each layer are the same, and the same mixed integer feasible region containing complete AC power flow constraints is used. The model is solved sequentially according to the optimization objectives to jointly optimize the active and reactive power output of the units, the upward and downward reserve capacity, the active and reactive power of wind and solar grid connection, the curtailed wind and solar power, and the phase shifter settings in the rolling time domain. The first layer minimizes the curtailed wind and solar power in the rolling time domain; the second layer maximizes the minimum line flexibility transmission margin of the critical lines in the rolling time domain within the optimal value tolerance of the first layer; and the third layer minimizes the unit operation, reserve, and phase shifter settings costs within the optimal value tolerance of the first two layers.

[0238] The first-level optimization objective of the three-layer rolling optimization model is to minimize the amount of wind and solar power curtailment, and the objective function is expressed as follows:

[0239] ,

[0240] In the formula, Indicates the rolling time The optimization variables include the active and reactive power output of conventional units, the upward and downward reserve capacity, the active and reactive power of wind power / solar power connected to the grid and the curtailed wind / solar power, the amplitude and phase angle of the phase shifter and the switching status. Represents the mixed-integer feasible region defined by the aforementioned AC power flow operation model; This represents the minimum amount of wind and solar power curtailed after the first layer of optimization. Represents the rolling time within the rolling time domain The amount of wind and solar power curtailed is expressed as:

[0241] ,

[0242] In the formula, Indicates from the rolling moment The initial rolling time domain, Indicates the duration of a time period within the rolling time domain; Indicates a collection of wind farms. Indicates a collection of photovoltaic power plants. Indicates wind farm During the period wind curtailment power, Indicates photovoltaic power station During the period The abandoned power; in the first-level optimization solution, node voltage, phase angle, active and reactive power and apparent power of the line and the transmission margin of line flexibility are operating states or auxiliary variables determined by the model constraints.

[0243] The second-level optimization objective of the three-layer rolling optimization model is to maximize the line flexibility transmission margin in the rolling time domain within the tolerance range of the minimum wind and solar power curtailment obtained from the first-level optimization. Its objective function is expressed as:

[0244] ,

[0245] In the formula, This represents the maximum line flexibility transmission margin after the second layer optimization; This indicates the allowable tolerance for abandoned power; the second layer optimizes the transmission margin of the line flexibility within the tolerance range of the first layer's optimal value, avoiding the infeasibility of the next layer caused by non-convex AC model and discrete variable solution errors;

[0246] The objective of the third layer of the three-layer rolling optimization model is to minimize operation and scheduling costs within the tolerance range of the minimum wind and solar power curtailment obtained from the first layer optimization and the margin tolerance range of the maximum line flexibility transmission margin obtained from the second layer optimization. Its objective function is expressed as:

[0247] (twenty one),

[0248] In the formula, This represents the minimum operating and adjustment cost after the third layer of optimization; This indicates the allowable line flexibility transmission margin tolerance. For the rolling moment The operating and file adjustment costs are expressed as:

[0249] (twenty two),

[0250] in:

[0251] ,

[0252] ,

[0253] ,

[0254] In the formula, Indicates the rolling time The unit operating cost; Indicates the duration of a time period within the rolling time domain; Indicates conventional units During the period Those who have made meritorious contributions; , and They represent the generating units. The coefficients of the quadratic term, the linear term, and the constant term of the quadratic function of operating cost are determined based on the actual operating cost characteristics of the unit and can be obtained by fitting cost characteristic parameters or historical operating data provided by the unit manufacturer. Indicates the rolling time The cost of spare capacity; and These represent the upward reserve unit cost and the downward reserve unit cost, respectively. , These represent conventional generating units. During the period Upward reserve capacity, downward reserve capacity; Indicates the rolling time Phase shifter adjustment cost; This represents a set of discrete amplitude and phase controllable phase shifters. and These represent phase shifters. Unit adjustment cost for amplitude and phase angle settings; , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period The phase angle settings of the phase shifter. The definition of shifting cost in the above formula is used to reduce unnecessary repeated switching in the current action range and in the future rolling time domain.

[0255] During the online operation phase of the method, after the current action is generated by the strategy network, only the phase shifter position of the current first time period is fixed. The active and reactive power output of the unit, reserve capacity, grid connection of new energy and curtailment power of the current and future time periods, as well as the phase shifter position of the future time period, are still optimization variables. Therefore, fixing the strategy action is not the same as fixing the complete source-grid scheme.

[0256] IV. Policy Network and Offline Security Reinforcement Learning Training:

[0257] In this embodiment, the policy network includes a shared feature extraction layer, amplitude action branches and phase angle action branches corresponding to each phase shifter, and a state value branch. The shared feature extraction layer is used to receive the input state vector and extract state features. The input state vector includes the normalized power grid prediction state, current operating state, node flexibility requirements, line allocation, and phase shifter cross-time domain state, etc. Each action branch is used to output the corresponding gear increment probability based on the state features under a preset action mask constraint, and the state value branch is used to output the current state value.

[0258] In some possible embodiments, the shared feature extraction layer can be set as a fully connected layer with 256, 128, and 64 neurons respectively, with each layer using the ReLU activation function; each phase shifter has an amplitude action branch and a phase angle action branch, and the output dimension of each branch is equal to the number of elements in the corresponding gear increment set; the state value branch outputs a scalar. The above network size is an example and can be adjusted while keeping the input, output, and physical constraint mapping relationships unchanged.

[0259] For the input state vector of the shared feature extraction layer, the active power, reactive power and line capacity can be normalized according to the system baseline capacity first, and then the continuous variables in the policy state vector can be standardized according to the mean and standard deviation of the training samples.

[0260] The strategy network is obtained through offline training, which uses historical continuous operating data and available rolling predictions to construct the training environment. Historical data provides exogenous quantities such as load, wind power, and photovoltaic power for multiple subsequent time periods; node flexibility requirements and their line allocation are calculated based on equations (15) and (17).

[0261] The inputs, outputs, and training feedback of the policy network all correspond to physical quantities of the power grid: the node voltages, line apparent power, unit outputs, and phase shifter states in the inputs come from the current operating state or AC power flow calculations; the outputs are the increments of the phase shifter amplitude and phase angle settings in the first period of the rolling time domain, rather than directly generating node voltages, line power flow, or unit continuous control quantities; the training feedback is generated from the hierarchical results of the same AC physical model under both fixed and non-fixed candidate action conditions.

[0262] For each training state, the current first time period adjustment action is not fixed. The three-layer rolling optimization model of equations (18), (20) and (21) is solved in sequence to obtain the wind and solar power curtailment, line flexibility transmission margin and operation and adjustment cost benchmarks in this state. Then, the policy network generates candidate adjustment actions for the first time period within the range allowed by the action mask. After fixing the candidate adjustment action, the three-layer model is solved again in sequence. The training reward is calculated based on the feasibility of the model solution and the difference between the optimization target result obtained when the model is feasible and the layered training benchmark. The unit output, phase shifter position, remaining switching times and the number of time periods held obtained in the first time period obtained by this rolling optimization are inherited as internal states to the next training time. Then, the AC power flow is recalculated under the new exogenous quantities.

[0263] Specifically, the training methods for policy networks include:

[0264] Based on basic power grid parameter data, historical power grid operation status data, and concurrent source-load forecast data, multiple sets of rolling time-domain data samples are constructed.

[0265] Based on the rolling time-domain data sample, at each rolling moment, the hierarchical rolling optimization model is solved, and the optimal solution and its corresponding optimization target value are used as a reference benchmark.

[0266] Based on the rolling time domain data samples, a strategy network input state vector is constructed corresponding to each rolling moment. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the current rolling time domain based on the input state vector under the constraints of a preset action mask, and obtains the corresponding state value through the state value branch.

[0267] The hierarchical rolling optimization model is solved based on the candidate phase shifter adjustment actions of the current first time period. The immediate reward is calculated based on the feasibility of the model solution and the comparison results between the optimization target value obtained when the model is feasible and the reference benchmark. The advantage estimate for policy update is obtained based on the immediate reward and state value corresponding to multiple sets of rolling time domain data samples.

[0268] Based on the aforementioned advantage estimation, the policy network is updated with feedback until the set iteration termination condition is met.

[0269] The input state vector of the policy network is represented as:

[0270] ,

[0271] In the formula, Indicates the rolling time The input state vector; Indicates the rolling time The corresponding rolling time domain consists of a predicted state sub-vector composed of load, wind power, and photovoltaic forecast data. Representing the rolling time respectively The corresponding scrolling time in the scrolling time domain To the rolling moment The active load vector, reactive load vector, available active power vector of wind farm, and available active power vector of photovoltaic power station are used, where the measured values ​​are used for the current first time period and the predicted values ​​are used for future time periods. , Representing the rolling time respectively The corresponding node flexibility requirement vector and node flexibility requirement line allocation vector, both of which include the rolling time. The corresponding rolling time domain yields node flexibility demand data and node flexibility demand line allocation data, obtained from the difference in net load between adjacent time periods and the prediction error correction. Node flexibility demand is calculated based on changes in net load between adjacent time periods and the prediction error correction. The node flexibility demand line allocation is obtained by mapping the node flexibility demand to each line using the power transmission distribution factor, which can be referenced from the corresponding formulas in the AC power flow operation model. The power transmission distribution factor is calculated based on the network topology, line parameters, and reference nodes at the current rolling time. When the network topology changes due to maintenance or faults, it is recalculated before entering the next rolling solution. Indicates the rolling time Current measured node voltage Phase angle Active power output of the unit and line apparent power The resulting sub-vectors of operating states; , These represent the amplitude range vector and phase angle range vector of each phase shifter at the previous rolling moment, respectively. Indicates the rolling time Vector of remaining allowable switching times for each phase shifter Indicates the rolling time Each phase shifter has continuously maintained the current gear's time period vector, and the three of them constitute the current phase shifter state sub-vector.

[0272] The magnitude action branch and phase action branch of the policy network are based on the rolling time. The input state vector is the decision output of the current first time period action vector. Represented as:

[0273] ,

[0274] In the formula, These represent the phase shifters for the current first time period. The amplitude increment and phase angle increment; Indicates the time of the roll The set of joint selectable actions is filtered according to a preset action mask constraint, wherein the preset action mask constraint defines gear boundaries, gear shifting steps, remaining switching times, and cross-time domain hold-up status.

[0275] The policy probability after filtering by action mask constraints is expressed as:

[0276] , In the formula, Indicates the policy network parameters; Indicates the action branch of any value. Or phase angle action branch ; The policy network represents the input state vector. After making decisions under the constraints of the action mask, the action vector is output. The joint strategy probability is obtained by multiplying the magnitude branch and phase angle branch probabilities of each phase shifter. and These are the corresponding amplitude action branch output and phase action branch output, respectively; Indicates phase shifter Corresponding amplitude action branch or phase action branch Based on the input state vector Output amplitude or phase motion under motion mask constraints. The probability of the strategy; Indicates phase shifter Corresponding amplitude action branch or phase action branch The original strategy probability; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The corresponding set of branch actions, The branch action variable represents the summation of the set of branch actions; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The action mask obtained by performing action mask constraint judgment on action* is obtained by judging each candidate action of the corresponding branch according to action mask constraints such as gear boundary, maximum shift steps, remaining switching times and minimum holding time. If the action meets the constraints, the action mask is 1, otherwise it is 0. This is used to limit each action branch to retain at least one non-shifting action, so that the normalized denominator is always greater than 0.

[0277] The instant rewards at each rolling moment are represented as follows:

[0278] (26)

[0279] In the formula, Indicates the rolling time Instant rewards; Indicates the time of the roll Fixed candidate phase shifter gear adjustment action Then, the feasible region is jointly defined by the constraints of the AC power flow operation model and the hierarchical rolling optimization model. Indicates fixed candidate actions The post-layered rolling optimization model is not feasible; , and These represent the optimization target values ​​obtained from solving the hierarchical rolling optimization model for candidate phase shifter adjustment actions based on policy network decision-making, and are relative to the normalized power abandonment difference, line flexibility transmission margin difference, and cost difference of the reference benchmark, all of which are limited to between 0 and 1. , , They represent action-based The rolling time obtained by solving the hierarchical rolling optimization model The amount of wind and solar power curtailed, the transmission margin of line flexibility, and the operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between wind and solar power curtailment. This represents the penalty constant when a candidate action causes a degradation in the amount of wind and solar power curtailed in the first layer. This represents the penalty constant when the transmission flexibility margin of the second-level optimization objective deteriorates, assuming no over-tolerance degradation occurs in the first-level optimization objective. Let represent the penalty constant when a candidate action makes the hierarchical rolling optimization model infeasible, and satisfy . , and .

[0280] Before evaluating the execution formula (26), this invention first determines whether the hierarchical optimization model corresponding to the fixed candidate action is feasible, i.e., whether there is an optimal solution. Then, it evaluates the abandoned power, line flexibility transmission margin and operating cost in turn. If any layer optimization objective is not feasible, the solution of the hierarchical optimization model is terminated, so that the improvement of the next layer cannot compensate for the degradation of the previous layer, and at the same time improves the efficiency of policy network training.

[0281] The advantage estimate for policy updating, obtained based on the immediate reward and state value corresponding to multiple sets of rolling time-domain data samples, is expressed by the following formula:

[0282] ,

[0283] In the formula, Indicates the rolling time The corresponding advantage estimate is used for policy updates; Indicates the corresponding rolling time The temporal difference residuals of the training rounds; This represents the advantage estimation length, a preset parameter for the training phase, whose value does not exceed the number of state transition steps available for advantage estimation in the corresponding continuous training trajectory. Indicates the discount factor; Indicates the advantage estimate attenuation coefficient; The state value branch parameter is... State value function;

[0284] The policy network is updated using a near-end policy optimization algorithm, wherein the near-end policy optimization objective is expressed as:

[0285] ,

[0286] In the formula, This indicates the policy network parameters that need to be updated. The policy network parameters are The corresponding objective function for optimizing the near-end strategy; This indicates taking the expected value; Represents the joint vector actions of the new and old policies. The probability ratio; This represents the network parameters before the update. The old policy is the policy used when generating the current batch of training data and remains fixed during this update round; the new policy is the policy that is currently updating the parameters using the current batch of training data. This is a truncation function. This represents the probability ratio cutoff coefficient. Represents the joint masking policy entropy. This represents the policy entropy coefficient.

[0287] In some possible embodiments, the learning rate of the proximal policy optimization algorithm is set to 3×10. -4 The discount factor is set to 0.99, the advantage estimation decay factor is set to 0.95, the probability ratio cutoff factor is set to 0.2, and the entropy factor is set to 0.01. Each update extracts a small batch of samples from the continuous running trajectory and performs multiple rounds of gradient updates. During training, non-gear shifting actions are retained, and actions that violate gear boundaries, maximum shifting steps, remaining shift counts, or shortest hold time are set to zero before probability normalization.

[0288] The training state transition adopts a first-period execution approach: for the current training state, if the hierarchical rolling optimization model after fixing candidate actions is feasible, then its first-period optimization result is used for state transition; if not, the first-period optimization result obtained by resolving the hierarchical rolling optimization model after removing the fixed candidate actions is used for state transition. The load, wind power, and available photovoltaic power at the next rolling moment are given by the corresponding historical measured sequence or simulated exogenous sequence, and then the AC power flow, node flexibility requirements, and line allocation are recalculated. This process ensures that the offline training process and the online rolling execution process maintain the same state update order.

[0289] The three-layer rolling optimization model can use a branch-and-bound framework to handle the discrete gear variables of the phase shifter, and then use the primal-dual interior-point method to solve the AC optimal power flow problem after fixing the integer variables; alternatively, it can be solved directly using a solver capable of handling nonlinear mixed integer models. The three-layer model is solved independently in the order of equations (18), (20), and (21). The optimal value of the previous layer is written into the constraint of the next layer within the preset numerical tolerance, and the weighted summation of objectives with different dimensions is not used.

[0290] V. Online scrolling optimization:

[0291] During online rolling optimization, this embodiment first acquires measured state and future prediction data for constructing the rolling time domain, and updates the rolling time domain and its corresponding system data at each rolling moment. Then, based on the acquired data, it uses the AC power flow operation model and its corresponding constraints to calculate the node flexibility requirements and corresponding line flexibility transmission margins for each time period, including:

[0292] For each time period, the node flexibility requirement is calculated based on the net load change with the adjacent future time period and the prediction error of load and new energy.

[0293] The power transmission distribution factor is calculated based on the basic parameters of the power grid, and the node flexibility requirements are mapped to the line allocation of each line based on the power transmission distribution factor.

[0294] The line flexibility transmission margin is determined based on the line allocation amount, line active power, and line active transmission power upper limit required for node flexibility.

[0295] Based on the current rolling time-domain system data, node flexibility requirements, and node flexibility requirement line allocation, an input state vector is constructed, comprising the current running state sub-vector, the predicted state sub-vector, the node flexibility requirement vector, the node flexibility requirement line allocation vector, and the current phase shifter state sub-vector. This vector serves as the input to the trained policy network. The parameters of the policy network are not updated during online runtime.

[0296] In this embodiment, the secure reinforcement learning strategy network is only used during the online operation phase to generate discrete phase shifter tuning actions for the current first time period based on the current actual state, future predictions, node flexibility requirements, and line allocation, without directly outputting a complete source-grid control scheme. The active and reactive power output of conventional units, upward and downward reserve capacity, active and reactive power of wind and solar grid connection, curtailed wind and solar power, node voltage, and future phase shifter tuning positions are all determined by a hybrid integer hierarchical rolling optimization model containing complete AC power flow constraints. The reinforcement learning involved in this embodiment is called secure reinforcement learning because actions violating tuning boundaries, maximum tuning steps, remaining switching times, and minimum hold time are excluded through action mask constraints before action generation. After action generation, feasibility correction is performed using a physical optimization model, and reserve coverage, AC power flow, and device constraints are uniformly reviewed before the command is issued.

[0297] The trained policy network generates candidate shift actions for the current rolling moment. Then, the candidate shift action is fixed and the mixed integer three-layer rolling optimization model with complete AC power flow constraints is solved sequentially.

[0298] The current online file adjustment action is represented as follows:

[0299] (29)

[0300] In the formula, Joint filing actions for online strategy output.

[0301] At this point, based on the candidate phase shifter adjustment actions of the current first time period output by the strategy network, the hierarchical rolling optimization model is used to solve for the output of conventional units, reserve capacity, grid-connected power of new energy sources, node voltage, line power flow, and phase shifter positions for future time periods. The feasibility of the mixed-integer hierarchical rolling optimization model with complete AC power flow constraints is then determined.

[0302] If a solution represents a feasible model, then the final action of the phase shifter in the current first time period is determined based on the candidate phase shifter adjustment actions. Refer to the following formula (30):

[0303] ,

[0304] In the formula, and These represent the increments of the phase shifter amplitude and phase angle corresponding to the final executed action, respectively. These are the amplitude and phase angle settings of the phase shifter at the previous rolling moment, respectively.

[0305] If there is no solution, it means that the model is not feasible. Then, discard the candidate phase shifter adjustment action and solve the hierarchical rolling optimization model again to obtain the phase shifter adjustment action in each time period in the rolling time domain of the optimal solution, and then calculate the final execution action of the phase shifter in the current first time period.

[0306] In this embodiment, the feasibility correction relationship of the strategy action is expressed as the following formula (31):

[0307] (31),

[0308] In the formula, Indicates the final action to be performed; This indicates the candidate phase shifter adjustment action generated by the policy network; When a fixed strategy action is not feasible, the final action to be executed is an optimized correction action obtained by a hierarchical rolling optimization model based on an AC power flow operation model. If no operating scheme that satisfies the constraints is obtained after removing the candidate phase shifter gear adjustment action, the current gear of the phase shifter is maintained, no new gear adjustment command is issued, and the dispatching agency handles the situation according to existing safety control measures.

[0309] refer to Figure 3 In this embodiment, after obtaining the final execution action, the AC power flow is first verified based on the optimal solution of the hierarchical rolling optimization model corresponding to the final execution action of the phase shifter. If the AC power flow verification is successful, the corresponding control command is issued.

[0310] In some possible embodiments, the AC power flow verification based on the optimal solution of the hierarchical rolling optimization model corresponding to the final execution action of the phase shifter is for the current rolling moment. The feasibility of the optimization results across the entire rolling time domain is verified using source network variables, including node power balance, line capacity, node voltage, line phase angle difference, and phase shifter operating constraints for each time period within the rolling time domain. Once the verification is successful, only the control commands for the current first time period are executed.

[0311] When performing a power flow review, it is necessary to determine whether the following power flow review conditions are met:

[0312] ,

[0313] In the formula, and Representing nodes respectively During the period The active power balance residual and the reactive power balance residual; , These represent the maximum allowable active power balance residual and the maximum reactive power balance residual, respectively. They represent the lines respectively. of End and End of time period Apparent power Indicates the line Rated apparent power; and Representing nodes respectively During the period voltage amplitude The lower and upper limits; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively; These represent phase shifters. During the period Amplitude and phase settings; and These represent the amplitude range set and the phase angle range set, respectively. Indicates phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain, Indicates the number of time periods included in the rolling time domain; Indicates the rolling time The remaining number of allowed switching attempts; Indicates phase shifter The shortest duration of holding time.

[0314] In some possible embodiments, the phase shifter position is only part of the current first-period source-grid control command; at the same time, commands for the active and reactive power output of conventional units, the upward and downward reserve capacity, and the active and reactive power grid connection of wind power and photovoltaic power are also issued simultaneously. Node voltage and line power flow are the operating states formed after executing the above control commands and are not issued as independent commands.

[0315] The execution of the first time period instruction and the rolling time domain update satisfy equation (33):

[0316] ,

[0317] In the formula, Indicates the obtained rolling time The final control scheme includes the control commands for the generating units, new energy sources, backup units, and phase shifters for the current first time period. This indicates that an actual instruction was issued; This indicates the updated rolling time domain.

[0318] In this embodiment, the method was compared with the operation modes of complete rolling optimization without strategy network, direct execution of strategy actions without physical correction, and operation without phase shifter in terms of wind and solar power curtailment, minimum line flexibility transmission margin in the rolling time domain, operation and rescheduling costs, number of rescheduling times, feasibility rate of fixed strategy actions, optimization correction trigger rate, and single online solution time. The results fully demonstrate that the method of the present invention has significant advantages in the evaluation of the above-mentioned indicators.

[0319] Example 2

[0320] Based on the same inventive concept as Embodiment 1, this embodiment introduces a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the discrete amplitude and phase controllable phase shifter control method based on secure reinforcement learning and hierarchical rolling optimization as described in Embodiment 1.

[0321] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0322] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0323] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0324] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0325] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A control method for a discrete amplitude- and phase-controllable phase shifter based on secure reinforcement learning and hierarchical rolling optimization, characterized in that, include, Within the scrolling optimization cycle corresponding to each scrolling moment, perform the following operations: Acquire basic power grid parameter data, power grid operation status data, and source-load prediction data; Based on the acquired data, the node flexibility requirements and corresponding line flexibility transmission margins for each time period in the rolling time domain are calculated using a pre-built AC power flow operation model. Based on the power grid operation status data, source load prediction data, node flexibility requirements and node flexibility requirement line allocation, a strategy network input state vector is constructed, which includes the current operation status sub-vector, prediction status sub-vector, node flexibility requirement vector, node flexibility requirement line allocation vector, and current phase shifter status sub-vector. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the rolling time domain based on the input state vector under the constraints of a preset action mask. Based on the candidate phase shifter adjustment actions in the current first time period, the source-grid variables and future time period phase shifter adjustment actions are solved using a pre-constructed hierarchical rolling optimization model. The source-grid variables include conventional unit output, reserve capacity, renewable energy grid-connected power, node voltage, line power flow, and future time period phase shifter settings. If a solution exists, the final action of the phase shifter in the current first time period is determined based on the candidate phase shifter adjustment action. If there is no solution, discard the candidate phase shifter adjustment action and solve the hierarchical rolling optimization model again to obtain the phase shifter adjustment action in each time period in the rolling time domain in the optimal solution, and then calculate the final execution action of the phase shifter in the current first time period. The AC power flow is verified based on the optimal solution of the hierarchical rolling optimization model corresponding to the final action of the phase shifter. If the AC power flow verification passes, the corresponding control command is issued.

2. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The basic power grid parameter data includes at least the power grid topology, line parameters, and conventional unit parameter data; the power grid operation status data includes at least the operation parameters of power grid nodes, branches, conventional units, wind farms, photovoltaic power stations, and amplitude and phase controllable phase shifters; the source and load forecast data includes at least wind power forecast, photovoltaic forecast, and load forecast data for multiple consecutive future periods. The rolling time domain corresponding to each rolling moment includes multiple consecutive time periods, represented as follows: ， in, Indicates from the rolling moment The initial rolling time domain; Indicates the number of time periods included in the rolling time domain; This represents the set of future time periods excluding the current first time period; Within any rolling optimization cycle, the data to be acquired uses the current measured values ​​for the current first time period and the predicted values ​​for the corresponding time period for future time periods.

3. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The calculation of node flexibility requirements and corresponding line flexibility transmission margins for each time period includes: For each time period, the node flexibility requirement is calculated based on the net load change with the adjacent future time period and the prediction error of load and new energy. The power transmission distribution factor is calculated based on the basic parameters of the power grid, and the node flexibility requirements are mapped to the line allocation of each line based on the power transmission distribution factor. The line flexibility transmission margin is determined based on the line allocation amount, line active power, and line active transmission power upper limit required for node flexibility.

4. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The hierarchical rolling optimization model is a three-layer rolling optimization model, where the source network optimization variables are the same for each layer, and the optimization is solved sequentially according to the optimization objective; wherein: The first-level optimization objective of the three-layer rolling optimization model is to minimize the amount of wind and solar power curtailment, and the objective function is expressed as follows: ， In the formula, Indicates the rolling time The optimization variables include the active and reactive power output of conventional units, the upward and downward reserve capacity, the active and reactive power of wind power / solar power connected to the grid and the curtailed wind / solar power, the amplitude and phase angle of the phase shifter and the switching status. Represents the mixed-integer feasible region defined by the aforementioned AC power flow operation model; This represents the minimum amount of wind and solar power curtailed after the first layer of optimization. Represents the rolling time within the rolling time domain The amount of wind and solar power curtailed is expressed as: ， In the formula, Indicates from the rolling moment The initial rolling time domain, Indicates the duration of a time period within the rolling time domain; Indicates a collection of wind farms. Indicates a collection of photovoltaic power plants. Indicates wind farm During the period wind curtailment power, Indicates photovoltaic power station During the period The power of discarded light; The second-level optimization objective of the three-layer rolling optimization model is to maximize the line flexibility transmission margin in the rolling time domain within the tolerance range of the minimum wind and solar power curtailment obtained from the first-level optimization. Its objective function is expressed as: ， In the formula, This represents the maximum line flexibility transmission margin after the second layer optimization. This represents the minimum value of the critical path flexibility propagation margin within the rolling time domain. This indicates the allowable tolerance for the amount of electricity that can be wasted. The objective of the third layer of the three-layer rolling optimization model is to minimize operation and scheduling costs within the tolerance range of the minimum curtailment of wind and solar power obtained from the first layer of optimization and the margin tolerance range of the maximum line flexibility transmission margin obtained from the second layer of optimization. Its objective function is expressed as: ， In the formula, This represents the minimum operating and adjustment cost after the third layer of optimization; This indicates the allowable line flexibility transmission margin tolerance. For the rolling moment The operating and file adjustment costs are expressed as: ， in: ， ， ， In the formula, Indicates the rolling time The unit operating cost; Indicates conventional units During the period Those who have made meritorious contributions; , and These represent conventional units. The coefficients of the quadratic term, the linear term, and the constant term of the quadratic function of operating cost; Indicates the rolling time The cost of spare capacity; and These represent the upward reserve unit cost and the downward reserve unit cost, respectively. , These represent conventional units. During the period Upward reserve capacity, downward reserve capacity; Indicates the rolling time The cost of phase shifter adjustment; This represents a set of discrete amplitude and phase controllable phase shifters. and These represent phase shifters. Unit adjustment cost for amplitude and phase angle settings; , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period The phase angle settings of the phase shifter.

5. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The construction of the AC power flow operation model includes: constructing a discrete amplitude and phase controllable phase shifter model, which uses discrete variables to define the phase shifter gear combination relationship, the amplitude ratio, phase shift angle, and equivalent admittance corresponding to each gear, and establishing constraints on gear changes and switching times, as well as constraints on the shortest hold time; wherein: The phase shifter range combination relationship is expressed as follows: ， In the formula, This represents a set of discrete amplitude and phase controllable phase shifters; Indicates the phase shifter number; and These represent the amplitude range set and the phase angle range set, respectively. and These represent the corresponding gear number; Indicates time period Do you want to select a gear combination? ; and These represent the amplitude range number and the phase angle range number, respectively. The amplitude ratio, phase shift angle, and equivalent admittance corresponding to the gear position are expressed as follows: ， In the formula, and These represent phase shifters. During the period The amplitude ratio and phase shift angle; and These represent phase shifters. Corresponding amplitude range Amplitude ratio and corresponding phase angle settings The phase shift angle; and These represent phase shifters. During the period The equivalent conductance and susceptance; and These represent gear combinations. The conductance and susceptance include the equivalent impedance of the line and phase shifter; The constraints on gear changes and switching frequency are expressed as follows: ， In the formula, , These represent phase shifters. In adjacent time periods Compared with the previous period Phase shifter amplitude settings; and These represent phase shifters. In adjacent time periods Compared with the previous period Phase angle settings of the phase shifter; and These represent phase shifters. The maximum number of adjustment steps for adjacent time periods of amplitude and phase; For phase shifter The upper bound constant of the gear change; For phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain; Indicates the rolling time The remaining number of allowed switching attempts; The minimum hold time constraint is expressed as follows: ， In the formula, Indicates phase shifter The shortest duration of holding; Indicates phase shifter At the rolling moment The number of consecutive periods that the current gear has been maintained; This indicates the time period number for summation.

6. The discrete amplitude and phase controllable phase shifter control method according to claim 5, characterized in that, The construction of the AC power flow operation model also includes: The phase shifter amplitude ratio, phase shift angle, and equivalent admittance are mapped into the corresponding AC branch through the installation line and expressed as follows: ， , Indicates phase shifter Installation wiring; This represents the set of lines on which phase shifters are installed; They represent the lines respectively. During the period Amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance; These represent the installation of phase shifters. The route during the time period The amplitude ratio, phase shift angle, equivalent conductance, and equivalent susceptance of the line without a phase shifter during the time period. The amplitude ratio, phase shift angle, conductance, and susceptance adopt fixed line parameters. ; Substituting the phase shifter complex ratio, line series admittance, and charging susceptance into the AC branch model, active, reactive, and apparent power constraints at both ends of the line are established, where: The product coefficient of line phase angle difference and voltage considering the complex ratio of phase shifter is expressed as: ， In the formula, , They represent the lines respectively. The first and last nodes; Indicates the time period The electrical phase angle difference of the line after phase shift is taken into account; , The lines are respectively Both ends in time period Electrical phase angle; Indicates the line During the period The voltage product coefficient; They represent the lines respectively. Both ends in time period The voltage amplitude; The active and reactive power flow constraints at both ends of the line are represented as follows: ， In the formula, , , , They represent the lines respectively. The two ends of the time period Active power and reactive power flowing into the line; Indicates the line Total charging current; The apparent power constraint at both ends of the line is expressed as: ， In the formula, Indicates the line of End of time period Apparent power; Indicates the line of End of time period Apparent power; Indicates the line Rated apparent power; The construction of the AC power flow operation model also includes: establishing unit output constraints, reserve capacity constraints, unit ramping constraints, wind / solar grid-connected power and wind / solar curtailment constraints, node active and reactive power balance constraints, node voltage and line phase angle difference constraints, node flexibility demand model, and line flexibility transmission margin constraints, among which: The unit output constraints and reserve capacity constraints are expressed as follows: ， In the formula, This represents a collection of conventional generating units. , These represent conventional units. During the period The contribution of effort that yields results and the contribution that yields no results; and These represent conventional units. Lower and upper limits of effort and contribution; and These represent conventional units. The lower and upper limits of reactive power output; , These represent conventional units. During the period Upward reserve capacity, downward reserve capacity; , These represent conventional units. Uphill climbing ability per unit time, downhill climbing ability per unit time; Indicates the duration of a time period within the rolling time domain; This represents the set of time periods in the rolling time domain, excluding the last time period. , Representing the rolling time respectively corresponding time period Upward and downward reserve requirements; The unit ramping constraint is expressed as follows: ， The grid-connected power of wind power and photovoltaic power and the constraints on wind curtailment and solar curtailment are expressed as follows: ， In the formula, Indicates wind farm During the period The actual grid-connected active power; Indicates wind farm During the period The wind curtailment power; Indicates photovoltaic power station During the period The actual grid-connected active power; Indicates photovoltaic power station During the period The power of discarded light; Indicates wind farm During the period Unproductive efforts; Indicates photovoltaic power station During the period Unproductive efforts; Indicates wind farm The rated apparent capacity of the converter; Indicates photovoltaic power station The rated apparent capacity of the converter; The active and reactive power balance constraint of the node is expressed as follows: ， In the formula, For a set of nodes, , and Representing nodes respectively A collection of conventional generating units, wind farms, and photovoltaic power stations; Represents nodes A set of connected lines; , Representing nodes respectively During the period Inner Scrolling Time Active load and reactive load; The node voltage and line phase angle difference constraints are expressed as follows: ， In the formula, and Representing nodes respectively voltage amplitude The lower and upper limits; Indicates reference node The voltage phase angle; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively; The node net load and the node flexibility requirement model are expressed as follows: ， In the formula, Represents a node At the rolling moment corresponding time period The projected net load; Represents a node At the rolling moment corresponding time period The need for flexibility; Indicates wind farm At the rolling moment corresponding time period Available active power; Indicates photovoltaic power station At the rolling moment corresponding time period Available active power; and Representing nodes respectively During the period The load forecasting error correction amount and the new energy forecasting error correction amount; Indicates the rolling time corresponding time period The system's net prediction error; The line flexibility transmission margin constraint is expressed as follows: ， in, Indicates the time of the roll corresponding time period line The required line allocation for node flexibility is expressed as: ， In the formula, Indicates the line At the rolling moment corresponding time period Line flexibility and transmission margin; Indicates the line According to the first node Point to the end node The positive reference direction in the time period The active power is taken from the active power at the beginning of the line; Indicates the line The upper limit of active transmission power; This represents the minimum value of the critical path flexibility propagation margin within the rolling time domain. Indicates the time of the roll corresponding time period Under the network structure and line parameters, nodes Changes in unit active power injection on the line Power transmission distribution factor of active power flow; Indicates the rolling time The set of critical paths, This is a collection of all routes.

7. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The policy network includes a shared feature extraction layer, amplitude action branches and phase angle action branches corresponding to each phase shifter, and a state value branch. The shared feature extraction layer is used to receive the input state vector and extract state features. Each action branch is used to output the corresponding gear increment probability based on the state features under a preset action mask constraint. The state value branch is used to output the current state value. The policy network is trained offline, and the training method includes: Based on basic power grid parameter data, historical power grid operation status data, and concurrent source-load forecast data, multiple sets of rolling time-domain data samples are constructed. Based on the rolling time-domain data sample, at each rolling time, the hierarchical rolling optimization model is solved, and the optimal solution and its corresponding optimization target value are used as a reference benchmark. Based on the rolling time domain data samples, a strategy network input state vector is constructed corresponding to each rolling moment. The strategy network generates candidate phase shifter adjustment actions for the current first time period in the current rolling time domain based on the input state vector under the constraints of a preset action mask, and obtains the corresponding state value through the state value branch. The hierarchical rolling optimization model is solved based on the candidate phase shifter adjustment actions of the current first time period. The immediate reward is calculated based on the feasibility of the model solution and the comparison between the optimization target value obtained when the model is feasible and the reference benchmark. The advantage estimate for policy update is obtained based on the immediate reward and state value corresponding to multiple sets of rolling time domain data samples. Based on the aforementioned advantage estimation, the policy network is updated with feedback until the set iteration termination condition is met.

8. The discrete amplitude and phase controllable phase shifter control method according to claim 7, characterized in that, The input state vector of the policy network is represented as: ， In the formula, Indicates the rolling time The input state vector; Indicates the rolling time The corresponding rolling time domain consists of a predicted state sub-vector composed of load, wind power, and photovoltaic forecast data. Representing the rolling time respectively The corresponding scrolling time in the scrolling time domain To the rolling moment The active load vector, reactive load vector, available active power vector of wind farm, and available active power vector of photovoltaic power station; , Representing the rolling time respectively The corresponding node flexibility requirement vector and node flexibility requirement line allocation vector; Indicates the rolling time Current measured node voltage Phase angle Active power output of the unit and line apparent power The resulting sub-vectors of operating states; , These represent the amplitude range vector and phase angle range vector of each phase shifter at the previous rolling moment, respectively. Indicates the rolling time Vector of remaining allowable switching times for each phase shifter Indicates the rolling time Each phase shifter has continuously maintained the current gear's time period vector, and the three of them constitute the current phase shifter state sub-vector; The magnitude action branch and phase action branch of the policy network are based on the rolling time. The input state vector is the decision output of the current first time period action vector. Represented as: ， In the formula, These represent the phase shifters for the current first time period. The amplitude increment and phase angle increment; Indicates the time of the roll The set of joint selectable actions is filtered according to a preset action mask constraint, wherein the preset action mask constraint defines gear boundaries, gear shifting steps, remaining switching times, and cross-time domain hold-up status. The policy probability after filtering by action mask constraints is expressed as: ， In the formula, Indicates the policy network parameters; Indicates the action branch of any image value Or phase angle action branch ; The policy representation network is based on the input state vector. After making decisions under the constraints of the action mask, the action vector is output. The probability of the joint strategy; and These are the corresponding amplitude action branch output and phase action branch output, respectively; Indicates phase shifter Corresponding amplitude action branch or phase action branch Based on the input state vector Output amplitude or phase motion under motion mask constraints. The probability of the strategy; Indicates phase shifter Corresponding amplitude action branch or phase action branch The original strategy probability; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The corresponding set of branch actions, Use it as the variable for the branch action; Indicates the time of the roll Phase shifter Amplitude action branch or phase action branch The action mask obtained by applying action mask constraints to the action*; The instant rewards at each rolling moment are represented as follows: ， In the formula, Indicates the rolling time Instant rewards; Indicates the time of the roll Fixed candidate phase shifter gear adjustment action Then, the feasible region is jointly defined by the constraints of the AC power flow operation model and the hierarchical rolling optimization model. Indicates fixed candidate actions The post-layered rolling optimization model is not feasible; , and These represent the optimization objective values ​​obtained from solving the hierarchical rolling optimization model for candidate phase shifter adjustment actions based on policy network decision-making, and are relative to the normalized power abandonment difference, line flexibility transmission margin difference, and cost difference of the reference benchmark, respectively. , , They represent action-based The rolling time obtained by solving the layered rolling model The amount of wind and solar power curtailed, the transmission margin of line flexibility, and the operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between operating and adjustment costs; This represents a positive reference value used for normalizing the scale of the difference between wind and solar power curtailment. This represents the penalty constant when a candidate action causes a degradation in the amount of wind and solar power curtailed in the first layer. This represents the penalty constant when the transmission flexibility margin of the second-level optimization objective deteriorates, assuming no over-tolerance degradation occurs in the first-level optimization objective. Let represent the penalty constant when a candidate action makes the hierarchical rolling optimization model infeasible, and satisfy . , and ; The advantage estimate for policy updating, obtained based on the immediate reward and state value corresponding to multiple sets of rolling time-domain data samples, is expressed by the following formula: ， In the formula, Indicates the rolling time The corresponding advantage estimate is used for policy updates; Indicates the corresponding rolling time The temporal difference residuals of the training rounds; Indicates the length of the advantage estimate; Indicates the discount factor; Indicates the advantage estimate attenuation coefficient; The state value branch parameter is State value function; The policy network is updated using a near-end policy optimization algorithm, wherein the near-end policy optimization objective is expressed as: ， In the formula, This indicates the policy network parameters that need to be updated. The policy network parameters are The corresponding objective function for optimizing the near-end policy, This indicates taking the expected value; Represents the joint vector actions of the new and old policies. The probability ratio; This indicates the policy network parameters before the update; This is a truncation function. This represents the probability ratio cutoff coefficient. Represents the joint masking policy entropy. This represents the policy entropy coefficient.

9. The discrete amplitude and phase controllable phase shifter control method according to claim 1, characterized in that, The AC power flow verification based on the optimal solution of the hierarchical rolling optimization model corresponding to the final execution action of the phase shifter includes determining whether the following AC power flow verification conditions are met: ， In the formula, and Representing nodes respectively During the period The active power balance residual and the reactive power balance residual; , These represent the maximum allowable active power balance residual and the maximum reactive power balance residual, respectively. They represent the lines respectively. of End and End of time period Apparent power Indicates the line Rated apparent power; and Representing nodes respectively During the period voltage amplitude The lower and upper limits; Indicates the time period The electrical phase angle difference of the line; and These represent the lower and upper limits of the electrical phase angle difference of the line, respectively; These represent phase shifters. During the period Amplitude and phase settings; and These represent the amplitude range set and the phase angle range set, respectively. Indicates phase shifter During the period The switching state; Indicates from the rolling moment The initial rolling time domain, Indicates the number of time periods included in the rolling time domain; Indicates the rolling time The remaining number of allowed switching attempts; Indicates phase shifter The shortest duration of holding time.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the discrete amplitude and phase controllable phase shifter control method based on security reinforcement learning and hierarchical rolling optimization as described in any one of claims 1 to 9.

Images