Dynamic optimization control system and method for power grid section limit based on reinforcement learning

Abstract

The present invention discloses a power grid section limit dynamic optimization control system and method based on reinforcement learning, which relates to the technical field of section limit dynamic control, and includes the following steps: identifying a controlled section set and constructing a section limit dynamic evaluation model; constructing a section response sensitivity matrix, and constructing a multi-objective optimization function in combination with the section limit dynamic evaluation model; solving the multi-objective optimization function based on a reinforcement learning method to obtain an optimal control instruction set, and predicting the section flow change results after control; feeding the prediction results back to the section limit dynamic evaluation model for judgment, issuing the control instruction set based on the judgment result, and realizing dynamic optimization of the control process based on a rolling optimization control mechanism; the present invention generates the optimal control instructions of the FACTS controller through reinforcement learning, and accurately controls the FACTS controller to adjust the key section flow distribution through the control instructions, thereby effectively alleviating the problem of insufficient section accommodation margin.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic control of section limits, and more specifically, to a dynamic optimization control system and method for power grid section limits based on reinforcement learning. Background Art

[0002] In the automated operation and management of complex systems, especially in the dispatching and control of large-scale, wide-area distributed networks like power systems, uneven power flow distribution across sections often leads to local channel overloads, restricting global operating margins and, in turn, impacting the overall control system's objective function achievement and robustness. This is especially true in scenarios where the system is subject to disturbances (such as load fluctuations, component removals, and planned maintenance) or where operational constraints change dynamically, traditional fixed threshold settings and rule-based control logic cannot accurately represent the real-time safety margins of sections, resulting in a lack of fine-grained dynamic adjustability in the control system.

[0003] At present, the section control functions of power systems mostly rely on sequential logic program control methods driven by manually set parameters or heuristic optimization control algorithms based on empirical rules.

[0004] For example, the invention patent with the announcement number CN119535989A discloses an adaptive control method for a multi-heat source thermal management system based on reinforcement learning model predictive control, including: receiving environmental state, reward function calculation, strategy optimization, decision correction, optimizing objective function and control law construction, temperature error calculation, action optimization, and device control. The present invention achieves real-time optimization of control strategies for fans, water pumps, valves, and other equipment in the cooling system by integrating the strategy optimization of intelligent agents and the action optimization in traditional model predictive control, and has the adaptability of reinforcement learning and the stability of model predictive control; by setting intelligent agents and feedback correction modules, it achieves adjustment of prediction model parameters according to the actual state of the system, thereby improving the robustness and adaptability of the control system; by combining the control law output by rolling optimization with multiple real-time feedback and prediction corrections, it achieves efficient temperature control of the system in dynamic changes.

[0005] For example, the invention patent publication number CN111864743B discloses a method for constructing a power grid dispatching and control model. The method includes the following steps: obtaining multiple historical power grid section flow data; constructing a power grid dispatching and control model based on a maximum entropy reinforcement learning algorithm based on preset safe operation requirements and control objectives; extracting training samples from the multiple historical power grid section flow data and inputting the training samples into the power grid dispatching and control model for model training to obtain various power grid control actions; updating the model parameters of the power grid dispatching and control model based on the power grid operation characteristics after executing each power grid control action based on the historical power grid section flow data; and returning to the step of extracting power grid operation characteristics corresponding to the current power grid operation indicators from the multiple historical power grid section flow data as training samples until all training samples are trained; and determining the optimal power grid dispatching and control model based on the training results. This method lays the foundation for improving the efficiency and performance of power grid dispatching and control.

[0006] The above disclosed technical solutions have at least the following technical problems:

[0007] Although some control strategies have been introduced into FACTS controllers to improve flexibility, current methods generally have shortcomings. For the control of FACTS controllers, traditional optimization methods often use static algorithms such as linear or nonlinear programming and genetic algorithms. It is difficult to quickly update the control strategy in dynamically changing scenarios, resulting in response lag and insufficient control accuracy. It is impossible to achieve accurate regulation of the flow of key sections to improve the section capacity margin. In addition, there is a lack of a predictive feedback mechanism for the system state after the FACTS control is applied, resulting in the inability to dynamically modify and optimize the control strategy according to the system response. In response to the above problems, the present invention proposes a solution. Summary of the Invention

[0008] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a dynamic optimization control system and method for power grid section limits based on reinforcement learning, which generates the optimal control instructions of the FACTS controller through reinforcement learning, and accurately controls the FACTS controller through the control instructions to adjust the power flow distribution of key sections, thereby effectively alleviating the problem of insufficient section accommodation margin.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A power grid section limit dynamic optimization control system and method based on reinforcement learning includes the following steps:

[0011] Based on sparse modeling and feature contribution analysis, the controlled section set is identified and a dynamic assessment model of section limits is constructed; based on the sensitivity of each FACTS controller to section tidal changes, a section response sensitivity matrix is ​​constructed, and a multi-objective optimization function is constructed in combination with the dynamic assessment model of section limits; based on the reinforcement learning method, the multi-objective optimization function is solved to obtain the optimal control instruction set of the FACTS controller, and the section tidal change results after the control action are predicted; the prediction results are fed back to the dynamic assessment model of section limits for judgment, and based on the judgment results, the control instruction set is issued and the dynamic optimization of the control process is achieved based on the rolling optimization control mechanism.

[0012] In a preferred embodiment, a controlled section set is identified based on sparse modeling and feature contribution analysis, specifically: constructing input features and output features, wherein the input features are the control variable control matrix of the FACTS controller, and the output features are the flow change matrix of each section; based on the sparse modeling method, a section flow change prediction model is constructed through the input and output features, all power grid sections are traversed, and sections with non-zero outputs are recorded as high-response sections to form a controlled section candidate set; the SHAP interpretation method is applied to the output of the prediction model to obtain the corresponding contribution value, and the contribution value index is obtained by statistical methods; for each section in the controlled section candidate set, a weighted score is performed based on the number of its non-zero model outputs and the contribution value index, and sections with scores greater than a preset threshold are screened to obtain a controlled section set.

[0013] In a preferred embodiment, a dynamic assessment model for section limits is constructed, specifically by: obtaining section limit impact data of a set of controlled sections, wherein the section limit impact data includes power offset rate characteristics, node voltage distribution gradient characteristics, and frequency disturbance response characteristics; inputting the section impact data as a feature vector, and constructing a dynamic assessment model for section limits based on the XGBoost method, and outputting the accommodation margin of each section.

[0014] In a preferred embodiment, a section response sensitivity matrix is ​​constructed based on the sensitivity of each FACTS controller to section tidal current changes, specifically: FACTS controller data is obtained, and an initial section response sensitivity matrix is ​​constructed through a tidal current sensitivity analysis method; a rolling time window is set to obtain historical equipment data within the time window; the historical equipment data set is input into a section tidal current change prediction model, and a response sensitivity correction coefficient matrix is ​​output; the response sensitivity correction coefficient matrix is ​​applied to the initial section response sensitivity matrix, the matrix is ​​dynamically corrected, and the corrected section response sensitivity matrix is ​​used as the final section response sensitivity matrix.

[0015] In a preferred embodiment, a multi-objective optimization function is constructed in combination with a dynamic evaluation model of section limits. Specifically, based on the section response sensitivity matrix and the dynamic evaluation model of section limits, a multi-objective optimization function is constructed with the goals of maximizing the section accommodation margin and minimizing the control cost, and constraints are set to obtain a constrained optimization model, wherein the constraints include node voltage constraints, power balance constraints and FACTS controller boundary constraints.

[0016] In a preferred embodiment, the optimal control instruction set of the FACTS controller is obtained by solving the multi-objective optimization function based on the reinforcement learning method, specifically: based on the intelligent agent, its state space and action space are defined, and the multi-objective optimization function is used as the reward function, where the positive reward represents the improvement of the section accommodation margin, and the negative penalty term represents the control cost; the FACTS controller instructions are output through the policy network method, the current strategy is evaluated through the value network method, and gradient updates and strategy iterations are performed; the section flow response, accommodation margin change and control cost feedback indicators after the execution of the control instructions are continuously collected to update the training data set; the trained reinforcement learning strategy is applied to the output FACTS controller to output the optimal control instruction set, and the optimal control instruction set includes the optimal control amount of each FACTS controller.

[0017] In a preferred embodiment, the section flow change result after the control action is predicted is specifically as follows: according to the optimal control instruction set, the parameter status of the corresponding FACTS controller in the power grid model is updated and the device parameter model is constructed; based on the AC power flow algorithm, the device parameter model is used to calculate the prediction result of the whole network flow distribution after the FACTS control action is implemented, and the current flow data of the controlled section is extracted.

[0018] In a preferred embodiment, the prediction results are fed back to the section limit dynamic assessment model for judgment, and the control instruction set is issued based on the judgment result, and the dynamic optimization of the control process is realized based on the rolling optimization control mechanism, specifically: the current flow data is used as an input parameter and fed back to the section limit dynamic assessment model, and the controlled accommodation margin of the controlled section is calculated; the controlled accommodation margin is judged to see whether it meets the preset safety margin threshold, if it is greater than the preset section safety margin threshold, the control instruction set is considered to be valid and the section safety margin meets the requirements; otherwise, the control instruction set is returned to be regenerated to form a closed-loop control mechanism; if all sections meet the safety margin threshold, the control process is terminated, the FACTS control instruction is issued and the equipment status is updated.

[0019] In a preferred embodiment, dynamic optimization of the control process is achieved based on a rolling optimization control mechanism, specifically: constructing a section limit optimization control process based on a rolling time window, setting a fixed-cycle control execution interval, and repeating the entire process of section status evaluation, control optimization decision-making, control instruction issuance and status feedback update in each cycle; at the beginning of each rolling cycle, updating the input data set; based on the current moment data, re-evaluating the section limit, constructing the FACTS response sensitivity matrix, and updating the weights of each section accommodation margin and control cost term in the objective function to form an optimization model for the current cycle; using an intelligent optimization algorithm to solve the optimization model, forming a control instruction set suitable for the current cycle, outputting and executing it; and feeding back the system status after control execution to the evaluation model and optimizer for parameter update.

[0020] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0021] 1. By introducing sparse modeling and feature contribution analysis, the set of controlled sections that are controllable and have a key impact on the safe operation of the system can be accurately identified, thereby avoiding invalid or redundant control resource allocation and improving control efficiency. A dynamic assessment model for section limits constructed by integrating power offset, node voltage gradient and frequency disturbance response characteristics can break the conservative bottleneck of traditional static limits and significantly improve the timeliness perception and dynamic accuracy of section safety margins. Combined with the section response sensitivity matrix, the marginal response capability of each FACTS controller to section currents is clarified, achieving highly responsive and directional control decisions. Finally, through a multi-objective optimization function with the goals of maximizing the accommodation margin and minimizing the control cost, control adaptation and dynamic balance for actual operation scenarios are achieved.

[0022] 2. A multi-objective optimization and rolling feedback control mechanism based on reinforcement learning, centered around the FACTS controller, was constructed to achieve intelligent, dynamic, and adaptive optimization of power grid section limit control. This solution, with the dual objectives of maximizing section capacity and minimizing control costs, effectively identifies bottleneck sections and generates optimal control instruction sets through state-action modeling and iterative policy learning, with the ability to continuously learn and update. Furthermore, closed-loop verification is performed after control through power flow forecasting and dynamic assessment models to ensure control effectiveness and safety margins, achieving real-time linkage between control, feedback, and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a method for dynamic optimization control of power grid section limits based on reinforcement learning provided in an embodiment of the present application.

[0024] Figure 2 Schematic diagram of the structure of the power grid section limit dynamic optimization control system based on reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0026] Example 1, Figure 1 The schematic flow chart of the method for dynamic optimization control of power grid section limits based on reinforcement learning provided in the embodiment of the present application includes the following steps:

[0027] S1, based on sparse modeling and feature contribution analysis, identifies the controlled section set and constructs a dynamic assessment model for section limits.

[0028] In this embodiment, a controlled section set refers to a group of network sections that can be effectively regulated within the control system. The power flow distribution within this set can be responsively regulated through control methods deployed within the system (e.g., FACTS devices, power output regulation, load redistribution, etc.). The identification process of this set is a key step in the control system's controllable object identification and screening mechanism, aiming to improve the efficiency of control resource allocation and system response performance. By identifying the controlled section set, the indiscriminate allocation of limited control resources to all network sections can be significantly avoided, effectively circumventing the problems of control system model expansion and increased computational complexity. Control strategy optimization is performed only on critical sections that can be accurately modeled and controlled and have a substantial impact on system operational bottlenecks, thereby improving the control strategy's target-oriented and real-time performance and providing targeted support for subsequent response modeling and optimized control.

[0029] Identify the controlled section set based on sparse modeling and feature contribution analysis, specifically:

[0030] Obtaining historical power grid operation data and constructing input features and output features, wherein the input features are the control variable control matrix of the FACTS controller, and the output features are the power flow change matrix of each section. The historical power grid operation data includes the FACTS controller action amount at multiple time moments and the power flow change amount at the corresponding time moments;

[0031] Based on the sparse modeling method, a section tidal flow change prediction model is constructed by input and output characteristics. All sections are traversed and the sections corresponding to at least one non-zero model output are recorded as high-response sections to obtain a candidate set of controlled sections.

[0032] Obtaining a corresponding contribution value based on the output of the cross-sectional tidal flow change prediction model based on the SHAP method, and obtaining a contribution value index through a statistical method, wherein the statistical method is to calculate an average;

[0033] For the sections in the controlled section candidate set, weighted scores are performed based on the number of their non-zero model outputs and contribution value indicators, and sections with a value greater than a preset threshold are screened to obtain the controlled section set.

[0034] The specific calculation formula of the cross-section tidal flow change prediction model is as follows:

[0035]

[0036] Where, For the The response sensitivity of each section to each feature, is the control variable control matrix, is the power flow variation matrix, is the regularization coefficient, is the residual sum of squares, is the L1 norm.

[0037] Furthermore, a dynamic assessment model for section limits was constructed primarily to dynamically evaluate and predict the maximum tolerable flow capacity (i.e., real-time limit) of each key section at the current moment. This model integrates the characteristics of power offset rate, node voltage distribution gradient, and frequency disturbance response, and can output the "accommodation margin" of each section, thereby providing the control system with a more fine-grained and dynamically perceptible basis for section operating boundaries. It also dynamically replaces static limit values, overcoming the conservative safety margin problem in conventional control. In subsequent steps, the response sensitivity ranking of controlled sections and the constrained optimization of FACTS control quantities must be based on the dynamic accommodation margin value of each section. Therefore, this model constitutes the basic assessment module for the entire control strategy optimization.

[0038] The construction of the dynamic assessment model for section limits is specifically as follows:

[0039] Acquiring section limit impact data of the controlled section set, wherein the section limit impact data includes power offset rate characteristics, node voltage distribution gradient characteristics, and frequency disturbance response characteristics;

[0040] The section impact data is input as a feature vector, and a section limit dynamic assessment model is constructed based on XGBoost to output the accommodation margin of each section.

[0041] Furthermore, the power offset rate characteristic is used to measure the intensity of power fluctuations in a section, which can reflect the impact of factors such as load disturbances and intermittent power output fluctuations on the operating boundary of the section. The larger the value, the more unstable the system is, and the more conservative the section limit should be. By constructing a dynamic assessment model for section limits based on the power offset rate characteristic, trend changes caused by fluctuations in section load or source end can be captured in real time, improving the assessment model's ability to perceive sudden power flow disturbances. This helps to enhance the control system's disturbance response feedforward adjustment capability, allowing the controller to implement control quantity adjustments in advance before sensing the potential instability trend of the key section, thereby improving the system's ability to suppress sudden disturbances.

[0042] The specific calculation formula of the power offset rate characteristic is as follows:

[0043]

[0044] Where, is the power offset rate characteristic, is the current power value of a controlled section, For time.

[0045] The node voltage distribution gradient reflects the difference in reactive power support capacity on either side of a section, indirectly indicating voltage stability risk. A larger voltage difference indicates a more severe voltage imbalance and poorer section operational stability. Constructing a dynamic section limit assessment model based on the node voltage distribution gradient helps reflect the stability margin of a section's voltage support capacity.

[0046] The specific calculation formula of the node voltage distribution gradient characteristic is as follows:

[0047]

[0048] Where, is the node voltage distribution gradient characteristic, is the average voltage in the left area, is the average voltage in the right area, is the node set in the left area of ​​the cross section, including nodes, is the node set in the right area of ​​the cross section, including nodes, The left area of ​​the cross section The voltage amplitude of each node, The right side of the cross section The voltage amplitude of each node.

[0049] The frequency disturbance response characteristic measures a section's sensitivity to system frequency disturbances. Larger values ​​indicate a more severe response to frequency disturbances, potentially leading to large energy surges and impacting grid transient stability. Building a dynamic section limit assessment model based on the frequency disturbance response characteristic can reflect the frequency fluctuation rate in the section's adjacent area during the initial stages of a disturbance, improving the ability to predict the section's dynamic safety margin.

[0050] The specific calculation formula of the frequency disturbance response characteristics is as follows:

[0051]

[0052] Where, is the frequency disturbance response characteristic, 、 is the tidal power value of the section at two moments before and after the disturbance occurs, 、 is the value of the whole network frequency at two moments before and after the disturbance.

[0053] The specific calculation formula of the dynamic assessment model of section limit is as follows:

[0054]

[0055] Where, is the cross-section accommodation margin, is the power offset rate characteristic, is the node voltage distribution gradient characteristic, is the frequency disturbance response characteristic, 、 、 is the weight coefficient after training.

[0056] S2, based on the sensitivity of each FACTS controller to the change of section flow, construct the section response sensitivity matrix, and combine it with the section limit dynamic assessment model to build a multi-objective optimization function.

[0057] In this embodiment, the section response sensitivity matrix refers to a two-dimensional matrix structure constructed through power flow sensitivity analysis that represents the degree of influence of various FACTS controller variables (such as injected reactive voltage and series impedance changes) on power flow changes at each controlled section in the power grid. Each element in the matrix represents the marginal impact of a FACTS controller on the power flow at a specific section, reflecting the linear or near-linear response relationship between the control action and the target response. By constructing this sensitivity matrix, the response strength of each FACTS controller to the power flow at each section can be systematically and quantitatively determined, distinguishing between devices with high control redundancy and weak response capabilities and those with strong target response and significant influence, providing a theoretical basis for subsequent control optimization.

[0058] Based on the sensitivity of each FACTS controller to the change of section flow, a section response sensitivity matrix is ​​constructed, which is specifically:

[0059] Acquire FACTS controller data deployed in the target power grid, wherein the FACTS controller data includes device type, installation location, adjustable parameter range, control cost, and real-time operating status;

[0060] Based on the FACTS controller data, the power flow sensitivity analysis method is used to obtain the influence of the control variables of each FACTS controller on the power flow changes of each section, and the initial section response sensitivity matrix is ​​constructed;

[0061] Setting a rolling time window and acquiring historical equipment data within the time window, wherein the historical equipment data includes historical power flow data, FACTS controller instruction data, and corresponding controlled section power flow response data;

[0062] Based on the section power flow change prediction model, historical equipment data is used as input to obtain the response sensitivity correction coefficient matrix;

[0063] Apply the response sensitivity correction coefficient matrix to the initial section response sensitivity matrix to dynamically correct the matrix;

[0064] The modified initial section response sensitivity matrix is ​​used as the final section response sensitivity matrix.

[0065] It should be noted that the response sensitivity correction coefficient matrix refers to the proportionality factor that quantitatively corrects the response sensitivity value obtained by preliminary calculation in order to more accurately evaluate the actual impact of the FACTS device on the flow changes in the controlled section in power grid control modeling and response analysis.

[0066] By introducing a dynamic assessment model for section limits, the current capacity margin of each section can be reflected in real time. This capacity margin indicator can be used as one of the optimization objectives to ensure that the control behavior is directly related to the system safety goal, no longer relying on static limits, and improving the responsiveness and pertinence of the control. By utilizing the section response sensitivity matrix, trial-and-error control can be avoided, and the predictability and linear solvability of the control action can be improved, so that the multi-objective optimization process has a clear physical basis and mathematical solvability. The multi-objective optimization function simultaneously considers the maximization of the capacity margin and the minimization of the control cost, avoiding the control imbalance or redundant control caused by the previous focus on the single dimension of section safety or control cost; it supports flexible adjustment of weight factors according to the operating scenario to achieve strategy adaptive control.

[0067] The multi-objective optimization function is constructed by combining the dynamic assessment model of the section limit, specifically:

[0068] Based on the section response sensitivity matrix and the section limit dynamic evaluation model, a multi-objective optimization function with the goals of maximizing the section accommodation margin and minimizing the control cost is constructed, and constraints are set to obtain a constrained optimization model. The constraints include node voltage constraints, power balance constraints and FACTS controller boundary constraints. The node voltage constraint is that after the FACTS controller is executed, the voltage amplitude of any node should be maintained within a preset safe allowable range. The power balance constraint is that after the control is executed, the sum of the active output of all power generation nodes should be equal to the sum of the active power of all load nodes to maintain the active power balance of the power grid. The FACTS controller boundary constraint is that the control value of each FACTS controller must meet the range defined by its minimum control capability and maximum control capability.

[0069] The specific calculation formula for minimizing the control cost is as follows:

[0070]

[0071] Where, To minimize the control cost, is the total number of FACTS controllers, For FACTS controller index, is the control variable of the FACTS controller, is the quadratic cost sensitivity of the FACTS controller variable, is the linear cost sensitivity of the FACTS controller variable.

[0072] The specific calculation formula of the multi-objective optimization function is as follows:

[0073]

[0074] Where, is a multi-objective optimization function, To minimize the control cost, To maximize the cross-sectional accommodation margin, 、 is the weight coefficient.

[0075] It should be noted that minimizing the control cost is based on the response sensitivity matrix as a constraint basis to measure the feasible control solution with the minimum cost.

[0076] S3, based on the reinforcement learning method, solves the multi-objective optimization function to obtain the optimal control instruction set of the FACTS controller and predicts the section flow change results after the control action.

[0077] In this embodiment, due to the limited number of FACTS devices and their restricted regulation capabilities, and the numerous tidal sections requiring regulation in the system, direct intervention would not only result in high control costs and a wide impact, but also face operational feasibility constraints. Therefore, a multi-objective optimization model is constructed, incorporating a reinforcement learning algorithm. This allows for dynamic and intelligent selection of the optimal control strategy combination from a global system perspective, thereby minimizing control costs while increasing the capacity of critical sections.

[0078] The section capacity margin can be defined as the amount of active power disturbance that a section can still withstand under its current operating conditions. By combining a real-time state estimation model with a control response sensitivity matrix, an intelligent solution mechanism can identify the bottleneck sections of the current system and direct control actions to the control path that best improves their capacity margin, avoiding the inefficient allocation of control resources to non-critical sections.

[0079] Unlike traditional control logic based on rule bases or manual policy formulation, intelligent algorithms such as reinforcement learning possess the ability to adapt policies and learn from environmental feedback. They can effectively explore and converge on the control combination that best improves the target section across multiple policy spaces. This approach not only improves the pertinence and effectiveness of control behavior, but also allows for continuous evolution of control strategies over a rolling time horizon, enabling feedback-driven self-optimization of control actions, ultimately improving the accuracy and stability of section capacity margin control.

[0080] The optimal control instruction set of the FACTS controller is obtained by solving the multi-objective optimization function based on the reinforcement learning method, specifically:

[0081] Based on the intelligent agent, the state space is defined as the current section capacity margin, the state of the FACTS control variables, and the network flow state. The action space is defined as the controllable quantity of the FACTS controller. The multi-objective optimization function is used as the reward function, where the positive reward represents the improvement of the section capacity margin and the negative penalty term represents the control cost.

[0082] Output FACTS controller instructions through the policy network method, evaluate the pros and cons of the current strategy through the value network method, and perform gradient updates and policy iterations;

[0083] Improve convergence speed and strategy stability through experience replay mechanism and target network stabilization training process;

[0084] Continuously collect cross-section tidal responses, capacity margin changes, and control cost feedback indicators after control instructions are executed, update training data sets, and improve the generalization ability and practicality of reinforcement learning models;

[0085] After the training is completed, the reinforcement learning strategy obtained from the training is applied to output the optimal control instruction set of the FACTS controller under the current power grid state, and the optimal control instruction set includes the optimal control quantity of each FACTS controller.

[0086] Furthermore, the design and implementation of control measures should not rely solely on the verification and evaluation of post-event results, but should have predictive control capabilities, that is, to conduct feedforward modeling and evaluation of the response changes of the system flow state before the control action is executed, so as to achieve the prediction of potential operating risks and pre-analysis of control effects, and improve the reliability and robustness of the control strategy in a dynamic operating environment.

[0087] Based on its ability to predict the evolution of power flows after control, the system can proactively identify and mitigate secondary risks caused by control, such as line overloads, voltage exceeding limits, and frequency anomalies. Furthermore, it enables coordinated control optimization among multiple FACTS devices. Supported by control coupling effect modeling, this multi-device collaboration improves system-wide operational safety and achieves a dynamic balance between control costs and system operational efficiency within constraints, ultimately achieving the unified goals of safety and economy.

[0088] The results of the cross-section tidal flow changes after the predictive control are as follows:

[0089] Update the parameter status of the corresponding FACTS controller in the power grid model according to the optimal control instruction set, form the adjusted topology structure and initial values ​​of power flow parameters, and build the equipment parameter model;

[0090] Based on the AC power flow algorithm and using the equipment parameter model, the power flow distribution prediction results of the entire network after the implementation of the FACTS control action are calculated, and the current power flow data of the controlled section are extracted.

[0091] S4, feeds the prediction results back to the section limit dynamic assessment model for judgment, issues the control instruction set based on the judgment result, and realizes dynamic optimization of the control process based on the rolling optimization control mechanism.

[0092] In this embodiment, the operating state of the power grid has a high degree of real-time and uncertainty characteristics, and the control instructions of any FACTS device may trigger new changes in the power flow distribution. If there is a lack of secondary effect evaluation of the system response after the control is executed, it may lead to ineffective control and even induce new section overload risks. Therefore, it is necessary to construct a feedback evaluation mechanism based on the predicted power flow to form a closed-loop control system of control-evaluation-correction to ensure the effectiveness and feasibility of the control instructions under dynamic operating conditions. Although the control strategy is initially derived from the multi-objective optimization solution model, aiming to strike a balance between safety and economy, due to factors such as certain structural approximations, input data disturbances and nonlinear dynamic responses of the power system, the actual control execution results often have the risk of deviating from expectations. By introducing a feedback link, online re-evaluation of whether the system reaches the preset section capacity margin threshold after the control is executed can be carried out, which can identify the control deviation and trigger the secondary optimization adjustment mechanism, thereby improving the robustness and accuracy of the control process.

[0093] The prediction results are fed back to the dynamic evaluation model of the section limit, and based on the judgment results output by the model, a control instruction set is issued to perform dynamic optimization of the section limit, specifically:

[0094] The current tidal flow data is used as input parameters and fed back into the dynamic assessment model of the section limit, and the controlled accommodation margin of the controlled section is calculated;

[0095] The controlled accommodation margin is judged to see whether it meets the preset safety margin threshold. If it is greater than the preset section safety margin threshold, the control instruction set is considered valid and the section safety margin meets the requirements; otherwise, the control instruction set is returned to be regenerated to form a closed-loop control mechanism;

[0096] If all sections meet the safety margin threshold, the control process is terminated, the FACTS control instructions are issued, and the equipment status is updated.

[0097] The specific calculation formula of the control accommodation margin is as follows:

[0098]

[0099] Where, is the accommodation margin after control, is the current flow data, It is the cross-section accommodation margin.

[0100] It is important to note that the post-controlled flow state is re-input into the dynamic assessment model to obtain a post-controlled capacity margin estimate based on the current state. This provides greater timeliness and accuracy, avoiding the error accumulation caused by static models or one-time predictions. If the control result does not meet the capacity margin threshold, the feedback optimization module recalculates the control strategy and adjusts the model parameters based on the new state, achieving the coordinated evolution of adaptive control optimization and system state. Control instructions are only issued when the control strategy is verified as effective in the assessment model (i.e., all section capacity margins meet the threshold), significantly improving the robustness and safety of control decisions. The closed-loop control mechanism prioritizes strategies with significant control effectiveness and lower costs, avoiding waste of control resources. Furthermore, through continuous feedback and optimization, potential overload risks in sections are proactively identified, enabling risk preemption and control preemption.

[0101] Furthermore, the fundamental purpose of designing a rolling optimization control mechanism is to build a real-time responsive control system for the evolution of complex power grid operating conditions, achieving dynamic and continuous optimization of section limit accommodation margins and risk pre-control. This step is not only the logical convergence of all the aforementioned module functions, but also a key link in achieving practical, intelligent, and stable operation of the entire control system. It has the following specific functions and technical benefits:

[0102] The grid operating environment is highly dynamic: factors such as grid load fluctuations, renewable energy fluctuations, and equipment status changes all lead to significant time-evolving characteristics in cross-section power flows and limit margins. Fixed control schemes are unable to cope with real-time changes, requiring a rolling control mechanism for dynamic adaptation.

[0103] Avoiding control failure and lag risks: If control strategies are based on outdated data or single-shot control models, control lag or deviation may occur during actual operation, leading to operational risks such as cross-section limit violations. The rolling update mechanism can effectively reduce the timeliness of control strategies.

[0104] The control strategy needs to be dynamically modified as the state changes: the FACTS controller response has certain nonlinearity and hysteresis, and the rolling mechanism can achieve continuous feedback on the control execution results, thereby dynamically evolving and adaptively modifying the control strategy.

[0105] Realize a self-closed-loop structure of control-feedback-re-optimization: This mechanism builds a closed-loop chain of section assessment, optimization decision-making, instruction issuance, status feedback, and re-evaluation, ensuring that each control operation has subsequent feedback evaluation and strategy correction, thereby improving system robustness.

[0106] The dynamic optimization of the control process is achieved based on the rolling optimization control mechanism, specifically:

[0107] Set a fixed control cycle, build a rolling time window, and execute the optimized control process within each control cycle;

[0108] At the beginning of each control cycle, the current system operation data is acquired and updated, including the section power flow, FACTS controller status, and available control resource capacity;

[0109] Based on the updated system operation data, perform real-time assessment of section limit status and reconstruct the FACTS controller response sensitivity matrix;

[0110] Based on the real-time evaluation results and sensitivity matrix, the weight coefficients of the cross-section accommodation margin and the control cost in the optimization objective function are updated to build an optimization model suitable for the current cycle;

[0111] Solving the optimization model using an intelligent optimization algorithm to generate a FACTS controller instruction set within a current cycle;

[0112] Send the control instructions to the corresponding FACTS controller and execute the corresponding control actions;

[0113] The system status data after control execution is obtained and used as feedback information input to update the section limit assessment model and optimization model parameters in the next control cycle. The system status data after control execution includes the section flow change results after the control action is executed, the new state of the FACTS controller after control, and the section limit dynamic assessment results.

[0114] Example 2, Figure 2 This is a schematic diagram of the structure of a power grid section limit dynamic optimization control system based on reinforcement learning provided in an embodiment of the present application, including a section state assessment module, a target optimization module, a control instruction generation and prediction module, and a feedback optimization module. There are connections between the modules:

[0115] Section status assessment module, used to identify the controlled section set and build a dynamic assessment model for section limits based on sparse modeling and feature contribution analysis;

[0116] The target optimization module is used to construct a section response sensitivity matrix based on the sensitivity of each FACTS controller to the change of section flow, and to build a multi-objective optimization function in combination with the dynamic evaluation model of section limit;

[0117] The control instruction generation and prediction module is used to solve the multi-objective optimization function based on the reinforcement learning method to obtain the optimal control instruction set of the FACTS controller and predict the change results of the cross-section flow after the control action;

[0118] The feedback optimization module is used to feed back the prediction results to the section limit dynamic assessment model for judgment, issue the control instruction set based on the judgment results, and realize dynamic optimization of the control process based on the rolling optimization control mechanism.

[0119] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0120] The above embodiments may be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0121] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0122] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0123] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0124] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic optimization control method for power grid section limits based on reinforcement learning, characterized in that: The steps include: Identify the controlled section set and build a dynamic assessment model for section limits based on sparse modeling and feature contribution analysis; Based on the sensitivity of each FACTS controller to the change of section flow, a section response sensitivity matrix is ​​constructed, and a multi-objective optimization function is constructed in combination with the section limit dynamic assessment model; The optimal control instruction set of the FACTS controller is obtained by solving the multi-objective optimization function based on the reinforcement learning method, and the cross-section flow change result after the control is applied is predicted; The prediction results are fed back to the section limit dynamic assessment model for judgment, and the control instruction set is issued based on the judgment results, and the dynamic optimization of the control process is achieved based on the rolling optimization control mechanism.

2. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 1, characterized in that: The identification of the controlled section set based on sparse modeling and feature contribution analysis is specifically as follows: Construct input characteristics and output characteristics, where the input characteristics are the control variable control matrix of the FACTS controller and the output characteristics are the flow change matrix of each section; Based on the sparse modeling method, a section power flow change prediction model is constructed through input and output characteristics. All power grid sections are traversed, and sections with non-zero output are recorded as high-response sections to form a candidate set of controlled sections. Apply the SHAP interpretation method to the output of the prediction model to obtain the corresponding contribution value, and obtain the contribution value index through statistical methods; For each section in the controlled section candidate set, a weighted score is performed based on the number of its non-zero model outputs and the contribution value index, and the sections with scores greater than the preset threshold are screened to obtain the controlled section set.

3. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 2, characterized in that: The construction of the dynamic assessment model for section limits is specifically as follows: Acquiring section limit impact data of the controlled section set, wherein the section limit impact data includes power offset rate characteristics, node voltage distribution gradient characteristics, and frequency disturbance response characteristics; The section impact data is input as a feature vector, and a section limit dynamic assessment model is constructed based on the XGBoost method to output the accommodation margin of each section.

4. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 3 is characterized in that: Based on the sensitivity of each FACTS controller to the change of section flow, a section response sensitivity matrix is ​​constructed, which is specifically: Obtain FACTS controller data and construct the initial section response sensitivity matrix through power flow sensitivity analysis method; Set a rolling time window and obtain historical device data within the time window; Input the historical equipment data set into the section power flow change prediction model and output the response sensitivity correction coefficient matrix; The response sensitivity correction coefficient matrix is ​​applied to the initial section response sensitivity matrix to dynamically correct the matrix, and the corrected section response sensitivity matrix is ​​used as the final section response sensitivity matrix.

5. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 4 is characterized in that: The multi-objective optimization function is constructed by combining the dynamic assessment model of the section limit, specifically: Based on the section response sensitivity matrix and the section limit dynamic evaluation model, a multi-objective optimization function is constructed with the goals of maximizing the section accommodation margin and minimizing the control cost. Constraints are set to obtain a constrained optimization model. The constraints include node voltage constraints, power balance constraints, and FACTS controller boundary constraints.

6. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 1, characterized in that: The optimal control instruction set of the FACTS controller is obtained by solving the multi-objective optimization function based on the reinforcement learning method, specifically: Based on the agent, its state space and action space are defined, and a multi-objective optimization function is used as the reward function, where the positive reward represents the improvement in the cross-section capacity margin, and the negative penalty term represents the control cost; Output FACTS controller instructions through the policy network method, evaluate the pros and cons of the current strategy through the value network method, and perform gradient updates and policy iterations; Continuously collect the cross-section flow response, accommodation margin change, and control cost feedback indicators after the control instructions are executed, and update the training data set; The trained reinforcement learning strategy is applied to the output FACTS controller to output an optimal control instruction set, wherein the optimal control instruction set includes the optimal control quantity of each FACTS controller.

7. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 1, characterized in that: The results of the cross-section tidal flow changes after the predictive control are as follows: Update the parameter status of the FACTS controller according to the optimal control instruction set and build the device parameter model; Based on the AC power flow algorithm and using the equipment parameter model, the power flow distribution prediction results of the entire network after the implementation of the FACTS control action are calculated, and the current power flow data of the controlled section are extracted.

8. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 7, characterized in that: The prediction results are fed back to the dynamic assessment model of the section limit for judgment, and the control instruction set is issued based on the judgment result, and the dynamic optimization of the control process is realized based on the rolling optimization control mechanism, specifically: The current tidal flow data is used as input parameters and fed back into the dynamic assessment model of the section limit, and the accommodation margin after control is calculated; The controlled accommodation margin is judged to see whether it meets the preset safety margin threshold. If it is greater than the preset section safety margin threshold, it is considered that the control instruction set is valid and the section safety margin meets the requirements; otherwise, the control instruction set is returned to be regenerated; If all sections meet the safety margin threshold, the control process is terminated, the FACTS control instructions are issued, and the equipment status is updated.

9. The method for dynamic optimization control of power grid section limits based on reinforcement learning according to claim 8, characterized in that: The dynamic optimization of the control process is achieved based on the rolling optimization control mechanism, specifically: Set a fixed control cycle, build a rolling time window, and execute the optimized control process within each control cycle; At the beginning of each control cycle, obtain and update the current system operation data; Based on the updated system operation data, perform real-time assessment of section limit status and reconstruct the FACTS controller response sensitivity matrix; Based on the real-time evaluation results and sensitivity matrix, the weight coefficients of the cross-section accommodation margin and control cost in the objective function are updated to build an optimization model suitable for the current cycle; An intelligent optimization algorithm is used to solve the optimization model and generate the FACTS controller instruction set for the current cycle; Send the control instructions to the corresponding FACTS controller and execute the corresponding control actions; The system status data after control execution is obtained and used as feedback information input to update the section limit assessment model and optimize model parameters in the next control cycle.

10. A system using the method for dynamic optimization control of power grid section limits based on reinforcement learning according to any one of claims 1 to 9, characterized in that: It includes section state assessment module, target optimization module, control instruction generation and prediction module and feedback optimization module. There are connections between modules: Section status assessment module, used to identify the controlled section set and build a dynamic assessment model for section limits based on sparse modeling and feature contribution analysis; The target optimization module is used to construct a section response sensitivity matrix based on the sensitivity of each FACTS controller to the change of section flow, and to build a multi-objective optimization function in combination with the dynamic evaluation model of section limit; The control instruction generation and prediction module is used to solve the multi-objective optimization function based on the reinforcement learning method to obtain the optimal control instruction set of the FACTS controller and predict the change results of the cross-section flow after the control action; The feedback optimization module is used to feed back the prediction results to the section limit dynamic assessment model for judgment, issue the control instruction set based on the judgment results, and realize dynamic optimization of the control process based on the rolling optimization control mechanism.

Images