Power grid key section screening method and system based on reinforcement learning, medium and processor

Abstract

The invention discloses a power grid key section screening method and system based on reinforcement learning, a processor and a medium, and belongs to the technical field of power grid safe operation. According to the method, a power grid dynamic state space is constructed by collecting multi-source data and utilizing a graph convolutional neural network, a partition risk is evaluated in combination with a deep Q network to obtain candidate key sections, meanwhile, a reward function and weight parameters thereof are adaptively optimized, a simulation fault scene is searched through a Monte Carlo tree to calculate a risk index, and the risk index is calculated. And finally, screening out a key section and carrying out online iterative optimization on model parameters. According to the method, the problems that a traditional method is difficult to process multi-source dynamic data, lacks complex fault scene simulation and cannot dynamically balance a multi-dimensional optimization target are solved, the dynamic risk characteristics of the power grid can be accurately described, the accuracy and the real-time performance of key section screening are improved, and effective support is provided for safe and stable operation of the power grid.

Description

Technical Field

[0001] This invention relates to the field of cross-section screening technology, and in particular to a method, system, medium, and processor for screening key cross-sections of power grids based on reinforcement learning. Background Technology

[0002] With the rapid development of smart grids, the scale of power grids is constantly expanding and their structure is becoming increasingly complex. The large-scale integration of new energy sources (such as wind power and photovoltaics) further exacerbates the uncertainty of power grid operation. As important transmission channels connecting different regional power grids, the safety and stability of critical sections of the power grid directly affect the reliable operation of the entire power system. Accurately selecting critical sections and assessing their risks is a core element in preventing cascading failures and ensuring power grid security.

[0003] Traditional methods for screening critical power grid sections mainly rely on static topology analysis or single-index assessment. These methods have significant limitations: First, they struggle to effectively handle multi-source dynamic data from real-time power grid operation (such as real-time power flow, equipment status, and fluctuations in renewable energy output), failing to accurately characterize the dynamic risk features of the power grid. Second, they lack systematic simulation of complex fault scenarios (such as multiple line outages and drastic fluctuations in renewable energy output), leading to a disconnect between risk assessment results and actual operating scenarios. Third, facing the uncertainties brought about by renewable energy integration, traditional methods struggle to adaptively adjust the weights of assessment indicators, failing to dynamically balance multi-dimensional optimization objectives such as safety margin and renewable energy absorption rate. Furthermore, existing technologies often employ offline analysis modes, unable to respond in real-time to rapid changes in power grid operating status, and exhibit low efficiency in risk assessment for large-scale power grid zoning.

[0004] Therefore, there is a need for a method, system, medium, and processor for screening critical sections of power grids based on reinforcement learning. Summary of the Invention

[0005] To address the limitations of existing technologies in handling multi-source dynamic data, simulating complex fault scenarios, and dynamically balancing multi-dimensional optimization objectives, this invention provides a reinforcement learning-based method, system, medium, and processor for selecting key power grid sections. This method can handle multi-source dynamic data, simulate complex fault scenarios, and dynamically balance multi-dimensional optimization objectives. The specific technical solution is as follows:

[0006] A reinforcement learning-based method for screening key power grid sections includes:

[0007] S1: Collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural networks;

[0008] S2: Based on the dynamic state space of the power grid and a deep Q-network, risk assessment is performed on each region of the power grid to obtain a set of candidate critical sections.

[0009] S3: Adaptively optimize the reward function and its weight parameters;

[0010] S4: For each candidate section in the candidate critical section set, calculate the risk index of the candidate section after simulating the power flow transfer under different fault scenarios F through MCTS. The candidate sections with risk indices exceeding the threshold are used as the final critical sections to form the critical section set.

[0011] S5: Online iterative optimization of deep Q-network model parameters.

[0012] Furthermore, in step S1, the process of collecting multi-source data and constructing the dynamic state space of the power grid based on a graph convolutional neural network includes the following steps:

[0013] S11: Collect raw data from multiple sources, including real-time power flow data, power grid topology parameters, equipment status data, and new energy data;

[0014] S12: Calculate line load rate based on power flow data The formula is as follows:

[0015]

[0016] In the above formula, P ij Active power; This represents the thermal stability limit of the line.

[0017] S13: Generate the power grid node embedding matrix X using a graph convolutional neural network based on multi-source raw data. GCN The formula is as follows:

[0018]

[0019] In the above formula, It is an adjacency matrix with self-loops; for H is the degree matrix; H is the input feature matrix; W is the trainable weight matrix;

[0020] S14: Generate the graph embedding vector z for each partition using an autoencoder. p As a dynamic state-space representation of the power grid, its formula is as follows:

[0021] z p =Autoencoder([Cr ij ,X GCN ,P renew ,σ renew ]);

[0022] In the above formula, P renew Forecast values ​​of wind / solar power output; σ renew Wind power / solar power output volatility.

[0023] Furthermore, in step S2, the step of performing risk assessment on each region of the power grid based on the dynamic state space of the power grid and a deep Q-network to obtain a set of candidate critical sections includes the following steps:

[0024] S21: Calculate the action value function for each partition based on a deep Q-network, and select the top K partitions by action value as high-risk partitions; the action value function formula is as follows:

[0025]

[0026] In the above formula, s represents the current power grid state, and z represents the graph embedding vector of each partition. p ; 'a' represents the action, and 'a' represents the value. p θ represents the filtering action performed on partition p; θ represents the reinforcement learning model parameters; γ∈[0,1] represents the discount factor; R t+1 The immediate reward after performing action a;

[0027] S22: Calculate the comprehensive risk index for the lines within each high-risk zone, using the following formula:

[0028]

[0029]

[0030] In the above formula, w1 and w2 are weights; G k-l This is the distribution factor matrix of line interruption, that is, the power flow transfer factor of line l interruption on line k; P represents the change in power flow after the line is disconnected. l Here are the active power parameters for line l;

[0031] S23: Use a greedy strategy to filter each high-risk partition. Above the threshold T RI The route combinations generate M candidate sections D; the candidate sections from all high-risk zones are then aggregated to form a set D of candidate key sections. candidate :

[0032] D candidate ={D1,D2,…,D n}

[0033] Furthermore, in step S3, the adaptive optimization of the reward function and its weight parameters includes the following steps:

[0034] S31: Adaptively adjust the weights of each indicator in the reward function according to the real-time operation scenario of the power grid;

[0035] S32: Calculate the penalty term P using the following formula:

[0036]

[0037] Where, l∈D candidate Let l represent candidate critical section D. candidate For any line within the range D, the summation range is D. candidate Includes all lines; max(0,Cr ij -0.9) means that the calculation only applies to overloaded lines; the output is 0 when there is no overload.

[0038] S33: Optimize the reward function R j The formula is as follows:

[0039] R j =α·S m +β·RED p +γ·ACC c +P;

[0040] Where α is the safety margin weight; β is the absorption rate weight; γ is the accuracy weight; S m RED represents the average safety margin of the cross section. p For the renewable energy consumption rate of the zone p; ACC c This refers to the accuracy of cross-section identification.

[0041] Furthermore, the formula for calculating the average safety margin of the cross-section is as follows:

[0042]

[0043] Where N is the total number of lines within the cross-section; Let be the load rate of line ij; This indicates the safety margin of a single line.

[0044] Furthermore, in step S4, for each candidate section in the candidate critical section set, after simulating power flow transfer under different fault scenarios F using MCTS, the risk index of the candidate section is calculated, and candidate sections with risk indices exceeding a threshold are selected as the final critical sections to form the critical section set. This includes the following steps:

[0045] S41: For each section D, power flow transfer under different fault scenarios F is simulated using MCTS to generate S. z The randomized simulation path and the obtained line load rate The number of times S exceeds the preset threshold c ;

[0046] S42: Calculate the line overload probability P for each section D. overload The formula is as follows:

[0047]

[0048] Among them, S c Line load rate The number of times the preset threshold is exceeded, S z P represents the total number of times the random path is simulated. overload This represents the probability of line overload.

[0049] S43: Calculate voltage stability margin V margin The formula is as above:

[0050]

[0051] Among them, |U i | represents the actual voltage amplitude at node i in the power grid. V is the rated voltage of node i in the power grid; margin For voltage stability margin;

[0052] S44: Calculate the renewable energy curtailment rate (RED) using the following formula:

[0053]

[0054] Among them, P curtail P represents the amount of renewable energy power that is abandoned in the power grid. renew To contribute actual power to new energy sources in the power grid;

[0055] S45: Risk index for generating candidate section D, formula as follows:

[0056] RI D =γ·P overload +α·(1-V margin )+β·RED;

[0057] S46: Select candidate sections whose risk index exceeds the threshold as the final critical sections to form the critical section set D. key :

[0058] D key ={D|RI D ≥T RI-final};

[0059] Among them, T RI-final This is the threshold for the risk index.

[0060] Furthermore, in step S5, the online iterative optimization of the deep Q-network model parameters θ includes the following steps:

[0061] S51: Add real-time fault data to the experience replay pool and update the sample distribution;

[0062] S52: Employs a near-end policy optimization algorithm, randomly sampling batch data from the replay pool, and calculating the policy gradient.

[0063]

[0064] In the above formula, θ represents the gradient of the policy network parameter θ; E[·] is the mathematical expectation operator; (s,a,r)~Replay indicates that the sample (s,a,r) is randomly sampled from the experience replay pool Replay; in the sample (s,a,r), s represents the current operating state of the power grid, a represents the agent's selection decision for the candidate section, and r is the reward; logπ θ (a|s) represents the policy network π. θ The logarithmic probability of taking action a in state s; R j The optimized reward function; γ is the discount factor; max a′ Q θ′ In (s′,a′), s′ represents the next state, and a′ represents a possible future action; Q θ′ (s′,a′) is the state-action value function estimated by the target network; max a′ To obtain the maximum Q value among all possible actions; Q θ (s,a) represents the state-action value function estimated by the current Q-network; θ represents the current policy network. The parameters of the network; θ′ are the parameters of the target network;

[0065] S53: Update the reinforcement learning model parameters θ in step S21 using policy gradients for the next round of cross-section selection.

[0066] A power grid critical section screening system based on reinforcement learning, applied to the above-described reinforcement learning-based power grid critical section screening method, includes:

[0067] The spatial module is used to collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural networks;

[0068] The candidate module is used to perform risk assessment on each region of the power grid based on the dynamic state space of the power grid and a deep Q-network to obtain a set of candidate critical sections.

[0069] The adjustment module is used for adaptive optimization of the reward function and its weight parameters;

[0070] The screening module is used to calculate the risk index of each candidate section in the candidate critical section set after simulating power flow transfer under different fault scenarios F through MCTS simulation. Candidate sections with risk indices exceeding the threshold are selected as the final critical sections to form the critical section set.

[0071] The optimization module is used for online iterative optimization of the parameters of the deep Q-network model.

[0072] A computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described reinforcement learning-based power grid critical section screening method.

[0073] A processor for running a program, wherein the program executes the above-described reinforcement learning-based method for screening critical power grid sections.

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0075] 1. It can process multi-source dynamic data, simulate complex fault scenarios, and dynamically balance multi-dimensional optimization objectives.

[0076] 2. Efficiently processes multi-source dynamic data to accurately depict the real-time state of the power grid.

[0077] Traditional methods rely on static topology analysis or single indicators, making it difficult to handle dynamic data such as real-time power flow, equipment status, and fluctuations in renewable energy output. This solution overcomes these limitations in the following ways:

[0078] Multi-source data fusion modeling: Graph Convolutional Neural Network (GCN) is used to integrate real-time power flow data, power grid topology parameters, equipment status and new energy data. The power grid topology and electrical characteristics are transformed into a computable node embedding matrix. Then, an autoencoder is used to generate partitioned graph embedding vectors to construct a dynamic state space and achieve accurate characterization of the power grid operation status.

[0079] Dynamic risk zoning assessment: Based on deep Q-network (DQN) to calculate the action value function of each zoning, high-risk zoning is dynamically screened. Compared with traditional static assessment, it can respond to changes in the power grid operation status in real time and improve the timeliness of risk identification.

[0080] 3. Simulate complex failure scenarios to improve the comprehensiveness of risk assessment.

[0081] Traditional methods lack systematic simulation of complex scenarios such as multiple line outages and drastic fluctuations in renewable energy sources, leading to a disconnect between assessment results and reality. This solution optimizes these methods in the following ways:

[0082] Monte Carlo Tree Search (MCTS) fault simulation: Simulate power flow transfer under multiple fault scenarios such as N-1 and N-2 for each candidate section, calculate multi-dimensional indicators such as line overload probability, voltage stability margin and renewable energy curtailment rate, and generate a comprehensive risk index to ensure that the evaluation results are consistent with the actual operation scenario.

[0083] Multi-dimensional risk comprehensive assessment: Combining overload risk, voltage stability and renewable energy absorption efficiency to generate a risk index, avoiding the one-sidedness of traditional single-indicator assessment, and accurately selecting the key sections with the greatest impact on power grid security.

[0084] 4. Dynamic balance multi-objective optimization to adapt to real-time power grid demands.

[0085] Traditional methods struggle to adaptively adjust the weights of evaluation indicators, failing to balance multiple objectives such as safety margin and renewable energy consumption. This solution improves upon these methods in the following ways:

[0086] Reward function adaptive optimization: The weights of indicators such as safety margin and absorption rate are dynamically adjusted based on real-time grid load levels and the proportion of renewable energy output. For example, the weight of safety margin is increased when the load is too high, and the absorption rate is optimized when the proportion of renewable energy exceeds the threshold, thus achieving a dynamic balance of multiple objectives.

[0087] The penalty mechanism strengthens safety constraints: penalties are imposed on lines with a load rate exceeding 90% in the candidate sections, driving the model to prioritize sections with high safety margins and avoiding the problem of insufficient safety assessment in traditional methods.

[0088] 5. Online iterative learning and modular design enhance system usability.

[0089] Real-time model optimization: By using the Proximal Policy Optimization (PPO) algorithm, real-time fault data is stored in an experience replay pool, and the parameters of the deep Q network are updated online iteratively. This enables the model to continuously learn the power grid operation rules, dynamically optimize the selection strategy, and solve the problem that traditional offline analysis cannot respond in real time.

[0090] Modular system architecture: The screening process is broken down into independent modules such as state space construction, candidate section generation, reward function adjustment, risk assessment and model optimization, which facilitates engineering deployment and maintenance, while improving the screening efficiency of large-scale power grid partitions and adapting to the complex engineering needs of smart grids. Attached Figure Description

[0091] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0092] Figure 1 This is a flowchart illustrating a method for selecting key sections of a power grid based on reinforcement learning.

[0093] Figure 2 This is a schematic diagram of a power grid critical section screening system based on reinforcement learning. Detailed Implementation

[0094] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0095] It should be understood that, when used in this application, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0096] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0097] It should also be further understood that the term “and / or” as used in this application refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes such combinations.

[0098] Example 1

[0099] like Figure 1 The diagram shows a flowchart of a reinforcement learning-based method for screening critical sections of a power grid. The method includes the following steps:

[0100] S1: Collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural network (GCN).

[0101] S11: Collect raw data from multiple sources, including real-time power flow data, power grid topology parameters, equipment status data, and renewable energy data.

[0102] Furthermore, real-time power flow data includes the active power P of each line. ij Reactive power Q ij Current amplitude I ij .

[0103] Furthermore, the power grid topology parameters include line reactance X. ij Resistance R ij Transformer turns ratio K t .

[0104] Furthermore, the equipment status, line maintenance status (0-normal / 1-maintenance), and generator output P...G Load power P L .

[0105] Furthermore, new energy data includes wind power / solar power output forecasts (P). renew and its volatility σ renew .

[0106] S12: Calculate line load rate based on power flow data in This represents the thermal stability limit of the line.

[0107] S13: Generate the power grid node embedding matrix using a graph convolutional neural network based on multi-source raw data: To integrate topological and electrical features.

[0108]

[0109] In the above formula, Let A be the adjacency matrix with self-loops, representing the adjacency matrix of the power grid topology (the line connection relationships between nodes, and the edge weights may include the line reactance X). ij or resistor R ij ), where I is the identity matrix; for The degree matrix satisfies:

[0110]

[0111] The input feature matrix incorporates the electrical characteristics of power grid nodes (such as line active power P). ij Reactive power Q ij , load rate Cr ij (etc.) and initial topological characteristics; For a trainable weight matrix, d g Embed the dimension of the target (denoted as d in the document). g ).

[0112] S14: Generate the graph embedding vector z for each partition using an autoencoder. p As a dynamic state-space representation of the power grid, its formula is as follows:

[0113] z p =Autoencoder([Cr ij ,X GCN ,P renew ,σ renew ]).

[0114] S2: Based on the dynamic state space of the power grid and using a deep Q-network, perform risk assessments on each region of the power grid to obtain a set of candidate critical sections D. candidate .

[0115] S21: Calculate the action value function Q for each partition based on a Deep Q-Network (DQN). θ (z p ,a p ), select the K partitions with the highest action value as high-risk partitions. p This represents the filtering action performed on partition p, which is a decision variable in the reinforcement learning model and an element in the action space. In the power grid critical section filtering scenario, a p This usually represents a binary action that indicates whether to select partition p as a high-risk area (e.g., select = 1, do not select = 0).

[0116] Furthermore, the general formula for the action value function is:

[0117]

[0118] s represents the current power grid state (including topology, power flow data, partition risks, etc., such as the graph embedding vector z constructed from S1). p ); 'a' represents the action (i.e., selecting a connecting line to include in the candidate section or removing it), for example, a p The filtering action is performed on partition p; θ represents the reinforcement learning model parameters; γ∈[0,1] is the discount factor used to balance the weights of current and future rewards; R t+1 The immediate reward after performing action a (based on the reward function formula R in S33) j ).

[0119] Action value function Q(z) p ,a p ) is used to evaluate "the state z in partition p". p Next, execute action a. p "Expected reward value after completion". By comparing different a p Based on the corresponding Q value, the model will select the K partitions with the highest action values ​​as high-risk partitions.

[0120] S22: Calculate the comprehensive risk index for lines within each high-risk zone:

[0121] (w1, w2 are weights);

[0122]

[0123] G k-l This is the distribution factor matrix of line interruption, that is, the power flow transfer factor of line l interruption on line k; P represents the change in power flow after the line is disconnected. l Here are the active power parameters for line l;

[0124] S23: Use a greedy strategy to filter. Above the threshold T RI The route combination can generate M candidate sections D for each high-risk zone; the candidate sections of all high-risk zones are aggregated to form a set of candidate key sections D. candidate :

[0125] D candidate ={D1,D2,…,D n}

[0126] S3: Adaptively optimize the reward function and its weight parameters. The core function of the reward function is to serve as a feedback signal for the reinforcement learning agent, driving policy optimization to achieve efficient selection and control of power grid sections.

[0127] S31: Dynamically adjust the weights of the reward function. Specifically, the weights of each indicator in the reward function are adaptively adjusted based on the real-time operation scenario of the power grid (load level, proportion of renewable energy output) to drive the reinforcement learning agent to prioritize key risk dimensions in the current scenario, as follows:

[0128] 1. When the power grid load level L is monitored level When it is "high": α←α+Δα, where Δα is a positive step size (e.g., 0.05~0.1), and the specific value can be preset according to the power grid risk sensitivity. L level The formula is as follows:

[0129]

[0130] 2. When the proportion of renewable energy output in the power grid is R renew When the percentage exceeds a set threshold (e.g., 30%): β←β+Δβ, where Δβ is a positive step size (e.g., 0.05~0.1). The higher the proportion of renewable energy, the larger Δβ can be. When renewable energy output fluctuates greatly, increasing β can encourage the agent to prioritize sections with less impact on renewable energy consumption, reducing power curtailment losses. Renewable energy output percentage R renew The calculation formula is as follows:

[0131]

[0132] S32: Calculate the penalty term P:

[0133]

[0134] Where, l∈D candidate Let l represent candidate critical section D. candidate For any line within the range D, the summation range is D. candidate Includes all lines; 0.9 represents a 90% load rate, lines exceeding this value are considered to be at risk of overload; max(0,Cr ij-0.9) Only for overloaded lines (Cr) ij >0.9) Calculate the excess portion, and output 0 when there is no overload; the penalty intensity coefficient is -10, the negative sign indicates penalty (reduced reward), and the coefficient value determines the extent of the impact of overload on the reward.

[0135] S33: Optimized reward function R j The formula is as follows:

[0136] R j =α·S m +β·RED p +γ·ACC c +P;

[0137] Where α is the safety margin weight; β is the absorption rate weight; γ is the accuracy weight; S m RED represents the average safety margin of the cross section. p For the regional renewable energy consumption rate; ACC c denoted as cross-section recognition accuracy; P represents the penalty term.

[0138] Furthermore, the formula for calculating the average safety margin of a cross-section is as follows:

[0139]

[0140] Where N is the total number of lines within the cross-section; Let be the load rate of line ij; This represents the safety margin of a single line. The larger the value, the further the line is from the risk of overload. The average safety margin of the section is obtained by averaging the values ​​of N lines.

[0141] Furthermore, the formula for calculating the cross-section recognition accuracy is as follows:

[0142]

[0143] Where M is the set of candidate key sections D candidate The total number of sections; K is the number of sections that are identified as the final critical sections after risk verification (i.e., those that meet the requirements). The number of cross sections (the percentage of candidate cross sections that truly meet the criteria for key cross sections) reflects the proportion of candidate cross sections that actually meet the criteria for key cross sections. A higher value indicates a higher accuracy of the screening strategy.

[0144] S4: For each candidate section D∈D in the candidate key section set... candidate After simulating power flow transfer under different fault scenarios F using MCTS, the risk index of the candidate section is calculated, and the candidate sections with risk indices exceeding the threshold are used as the final critical sections to form a set of critical sections.

[0145] Furthermore, the MCTS stands for Monte Carlo Tree Search. Specifically, it includes the following steps:

[0146] S41: For each section D, power flow transfer under different fault scenarios F is simulated using MCTS to generate S. z The randomized simulation path and the obtained line load rate Exceeding a preset threshold (e.g.) The number of times S c ;

[0147] S42: Calculate the line overload probability P for each section D. overload : Where S c Line load rate The number of times the preset threshold is exceeded, S z P represents the total number of times the random path is simulated. overload This represents the probability of line overload.

[0148] S43: Calculate voltage stability margin V margin : Among them |U i | represents the actual voltage amplitude at node i in the power grid. V is the rated voltage of node i in the power grid; margin This represents the voltage stability margin.

[0149] S44: Calculate the curtailment rate of renewable energy sources. Where P curtail P represents the amount of renewable energy power that is abandoned in the power grid. renew For the actual power output of new energy sources in the power grid; among which:

[0150] This represents the maximum renewable energy power that the power grid can absorb under its current conditions.

[0151] S45: The generation risk index of candidate section D, the formula is as follows:

[0152] RI D =γ·P overload +α·(1-V margin )+β·RED;

[0153] Where α is the safety margin weight; β is the absorption rate weight; and γ is the accuracy weight.

[0154] S46: Select candidate sections whose risk index exceeds the threshold as the final critical sections to form the critical section set D. key :

[0155] D key ={D|RI D ≥T RI-final}, TRI-final This is the threshold for the risk index.

[0156] D key That is, the final set of critical sections that have passed risk verification.

[0157] Furthermore, the F scenario is a set of fault scenarios: F = {N-1, N-2 fault combinations}, where:

[0158] Fault combination F N-1 :F N-1 ={f l |f l = Line l is interrupted, l∈L}, where L is the set of lines in the current section D, and each f l This indicates a fault scenario where a single line l is disconnected.

[0159] Fault combination F N-2 :F N-2 ={f l1，l2 |f l1，l2 = Lines l1 and l2 are simultaneously disconnected, l1, l2 ∈ L, l1 ≠ l2}, each f l1，l2 This represents a fault scenario where two different lines, l1 and l2, are simultaneously disconnected.

[0160] S5: Online iterative optimization of parameters θ of the deep Q-network model.

[0161] S51: Add real-time fault data to the experience replay pool and update the sample distribution.

[0162] Furthermore, real-time fault data includes actual line tripping events. fault Overload log

[0163] Furthermore, the experience replay pool is a historical state-action-reward triple (s, a, r).

[0164] S52: Employs the Proximal Policy Optimization (PPO) algorithm, randomly sampling batch data from the replay pool, and calculating the policy gradient:

[0165]

[0166] In the above formula, The gradient of the policy network parameter θ is used to guide the update direction of the model parameters. The policy network is adjusted using gradient descent (or ascent) algorithms to gradually optimize the agent's policy. E[·] is the mathematical expectation operator, which averages the samples in the experience replay buffer. Subscript: (s,a,r)~Replay indicates that the sample (s,a,r) is randomly sampled from the experience replay buffer. In the sample (s,a,r):

[0167] s: State, representing the current operating state of the power grid (such as power flow data, topology characteristics, etc.).

[0168] a: Action, representing the agent's decision on selecting candidate cross-sections.

[0169] r: Reward, i.e., the reward function R defined in step S3. j The logπ is used to evaluate the quality of actions. Replay is the experience replay pool, storing historical samples (s, a, r, s′) of the agent's interactions with the environment, used for repeated training to improve data utilization and stability. θ (a|s) represents the policy network π. θ The logarithmic probability of taking action a in state s is used to measure the current policy's preference for action a, and is combined with the reward to guide policy updates. R j This is the optimized reward function. γ is the discount factor, used to calculate the present value of future rewards. Its value is typically in the range [0,1], and it is used to make the agent focus more on near-term rewards, balancing long-term and short-term gains. max a′ Q θ′ In (s′,a′), s′ represents the next state, i.e., the new state of the power grid after action a; a′ represents a possible future action; Q θ′ (s′,a′) is the state-action value function estimated by the target network (with parameter θ′), representing the expected cumulative reward for taking action a′ in state s′; max a′ To obtain the maximum Q value among all possible actions, it is used to estimate the value of state s′ (i.e., the V function). θ (s,a) is the state-action value function estimated by the current Q-network (with parameter θ), representing the expected cumulative reward for taking action a in state s. θ is the sum of the current policy network and... The parameters of the network; θ′ are the parameters of the target network, which are usually updated periodically by θ (e.g., soft updates) to stabilize the training process.

[0170] S53: Update the reinforcement learning model parameters θ from step S21 using policy gradient (generally θ and θ). The new θ is obtained and used for the next round of cross-section screening.

[0171] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0172] 1. It can process multi-source dynamic data, simulate complex fault scenarios, and dynamically balance multi-dimensional optimization objectives.

[0173] 2. Efficiently processes multi-source dynamic data to accurately depict the real-time state of the power grid.

[0174] Traditional methods rely on static topology analysis or single indicators, making it difficult to handle dynamic data such as real-time power flow, equipment status, and fluctuations in renewable energy output. This solution overcomes these limitations in the following ways:

[0175] Multi-source data fusion modeling: Graph Convolutional Neural Network (GCN) is used to integrate real-time power flow data, power grid topology parameters, equipment status and new energy data. The power grid topology and electrical characteristics are transformed into a computable node embedding matrix. Then, an autoencoder is used to generate partitioned graph embedding vectors to construct a dynamic state space and achieve accurate characterization of the power grid operation status.

[0176] Dynamic risk zoning assessment: Based on deep Q-network (DQN) to calculate the action value function of each zoning, high-risk zoning is dynamically screened. Compared with traditional static assessment, it can respond to changes in the power grid operation status in real time and improve the timeliness of risk identification.

[0177] 3. Simulate complex failure scenarios to improve the comprehensiveness of risk assessment.

[0178] Traditional methods lack systematic simulation of complex scenarios such as multiple line outages and drastic fluctuations in renewable energy sources, leading to a disconnect between assessment results and reality. This solution optimizes these methods in the following ways:

[0179] Monte Carlo Tree Search (MCTS) fault simulation: Simulate power flow transfer under multiple fault scenarios such as N-1 and N-2 for each candidate section, calculate multi-dimensional indicators such as line overload probability, voltage stability margin and renewable energy curtailment rate, and generate a comprehensive risk index to ensure that the evaluation results are consistent with the actual operation scenario.

[0180] Multi-dimensional risk comprehensive assessment: The risk index is generated by combining overload risk, voltage stability and renewable energy consumption efficiency, avoiding the one-sidedness of traditional single-indicator assessment, and accurately screening the key sections that have the greatest impact on power grid security.

[0181] 4. Dynamic balance multi-objective optimization to adapt to real-time power grid demands.

[0182] Traditional methods struggle to adaptively adjust the weights of evaluation indicators, failing to balance multiple objectives such as safety margin and renewable energy consumption. This solution improves upon these methods in the following ways:

[0183] Reward function adaptive optimization: The weights of indicators such as safety margin and absorption rate are dynamically adjusted based on real-time grid load levels and the proportion of renewable energy output. For example, the weight of safety margin is increased when the load is too high, and the absorption rate is optimized when the proportion of renewable energy exceeds the threshold, thus achieving a dynamic balance of multiple objectives.

[0184] The penalty mechanism strengthens safety constraints: penalties are imposed on lines with a load rate exceeding 90% in the candidate sections, driving the model to prioritize sections with high safety margins and avoiding the problem of insufficient safety assessment in traditional methods.

[0185] 5. Online iterative learning and modular design enhance system usability.

[0186] Real-time model optimization: By using the Proximal Policy Optimization (PPO) algorithm, real-time fault data is stored in an experience replay pool, and the parameters of the deep Q network are updated iteratively online. This enables the model to continuously learn the power grid operation rules, dynamically optimize the selection strategy, and solve the problem that traditional offline analysis cannot respond in real time.

[0187] Modular system architecture: The screening process is broken down into independent modules such as state space construction, candidate section generation, reward function adjustment, risk assessment and model optimization, which facilitates engineering deployment and maintenance, while improving the screening efficiency of large-scale power grid partitions and adapting to the complex engineering needs of smart grids.

[0188] Example 2

[0189] like Figure 2 The diagram shows a structural schematic of a power grid critical section screening system based on reinforcement learning, applied to the aforementioned power grid critical section screening method based on reinforcement learning, including:

[0190] The spatial module is used to collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural networks;

[0191] The candidate module is used to perform risk assessment on each region of the power grid based on the dynamic state space of the power grid and a deep Q-network to obtain a set of candidate critical sections.

[0192] The adjustment module is used for adaptive optimization of the reward function and its weight parameters;

[0193] The screening module is used to calculate the risk index of each candidate section in the candidate critical section set after simulating power flow transfer under different fault scenarios F through MCTS simulation. Candidate sections with risk indices exceeding the threshold are selected as the final critical sections to form the critical section set.

[0194] The optimization module is used for online iterative optimization of the parameters of the deep Q-network model.

[0195] Example 3

[0196] A computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the above-described reinforcement learning-based power grid critical section screening method.

[0197] Example 4

[0198] A processor for running a program, wherein the program executes the above-described reinforcement learning-based method for screening critical power grid sections.

[0199] This application discloses a method, system, processor, and medium for selecting key sections of a power grid based on reinforcement learning, belonging to the field of power grid safe operation technology. The method collects multi-source data and constructs a dynamic state space of the power grid using a graph convolutional neural network. It then combines this with a deep Q-network to evaluate regional risks and obtain candidate key sections. Simultaneously, it adaptively optimizes the reward function and its weight parameters, and uses Monte Carlo tree search to simulate fault scenarios and calculate the risk index. Finally, it selects key sections and iteratively optimizes the model parameters online. This invention solves the problems of traditional methods, such as difficulty in handling multi-source dynamic data, lack of complex fault scenario simulation, and inability to dynamically balance multi-dimensional optimization objectives. It can accurately characterize the dynamic risk features of the power grid, improve the accuracy and real-time performance of key section selection, and provide effective support for the safe and stable operation of the power grid.

[0200] Those skilled in the art will recognize that the units of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0201] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical functional division. In actual implementation, there may be other division methods, such as multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored.

[0202] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0203] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of this application.

Claims

1. A method for screening key sections of a power grid based on reinforcement learning, characterized in that, include: S1: Collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural networks; S2: Based on the dynamic state space of the power grid and a deep Q-network, risk assessment is performed on each region of the power grid to obtain a set of candidate critical sections. S3: Adaptively optimize the reward function and its weight parameters; S4: For each candidate section in the candidate critical section set, calculate the risk index of the candidate section after simulating the power flow transfer under different fault scenarios F through MCTS. The candidate sections with risk indices exceeding the threshold are used as the final critical sections to form the critical section set. S5: Online iterative optimization of deep Q-network model parameters.

2. The power grid critical section screening method based on reinforcement learning according to claim 1, characterized in that, In step S1, the process of collecting multi-source data and constructing the dynamic state space of the power grid based on a graph convolutional neural network includes the following steps: S11: Collect raw data from multiple sources, including real-time power flow data, power grid topology parameters, equipment status data, and new energy data; S12: Calculate the line load rate Cr based on power flow data. ij The formula is as follows: In the above formula, P ij Active power; This represents the thermal stability limit of the line. S13: Generate the power grid node embedding matrix X using a graph convolutional neural network based on multi-source raw data. GCN The formula is as follows: In the above formula, It is an adjacency matrix with self-loops; for H is the degree matrix; H is the input feature matrix; W is the trainable weight matrix; S14: Generate the graph embedding vector z for each partition using an autoencoder. p As a dynamic state-space representation of the power grid, its formula is as follows: z p =Autoencoder([Cr ij ,X GCN ,P renew ,σ renew ]); In the above formula, P renew Forecast values ​​of wind / solar power output; σ renew Wind power / solar power output volatility.

3. The power grid critical section screening method based on reinforcement learning according to claim 1, characterized in that, In step S2, the step of performing risk assessment on each region of the power grid based on the dynamic state space of the power grid and a deep Q-network to obtain a set of candidate critical sections includes the following steps: S21: Calculate the action value function for each partition based on a deep Q-network, and select the top K partitions by action value as high-risk partitions; the action value function formula is as follows: In the above formula, s represents the current power grid state, and z represents the graph embedding vector of each partition. p ; 'a' represents the action, and 'a' represents the value. p θ represents the filtering action performed on partition p; θ represents the reinforcement learning model parameters; γ∈[0,1] represents the discount factor; R t+1 The immediate reward after performing action a; S22: Calculate the comprehensive risk index for the lines within each high-risk zone, using the following formula: In the above formula, w1 and w2 are weights; G k-l This is the distribution factor matrix of line interruption, that is, the power flow transfer factor of line l interruption on line k; P represents the change in power flow after the line is disconnected. l Here are the active power parameters for line l; S23: Use a greedy strategy to filter RI for each high-risk partition. ij Above the threshold T RI The route combinations generate M candidate sections D; the candidate sections from all high-risk zones are then aggregated to form a set D of candidate key sections. candidate : D candidate ={D1,D2,…,D n }。 4. The power grid critical section screening method based on reinforcement learning according to claim 1, characterized in that, In step S3, the adaptive optimization of the reward function and its weight parameters includes the following steps: S31: Adaptively adjust the weights of each indicator in the reward function according to the real-time operation scenario of the power grid; S32: Calculate the penalty term P using the following formula: Where, l∈D candidate Let l represent candidate critical section D. candidate For any line within the range D, the summation range is D. candidate Includes all lines; max(0,Cr ij -0.9) means that the calculation only applies to overloaded lines; the output is 0 when there is no overload. S33: Optimize the reward function R j The formula is as follows: R j =α·S m +β·RED p +γ·ACC c +P; Where α is the safety margin weight; β is the absorption rate weight; γ is the accuracy weight; S m RED represents the average safety margin of the cross section. p For the renewable energy consumption rate of the zone p; ACC c This refers to the accuracy of cross-section identification.

5. The power grid critical section screening method based on reinforcement learning according to claim 4, characterized in that, The formula for calculating the average safety margin of the cross-section is as follows: Where N is the total number of lines within the cross-section; Let be the load rate of line ij; This indicates the safety margin of a single line.

6. The power grid critical section screening method based on reinforcement learning according to claim 4, characterized in that, In step S4, for each candidate section in the candidate critical section set, after simulating power flow transfer under different fault scenarios F using MCTS, the risk index of the candidate section is calculated, and candidate sections with risk indices exceeding a threshold are selected as the final critical sections to form the critical section set. This includes the following steps: S41: For each section D, power flow transfer under different fault scenarios F is simulated using MCTS to generate S. z The randomized simulation path and the obtained line load rate The number of times S exceeds the preset threshold c ; S42: Calculate the line overload probability P for each section D. overload The formula is as follows: Among them, S c Line load rate The number of times the preset threshold is exceeded, S z P represents the total number of times the random path is simulated. overload This represents the probability of line overload. S43: Calculate voltage stability margin V margin The formula is as above: Among them, |U i | represents the actual voltage amplitude at node i in the power grid. V is the rated voltage of node i in the power grid; margin For voltage stability margin; S44: Calculate the renewable energy curtailment rate (RED) using the following formula: Among them, P curtail P represents the amount of renewable energy power that is abandoned in the power grid. renew To contribute actual power to new energy sources in the power grid; S45: Risk index for generating candidate section D, formula as follows: RI D =γ·P overload +α·(1-V margin )+β·RED; S46: Select candidate sections whose risk index exceeds the threshold as the final critical sections to form the critical section set D. key : D key ={D|RI D ≥T RI-final }; Among them, T RI-final This is the threshold for the risk index.

7. The power grid critical section screening method based on reinforcement learning according to claim 3, characterized in that, In step S5, the online iterative optimization of the deep Q-network model parameters θ includes the following steps: S51: Add real-time fault data to the experience replay pool and update the sample distribution; S52: Employs a near-end policy optimization algorithm, randomly sampling batch data from the replay pool, and calculating the policy gradient. In the above formula, θ represents the gradient of the policy network parameter θ; E[·] is the mathematical expectation operator; (s,a,r)~Replay indicates that the sample (s,a,r) is randomly sampled from the experience replay pool Replay; in the sample (s,a,r), s represents the current operating state of the power grid, a represents the agent's selection decision for the candidate section, and r is the reward; logπ θ (a|s) represents the policy network π. θ The logarithmic probability of taking action a in state s; R j The optimized reward function; γ is the discount factor; max a′ Q θ′ In (s′,a′), s′ represents the next state, and a′ represents a possible future action; Q θ′ (s′,a′) is the state-action value function estimated by the target network; max a′ To obtain the maximum Q value among all possible actions; Q θ (s,a) represents the state-action value function estimated by the current Q-network; θ represents the current policy network. The parameters of the network; θ′ are the parameters of the target network; S53: Update the reinforcement learning model parameters θ in step S21 using policy gradients for the next round of cross-section selection.

8. A power grid critical section screening system based on reinforcement learning, characterized in that, The method for screening critical power grid sections based on reinforcement learning, as described in any one of claims 1 to 7, includes: The spatial module is used to collect multi-source data and construct the dynamic state space of the power grid based on graph convolutional neural networks; The candidate module is used to perform risk assessment on each region of the power grid based on the dynamic state space of the power grid and a deep Q-network to obtain a set of candidate critical sections. The adjustment module is used for adaptive optimization of the reward function and its weight parameters; The screening module is used to calculate the risk index of each candidate section in the candidate critical section set after simulating power flow transfer under different fault scenarios F through MCTS simulation. Candidate sections with risk indices exceeding the threshold are selected as the final critical sections to form the critical section set. The optimization module is used for online iterative optimization of the parameters of the deep Q-network model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the power grid critical section screening method according to any one of claims 1 to 7.

10. A processor, characterized in that, The processor is used to run a program, wherein the program executes the power grid critical section screening method based on reinforcement learning as described in any one of claims 1 to 7.

Images