An emergency control strategy hierarchical efficient learning method for overloaded lines

Abstract

This invention provides a hierarchical and efficient learning method for emergency control strategies of overloaded lines, comprising: searching for overload operation modes within a future time period; using an initial policy network for decision-making to filter out overload operation modes that fail to be handled; performing two-stage reinforcement learning on the failed overload operation modes, wherein the first stage of reinforcement learning learns overload operation modes with typical characteristics, and the second stage of reinforcement learning learns overload operation modes that still fail to be handled; designing and learning the structure of a selection network based on all policy networks learned in the two-stage reinforcement learning and the initial policy network; and concatenating the selection network with the learned policy networks and the initial policy network to form a multi-strategy hybrid emergency control strategy model. This invention can achieve hierarchical and efficient learning of emergency control strategies for overloaded lines and can efficiently learn a multi-strategy hybrid emergency control model for a massive number of overload operation modes.

Description

Technical Field

[0001] This invention belongs to the field of emergency control in power systems, and particularly relates to a hierarchical and efficient learning method for emergency control strategies for overloaded lines. Background Technology

[0002] In the field of power system emergency control, the continuous integration of new energy sources leads to uncertainty in grid overload operation modes. For a given time period, there may be numerous overload operation modes and potential risk overload operation modes. Therefore, emergency control strategies for risky overload operation modes need to cover a wider range of overload operation modes, placing high demands on strategy generation efficiency. With the development of artificial intelligence technology, reinforcement learning has been gradually applied to the generation of emergency control strategies. It can learn emergency control strategies based on pre-searched potential risk overload operation modes for that time period.

[0003] When learning a large number of overload operation modes, the time required to learn emergency control strategies is significant, making learning efficiency crucial. However, most current reinforcement learning methods have limited scope of emergency control strategies when faced with a large number of overload operation modes, making it difficult to handle massive overload operation scenarios simultaneously, and their learning efficiency is low. Therefore, a more efficient emergency control strategy learning method needs to be designed specifically for this purpose. Summary of the Invention

[0004] To address at least one of the problems existing in the prior art, this invention provides a hierarchical and efficient learning method for emergency control strategies of overloaded circuits, which can handle a large number of overload operation scenarios and has high learning efficiency.

[0005] To achieve the objectives of this invention, a hierarchical and efficient learning method for emergency control strategies of overloaded lines is provided, comprising the following steps:

[0006] S1: Search for overload operation modes within a future time period, denoted as D. S A policy network π0 is used to make decisions on overload operation modes, and the overload operation modes that fail to be handled are selected and denoted as D'. S ;

[0007] S2: For D' S The system operates under overload conditions, and performs the first stage of reinforcement learning to learn the policy network. The first stage mainly learns samples with typical characteristics.

[0008] S3: Records the policy network learned in the first stage of reinforcement learning in S2. Assume that K policies were learned, denoted as π1 to π1. K For a network with K policies in D' SThe decision is made during the process, identifying the overload operation mode that still cannot be handled successfully, denoted as "D". S ;

[0009] S4: Regarding D” S The overload operation mode in the middle is used for the second stage of reinforcement learning to learn the policy network. This stage mainly learns the overload operation mode that still fails.

[0010] S5: Records the policy network learned in the second stage of reinforcement learning in S4. Assume that L policies are learned, denoted as π. K +1~π K +L;

[0011] S6: Analyze the above 1+K+L policy networks π0~π K+L In the overload operation mode set D S Based on the processing performance of each overload operation mode, the strategy network model with the best processing performance is selected to form the learning sample set D of the selection network. E ;

[0012] S7: Based on the above 1+K+L policy networks, design the structure of the selection network model, and based on the learning sample set D E Learn to choose the right network, and you will learn to choose the right network.

[0013] S8: Combine the learned choice network with a 1+K+L policy network π0~π K+L By connecting them, a multi-strategy hybrid emergency control model is obtained.

[0014] Furthermore, the specific methods for making decisions are as follows:

[0015] The input data of the strategy network is extracted from the overload operation mode. The input data is data reflecting the grid operation status, including node voltage V and node phase. The active power P injected at each node, the reactive power Q injected at each node, and the adjacency matrix A of the power grid are concatenated to form the overload operation mode state s={X,A}.

[0016] Input the state s of the overload operation mode into the policy network π0 to obtain the control scheme under the overload operation mode;

[0017] The overload operation mode of the power grid is modified according to the control scheme, and the power flow calculation and time-domain simulation of the overload operation mode are re-performed to obtain the adjustment effect. The adjustment effect is whether the power grid still experiences line overload or node voltage exceeding the limit after the decision. If there are overloaded lines in the adjustment effect of the overload operation mode, the overload operation mode is considered to be a failed overload operation mode.

[0018] Furthermore, in the first phase of reinforcement learning, firstly based on D' S The system performs similar sample analysis to generate samples for the first stage of reinforcement learning, and then uses deep reinforcement learning methods to learn the policy network.

[0019] Furthermore, the specific process of the similar sample analysis is as follows: analyze the similarity of overload operation modes according to the similar sample classification principle, classify the overload operation modes that satisfy all similar sample classification principles into one class of similar overload operation modes, and select the overload operation mode with the most severe risk from one class of similar overload operation modes as the typical overload operation mode of the first stage reinforcement learning sample.

[0020] Furthermore, the classification principle for similar samples is that they have the same topology, the same line commissioning / shutdown status, and the same overloaded lines.

[0021] Furthermore, the second stage of reinforcement learning addresses the failure of the policy network learned in the first stage to process the "D" error. S We will learn about the overload operation mode in the system. The specific process is as follows:

[0022] S41: Initialize the data, initialize the number of learning iterations for the second stage of reinforcement learning to i, i = 0;

[0023] S42: From D” S A predetermined number of samples are selected as the samples for the i-th reinforcement learning iteration, and reinforcement learning is performed to learn a policy network denoted as π. i ;

[0024] S43: Using the policy network π i "To D" S The decision is made regarding the overload operation mode in the policy network π. i The overload operation mode that can be handled ranges from D” S Remove from the middle;

[0025] S44: Determine D” S If there is an unhandled overload operation mode, proceed to S42; otherwise, proceed to S45.

[0026] S45: Learning ends. In this stage, L policy networks will be learned and output.

[0027] Furthermore, the evaluation of the processing effect considers power flow convergence, node voltage overrun, and line overload. The evaluation process is as follows: under the conditions of power flow convergence after decision-making, no node voltage overrun, and no line overload, the further the current of the overloaded line before decision-making is from the line limit, the better the processing effect.

[0028] Furthermore, the structure of the selection network model is based on a neural network and consists of three parts: a feature extraction part, a fitting part, and a selector part. The feature extraction part includes alternating Transformer convolutional layers and batch normalization layers.

[0029] Furthermore, the fitting part includes a fully connected neural network and a sigmoid function transformation. The fully connected neural network is used to fit the data, and the sigmoid function transformation is used to transform the data to the range of 0-1. The output of the fitting part is a selection vector d, which is a 1+K+L dimensional vector. The value of this vector is a 0-1 vector, where an element value of 0 represents that the selector behind the vector is not activated, and an element value of 1 represents that the selector behind the vector is activated.

[0030] Furthermore, in the selector section, the number of selectors is the same as the dimension of the selection vector, which is 1+K+L. The input of each selector is the corresponding element value in the selection vector d. When the selector input is 1, the corresponding strategy network behind the selector is activated, so that it reads the power grid operating status and performs the calculation of the strategy network to obtain the control scheme. When the selector input is 0, the corresponding strategy network behind the selector is not activated.

[0031] Compared with the prior art, the present invention can achieve at least the following beneficial effects:

[0032] This invention provides a hierarchical and efficient learning method for emergency control strategies of overloaded lines. First, it searches for overload operation modes within a future time period, using an initial policy network to make decisions and filter out overload operation modes that fail to be handled. Then, it performs two-stage reinforcement learning on these failed overload operation modes. The first stage learns overload operation modes with typical characteristics, and the second stage learns overload operation modes that still fail to be handled. Next, based on all policy networks learned in the two-stage reinforcement learning and the initial policy network, it designs and learns the structure of a selection network. Finally, it concatenates the selection network with the learned policy networks and the initial policy network to form a multi-strategy hybrid emergency control strategy model. This invention can achieve hierarchical and efficient learning of emergency control strategies for overloaded lines and can efficiently learn a multi-strategy hybrid emergency control model for a massive number of overload operation modes. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the hierarchical and efficient learning method for emergency control strategies for overloaded lines in an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of the selected network structure in an embodiment of the present invention.

[0035] Figure 3 This is the IEEE-39 node topology diagram. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. These embodiments were obtained by those skilled in the art based on the embodiments of the present invention without creative effort.

[0037] Please see Figure 1 The present invention provides a hierarchical and efficient learning method for emergency control strategies of overloaded lines, comprising the following steps:

[0038] S1: Search for overload operation modes within a future time period, denoted as D. S A policy network π0 is used to make decisions on the set of overload operation modes, and the overload operation modes that fail to be handled are selected and denoted as D'. S .

[0039] S2: Set D' for handling overload operation modes that fail. S The system operates under overload conditions, and performs the first stage of reinforcement learning to learn a policy network. This first stage mainly learns samples with typical characteristics.

[0040] S3: Records the policy network learned in the first stage of reinforcement learning in S2. Assume that K policies were learned, denoted as π1 to π1. K For K policy networks, consider the set D' of failed overload operation modes. S The decision is made by identifying the overload operation modes that still cannot be handled successfully, denoted as set D. S .

[0041] S4: For set D” S The overload operation mode in the middle is used for the second stage of reinforcement learning to learn the policy network. This second stage mainly learns the overload operation mode that still fails, which is the backup stage.

[0042] S5: Records the policy network learned in the second stage of reinforcement learning in S4. Assume that L policies are learned, denoted as π. K +1~π K +L.

[0043] S6: Analyze the above 1+K+L policy networks π0~π K+L In the overload operation mode set D SBased on the processing performance of each overload operation mode, the strategy network with the best processing performance is selected, and a learning sample set D for the selection network is formed. E .

[0044] S7: Based on the above 1+K+L policy networks π0~π K+L Design and select the structure of the network model, and based on the learning sample set D E Learn to select networks, and obtain a well-learned selection network, denoted as ζ.

[0045] S8: Connect the selection network ζ with 1+K+L policy networks π0~π K+L By connecting these components, a multi-strategy hybrid emergency control strategy model is obtained. Finally, the multi-strategy hybrid emergency control strategy model is output.

[0046] In step S1, the policy network π0 is the policy network that has been learned before this time period. The policy network π0 is used to apply the overload operation mode set D. S The specific decision-making method for an overload operation mode is as follows: First, the input data of the strategy network is extracted from the overload operation mode. This input data reflects the power grid operating status, including node voltage V and node phase. The active power injected at each node (P), the reactive power injected at each node (Q), and the adjacency matrix A of the power grid are concatenated to form the overload operation state s = {X, A}. Then, the state s of the overload operation mode is input into the policy network π0 to obtain the corresponding control scheme under the overload operation mode, denoted as a. Next, the overload operation mode of the power grid is modified according to the control scheme a, and power flow calculation and time-domain simulation are performed again for this overload operation mode to obtain the adjustment effect. This adjustment effect is whether the power grid still experiences line overload or node voltage exceeding the limit after the decision. Therefore, assume that the set of overload operation modes D S There is N S Given several overload operation modes, after making decisions on all overload operation modes using the above decision-making method, N will be obtained. S The adjustment effect of each overload operation mode is evaluated. If an overloaded line exists in the adjustment effect of an overload operation mode, then the overload operation mode is considered a failed overload operation mode, and all failed overload operation modes are denoted as set D'. S .

[0047] The modification of the power grid's overload operation mode based on control scheme a varies depending on the controlled object. Generally, control scheme a includes three measures: generator power regulation measures, load shedding measures, and generator tripping measures. The generator power regulation measures correspond to a. G The power regulation expression for the m-th generator node under this measure is shown in equation (1), that is, according to the maximum adjustable power P of generator node m.m,max With a G,m The product of these factors is used for adjustment; the load shedding measure corresponds to a. L The power regulation expression for the d-th load node under this measure is shown in equations (2) and (3); the corresponding tripping measure is a. GC The power regulation expression for the m-th generator node under this measure is shown in equation (4), that is, when a GC When P is greater than or equal to 0.5, G,m Set it to 0, otherwise leave it unchanged.

[0048] P G,m =a G,m ·P m,max (1)

[0049] P′ L,d =a L,d ·(P L,d -P L,d,min (2)

[0050] Q′ L,d =a L,d ·(Q L,d -Q L,d,min (3)

[0051]

[0052] Among them, P G,m a represents the power of the m-th generator node; G,m P′ represents the adjustment amount of the power regulation measure for the m-th generator node. L,d a represents the active power after the d-th load node sheds its load; L,d P represents the load shedding amount for the d-th load node; L,d P represents the active power of the d-th load node; L,d,min Q′ represents the minimum active power of the d-th load node. L,d Q represents the reactive power after the d-th load node disconnects its load; L,d The reactive power of the d-th load node; Q L,d,min a represents the minimum reactive power of the d-th load node; GC,m This represents the probability of the m-th generator node taking a generator-switching action.

[0053] In step S2, firstly, based on the set D' of overload operation modes that failed to be processed... SThe overload operation modes are used to generate samples for the first stage of reinforcement learning. Since this step primarily learns samples with typical features, it's necessary to analyze the similarity of overload operation modes, grouping similar overload operation modes into one class. Then, from this class of similar overload operation modes, the most dangerous overload operation mode is selected as a typical sample. The aim is to enable the learned policy network to handle other similar overload operation modes through learning from a small number of typical overload operation modes. The classification principle for similar samples is as follows:

[0054] 1) Same topology and same line on / off status. The reason for this selection is that when the topology and line on / off status are inconsistent, even with the same source load distribution, the power flow will differ significantly.

[0055] 2) Identical overload circuits. The reason for this selection is that the overload circuit is the control target, and when the overload circuits are the same in overload operation mode, their control targets are the same.

[0056] In one embodiment of the present invention, when all the above principles are satisfied simultaneously, the set D' of overload operation modes that fail to process can be determined. S Risk overload operation modes with the same topology, the same line commissioning / shutdown status, and the same overloaded lines are considered similar overload operation modes. Then, the overload operation mode with the most severe overload is selected from the similar overload operation modes as the typical overload operation mode of the first stage reinforcement learning sample, denoted as D'. S1 After selecting typical overload operation modes, common deep reinforcement learning methods are used to efficiently learn K policy networks. It's important to note that multiple policy networks are needed because their processing power is limited. To ensure that all policy networks can handle all overload operation modes, multiple policy networks need to be used in combination. The number of policy networks learned is determined by D'. S1 The number of overload operation modes is determined by D' S1 If there are fewer overload operation modes in D', then fewer policy networks may be learned; if D' S1 If there are many overload operation modes, then there will be more policy networks learned.

[0057] In step S3, the K policy networks learned in S2 are recorded. Assuming the first policy network is numbered 1 and the last policy network is numbered K, then the K policy networks are denoted as π1 to π2. K For K policy networks, consider the set D' of failed overload operation modes. S The decision-making process is the same as in step S1, and will not be repeated here. After the decision-making process, identify the K policy networks π1 to π1 that are not affected by the policies learned in S2. K In handling the set D' of failed overload operation modes SThe overload operation modes that cannot be handled by the system are denoted as set D. S .

[0058] Step S4 mainly targets set D” S The second stage of reinforcement learning is performed on the overload operation mode. This stage mainly focuses on learning overload operation modes that still fail to be handled, serving as a safety net. This stage employs common deep reinforcement learning methods to efficiently learn L policy networks. The specific process is as follows:

[0059] S41: Initialize the data, initialize the number of learning iterations for the second stage of reinforcement learning to i, i = 0.

[0060] S42: From set D” S A certain number of samples are selected as samples for the i-th reinforcement learning iteration, and reinforcement learning is performed to learn a policy network denoted as π. i In this embodiment of the invention, the reinforcement learning method employs conventional reinforcement learning techniques. The aforementioned number of samples needs to be determined based on the scenario requirements, and is generally set through a trial-and-error approach.

[0061] S43: Using the policy network π i For set D” S The decision-making process for overload operation mode is the same as that in S1, and the policy network π is used. i The overload operation modes that can be handled are from set D” S Remove it from the list.

[0062] S44: Determine set D” S If there is an unhandled overload operation mode, jump to S42; otherwise, jump to S45.

[0063] S45: Learning ends. In this stage, L policy networks will be learned and output.

[0064] Similarly, this step requires learning multiple policy networks because policy networks have limited processing power. To enable a policy network to fully handle all overload operation modes, it needs to be achieved by stacking multiple policy networks. The number of policy networks learned is determined by the set D. S The number of overload operation modes in the set D is determined by the set D. S If there are fewer overloaded operating modes in the set D, then fewer policy networks may be learned; if the set D” S If there are many overload operation modes, then there will be more policy networks learned.

[0065] In step S5, record the L policy networks learned in S4, where the first policy network is numbered K+1 and the last policy network is numbered K+L. These L policy networks are then denoted as π. K+1 ~π K+L .

[0066] From the above steps, a total of 1+K+L policy networks are formed, including the policy network π0 that was learned before the process started, and the K policy networks π1 to π2 learned in step S2 through the first stage of reinforcement learning. K And the L policy networks π learned through the second stage of reinforcement learning in step S4. K+1 ~π K+L In step S6, the above 1+K+L policy networks are placed in the overload operation mode set D. S Decisions are made in this way, and the overload operation mode set D of each policy network is analyzed. S The processing effects of various overload operation modes are evaluated, and the policy network model with the best processing effect among all overload operation modes is selected. Assume that the set of overload operation modes is D. S The state of the k-th overload operation mode is s k If the policy network with the best processing effect is numbered j, then a mapping pair {s} is formed. k The mapping pairs of all overload operation modes will form the learning sample set D of the selection network. E .

[0067] In one embodiment of the present invention, the evaluation of the above-mentioned processing effect mainly considers power flow convergence, node voltage overrun, and line overload. The evaluation process is as follows: under the conditions that the power flow converges after the decision, the node voltage does not exceed the limit, and the line is not overloaded, the further the current of the overloaded line before the decision is from the line limit, the better the processing effect.

[0068] In step S7, the first step is to design and select the structure of the network ζ.

[0069] Choose the network ζ structure as follows: Figure 2As shown, it comprises three parts: a feature extraction part, a fitting part, and a selector part. The feature extraction part uses a Transformer-based graph neural network, which is highly capable of extracting grid topology features. The activation function is the ReLU function. The feature extraction part includes Transformer Convolution (TC) layers and Batch Normalization (BN) layers. The TC layer is used to extract graph features, and the BN layer is used to adjust the data distribution to make it more suitable for training. The number of each layer is determined based on specific business requirements. Currently, the mainstream method for determining the number of layers in the power industry is trial and error, i.e., trying different configurations and selecting the number of layers with better performance. In one embodiment of the invention, three TC layers and four BN layers are used, with the BN and TC layers alternating. Other numbers can be used in other embodiments.

[0070] The fitting part includes a fully connected network (FCN) and a sigmoid function transformation. The FCN layer is used to fit the data, and the sigmoid function transformation is used to transform the data to the range of 0-1. The final output of the fitting part is the selection vector d. The dimension of this vector is the number of policy networks in step S6, which is a 1+K+L dimensional vector. The value of this vector is a 0-1 vector. An element value of 0 means that the selector behind the vector is not activated, and an element value of 1 means that the selector behind the vector is activated.

[0071] In the selector section, the number of selectors is the same as the dimension of the selection vector d, which is 1+K+L. The input of each selector is the corresponding element value in the selection vector d. When the input of a selector is 1, the corresponding strategy network behind that selector is activated, so that it reads the power grid operating state s and executes the calculation of the strategy network to obtain the control scheme a; when the input of a selector is 0, the corresponding strategy network behind that selector is not activated.

[0072] After designing the structure of the selection network ζ as described above, based on the learning sample set D... E The learning process involves selecting a network and obtaining a well-learned network. The learning method employed is a conventional graph deep learning approach.

[0073] In step S8, the selection network ζ is compared with 1+K+L policy networks π0~π K+L Connected, such as Figure 2 As shown, a multi-strategy hybrid emergency control strategy model is obtained. Finally, the multi-strategy hybrid emergency control strategy model is output.

[0074] In one embodiment of the present invention, the above method is verified in the IEEE-39 node system.

[0075] like Figure 3 The diagram shows the topology of a 39-node system. Given a set of overload operation modes for a certain time period, denoted as D0, the overload operation modes in set D0 are processed by line disconnection, generator disconnection, local load fluctuations, etc., to generate a massive number of overload operation modes containing line overload, denoted as the overload operation mode set D. S Given a policy network π0, which represents the policy learned prior to this time period, the aforementioned methods and steps are applied to the overload operation mode set D. S Emergency control strategies are learned on the basis of the model, resulting in a multi-strategy hybrid emergency control strategy model. The model is then analyzed in the overload operation mode set D. S The effectiveness of the proposed method is verified by examining its impact on decision-making.

[0076] For the 39-node system, a massive set of overload operation modes D, including line overload, is generated through line disconnection, generator disconnection, and local load fluctuations. S Overload operation mode set D S There are a total of 2036 overload operation modes.

[0077] The method proposed in the aforementioned embodiments is used in the overload operation mode set D. S The model employs hierarchical and efficient learning of emergency control strategies to obtain a multi-strategy hybrid emergency control strategy model. The strategy composition of the model and the decision data of each strategy are shown in Table 1.

[0078] As shown in Table 1, in the initial policy network decision-making in S1, the policy network π0 decision-making method can solve 1775 out of 2036 overload operation modes, achieving a success rate of 87.2%. Only 261 overload operation modes need to be relearned. Let D be the set of these 261 overload operation modes. S .

[0079] Then, the set of overload operation modes D S The first stage of reinforcement learning is performed on 261 unprocessed overload operation modes. Based on the classification principle in step S2, 38 overload operation modes with typical characteristics are selected from the 261 for further learning. The number of learned overload operation modes is denoted as N. L N L =38. After learning, a policy network is learned, i.e., K=1, and the learned policy network is denoted as π1. From the table, it can be seen that policy network π1 can handle 205 overload operation modes with a success rate of 78.5%. Let N be the number of overload operation modes that policy network π1 can handle. P N P =205. Further set p PL The metric is used to measure learning effectiveness, and it is N.P With N L The ratio of p to 1 reflects the generalization ability of the learned network; the larger the value, the stronger the generalization ability. As can be seen from the table... PL The value of 5.39 indicates that the learned π1 has good generalization ability.

[0080] After step S3, identify the overload operation mode D that still fails to resolve the issue. S =56, for 56 overload operation modes, perform the second stage of reinforcement learning in step S4, N L =56. After learning, a policy network is learned, i.e., L=1, and the learned policy network is denoted as π2. From the table, it can be seen that π2 can handle D”. S The system operates under 56 overload conditions with a 100% success rate.

[0081] Finally, after S5-S8, a multi-strategy hybrid emergency control strategy model π0+π1+π2 can be obtained, which is then applied to the overload operation mode set D. S Decision-making is performed based on the multi-strategy hybrid emergency control strategy model π0+π1+π2, which can handle the overload operation mode set D, as shown in Table 1. S The effectiveness of the proposed method was verified by testing all overload operation modes.

[0082] Table 1. Composition and Related Decision Statistics of the Multi-Strategy Hybrid Emergency Control Model

[0083]

[0084] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hierarchical and efficient learning method for emergency control strategies of overloaded lines, characterized in that, Includes the following steps: S1: Search for overload operation modes within a future time period, denoted as D. S Policy network π 0 makes a decision on the overload operation mode, and filters out the overload operation modes that fail to handle the overload, denoted as D'. S ; S2: For D' S The overload operation mode in the middle is used to carry out the first stage of reinforcement learning to learn the policy network. The first stage is used to learn samples with typical features. S3: Records the policy network learned in the first stage of reinforcement learning in S2. Assume K policies are learned, denoted as follows: π 1~ π K For a network with K policies in D' S During the decision-making process, identify the overload operation mode that still cannot be successfully handled, denoted as D''. S ; S4: Targeting D'' S The overload operation mode in the middle is used to carry out the second stage of reinforcement learning to learn the policy network. This second stage is used to learn the overload operation mode that still fails to handle the overload operation mode. S5: Records the policy network learned in the second stage of reinforcement learning in S4. Assume L networks are learned, denoted as follows: π K+1 ~ π K+L ; S6: Analyze the above 1+K+L policy networks π 0~ π K+L In the overload operation mode set D S Based on the processing performance of each overload operation mode, the strategy network model with the best processing performance is selected to form the learning sample set D of the selection network. E ; S7: Based on the above 1+K+L policy networks, design the structure of the selection network model, and based on the learning sample set D E Learn to choose the right network, and you will learn to choose the right network. S8: Combine the learned choice network with a 1+K+L policy network π 0~ π K+L By connecting them, a multi-strategy hybrid emergency control model is obtained.

2. The hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 1, characterized in that, The specific methods for making decisions are as follows: The input data of the strategy network is extracted from the overload operation mode. The input data is data reflecting the grid operating status, including node voltage. V Node phase φ Node injection active P and node injection reactive power Q and the adjacency matrix of the power grid A The above data is then combined to form the overload operation mode. s ={ X , A }, X ={ V , φ , P , Q }; To policy network π Enter the status of this overload operation mode in field 0. s Thus, the control scheme under this overload operation mode is obtained; The overload operation mode of the power grid is modified according to the control scheme, and the power flow calculation and time-domain simulation of the overload operation mode are re-performed to obtain the adjustment effect. The adjustment effect is whether the power grid still experiences line overload or node voltage exceeding the limit after the decision. If there are overloaded lines in the adjustment effect of the overload operation mode, the overload operation mode is considered to be a failed overload operation mode.

3. The hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 1, characterized in that, In the first phase of reinforcement learning, firstly according to D' S The system performs similar sample analysis to generate samples for the first stage of reinforcement learning, and then uses deep reinforcement learning methods to learn the policy network.

4. The hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 3, characterized in that, The specific process of the similar sample analysis is as follows: analyze the similarity of overload operation modes according to the similar sample classification principle, classify the overload operation modes that meet all similar sample classification principles into one class of similar overload operation modes, and select the overload operation mode with the most severe risk from one class of similar overload operation modes as the typical overload operation mode of the first stage reinforcement learning sample.

5. A hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 4, characterized in that, The classification principle for similar samples is that they have the same topology, the same line commissioning / shutdown status, and the same overloaded lines.

6. The hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 1, characterized in that, The second stage of reinforcement learning addresses the issue of D'', which the policy network learned in the first stage of reinforcement learning still fails to process. S We will learn about the overload operation mode in the system. The specific process is as follows: S41: Initialize data, initialize the number of learning iterations for the second stage of reinforcement learning. i , i =0; S42: From D'' S Select a predetermined number of samples as the first i The samples for reinforcement learning are used, and reinforcement learning is performed to learn a policy network, denoted as... π i ; S43: Using Policy Networks π i To D'' S The decision is made regarding the overload operation mode in the policy network. π i The overload operation mode that can be handled is from D'' S Remove from the middle; S44: Determine D'' S If there are any unhandled overload operation modes, then jump to S42; If not, proceed to S45; S45: Learning session complete.

7. A hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 1, characterized in that, The evaluation of the processing effect considers power flow convergence, node voltage overrun, and line overload. The evaluation process is as follows: under the conditions of power flow convergence after decision-making, no node voltage overrun, and no line overload, the further the current of the overloaded line before decision-making is from the line limit, the better the processing effect.

8. A hierarchical and efficient learning method for emergency control strategies of overloaded lines according to any one of claims 1-7, characterized in that, The selected network model is based on a neural network and consists of three parts: a feature extraction part, a fitting part, and a selector part. The feature extraction part includes alternating Transformer convolutional layers and batch normalization layers.

9. A hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 8, characterized in that, The fitting component includes a fully connected neural network and a sigmoid function transformation. The fully connected neural network is used to fit the data, and the sigmoid function transformation is used to transform the data to the range of 0-1. The output of the fitting component is a selection vector. d Choose vector d The dimension is a 1+K+L dimensional vector, where the value of the vector is a 0-1 vector. An element value of 0 means that the selector behind the vector is not activated, and an element value of 1 means that the selector behind the vector is activated.

10. A hierarchical and efficient learning method for emergency control strategies of overloaded lines according to claim 9, characterized in that, In the selector section, the number of selectors is the same as the dimension of the selection vector, which is 1+K+L. The input of each selector is the selection vector. d When the selector input is 1, the corresponding strategy network behind the selector is activated, enabling it to read the power grid operating status and perform calculations to obtain the control scheme; when the selector input is 0, the corresponding strategy network behind the selector is not activated.

Images