Novel power system online pre-decision method and system based on knowledge fusion

Abstract

The invention provides a novel power system online pre-decision method and system based on knowledge fusion, and the method comprises the steps: obtaining the first operation information of a power system in a first decision period; if it is judged that the first operation information meets a first decision-making condition, the first operation information is input to a preset upper-layer intelligent agent, so that the upper-layer intelligent agent outputs a first pre-decision-making instruction according to the first operation information, and the first pre-decision-making instruction comprises the power adjustment amount and the load removal amount of each unit; in a second decision period, obtaining second operation information of the power system; if it is judged that the power system is in a transient instability stage according to the second operation information, inputting instability characteristics of the power system to a preset lower-layer agent, so that the lower-layer agent outputs a second pre-decision instruction according to the second operation information, the second pre-decision instruction comprises a plurality of generator tripping instructions, so that the operation stability of the power system is improved.

Description

Technical Field

[0001] The present application relates to the field of power system control decision-making and artificial intelligence technology, and in particular to a novel power system online pre-decision-making method and system based on knowledge fusion. Background Art

[0002] The high proportion of new energy penetration and high proportion of power electronics access bring new challenges to the control decision-making problems of new power systems. The inertia of the system is reduced, the uncertainty of the operating state is increased, the predictability is reduced, the internal mechanism of the system stability has undergone profound changes, and the voltage, power angle and frequency of the power system have become increasingly prominent. In the environment of high proportion of renewable energy and high proportion of power electronics, how to quickly and accurately evaluate the stability of the power system and help operation and maintenance personnel to make correct decisions quickly is a difficult problem that needs to be solved in the development of new power systems. At present, the pre-decision of the power system can be divided into offline pre-decision and online pre-decision. Offline pre-decision is generally performed by power grid operation technical experts. They analyze the transient simulation results based on domain knowledge and operation experience, and obtain effective measures through continuous trial and error and adjustment. This control scheme is called "offline pre-decision, real-time matching". However, due to the large number of typical operating conditions and transient voltage instability faults in modern large-scale interconnected complex power systems, the manual offline pre-setting working mode has become very time-consuming and labor-intensive. The competitive access to the grid in modern power systems places increasing emphasis on competitive use of power systems, requiring dynamic correction of operating limits and control strategies according to online operating conditions, achieving safe, stable and efficient real-time tracking of the system, and forming a stable control decision-making framework of “online pre-decision-making and real-time matching”.

[0003] The existing technology establishes a physical model of the controlled object of the power system, analyzes its internal operating mechanism, forms an online pre-decision transient stability objective function, and realizes online adjustment of the unit power by optimizing the solution. However, in the context of energy transformation, the scale of new power systems continues to increase, the strong randomness of wind and solar power, and the tight coupling of energy networks have introduced a lot of uncertainty and unclear mechanisms into conventional power grid decision-making problems. At the same time, the increase in the dimension of control variables and the addition of nonlinear non-convex objective functions / constraints have also led to doubts about the real-time performance of existing pure mechanism decision-making technologies.

[0004] In recent years, the rapid development of artificial intelligence has provided a new approach to solving the above challenges. As a representative technology in the field of artificial intelligence, reinforcement learning can achieve efficient and optimal decision-making by continuously improving strategies through the interaction between intelligent agents and the environment to obtain the maximum expected reward. However, although artificial intelligence methods such as reinforcement learning do not require strict analysis of the internal mechanisms and mechanisms of the research object, they are based on a large amount of data, build relevant empirical models by analyzing the correlation between data, and mine the operating mode of problem features from the data. In practical applications, this causes problems such as weak decision-making interpretability and unclear mechanisms. Summary of the invention

[0005] In response to the above technical problems, the present application provides a new online pre-decision-making method and system for power systems based on knowledge fusion, which can realize online control of system transient stability and improve the stability of power system operation.

[0006] In a first aspect, the present application provides a novel online pre-decision-making method for a power system based on knowledge fusion, comprising:

[0007] In a first decision cycle, first operation information of the power system is obtained, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set of the power system at a current moment;

[0008] If it is determined that the first operation information satisfies a first decision condition, the first operation information is input into a preset upper-level intelligent agent, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, wherein the first pre-decision instruction includes a power adjustment amount and a load removal amount of each unit;

[0009] In a second decision cycle, second operation information of the power system is obtained, wherein the second operation information includes a generator terminal voltage, a network node voltage, and a power angle state of the power system at a current moment;

[0010] If it is determined according to the second operation information that the power system is in a transient instability stage, the instability characteristics of the power system are input into a preset lower-level intelligent agent, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operation information, wherein the second pre-decision instruction includes a shutdown instruction of a plurality of generators;

[0011] Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training.

[0012] The embodiment of the present application provides a novel online pre-decision method for power systems based on knowledge fusion, a multi-agent hierarchical Markov decision process based on deep reinforcement learning of knowledge fusion, training to obtain upper-level agents and lower-level agents, and using different agents to make decisions in different decision cycles, wherein the upper-level agent is responsible for deciding the power adjustment amount and load removal amount of each unit, and the lower-level agent is responsible for deciding the amount of each generator to be cut. The embodiment of the present application improves the pertinence of the decisions of each agent by making decisions on different actions in different periods, so that different agents only need to focus on their corresponding decision problems and system states, improves the computing efficiency and decision rationality of the agents, and enables the power system to respond to various operating conditions and transient voltage instability faults in a timely manner, thereby improving the stability of the power system operation.

[0013] Furthermore, the first decision cycle and the second decision cycle run in parallel, the time scale of the first decision cycle is in minutes, and the time scale of the second decision cycle is in seconds.

[0014] In the embodiment of the present application, a first decision cycle of minute level and a second decision cycle of second level are constructed, so that the upper-level intelligent agent corresponding to the first decision cycle is mainly responsible for the pre-decision of long time scale, mainly for prevention; while the lower-level intelligent agent corresponding to the lower-level second decision cycle is mainly responsible for the pre-decision of segment time scale to solve emergency situations, and realize the hierarchical processing of problems. When the first operation information meets the first decision condition, the intelligent agent is required to output the corresponding power adjustment amount and load removal amount according to the current system status, and then adjust the power and load of each unit of the power system. This process is not urgent, because the power system itself has a large power elasticity and load elasticity, so it can be adjusted accordingly through the pre-decision of long time scale, and the decision is made every few minutes. When the power system is in the transient instability stage, there may be emergency situations such as generator power angle swing and node voltage exceeding the rated range. It is necessary to start the emergency control mechanism, and the lower-level intelligent agent outputs the corresponding machine cutting action instruction set according to the current system status, and then cuts the machine for several generators in the power system, and quickly responds to solve the problem. Therefore, this process requires a short-time scale pre-decision to make corresponding adjustments, and the decision is made every few seconds. The embodiment of the present application divides the two decision cycles into minute level and second level, and performs them simultaneously, thereby achieving effective allocation of system resources, improving the operating efficiency of the intelligent body and the real-time nature of decision-making, and further improving the stability of power system operation.

[0015] Further, the determining whether the first operation information satisfies a first decision condition includes:

[0016] Determine a node voltage value and a load amount of the power system according to the first operation information;

[0017] If the node voltage value is greater than or equal to a first preset threshold or the load amount is greater than or equal to a second preset threshold, it is determined that the power system meets the first decision condition.

[0018] In an embodiment of the present application, it is further described how to determine whether the power system meets the first decision condition based on the first operating information. First, the current transient voltage and load of the power system are determined, and then the transient voltage and load are compared with the first preset threshold and the second preset threshold respectively. As long as one of the values ​​of the transient voltage and load exceeds the corresponding preset threshold, the first decision condition is met, and then the upper-level intelligent agent is controlled to make a decision.

[0019] In a possible implementation manner, the upper-layer agent outputs a first pre-decision instruction according to the first operation information, including:

[0020] Update the first system state space according to the first operation information to obtain the current first system state space;

[0021] Inputting the current first system state space into the upper-layer deep reinforcement learning network of the upper-layer agent, so that the upper-layer deep reinforcement learning network maps the current first system state space to the corresponding first action space;

[0022] Extracting the power adjustment amount and load shedding amount of each unit from the first action space;

[0023] The power adjustment amount and the load shedding amount are output as the first pre-decision instruction.

[0024] An embodiment of the present application provides a method for outputting a corresponding action matrix through an upper-level intelligent agent. First, the first system state space inside the upper-level intelligent agent is updated through the first operating information to obtain the current first system state space. Then the upper-level intelligent agent maps the current first system state space through the internal upper-level deep reinforcement learning network to generate a first action space. Finally, the required power adjustment amount and load removal amount are extracted from the first action space and an action matrix is ​​constructed for output. The embodiment of the present application realizes the mapping of state space to action space through the deep reinforcement learning network inside the intelligent agent, so that the intelligent agent can make decisions based on the first operating information, output the corresponding action matrix, and improve the stability of the power system operation.

[0025] In a possible implementation manner, the lower-layer agent outputs a second pre-decision instruction according to the second operation information, including:

[0026] Update the second system state space according to the second operation information to obtain the current second system state space;

[0027] Inputting the current second system state space into the lower-layer deep reinforcement learning network of the lower-layer agent, so that the lower-layer deep reinforcement learning network maps the current second system state space to the corresponding second action space;

[0028] Extracting the number of unit cuts and the corresponding positions from the second action space;

[0029] The number of the unit cut-offs and the corresponding positions are output as the second pre-decision instruction.

[0030] An embodiment of the present application provides a method for outputting a corresponding action matrix through a lower-level intelligent agent. First, the second system state space inside the lower-level intelligent agent is updated through the second operating information to obtain the current second system state space. Then the lower-level intelligent agent maps the current second system state space through the internal lower-level deep reinforcement learning network to generate a second action space. Finally, the required power adjustment amount and load removal amount are extracted from the second action space and an action matrix is ​​constructed for output. The embodiment of the present application realizes the mapping of state space to action space through the deep reinforcement learning network inside the intelligent agent, so that the intelligent agent can make decisions based on the second operating information, output the corresponding action matrix, and improve the stability of the power system operation.

[0031] Furthermore, the deep reinforcement learning multi-agent hierarchical Markov decision process based on knowledge fusion obtains the upper-layer agent and the lower-layer agent through offline training, including:

[0032] Acquiring historical operation data and simulation data of the power system;

[0033] According to the historical operation data and simulation data, combined with Lyapunov theory, the transient voltage stability mechanism constraints, voltage stability margin range and power angle stability margin range of the power system are established;

[0034] Constructing an initial upper-layer agent and an initial lower-layer agent;

[0035] According to the historical operation data and simulation data of the power system, the initial upper-layer agent is trained using a double-layer deep Q network to obtain the upper-layer agent, wherein the loss function of the double-layer deep Q network is constructed according to the transient voltage stability mechanism constraint;

[0036] According to the historical operation data and simulation data of the power system, the initial lower-level intelligent agent is trained using a branch competition network to obtain the lower-level intelligent agent, wherein the loss function of the branch competition network is constructed based on the voltage stability margin range and the power angle stability margin range.

[0037] The embodiment of the present application provides a method for obtaining an upper-level agent and a lower-level agent through offline training. First, the historical operation data and simulation data of the power system determine the transient voltage stability mechanism constraints, voltage stability margin range, and power angle stability margin range of the power system. At the same time, an initial upper-level agent and an initial lower-level agent are constructed as objects of offline training. Considering that the upper-level agent and the lower-level agent are responsible for making different decisions, and the decision cycles of the two agents are also different, the embodiment of the present application uses a double-layer deep Q network to train the initial upper-level agent according to these characteristics to obtain the upper-level agent; and uses a branch competition network to train the initial lower-level agent to obtain the lower-level agent. In addition, during the training process, the loss function of the double-layer deep Q network is constructed using the transient voltage stability mechanism constraints corresponding to the decision content of each agent, which is used to train the initial upper-level agent; and the loss function of the branch competition network is constructed using the voltage stability margin range and the power angle stability margin range to train the initial lower-level agent. Therefore, the embodiments of the present application provide a two-layer reinforcement learning scheme based on knowledge fusion to address the problems in the existing new power system pre-decision-making technology, such as the poor interpretability of decisions caused by relying on pure data-driven methods and the lack of mechanism support for black box models. While utilizing the data-driven nonlinear system characterization capabilities, important mechanism conditions in the transient pre-decision-making process are added to ensure that the training-decision-making results are executable and interpretable.

[0038] In a possible implementation, the transient voltage stability mechanism constraint, voltage stability margin range, and power angle stability margin range of the power system are established based on the historical operation data and simulation data in combination with Lyapunov theory, including:

[0039] Determine sufficient conditions for voltage stability of the power system according to Lyapunov theory;

[0040] Determining a transient voltage stability mechanism constraint of the power system according to the sufficient condition for voltage stability;

[0041] Determining a power angle stability margin range of the power system according to the historical operation data and the simulation data;

[0042] According to the historical operation data, simulation data and preset power industry standard transient stability criteria, the bus transient minimum voltage, the maximum sustainable time value of the voltage drop and the bus voltage value after restoration of stability are combined to determine the voltage stability margin range of the power system.

[0043] In the embodiment of the present application, the historical operation data and simulation data of the power system are combined with Lyapunov theory to establish the transient voltage stability mechanism constraints, voltage stability margin range and power angle stability margin range of the power system. These indicators constitute the important power system rules and knowledge in transient stability analysis, providing a mechanism basis for the design of knowledge fusion reinforcement learning.

[0044] In a possible implementation, the initial upper-layer agent is trained using a double-layer deep Q network according to the historical operation data and simulation data of the power system to obtain the upper-layer agent, including:

[0045] Constructing the historical operation data and simulation data into a training data set;

[0046] Constructing a reward function of the initial upper-layer agent according to the fixed cost of single-step dispatch of the power system and the dispatch cost of the unit;

[0047] Constructing the two-layer deep Q network according to the reward function;

[0048] The training data in the training data set are sequentially input into the initial upper-layer agent for training to obtain the upper-layer agent, wherein, during each training, the initial upper-layer agent outputs corresponding upper-layer decisions according to the input training data, and inputs the upper-layer decisions into the double-layer deep Q network, so that the double-layer deep Q network predicts an estimated Q value and a target Q value, and updates the deep reinforcement learning network parameters in the initial upper-layer agent according to the corresponding loss function.

[0049] In a possible implementation, the training of the initial lower-layer agent using a branch competition network according to the historical operation data and simulation data of the power system to obtain the lower-layer agent includes:

[0050] Constructing the historical operation data and simulation data into a training data set;

[0051] Constructing a reward function of the lower-layer agent according to whether the machine-cutting decision is effective and the machine-cutting cost of the power system;

[0052] Constructing the branch competition network according to the reward function;

[0053] The training data in the training data set are sequentially input into the initial lower-layer agent for training to obtain the lower-layer agent, wherein, during each training, the initial lower-layer agent outputs corresponding lower-layer decisions according to the input training data, and inputs the lower-layer decisions into the branch competition network, so that the branch competition network predicts the target Q value and the Q values ​​of all branches, and updates the deep reinforcement learning network parameters in the initial lower-layer agent according to the corresponding loss function.

[0054] In the second aspect, accordingly, the present application provides a novel online pre-decision system for electric power system based on knowledge fusion, comprising a first acquisition module, a first decision module, a second acquisition module and a second decision module;

[0055] The first acquisition module is used to acquire first operation information of the power system in a first decision cycle, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set at the current moment of the power system;

[0056] The first decision module is used to input the first operation information to a preset upper-level intelligent agent if it is determined that the first operation information meets a first decision condition, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, wherein the first pre-decision instruction includes a power adjustment amount and a load removal amount of each unit;

[0057] The second acquisition module is used to acquire second operation information of the power system in a second decision cycle, wherein the second operation information includes the generator terminal voltage, network node voltage and power angle state of the power system at the current moment;

[0058] The second decision module is used for inputting the instability characteristics of the power system into a preset lower-level intelligent agent if it is determined according to the second operating information that the power system is in a transient instability stage, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operating information, wherein the second pre-decision instruction includes a shutdown instruction of a plurality of generators;

[0059] Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 : A flow chart of a novel online pre-decision-making method for a power system based on knowledge fusion provided in an embodiment of the present application.

[0061] Figure 2: A flow chart of intelligent agent training in a novel online pre-decision-making method for a power system based on knowledge fusion provided in an embodiment of the present application.

[0062] Figure 3 : A flow chart of the collective agent training and pre-decision process in a new online pre-decision method for a power system based on knowledge fusion provided in an embodiment of the present application.

[0063] Figure 4 : A structural diagram of a new type of power system online pre-decision system based on knowledge fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments and do not serve to limit the order in which the steps are executed. In the description of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0066] Throughout the entire text of the specification, the new power system described in this application is based on the inherent requirements of implementing the new development concept, building a new development pattern, and promoting high-quality development, with ensuring energy and power security as the basic premise, meeting the power needs of economic and social development as the primary goal, and maximizing the absorption of new energy as the main task. It has a strong smart grid as the hub platform, and is supported by source-grid-load-storage interaction and multi-energy complementarity. It is a power system with the basic characteristics of clean and low-carbon, safe and controllable, flexible and efficient, intelligent and friendly, and open and interactive.

[0067] Embodiment 1:

[0068] like Figure 1 As shown, the first embodiment provides a novel online pre-decision-making method for a power system based on knowledge fusion, including steps S1-S4:

[0069] Step S1, in a first decision cycle, obtaining first operation information of the power system, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set at the current moment of the power system;

[0070] Step S2: if it is determined that the first operation information satisfies the first decision condition, the first operation information is input into a preset upper-level intelligent agent, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, wherein the first pre-decision instruction includes a power adjustment amount and a load removal amount of each unit;

[0071] Step S3, in a second decision cycle, obtaining second operation information of the power system, wherein the second operation information includes the generator terminal voltage, network node voltage and power angle state of the power system at the current moment;

[0072] Step S4: if it is determined according to the second operation information that the power system is in a transient instability stage, the instability characteristics of the power system are input into a preset lower-level intelligent agent, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operation information, wherein the second pre-decision instruction includes a shutdown instruction of a plurality of generators;

[0073] Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training.

[0074] The embodiment of the present application provides a novel online pre-decision method for power systems based on knowledge fusion, a multi-agent hierarchical Markov decision process based on deep reinforcement learning of knowledge fusion, training to obtain upper-level agents and lower-level agents, and using different agents to make decisions in different decision cycles, wherein the upper-level agent is responsible for deciding the power adjustment amount and load removal amount of each unit, and the lower-level agent is responsible for deciding the amount of each generator to be cut. The embodiment of the present application improves the pertinence of the decisions of each agent by making decisions on different actions in different periods, so that different agents only need to focus on their corresponding decision problems and system states, improves the computing efficiency and decision rationality of the agents, and enables the power system to respond to various operating conditions and transient voltage instability faults in a timely manner, thereby improving the stability of the power system operation.

[0075] Furthermore, the first decision cycle and the second decision cycle are run in parallel, the time scale of the first decision cycle is minute level, and the time scale of the second decision cycle is second level. Usually, the first decision cycle is set to n minutes, and the second decision cycle is set to m seconds.

[0076] In the embodiment of the present application, a first decision cycle of minute level and a second decision cycle of second level are constructed, so that the upper-level intelligent agent corresponding to the first decision cycle is mainly responsible for the pre-decision of long time scale, mainly for prevention; while the lower-level intelligent agent corresponding to the lower-level second decision cycle is mainly responsible for the pre-decision of segment time scale to solve emergency situations, and realize the hierarchical processing of problems. When the first operation information meets the first decision condition, the intelligent agent is required to output the corresponding power adjustment amount and load removal amount according to the current system status, and then adjust the power and load of each unit of the power system. This process is not urgent, because the power system itself has a large power elasticity and load elasticity, so it can be adjusted accordingly through the pre-decision of long time scale, and the decision is made every few minutes. When the power system is in the transient instability stage, there may be emergency situations such as generator power angle swing and node voltage exceeding the rated range. It is necessary to start the emergency control mechanism, and the lower-level intelligent agent outputs the corresponding machine cutting action instruction set according to the current system status, and then cuts the machine for several generators in the power system, and quickly responds to solve the problem. Therefore, this process requires a short-time scale pre-decision to make corresponding adjustments, and the decision is made every few seconds. The embodiment of the present application divides the two decision cycles into minute level and second level, and performs them simultaneously, thereby achieving effective allocation of system resources, improving the operating efficiency of the intelligent body and the real-time nature of decision-making, and further improving the stability of power system operation.

[0077] Further, in step S2, the determining whether the first operation information satisfies a first decision condition includes:

[0078] Determine a node voltage value and a load amount of the power system according to the first operation information;

[0079] If the node voltage value is greater than or equal to a first preset threshold or the load amount is greater than or equal to a second preset threshold, it is determined that the power system meets the first decision condition.

[0080] In an embodiment of the present application, it is further described how to determine whether the power system meets the first decision condition based on the first operating information. First, the current transient voltage and load of the power system are determined, and then the transient voltage and load are compared with the first preset threshold and the second preset threshold respectively. As long as one of the values ​​of the transient voltage and load exceeds the corresponding preset threshold, the first decision condition is met, and then the upper-level intelligent agent is controlled to make a decision.

[0081] In a possible implementation, in step S2, the upper-layer agent outputs a first pre-decision instruction according to the first operation information, including:

[0082] Update the first system state space according to the first operation information to obtain the current first system state space;

[0083] Inputting the current first system state space into the upper-layer deep reinforcement learning network of the upper-layer agent, so that the upper-layer deep reinforcement learning network maps the current first system state space to the corresponding first action space;

[0084] Extracting the power adjustment amount and load shedding amount of each unit from the first action space;

[0085] The power adjustment amount and the load shedding amount are output as the first pre-decision instruction.

[0086] An embodiment of the present application provides a method for outputting a corresponding action matrix through an upper-level intelligent agent. First, the first system state space inside the upper-level intelligent agent is updated through the first operating information to obtain the current first system state space. Then the upper-level intelligent agent maps the current first system state space through the internal upper-level deep reinforcement learning network to generate a first action space. Finally, the required power adjustment amount and load removal amount are extracted from the first action space and an action matrix is ​​constructed for output. The embodiment of the present application realizes the mapping of state space to action space through the deep reinforcement learning network inside the intelligent agent, so that the intelligent agent can make decisions based on the first operating information, output the corresponding action matrix, and improve the stability of the power system operation.

[0087] In a preferred embodiment, the specific execution process of step S2 is as follows: in the long time scale stage (minute level), the upper intelligent agent collects the first operation information of the regional new power system through the sensing device, including the current generator terminal voltage, network node voltage, generator set active power and reactive power, etc., to form an online state matrix. When it is detected that the transient voltage runs to the stable boundary and the load exceeds the predicted demand, the online state matrix is ​​input into the knowledge fusion deep Q network inside the upper intelligent agent, and the power adjustment amount and load removal amount action matrix are output, and sent by the dispatching center to each unit and load controller to execute the pre-decision instruction.

[0088] In a possible implementation, in step S3, the lower-layer agent outputs a second pre-decision instruction according to the second operation information, including:

[0089] Update the second system state space according to the second operation information to obtain the current second system state space;

[0090] Inputting the current second system state space into the lower-layer deep reinforcement learning network of the lower-layer agent, so that the lower-layer deep reinforcement learning network maps the current second system state space to the corresponding second action space;

[0091] Extracting the number of unit cuts and the corresponding positions from the second action space;

[0092] The number of the unit cut-offs and the corresponding positions are output as the second pre-decision instruction.

[0093] An embodiment of the present application provides a method for outputting a corresponding action matrix through a lower-level intelligent agent. First, the second system state space inside the lower-level intelligent agent is updated through the second operating information to obtain the current second system state space. Then the lower-level intelligent agent maps the current second system state space through the internal lower-level deep reinforcement learning network to generate a second action space. Finally, the required power adjustment amount and load removal amount are extracted from the second action space and an action matrix is ​​constructed for output. The embodiment of the present application realizes the mapping of state space to action space through the deep reinforcement learning network inside the intelligent agent, so that the intelligent agent can make decisions based on the second operating information, output the corresponding action matrix, and improve the stability of the power system operation.

[0094] In a preferred embodiment, the specific execution process of step S3 is as follows: in the short time scale stage (second level), the lower-level intelligent agent collects the second real-time operation information of the regional new power system through the sensing device, including the generator terminal voltage, power angle state, and network node voltage, and judges the transient instability condition. When it is detected that the system is in the transient instability stage, the emergency control mechanism is activated, and the system instability characteristics such as the generator power angle swing and the node voltage exceeding the rated range are input into the knowledge fusion branch competition network inside the lower-level intelligent agent, and the node generator cutting action instruction set is output online to achieve rapid removal of transient faults.

[0095] Further, such as Figure 2 As shown, the deep reinforcement learning multi-agent hierarchical Markov decision process based on knowledge fusion obtains the upper-layer agent and the lower-layer agent through offline training, including steps S501-S505:

[0096] Step S501, obtaining historical operation data and simulation data of the power system;

[0097] Step S502: establishing the transient voltage stability mechanism constraint, voltage stability margin range, and power angle stability margin range of the power system based on the historical operation data and simulation data and in combination with Lyapunov theory;

[0098] Step S503: construct an initial upper-layer agent and an initial lower-layer agent;

[0099] Step S504: training the initial upper-layer agent using a double-layer deep Q network according to the historical operation data and simulation data of the power system to obtain the upper-layer agent, wherein the loss function of the double-layer deep Q network is constructed according to the transient voltage stability mechanism constraint;

[0100] Step S505: According to the historical operation data and simulation data of the power system, the initial lower-level intelligent agent is trained using a branch competition network to obtain the lower-level intelligent agent, wherein the loss function of the branch competition network is constructed based on the voltage stability margin range and the power angle stability margin range.

[0101] The embodiment of the present application provides a method for obtaining an upper-level agent and a lower-level agent through offline training. First, the historical operation data and simulation data of the power system determine the transient voltage stability mechanism constraints, voltage stability margin range, and power angle stability margin range of the power system. At the same time, an initial upper-level agent and an initial lower-level agent are constructed as objects of offline training. Considering that the upper-level agent and the lower-level agent are responsible for making different decisions, and the decision cycles of the two agents are also different, the embodiment of the present application uses a double-layer deep Q network to train the initial upper-level agent according to these characteristics to obtain the upper-level agent; and uses a branch competition network to train the initial lower-level agent to obtain the lower-level agent. In addition, during the training process, the loss function of the double-layer deep Q network is constructed using the transient voltage stability mechanism constraints corresponding to the decision content of each agent, which is used to train the initial upper-level agent; and the loss function of the branch competition network is constructed using the voltage stability margin range and the power angle stability margin range to train the initial lower-level agent. Therefore, the embodiments of the present application provide a two-layer reinforcement learning scheme based on knowledge fusion to address the problems in the existing new power system pre-decision-making technology, such as the poor interpretability of decisions caused by relying on pure data-driven methods and the lack of mechanism support for black box models. While utilizing the data-driven nonlinear system characterization capabilities, important mechanism conditions in the transient pre-decision-making process are added to ensure that the training-decision-making results are executable and interpretable.

[0102] In a preferred embodiment, in step S501, based on the new energy generator set at the source end of the new power system, the energy storage system unit, the synchronous generator unit, the transmission line at the sending end and the historical operation data and simulation data of the multiple loads at the receiving end.

[0103] In a possible implementation, in step S502, the transient voltage stability mechanism constraint, voltage stability margin range, and power angle stability margin range of the power system are established based on the historical operation data and the simulation data in combination with the Lyapunov theory, including:

[0104] Determine sufficient conditions for voltage stability of the power system according to Lyapunov theory;

[0105] Determining a transient voltage stability mechanism constraint of the power system according to the sufficient condition for voltage stability;

[0106] Determining a power angle stability margin range of the power system according to the historical operation data and the simulation data;

[0107] According to the historical operation data, simulation data and preset power industry standard transient stability criteria, the bus transient minimum voltage, the maximum sustainable time value of the voltage drop and the bus voltage value after restoration of stability are combined to determine the voltage stability margin range of the power system.

[0108] In a preferred embodiment, the specific implementation process of step S502 is: according to the historical operation data and simulation data of the new power system source-end new energy generator set, energy storage system unit, synchronous generator unit, transmission line at the sending end and multiple loads at the receiving end, identify the robust operation voltage range interval of each node in the system Voltage stability can be defined as: For voltage v within the above range, assuming that all nodes i, the control strategy g i is to satisfy g i (v i )=0, assuming that each g i is in [-∞,v] and Strictly monotonically decreasing and satisfies Then voltage stability holds. According to the Lyapunov function, the sufficient condition for system voltage stability is:

[0109]

[0110] Where V is the Lyapunov energy function and v is the node voltage vector. The voltage stability constraint expression in the preventive control process can be obtained as follows:

[0111]

[0112] Similarly, the power-power angle plane is drawn based on historical operation data and simulation data to calculate the kinetic energy increase area A inc And the area A where the kinetic energy is reduced dec If A dec >A inc , then the system power angle is stable; if A dec inc , the system loses power angle stability; A dec =A inc When , the system power angle is critically stable. The stability margin η of the system transient power angle can be described as:

[0113]

[0114] ​Furthermore, based on historical operation data, simulation data and China's power industry standard transient stability criteria, the bus transient minimum voltage, the maximum sustainable time of voltage drop and the bus voltage value after restoration of stability are combined with each other. Specifically, according to China's power industry standard "Guidelines for Safety and Stability Calculation and Analysis of Southern Power Grid", the transient voltage stability criteria are defined as follows: When the power system is in the recovery period after a fault occurs, the central point bus voltage drops below 0.75pu for no more than 1s, and after the system is restored to stability, the central point bus voltage of 220kV and above is not less than 0.9pu. The transient voltage stability margin of the system can be described as:

[0115]

[0116] Where U min It is the lowest voltage of the central bus during the recovery period after the transient state ends; U cr , T cr The voltage is the standard given value of the power grid voltage and the maximum sustainable time value below the given value after the voltage drops; T b The bus voltage drops below U cr The maximum duration of U R k is the voltage value during the recovery period after the transient state ends; t is the critical voltage deviation time factor. The above transient stability margin indicators constitute important power system rules and knowledge in transient stability analysis, and provide a mechanism basis for the design of knowledge fusion reinforcement learning.

[0117] In the embodiment of the present application, the historical operation data and simulation data of the power system are combined with Lyapunov theory to establish the transient voltage stability mechanism constraints, voltage stability margin range and power angle stability margin range of the power system. These indicators constitute the important power system rules and knowledge in transient stability analysis, providing a mechanism basis for the design of knowledge fusion reinforcement learning.

[0118] In a preferred embodiment, the construction of the initial upper-level intelligent agent and the initial lower-level intelligent agent described in step S503 is specifically as follows: a regional intelligent agent entity model is constructed based on the prefecture-level city dispatching center, and multiple regional intelligent agent entity models constitute a new type of power system multi-agent system, each intelligent agent respectively includes an upper-level preventive control decision-making mechanism and a lower-level emergency control decision-making mechanism, and the levels are executed at different time scales.

[0119] In a possible implementation, in step S504, the initial upper-layer agent is trained using a double-layer deep Q network according to the historical operation data and simulation data of the power system to obtain the upper-layer agent, including:

[0120] Constructing the historical operation data and simulation data into a training data set;

[0121] Constructing a reward function of the initial upper-layer agent according to the fixed cost of single-step dispatch of the power system and the dispatch cost of the unit;

[0122] Constructing the two-layer deep Q network according to the reward function;

[0123] The training data in the training data set are sequentially input into the initial upper-layer agent for training to obtain the upper-layer agent, wherein, during each training, the initial upper-layer agent outputs corresponding upper-layer decisions according to the input training data, and inputs the upper-layer decisions into the double-layer deep Q network, so that the double-layer deep Q network predicts an estimated Q value and a target Q value, and updates the deep reinforcement learning network parameters in the initial upper-layer agent according to the corresponding loss function.

[0124] In a preferred embodiment, the initial upper-layer agent includes a system state space S upper , action space A upper And the reward function R upper The system state space information includes the current mechanical power P m , electromagnetic power P E , rotor angle δ, speed w, and generator voltage V g , network node voltage V i , the active power P of the unit g And reactive power Q g When the voltage has reached the stable boundary or is unstable, the system state space is input into the deep reinforcement learning network of the initial upper-layer agent and mapped to the controllable unit power ΔP that needs to be adjusted. g and the power ΔP that can be removed from the load bus load , which constitutes the action space of the agent.

[0125] The role of the reward function is to evaluate the superiority of the action based on the effectiveness of the action and the total adjustment amount of the decision after the system performs power adjustment or load reduction. The cumulative reward function aims to maximize the cumulative reward value, that is, minimize the total cost, as shown in the following formula. con The fixed cost of single-step scheduling is set to a negative number. The more decisions are made, the greater the penalty. is the dispatching cost of the j-th unit, which is set to a positive number; m and n are the total number of decisions and the total number of dispatched units, respectively.

[0126]

[0127] The initial upper agent is trained using a two-layer deep Q network. It has two networks with the same structure but different parameters, one for predicting Q estimates and the other for predicting Q targets.

[0128] Q * (s,a)=Q(s,a)+α(r+γmax(Q(s′,a′)-Q(s,a))

[0129] Q target =r+γQ′′ max

[0130] In the formula, Q represents the predicted estimated Q value, Q target represents the predicted target Q value, s represents the state set, a represents the action set, r represents the reward, α and γ represent the weight coefficients, and θ is the Q network training parameter. The loss function is defined based on the Q target and Q estimate, which is expressed as:

[0131]

[0132] Where n represents the minimum batch size.

[0133] In order to describe the dynamic characteristics of high-dimensional systems, a physical information deep reinforcement learning network f(t,x) is defined, where t is the time variable and x is the input variable, which can be expressed as:

[0134]

[0135] Where u represents the unknown function to be approximated and N represents the nonlinear differential operator. The transient voltage physical constraints are learned by minimizing the mean square error loss to form the shared parameters between u and f. The loss function of its training is:

[0136] MSE PINN =MSE u +MSE f

[0137]

[0138]

[0139] Among them, MSE u and MSE fThey are data error and physical information error of the deep reinforcement learning network respectively. During the training process, the calculated error value is fed back to the optimizer to update the parameters of the deep reinforcement learning network, and finally form a trained constrained deep reinforcement learning network model. The loss function of the above deep reinforcement learning network describes the mechanism characteristics of Lyapunov transient voltage stability constraints, and embeds it into the loss function of the deep Q learning network to accelerate the training process. The above steps finally form a knowledge-fused upper-level pre-decision reinforcement learning network after offline training.

[0140] In a possible implementation, in step S505, the initial lower-layer agent is trained using a branch competition network according to the historical operation data and simulation data of the power system to obtain the lower-layer agent, including:

[0141] Constructing the historical operation data and simulation data into a training data set;

[0142] Constructing a reward function of the lower-layer agent according to whether the machine-cutting decision is effective and the machine-cutting cost of the power system;

[0143] Constructing the branch competition network according to the reward function;

[0144] The training data in the training data set are sequentially input into the initial lower-layer agent for training to obtain the lower-layer agent, wherein, during each training, the initial lower-layer agent outputs corresponding lower-layer decisions according to the input training data, and inputs the lower-layer decisions into the branch competition network, so that the branch competition network predicts the target Q value and the Q values ​​of all branches, and updates the deep reinforcement learning network parameters in the initial lower-layer agent according to the corresponding loss function.

[0145] In a preferred embodiment, the initial lower-level agent includes a system state space S lower , action space A lower And the reward function R lower The system state space information includes the current generator voltage V g , network node voltage V i , the active power P of the unit g And reactive power Q g , load active power P load , reactive power Q load Equal perception parameters. Action space A lower Including the number k and location l of the unit cut off. Reward R lower It consists of two parts. The first part is the reward for a successful machine-cutting decision, which is used to judge whether the machine-cutting decision is effective. The second part is the penalty determined according to the actual amount of machine cuts, which is used to evaluate the superiority of the machine-cutting strategy.

[0146] In order to avoid the agent action space dimension being too high and the training difficulty increasing, the initial lower-level agent is trained using the branch competition (BDQ) reinforcement learning model. lower After the deep reinforcement learning network outputs the shared representation f lower , then the value branch is from the representation f lower Mapping to value V lower (S lower ), N lower The dominant branch is represented by f lower Mapping to N lower Advantages of individual actions Ada(S lower ,A lower ). The Q value is calculated using the competitive Q network method:

[0147]

[0148] In the formula, A d is the set of machine switching actions. When updating the Q network, the target Q value of branch d is Y d , Y d It can be calculated in the way of double Q network. The loss function L of the branch competition network is defined as the mean square error between the Q value of all branches and the target Q value. At the same time, the transient power angle stability margin and voltage stability margin penalty terms are added to the loss function to limit the network's optimization process.

[0149]

[0150] In the formula, γ lower is the weight coefficient, R Phy is the transient power angle stability margin and voltage stability margin penalty term. According to the above process, the agent is still completely random in decision exploration, and there is no restriction on the number of machine cutting nodes and removed units. Therefore, it is easy to make invalid decisions or make too many decisions in actual training, and the sample decision quality of the experience pool is not high enough. When trying to make decisions, human experts usually use knowledge and experience to narrow the decision range and reduce invalid exploration. According to experience, it is generally believed that it is more likely to find an effective decision by cutting the generator from the power angle swinging generator node, so the selection of the cutting node is restricted. The cutting node restriction can be achieved by adding an action shield before the agent outputs the action. Each time, only one node is randomly selected from the power angle swinging cutting node set to cut the machine, and the number of cut units is also randomly given.

[0151] The flowchart of the process of constructing, training and pre-decision-making of the collective agent in the embodiment of the present application is as follows: Figure 3As shown, the embodiment of the present application constructs an initial upper-level intelligent agent for preventive control and an initial lower-level intelligent agent for emergency control based on the historical operation data and simulation data of the power system and based on Lyapunov theory. Then, the initial upper-level intelligent agent and the initial lower-level intelligent agent are trained respectively using reinforcement learning based on a deep Q network and reinforcement learning based on a branch competition network to obtain the trained upper-level intelligent agent and lower-level intelligent agent. Finally, the upper-level intelligent agent and the lower-level intelligent agent are applied to the daily operation of the power system. Through the parallel operation of the upper-level intelligent agent and the lower-level intelligent agent, long-time scale preventive control and short-time scale emergency control of the power system are simultaneously realized to ensure the stable operation of the power system.

[0152] Embodiment 2:

[0153] like Figure 4 As shown, accordingly, embodiment 2 provides a novel online pre-decision system for power system based on knowledge fusion, including a first acquisition module 10, a first decision module 20, a second acquisition module 30 and a second decision module 40;

[0154] The first acquisition module 10 is used to acquire first operation information of the power system in a first decision cycle, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set at the current moment of the power system;

[0155] The first decision module 20 is used to input the first operation information to a preset upper-level intelligent agent if it is determined that the first operation information meets the first decision condition, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, and the first pre-decision instruction includes the power adjustment amount and load removal amount of each unit;

[0156] The second acquisition module 30 is used to acquire second operation information of the power system in a second decision cycle, wherein the second operation information includes the generator terminal voltage, network node voltage and power angle state of the power system at a current moment;

[0157] The second decision module 40 is used for inputting the instability characteristics of the power system into a preset lower-level intelligent agent if it is determined that the power system is in a transient instability stage according to the second operating information, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operating information, and the second pre-decision instruction includes a shutdown instruction of a plurality of generators;

[0158] Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training.

[0159] The embodiment of the present application provides a novel online pre-decision system for power systems based on knowledge fusion, a multi-agent hierarchical Markov decision process based on deep reinforcement learning of knowledge fusion, training to obtain upper-level agents and lower-level agents, and using different agents to make decisions in different decision cycles, wherein the upper-level agent is responsible for deciding the power adjustment amount and load removal amount of each unit, and the lower-level agent is responsible for deciding the amount of machine cuts of each generator. The embodiment of the present application improves the pertinence of the decisions of each agent by making decisions on different actions in different periods, so that different agents only need to focus on their corresponding decision problems and system states, improves the computing efficiency and decision rationality of the agents, and enables the power system to respond to various operating conditions and transient voltage instability faults in a timely manner, thereby improving the stability of the power system operation.

[0160] The more detailed working principle and step flow of this embodiment can refer to, but are not limited to, the relevant records of Embodiment 1.

[0161] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A novel online pre-decision-making method for power system based on knowledge fusion, characterized in that: include: In a first decision cycle, first operation information of the power system is obtained, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set of the power system at a current moment; If it is determined that the first operation information satisfies a first decision condition, the first operation information is input into a preset upper-level intelligent agent, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, wherein the first pre-decision instruction includes a power adjustment amount and a load removal amount of each unit; In a second decision cycle, second operation information of the power system is obtained, wherein the second operation information includes a generator terminal voltage, a network node voltage, and a power angle state of the power system at a current moment; If it is determined according to the second operation information that the power system is in a transient instability stage, the instability characteristics of the power system are input into a preset lower-level intelligent agent, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operation information, wherein the second pre-decision instruction includes a shutdown instruction of a plurality of generators; Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training.

2. A novel online pre-decision-making method for power system based on knowledge fusion as claimed in claim 1, characterized in that: The first decision cycle and the second decision cycle run in parallel, the time scale of the first decision cycle is in minutes, and the time scale of the second decision cycle is in seconds.

3. A novel online pre-decision-making method for power system based on knowledge fusion as claimed in claim 1, characterized in that: The determining that the first operation information satisfies a first decision condition includes: Determine a node voltage value and a load amount of the power system according to the first operation information; If the node voltage value is greater than or equal to a first preset threshold or the load amount is greater than or equal to a second preset threshold, it is determined that the power system meets the first decision condition.

4. A novel online pre-decision making method for power system based on knowledge fusion as claimed in claim 1, characterized in that: The upper-layer agent outputs a first pre-decision instruction according to the first operation information, including: Update the first system state space according to the first operation information to obtain the current first system state matrix space; Inputting the current first system state space into the upper-layer deep reinforcement learning network of the upper-layer agent, so that the upper-layer deep reinforcement learning network maps the current first system state space to the corresponding first action space; Extracting the power adjustment amount and load shedding amount of each unit from the first action space; The power adjustment amount and the load shedding amount are output as the first pre-decision instruction.

5. A novel online pre-decision-making method for power system based on knowledge fusion as claimed in claim 1, characterized in that: The lower-layer agent outputs a second pre-decision instruction according to the second operation information, including: Update the second system state space according to the second operation information to obtain the current second system state space; Inputting the current second system state space into the lower-layer deep reinforcement learning network of the lower-layer agent, so that the lower-layer deep reinforcement learning network maps the current second system state space to the corresponding second action space; Extracting the number of unit cuts and the corresponding positions from the second action space; The number of the unit cut-offs and the corresponding positions are output as the second pre-decision instruction.

6. A novel online pre-decision-making method for power system based on knowledge fusion as claimed in any one of claims 1 to 5, characterized in that: The deep reinforcement learning multi-agent hierarchical Markov decision process based on knowledge fusion obtains the upper-layer agent and the lower-layer agent through offline training, including: Acquiring historical operation data and simulation data of the power system; According to the historical operation data and simulation data, combined with Lyapunov theory, the transient voltage stability mechanism constraints, voltage stability margin range and power angle stability margin range of the power system are established; Constructing an initial upper-layer agent and an initial lower-layer agent; According to the historical operation data and simulation data of the power system, the initial upper-layer agent is trained using a double-layer deep Q network to obtain the upper-layer agent, wherein the loss function of the double-layer deep Q network is constructed according to the transient voltage stability mechanism constraint; According to the historical operation data and simulation data of the power system, the initial lower-level intelligent agent is trained using a branch competition network to obtain the lower-level intelligent agent, wherein the loss function of the branch competition network is constructed based on the voltage stability margin range and the power angle stability margin range.

7. A novel online pre-decision making method for power system based on knowledge fusion as claimed in claim 6, characterized in that: The transient voltage stability mechanism constraint, voltage stability margin range and power angle stability margin range of the power system are established based on the historical operation data and simulation data in combination with Lyapunov theory, including: Determine sufficient conditions for voltage stability of the power system according to Lyapunov theory; Determining a transient voltage stability mechanism constraint of the power system according to the sufficient condition for voltage stability; Determining a power angle stability margin range of the power system according to the historical operation data and the simulation data; According to the historical operation data, simulation data and preset power industry standard transient stability criteria, the bus transient minimum voltage, the maximum sustainable time value of the voltage drop and the bus voltage value after restoration of stability are combined to determine the voltage stability margin range of the power system.

8. A novel online pre-decision making method for power system based on knowledge fusion as claimed in claim 6, characterized in that: The method of training the initial upper-layer agent using a double-layer deep Q network according to the historical operation data and simulation data of the power system to obtain the upper-layer agent includes: Constructing the historical operation data and simulation data into a training data set; Constructing a reward function of the initial upper-layer agent according to the fixed cost of single-step dispatch of the power system and the dispatch cost of the unit; Constructing the two-layer deep Q network according to the reward function; The training data in the training data set are sequentially input into the initial upper-layer agent for training to obtain the upper-layer agent, wherein, during each training, the initial upper-layer agent outputs corresponding upper-layer decisions according to the input training data, and inputs the upper-layer decisions into the double-layer deep Q network, so that the double-layer deep Q network predicts an estimated Q value and a target Q value, and updates the deep reinforcement learning network parameters in the initial upper-layer agent according to the corresponding loss function.

9. A novel online pre-decision making method for power system based on knowledge fusion as claimed in claim 6, characterized in that: The method of training the initial lower-layer agent using a branch competition network according to the historical operation data and simulation data of the power system to obtain the lower-layer agent includes: Constructing the historical operation data and simulation data into a training data set; Constructing a reward function of the lower-layer agent according to whether the machine-cutting decision is effective and the machine-cutting cost of the power system; Constructing the branch competition network according to the reward function; The training data in the training data set are sequentially input into the initial lower-layer agent for training to obtain the lower-layer agent, wherein, during each training, the initial lower-layer agent outputs corresponding lower-layer decisions according to the input training data, and inputs the lower-layer decisions into the branch competition network, so that the branch competition network predicts the target Q value and the Q values ​​of all branches, and updates the deep reinforcement learning network parameters in the initial lower-layer agent according to the corresponding loss function.

10. A new type of power system online pre-decision system based on knowledge fusion, characterized in that: It includes a first acquisition module, a first decision module, a second acquisition module and a second decision module; The first acquisition module is used to acquire first operation information of the power system in a first decision cycle, wherein the first operation information includes the generator terminal voltage, network node voltage, active power and reactive power of the generator set at the current moment of the power system; The first decision module is used to input the first operation information to a preset upper-level intelligent agent if it is determined that the first operation information meets a first decision condition, so that the upper-level intelligent agent outputs a first pre-decision instruction according to the first operation information, wherein the first pre-decision instruction includes a power adjustment amount and a load removal amount of each unit; The second acquisition module is used to acquire second operation information of the power system in a second decision cycle, wherein the second operation information includes the generator terminal voltage, network node voltage and power angle state of the power system at the current moment; The second decision module is used for inputting the instability characteristics of the power system into a preset lower-level intelligent agent if it is determined according to the second operating information that the power system is in a transient instability stage, so that the lower-level intelligent agent outputs a second pre-decision instruction according to the second operating information, wherein the second pre-decision instruction includes a shutdown instruction of a plurality of generators; Among them, the upper-layer agent and the lower-layer agent are deep reinforcement learning multi-agent hierarchical Markov decision processes based on knowledge fusion, which are obtained through offline training.

Images