A Power Flow Analysis Method Based on Reinforcement Learning
Through the grid trend analysis method based on reinforcement learning, and the multi-agent system and conflict resolution mechanism are used to solve the limitations of traditional methods in dealing with complex grid structures and dynamic changes, more accurate trend prediction and optimized scheduling are achieved, and the operation efficiency and economics of the power system are improved.
Patent Information
- Application Number
- CN202410757137.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-13
- Publication Date
- 2025-03-07
- Estimated Expiration
- 2044-06-13
AI Technical Summary
Traditional grid trend analysis methods rely on static models and manual rules, and it is difficult to effectively deal with complex grid structures and dynamic changes, resulting in the impact of the accuracy of priority allocation results.
The power grid current analysis method based on reinforcement learning is adopted to optimize the overall power grid current distribution through grid topology modeling, multi-agent system construction, collaborative learning and conflict resolution mechanisms.
It realizes more accurate trend prediction and optimized scheduling, improves the operating efficiency and economy of the power system, has strong real-time and adaptability, and can quickly respond to system changes and make adjustments to ensure the stable operation of the system.
Smart Images

Figure CN118748402B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power distribution networks, and in particular to a power grid power flow analysis method based on reinforcement learning. Background Art
[0002] In the current power system, power flow analysis is a crucial task, which is used to evaluate the voltage, current and other parameters at each node and line in the power system to ensure the stable operation of the system. Traditional power flow analysis methods are usually based on static models and mathematical solution techniques, which have certain limitations when dealing with complex power grid structures and dynamic changes. Therefore, the power grid power flow analysis method based on reinforcement learning came into being, bringing new opportunities and challenges to the operation and management of power systems.
[0003] Although traditional power grid flow analysis methods can achieve intelligent decision-making and collaborative optimization, they often rely on static models and manually formulated rules. The priority allocation method in decision-making relies on a large amount of data and model training. However, in actual situations, the quality and reliability of the data may be difficult to guarantee, resulting in the accuracy of the priority allocation results being affected. Summary of the invention
[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.
[0005] In view of the above problems existing in the existing power grid power flow analysis method based on reinforcement learning, the present invention is proposed.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a power grid flow analysis method based on reinforcement learning, the method comprising the following steps:
[0007] S1: Grid topology modeling: Collect the topological structure data of the grid, use graph neural network to model the grid topology, and generate a topological graph representation, where nodes represent each device in the grid and edges represent the connection relationship between them.
[0008] S2: Build a multi-agent system, divide the power grid into several areas, each area is managed by an intelligent agent, and let multiple agents learn collaboratively;
[0009] S3: Make decisions based on collaborative learning results to optimize the overall power grid flow distribution;
[0010] S4: Build a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment.
[0011] As a preferred solution of the power grid flow analysis method based on reinforcement learning described in the present invention, each of the intelligent agents includes the following functional modules:
[0012] Perception module: collects real-time data in the area;
[0013] Decision-making module: Make decisions based on reinforcement learning algorithms to optimize the distribution of power flow in the area;
[0014] Communication module: collaborate with other agents, exchange information and achieve global optimization.
[0015] As a preferred solution of the power grid flow analysis method based on reinforcement learning described in the present invention, the calculation formula of the reinforcement learning algorithm is as follows:
[0016] ;
[0017] Where N is the set of nodes in the power grid, is the voltage at node i, is the expected voltage, is the set of lines between nodes, For line The power loss is t, time, T is the total operation time of the power grid, is a tuning parameter.
[0018] As a preferred solution of the power grid flow analysis method based on reinforcement learning described in the present invention, the specific method of the conflict resolution mechanism is:
[0019] S41: Compare the decisions of each agent to detect whether there is a conflict; the conflict includes: whether the voltage adjustment decision of the same node is contradictory; whether the power dispatch decision of the same line exceeds the line capacity; whether the load dispatch of different areas affects each other;
[0020] S42: Each agent shares its decision information on a public communication platform for all relevant agents to view; once a conflict is detected, the specific content of the conflict and the agents involved are determined through the shared information;
[0021] S43: After determining the conflict content, a set of priority rules are defined according to the importance and urgency of power grid operation. The rules are as follows:
[0022] Key impact: such as trunk lines and important nodes;
[0023] Impact of grid operation status: such as current load level;
[0024] Emergency impact: whether it involves emergency load adjustment or sudden fault handling;
[0025] S44: Decisions of agents with higher priorities are executed first, and the remaining agents need to adjust their decisions to avoid conflicts.
[0026] As a preferred solution of the power grid flow analysis method based on reinforcement learning described in the present invention, the calculation formula of the influence value of each priority rule is:
[0027] Critical impact value A= ;
[0028] in, is the criticality coefficient of node i; is the criticality coefficient of line (i, j), parameter Controls the decay speed, is the voltage difference between node i and node j;
[0029] The impact value of power grid operation status B= ;
[0030] in, is the load level of node i at time t, is the load level of line (i, j) at time t;
[0031] Urgency impact value C= Γ(t):
[0032] in, is the decision urgency coefficient of node i, Γ(t) is the emergency fault condition function of the system at time t, is the weight.
[0033] A power grid flow analysis system based on reinforcement learning, based on the above-mentioned power grid flow analysis method based on reinforcement learning, the system comprises:
[0034] The power grid topology modeling module collects the topological structure data of the power grid, including information such as nodes, lines, and transformers, and uses graph neural networks to model the power grid topology and generate a topological graph representation, where nodes represent various devices in the power grid and edges represent the connection relationships between them;
[0035] The multi-agent system collaboration module divides the power grid into several areas, each of which is managed by an intelligent agent, and allows multiple agents to learn collaboratively;
[0036] The conflict resolution module builds a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment.
[0037] System integration and testing, system integration: integrate multi-agent systems, reinforcement learning algorithms and distributed computing frameworks into the power grid management system; simulation testing: test system performance in a power grid simulation environment to verify its performance under different loads and fault conditions.
[0038] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned power grid power flow analysis method based on reinforcement learning when executing the computer program.
[0039] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned power grid power flow analysis method based on reinforcement learning.
[0040] Beneficial effects of the present invention:
[0041] 1. The present invention comprehensively considers the voltage deviation of the grid node and the power loss of the line through the loss function, so that the optimization target is more comprehensive and can effectively balance the node voltage stability and power transmission efficiency.
[0042] 2. The present invention takes into account various factors such as the criticality of nodes and lines, load levels and emergency fault conditions, and can comprehensively evaluate the importance of each part of the power grid, which helps to determine the nodes and lines that are prioritized.
[0043] 3. In summary, the reinforcement learning-based method of the present invention can fully explore the hidden rules of data, achieve more accurate power flow prediction and optimal scheduling, and improve the operating efficiency and economy of the power system. The reinforcement learning-based method has strong real-time and adaptability, and can quickly respond to system changes and make adjustments to ensure the stable operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0045] Figure 1 A schematic diagram of the flow chart of a power grid power flow analysis method based on reinforcement learning proposed in the present invention. DETAILED DESCRIPTION
[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0049] Reference Figure 1 , is an embodiment of the present invention, and provides a power grid flow analysis method based on reinforcement learning, the method comprising the following steps:
[0050] S1: Grid topology modeling: Collect the topological structure data of the grid, including nodes, lines, transformers and other information, use graph neural network to model the grid topology, and generate a topological graph representation, where nodes represent the various devices in the grid and edges represent the connection relationship between them.
[0051] S2: Build a multi-agent system, divide the power grid into several areas, each area is managed by an intelligent agent, and let multiple agents learn collaboratively;
[0052] Specifically, each intelligent agent includes the following functional modules:
[0053] Sensing module: collects real-time data in the area, including voltage, current, and load;
[0054] Decision-making module: Make decisions based on reinforcement learning algorithms to optimize the distribution of power flow in the area;
[0055] The calculation formula of the reinforcement learning algorithm is as follows:
[0056] ;
[0057] Where N is the set of nodes in the power grid, is the voltage at node i, is the expected voltage, is the set of lines between nodes, For Line The power loss is t, time, T is the total operation time of the power grid, is a tuning parameter.
[0058] Specifically, the value range of formula L is a non-negative real number, which represents the comprehensive loss of the power grid during the entire operation time. By minimizing L, we can optimize the operation state of the power grid, make the node voltage closer to the expected value, reduce power loss and balance the load. The parameter Used to weigh the relative importance of voltage deviation and power loss in the loss function.
[0059] Communication module: collaborate with other agents, exchange information, and achieve global optimization
[0060] S3: Make decisions based on collaborative learning results to optimize the overall power grid flow distribution;
[0061] S4: Build a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment.
[0062] The specific methods of the conflict resolution mechanism are:
[0063] S41: Compare the decisions of each agent to detect whether there is a conflict; the conflict includes: whether the voltage adjustment decision of the same node is contradictory; whether the power dispatch decision of the same line exceeds the line capacity; whether the load dispatch of different areas affects each other;
[0064] S42: Each agent shares its decision information on a public communication platform for all relevant agents to view; once a conflict is detected, the specific content of the conflict and the agents involved are determined through the shared information;
[0065] S43: After determining the conflict content, a set of priority rules are defined according to the importance and urgency of power grid operation. The rules are as follows:
[0066] Key impact: such as trunk lines and important nodes;
[0067] Impact of grid operation status: such as current load level;
[0068] Emergency impact: whether it involves emergency load adjustment or sudden fault handling;
[0069] Specifically, the calculation formula for the impact value of each priority rule is:
[0070] Critical impact value A= ;
[0071] in, is the criticality coefficient of node i; is the criticality coefficient of line (i, j), parameter Controls the decay speed, is the voltage difference between node i and node j. This term indicates that the criticality of nodes and lines is affected by the exponential decay of the voltage difference. The weight is ;
[0072] The impact value of power grid operation status B= ;
[0073] in, is the load level of node i at time t, is the load level of line (i, j) at time t, which means that the node load and line load are adjusted by the voltage difference;
[0074] Urgency impact value C= Γ(t);
[0075] in, is the decision urgency coefficient of node i, Γ(t) is the emergency fault condition function of the system at time t, is the weight;
[0076] If there is a power equipment failure or sudden load change in the area that node i is responsible for, the decision urgency coefficient of node i will increase accordingly. This coefficient can be defined as a dynamically changing function and adjusted according to the actual situation. Usually, it may be affected by factors such as the load level of the node area, the state of the power grid equipment, and the quality of power supply. Γ(t) can be a binary function, indicating whether an emergency failure occurs at time t, or it can be a continuous function, indicating the magnitude of the failure. This function usually increases when the system has an emergency failure at time t, reflecting the emergency situation of the system. Now take a binary function as an example;
[0077] Γ(t)
[0078] S44: Decisions of agents with higher priorities are executed first, and the remaining agents need to adjust their decisions to avoid conflicts.
[0079] In addition, this embodiment also provides a power flow analysis system based on reinforcement learning. Based on the above-mentioned power flow analysis method based on reinforcement learning, the system includes:
[0080] The power grid topology modeling module collects the topological structure data of the power grid, including information such as nodes, lines, and transformers, and uses graph neural networks to model the power grid topology and generate a topological graph representation, where nodes represent various devices in the power grid and edges represent the connection relationships between them;
[0081] The multi-agent system collaboration module divides the power grid into several areas, each of which is managed by an intelligent agent, and allows multiple agents to learn collaboratively;
[0082] The conflict resolution module builds a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment.
[0083] System integration and testing, system integration: integrate multi-agent systems, reinforcement learning algorithms and distributed computing frameworks into the power grid management system; simulation testing: test system performance in a power grid simulation environment to verify its performance under different loads and fault conditions.
[0084] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned power grid power flow analysis method based on reinforcement learning are implemented.
[0085] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned power grid power flow analysis method based on reinforcement learning are implemented.
[0086] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0087] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for implementing a power grid power flow analysis method based on reinforcement learning as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (Static Random Access Memory, referred to as SRAM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), an erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), a programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), a read-only memory (Read-Only Memory, referred to as ROM), a magnetic storage device, a flash memory, a magnetic disk or an optical disk.
[0088] In summary, the present invention comprehensively considers the voltage deviation of the grid node and the power loss of the line through the loss function, making the optimization target more comprehensive and being able to effectively balance the node voltage stability and power transmission efficiency.
[0089] The present invention comprehensively considers multiple factors such as the criticality of nodes and lines, load levels, and emergency fault conditions, and can comprehensively evaluate the importance of each part of the power grid, which helps to determine the nodes and lines that are prioritized. The reinforcement learning-based method of the present invention can fully explore the hidden laws of data, achieve more accurate power flow prediction and optimized scheduling, and improve the operating efficiency and economy of the power system. The reinforcement learning-based method has strong real-time and adaptability, can quickly respond to system changes and make adjustments, and ensure the stable operation of the system.
[0090] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A power grid flow analysis method based on reinforcement learning, characterized in that: The method comprises the following steps: S1: Grid topology modeling: Collect the topological structure data of the grid, use graph neural network to model the grid topology, and generate a topological graph representation, where nodes represent the various devices in the grid and edges represent the connection relationship between them; S2: Build a multi-agent system, divide the power grid into several areas, each area is managed by an intelligent agent, and let multiple agents learn collaboratively; Each of the intelligent agents includes the following functional modules: Perception module: collects real-time data in the area; Decision-making module: Make decisions based on reinforcement learning algorithms to optimize the distribution of power flow in the area; Communication module: collaborate with other agents, exchange information, and achieve global optimization; The calculation formula of the reinforcement learning algorithm is as follows: ; Where N is the set of nodes in the power grid, is the voltage at node i, is the expected voltage, is the set of lines between nodes, For line The power loss, is time, T is the total operation time of the power grid, is the adjustment parameter; S3: Make decisions based on collaborative learning results to optimize the overall power grid flow distribution; S4: Construct a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment; The specific method of the conflict resolution mechanism is as follows: S41: Compare the decisions of each agent to detect whether there is a conflict; the conflict includes: whether the voltage adjustment decision of the same node is contradictory; whether the power dispatch decision of the same line exceeds the line capacity; whether the load dispatch of different areas affects each other; S42: Each agent shares its decision information on a public communication platform for all relevant agents to view; once a conflict is detected, the specific content of the conflict and the agents involved are determined through the shared information; S43: After determining the conflict content, a set of priority rules are defined according to the importance and urgency of power grid operation. The rules are as follows: Key impact: such as trunk lines and important nodes; Impact of grid operation status: such as current load level; Emergency impact: whether it involves emergency load adjustment or sudden fault handling; S44: The decision of the agent with the highest priority is executed first, and the decisions of the other agents need to be adjusted to avoid conflicts. The calculation formula of the impact value of each priority rule is: Critical impact value A= ; in, is the criticality coefficient of node i; is the criticality coefficient of line (i, j), parameter Controls the decay speed, is the voltage difference between node i and node j; The impact value of power grid operation status B= ; in, is the load level of node i at time t, is the load level of line (i, j) at time t; Urgency impact value C= ; in, is the decision urgency coefficient of node i, is the emergency fault condition function of the system at time t, is the weight.
2. A power flow analysis system based on reinforcement learning, based on the power flow analysis method based on reinforcement learning according to claim 1, characterized in that: The system includes: The power grid topology modeling module collects the topological structure data of the power grid, uses graph neural network to model the power grid topology, and generates a topological graph representation, in which nodes represent various devices in the power grid and edges represent the connection relationship between them; The multi-agent system collaboration module divides the power grid into several areas, each of which is managed by an intelligent agent, and allows multiple agents to learn collaboratively; The conflict resolution module builds a conflict resolution mechanism to resolve conflicts among multiple agents through priority judgment. System integration and testing, system integration: integrate multi-agent systems, reinforcement learning algorithms and distributed computing frameworks into the power grid management system; simulation testing: test system performance in a power grid simulation environment to verify its performance under different loads and fault conditions.
3. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the power grid power flow analysis method based on reinforcement learning described in claim 1 are implemented.
4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the power grid power flow analysis method based on reinforcement learning described in claim 1 are implemented.
Citation Information
Patent Citations
Multi-regional power grid collaborative optimization method, system and device and readable storage medium
CN115333111A
Electric power system power flow analysis method and device based on graph network big data model
CN117650533A
