Spark-based power grid topology loop detection method and system

Abstract

The invention discloses a spark-based power grid topology loop detection method and system. The method comprises the following steps: constructing an original graph model library based on power grid equipment information and connection information of a power grid full-amount real-time library; power grid topological data is obtained from the original graph model library, the power grid topological data comprises a power grid topological relation and topological equipment information, and a Graphx topological graph is constructed in a spark framework based on the power grid topological data; and based on the Graphx topological graph, detecting a topological loop by using a loop identification algorithm, and applying the detected topological loop to a display platform interface. According to the scheme, the power grid topology loop is detected more efficiently and quickly by using the computing resources of the spark big data cluster.

Description

Technical Field

[0001] The invention belongs to the field of power grid topology engineering, and in particular relates to a power grid topology loop detection method and system based on Spark. Background Art

[0002] A topological loop in a power grid is a group of connected lines or nodes that form a closed loop in a power grid. In a power system, these loops may be formed by substations, transmission lines, and distribution lines. Topological loop problems in power grids are usually related to the stability, fault tolerance, and safety of power systems. Topological loops may cause unstable behaviors in the power grid, such as overcurrent, short circuit, and other problems. Therefore, it is very important to detect and identify potential topological loops when the power system is in operation.

[0003] In the case of topological loop problems, the power grid system must first be abstracted into a graph model. The graph model here is used to represent the various components in the power system (such as generators, transformers, lines, loads, etc.) and the relationships between them. The two key concepts in the graph model are nodes and edges. Among them, nodes represent various devices in the power system, such as generators, transformers, loads, etc. Each node usually has a unique identifier. Edges represent the electrical connection between various devices in the power system. This can be wires, cables, transformers, etc. According to the direction of the edge, the graph model can be divided into directed graphs and undirected graphs. In an undirected graph, each edge is bidirectional and has no clear direction. This means that the edge from node A to node B is the same as the edge from node B to node A. In the case of power grid topological loop problems, the edges are usually undirected.

[0004] In addition, some loops in the power grid diagram model are reasonable. For example, although loops occur, the devices are isolated by disconnectors, load switches, circuit breakers, etc., and there will be no safety issues. Therefore, the power grid topology loop usually requires finding a loop without a specific device.

[0005] The calculation of power grid topology usually requires mapping the power grid data into the data in the form of points and edges of the graph model, using the graph calculation algorithm to solve the loop set, and then mapping the calculation results back to the power grid data and outputting them. Among them, solving the loop set has the largest amount of calculation, the most complex calculation, and requires the most computing resources, which is the bottleneck of the entire calculation process. Summary of the invention

[0006] In order to solve the deficiencies in the prior art, the present invention provides a Spark-based power grid topology loop detection method and system to solve the technical problem of finding and outputting topology loops that may have problems in the power system on Spark.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0008] The present invention first discloses a power grid topology loop detection method based on Spark, which comprises the following steps: Step 1: construct an original graph model library based on the grid equipment information and connection information of the full real-time database of the grid; Step 2, acquiring power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and constructing a Graphx topology graph in a spark framework based on the power grid topology data; Step 3: Based on the Graphx topology graph, a loop identification algorithm is used to detect topology loops, and the detected topology loops are applied to the display platform interface.

[0009] The present invention further includes the following preferred embodiments: The original graph model library adopts a graph database storage system, and the original graph model library is constructed based on the grid equipment information and connection information of the full grid real-time library, further comprising: Parse the full amount of power grid equipment information and connection information XML files, obtain the point esv files and edge csv files required by the original graph model library, define the Schema Json files of the Vertex and Edge entities of the graph database storage system, process the power grid equipment information and connection information data tables, and import them into the graph database storage system in batches according to the defined Schema.

[0010] The construction of the original graph model library based on the grid equipment information and connection information of the full real-time database of the grid further includes: Predefine point entities and edge entities of the graph model according to different device types and connection types, wherein the device types and connection types are associated with topological devices and topological connection relationships; According to the defined point entities and edge entities, data is exported from the full real-time database of the power grid, the exported data is preprocessed, and stored in the graph database storage system.

[0011] The step of constructing a Graphx topology graph in a spark framework based on the power grid topology data further includes: Read the topology device information table data and topology device information table in the Hive table through Spark, and abstractly convert them into Graphx relational RDD and point RDD; Construct a Graphx original topology graph based on the relationship RDD and the point RDD; Points not involved in calculation are filtered out from the point RDD, and edges associated with the points not involved in calculation are removed from the original topological graph to obtain a final Graphx graph.

[0012] The detecting of topological loops by using a loop identification algorithm based on the Graphx topological graph further comprises: Step 3.1: Build the spark Graphx graph G; Step 3.2: Select a starting node in G, start breadth-first traversal from the starting node, mark the depth of each leaf node from its respective root node, and the corresponding subordinate parent node; Step 3.3: Find an edge of the potential loop based on the root node and leaf node information. This edge is called an identification edge, and its endpoint is called an identification point. If (u, v) is an identification edge, then u and v are not each other's subordinate parent nodes, u and v are identification points, and u and v are set to belong to the same unique loop ID; Step 3.4: Trace back all the identification points along the parent node and collect the backtracking paths. If the current node receives a backtracking path set with the same loop ID and the same depth as the upstream node, the loop backtracking of the loop ID is completed, and the sum of all the backtracking path sets is used as all the constituent paths of the loop of the loop ID. Step 3.5: Collect all loops, traverse all edges and output the edges of all loops to the Hive database to form a topological loop result table.

[0013] The step of applying the detected topological loop to the display platform interface further includes: An application platform topology loop model library is constructed, the topology loop is preprocessed and stored in the application platform topology loop model library, and the application platform topology loop model library is associated with the display platform, thereby applying the topology loop to the display platform interface.

[0014] The applying the topology loop to the display platform interface further includes: Using the spark framework to read the topology loop result table and the topology device information table, wherein the topology loop result table belongs to an edge table, and each edge is a topology connection edge; Read the topology connection edge, obtain the topology device entity table by associating the topology device information table and store it in the application platform topology loop model library. A topology connection edge belongs to one or more topology loops, that is, the topology connection edge will be expanded into multiple topology connection edges, and the expanded topology connection edges belong to different topology loops; By expanding and processing the topological loop result table, a topological loop edge entity table is obtained and stored in the topological loop model library of the application platform.

[0015] The present invention also discloses a Spark-based power grid topology loop detection system using the aforementioned Spark-based power grid topology loop detection method, comprising: A graph model library construction module is used to construct an original graph model library based on the grid equipment information and connection information of the full real-time database of the grid; A topology graph construction module is used to obtain power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and to construct a Graphx topology graph in a spark framework based on the power grid topology data; A topological loop identification module is used to detect topological loops based on the Graphx topological graph using a loop identification algorithm, and apply the detected topological loops to the display platform interface.

[0016] Accordingly, the present application also discloses a terminal, including a processor and a storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the aforementioned spark-based power grid topology loop detection method.

[0017] Correspondingly, the present application also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the aforementioned Spark-based power grid topology loop detection method are implemented.

[0018] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a Spark-based power grid topology loop detection method and system, which realizes Spark-based power grid topology loop detection, fully and flexibly utilizes the computing resources of the big data cluster according to the size of the data, detects power grid topology loops more efficiently and quickly, and discovers potential hidden dangers and problems early, and is highly scalable, fault-tolerant and universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the power grid topology loop detection method based on spark in the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0021] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0022] Apache spark is an open source, distributed framework for big data processing. It provides high-level APIs and built-in libraries to support a variety of big data processing tasks, including batch processing, interactive query, stream processing, and machine learning. The core data structure of spark is the Resilient Distributed Dataset (RDD), which represents an immutable, partitionable dataset distributed on a cluster. Spark is a pure memory computing framework that stores data in memory to increase processing speed. GraphX ​​is spark's graph processing library, which provides APIs and algorithms for graph computing. It is built on spark's RDD, allowing users to naturally integrate graph algorithms with other spark tasks. In GraphX, a graph consists of VertexRDD and EdgeRDD. VertexRDD represents the nodes and node attributes of the graph, and EdgeRDD represents the edges and edge attributes of the graph. Through the spark computing framework, the resources of the big data platform can be maximized to efficiently and reliably complete the calculation of topological loops.

[0023] However, the graph computing engine GraphX ​​that comes with the Spark computing framework uses the Pregel computing model, which is a point-centric computing model and cannot be directly used to solve graph mining algorithms such as loop enumeration.

[0024] In view of the shortcomings of the prior art, the present invention proposes a power grid topology loop detection method and system based on Spark, which effectively utilizes the massive computing resources, fault tolerance and elastic resource utilization capabilities of a large-scale Spark cluster, and uses the Spark computing framework to perform efficient parallel computing solutions to the topology loop problem of the power grid to find out and output topological loops that may have problems.

[0025] The cycle basis of an undirected graph is a set of basic cycles, each of which cannot be formed by linear combination of other cycles. This set of basic cycles can be used to describe the cycle structure in the graph. Since the number of cycles in an undirected graph may be astronomical, it is not practical to enumerate and output all cycles in many cases. To facilitate the analysis of downstream tasks, the present invention only enumerates and outputs all basic cycles contained in the cycle basis of the undirected graph in the graph. Based on the cycle basis of the undirected graph, any cycle in the graph can be obtained by linear combination of one or more basic cycles.

[0026] See also Figure 1 As shown, the power grid topology loop detection method based on Spark disclosed in the present invention includes the following steps: Step 1: Build the original graph model library based on the grid equipment information and connection information of the full real-time database of the grid.

[0027] The original graph model library of the present invention adopts a graph database storage system. The full amount of power grid equipment information and connection information XML files are parsed to obtain the point esv files and edge csv files required by the original graph model library. According to the application requirements, the Schema Json files of the Vertex and Edge entities of the graph database storage system are defined, the power grid equipment information and connection information data tables are processed, and batches are imported into the graph database storage system according to the defined Schema.

[0028] Before constructing the original graph model library, point entities and edge entities of the graph model are defined in advance according to different device types and connection types, and the device type and connection type are associated with topological devices and topological connection relationships.

[0029] According to the defined point entities and edge entities, data is exported from the full real-time database of the power grid, the exported data is preprocessed, and stored in the original graph model library.

[0030] Step 2: Acquire power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and construct a Graphx topology graph in a spark framework based on the power grid topology data.

[0031] First, based on the original graph model library, obtain the csv files of the topology device entity table and the topology connection edge entity table; preprocess the data of the topology device entity table and the topology connection edge entity table, store them in the big data relational data table library, and obtain the power grid topology data. The big data relational data table library uses the Hive database, which has good integration with spark. Spark can quickly read Hive data and map it to the RDD memory data type of spark. Read the csv file through Hive and perform field conversion processing to meet the input data format of Spark calculation.

[0032] The step of constructing a Graphx topology graph in a spark framework based on the power grid topology data further includes: The topology device information table data and the topology device information table in the Hive table are read through Spark, and abstractly converted into Graphx's relational RDD (Edge RDD) and vertex RDD (Vertex RDD); the Graphx original topology graph is constructed based on the relational RDD and vertex RDD.

[0033] Filter the Vertex RDD to get the Vertex points that are not involved in the calculation, including disconnectors, load switches, and circuit breakers. Remove the edges associated with the Vertex points that are not involved in the calculation from the original topology graph to get the final Graphx graph.

[0034] Step 3: Based on the Graphx topology graph, a loop identification algorithm is used to detect topological loops, and the detected topological loops are applied to the display platform interface.

[0035] The loop identification algorithm is applied to the Graphx topology graph to calculate all loops. The topology loop algorithm is implemented by Graphx, which is an API component of spark, and the underlying computing layer is distributed computing. The graph of Graphx is a distributed attribute graph, and the core implementation of topology loop detection uses the Pregel function interface calculation of Graphx. In a preferred embodiment, the entire process of the topology loop algorithm includes: Step 3.1: Build the spark Graphx graph G; Step 3.2: Select a starting node in G, start breadth-first traversal from the starting node, mark the depth of each leaf node from its respective root node, and the corresponding subordinate parent node; Step 3.3: Find an edge of the potential loop based on the root node and leaf node information. This edge is called an identification edge, and its endpoint is called an identification point. If (u, v) is an identification edge, then u and v are not each other's subordinate parent nodes, u and v are identification points, and u and v are set to belong to the same unique loop ID; Step 3.4: Trace back all the identification points along the parent node and collect the backtracking paths. If the current node receives a backtracking path set with the same loop ID and the same depth as the upstream node, the loop backtracking of the loop ID is completed, and the sum of all the backtracking path sets is used as all the constituent paths of the loop of the loop ID. Step 3.5: Collect all loops, traverse all edges and output the edges of all loops to the Hive database to form a topological loop result table.

[0036] The applying the topological loop to the display platform interface further includes: An application platform topology loop model library is constructed, and the topology loop is preprocessed and stored in the application platform topology loop model library. The model library is associated with the display platform, thereby applying the topology loop to the display platform interface.

[0037] The application process of the topological loop includes using the spark framework to read the topological loop result table and the topological device information table, wherein the topological loop result table belongs to the edge table, and each edge is a topological connection edge. The topological connection edge is read, and the topological device entity table is obtained by associating the topological device information table and stored in the application platform topological loop model library. A topological connection edge belongs to one or more topological loops, that is, the topological connection edge will be expanded into multiple topological connection edges, and the expanded topological connection edges belong to different topological loops. By expanding and processing the topological loop result table, a topological loop edge entity table is obtained and stored in the application platform topological loop model library.

[0038] In a specific embodiment, the underlying storage of the topology loop model library of the application platform uses Elasticsearch, and the application display of the topology loop is realized through the Elasticsearch indexing function.

[0039] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a Spark-based power grid topology loop detection method and system, which realizes Spark-based power grid topology loop detection, fully and flexibly utilizes the computing resources of the big data cluster according to the size of the data, detects power grid topology loops more efficiently and quickly, and discovers potential hidden dangers and problems early, and is highly scalable, fault-tolerant and universal.

[0040] The present invention may be a system, a method and / or a computer program product. The present invention also discloses a Spark-based power grid topology loop detection system based on the aforementioned Spark-based power grid topology loop detection method, comprising: A graph model library construction module is used to construct an original graph model library based on the grid equipment information and connection information of the full real-time database of the grid; A topology graph construction module is used to obtain power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and to construct a Graphx topology graph in a spark framework based on the power grid topology data; A topological loop identification module is used to detect topological loops based on the Graphx topological graph using a loop identification algorithm, and apply the detected topological loops to the display platform interface.

[0041] Based on the spirit of the present invention, those skilled in the art can easily think that a computer program product can be obtained based on the aforementioned spark-based power grid topology loop detection method. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the aforementioned spark-based power grid topology loop detection method.

[0042] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0043] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0044] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A power grid topology loop detection method based on Spark, characterized in that: The following steps are involved: Step 1: construct an original graph model library based on the grid equipment information and connection information of the full real-time database of the grid; Step 2, acquiring power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and constructing a Graphx topology graph in a spark framework based on the power grid topology data; Step 3: Based on the Graphx topology graph, a loop identification algorithm is used to detect topology loops, and the detected topology loops are applied to the display platform interface.

2. The power grid topology loop detection method based on spark according to claim 1, characterized in that: The original graph model library adopts a graph database storage system, and the original graph model library is constructed based on the grid equipment information and connection information of the full grid real-time library, further comprising: Parse the full amount of power grid equipment information and connection information XML files, obtain the point esv files and edge csv files required by the original graph model library, define the Schema Json files of the Vertex and Edge entities of the graph database storage system, process the power grid equipment information and connection information data tables, and import them into the graph database storage system in batches according to the defined Schema.

3. The power grid topology loop detection method based on spark according to claim 2 is characterized in that: The construction of the original graph model library based on the grid equipment information and connection information of the full real-time database of the grid further includes: Predefine point entities and edge entities of the graph model according to different device types and connection types, wherein the device types and connection types are associated with topological devices and topological connection relationships; According to the defined point entities and edge entities, data is exported from the full real-time database of the power grid, the exported data is preprocessed, and stored in the graph database storage system.

4. The power grid topology loop detection method based on spark according to claim 3 is characterized in that: The step of constructing a Graphx topology graph in a spark framework based on the power grid topology data further includes: Read the topology device information table data and topology device information table in the Hive table through Spark, and abstractly convert them into Graphx relational RDD and point RDD; Construct a Graphx original topology graph based on the relationship RDD and the point RDD; Points not involved in calculation are filtered out from the point RDD, and edges associated with the points not involved in calculation are removed from the original topological graph to obtain a final Graphx graph.

5. The power grid topology loop detection method based on spark according to claim 4 is characterized in that: The detecting of topological loops by using a loop identification algorithm based on the Graphx topological graph further comprises: Step 3.1: Build the spark Graphx graph G; Step 3.2: Select a starting node in G, start breadth-first traversal from the starting node, mark the depth of each leaf node from its respective root node, and the corresponding subordinate parent node; Step 3.3: Find an edge of the potential loop based on the root node and leaf node information. This edge is called an identification edge, and its endpoint is called an identification point. If (u, v) is an identification edge, then u and v are not each other's subordinate parent nodes, u and v are identification points, and u and v are set to belong to the same unique loop ID; Step 3.4: Trace back all the identification points along the parent node and collect the backtracking paths. If the current node receives a backtracking path set with the same loop ID and the same depth as the upstream node, the loop backtracking of the loop ID is completed, and the sum of all the backtracking path sets is used as all the constituent paths of the loop of the loop ID. Step 3.5: Collect all loops, traverse all edges and output the edges of all loops to the Hive database to form a topological loop result table.

6. The power grid topology loop detection method based on spark according to claim 5 is characterized in that: The step of applying the detected topological loop to the display platform interface further includes: An application platform topology loop model library is constructed, the topology loop is preprocessed and stored in the application platform topology loop model library, and the application platform topology loop model library is associated with the display platform, thereby applying the topology loop to the display platform interface.

7. The power grid topology loop detection method based on spark according to claim 6 is characterized in that: The applying the topology loop to the display platform interface further includes: Using the spark framework to read the topology loop result table and the topology device information table, wherein the topology loop result table belongs to an edge table, and each edge is a topology connection edge; Read the topology connection edge, obtain the topology device entity table by associating the topology device information table and store it in the application platform topology loop model library. A topology connection edge belongs to one or more topology loops, that is, the topology connection edge will be expanded into multiple topology connection edges, and the expanded topology connection edges belong to different topology loops; By expanding and processing the topological loop result table, a topological loop edge entity table is obtained and stored in the topological loop model library of the application platform.

8. A power grid topology loop detection system based on Spark, characterized in that: include: A graph model library construction module is used to construct an original graph model library based on the grid equipment information and connection information of the full real-time database of the grid; A topology graph construction module is used to obtain power grid topology data from the original graph model library, wherein the power grid topology data includes power grid topology relationships and topology device information, and to construct a Graphx topology graph in a spark framework based on the power grid topology data; A topological loop identification module is used to detect topological loops based on the Graphx topological graph using a loop identification algorithm, and apply the detected topological loops to the display platform interface.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the Spark-based power grid topology loop detection method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the Spark-based power grid topology loop detection method described in any one of claims 1 to 7 are implemented.

Images