DMHS-based power grid operation data increment collection method and system

Abstract

The invention discloses a DMHS-based power grid operation data increment collection method and system, and the method comprises the steps: monitoring the increment change of an operation data table in a source end system, such as an EMS, an OMS and other systems, through DMHS real-time synchronization software, and generating a DMHS message; through operation data standardization, the DMHS message is packaged into an operation data information message, and the operation data information message is sent to the regulation and control cloud platform through a message bus; and receiving the information message of the operation data, verifying and processing the operation data, and uniformly storing the operation data in the regulation and control cloud database. According to the method, minute-level collection and storage of the power grid operation data from each source end to the regulation and control cloud platform are realized, the operation data collection efficiency is effectively improved, and high-real-time operation data basic guarantee is provided for various power grid services.

Description

Technical Field

[0001] The present invention relates to the field of smart grids, and in particular to a method and system for collecting incremental grid operation data based on DMHS. Background Art

[0002] The power grid operation data mainly includes measurement data, alarm data, fault and operation event data, power data, plan forecast data and external environment data. These seven types of data are stored in EMS system, OMS system and other external systems, collectively referred to as source end systems. The power grid operation data collection refers to the extraction of operation data by each source end system, encapsulation into operation data message, and sending it to the control cloud platform (hereinafter referred to as the cloud) through the message bus, parsing it in the cloud and uniformly storing it in the database.

[0003] The types of grid operation data collected are numerous and the amount of data is large. The operation data will be extracted from the source system regularly and sent to the cloud via the message bus. With the rapid development of various business applications such as cloud grid data analysis and early warning, intelligent dispatching decision-making, and power marketing, the real-time requirements for operation data have been greatly improved. Under the current method, it generally takes no less than 15 minutes for grid operation data to be collected from the source to the cloud, which is difficult to meet the high real-time requirements of the above business applications for operation data. In addition, the source and the cloud, as well as different sources, are heterogeneous systems with different technical architectures and development languages, and different operation data storage structures, device models, etc. The current operation data extraction depends on the system architecture and operation mode of each source, and has poor portability and adaptability. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method and system for incremental collection of power grid operation data based on DMHS, which can collect and store operation data from various source ends to the cloud within minutes, and is independent of the source end system architecture and operation mode.

[0005] Technical solution: A method for incrementally collecting power grid operation data based on DMHS, the method comprising the following steps:

[0006] Step 1: Establish DMHS monitoring message rules;

[0007] Step 2: DMHS captures the incremental changes of each source end operation data table in real time according to the monitoring message rules, generates DMHS messages and sends them to the message topic topic in message bus A. A0 ;

[0008] Step 3: Formulate the mapping rules between the source end and the control cloud platform device model;

[0009] Step 4: From the message topic in message bus A A0Receive, parse and verify DMHS messages;

[0010] Step 5: Run data standardization conversion, convert the DMHS message into an operation data message according to the device model mapping rules, and send it to the message topic topic in message bus B B0 ;

[0011] Step 6: From the message topic in message bus B B0 Receive, parse and verify operation data message;

[0012] Step 7: Classify the operation data and store it in the cloud operation data collection table.

[0013] Preferably, the DMHS monitoring message rules in step 1 include two monitoring modes, monitoring according to the name of the table and monitoring according to the operation type of the table.

[0014] Preferably, the monitoring according to the table name refers to setting the table name matching rules and only monitoring the incremental messages of the table that matches successfully; the monitoring according to the table operation type refers to setting the data of the specific operation type of the monitoring table, including "insert", "update" and "delete" operation types.

[0015] Preferably, the two monitoring modes included in the DMHS monitoring message rule, monitoring according to the name of the table can be used alone, and monitoring according to the operation type of the table can be used in addition to monitoring according to the name of the table.

[0016] Preferably, the device model mapping rule in step 3 is a device code mapping rule between the source end and the control cloud platform, and the device code mapping rule includes a standard model mapping rule and a non-standard model mapping rule.

[0017] Preferably, the standard model mapping rule is to directly obtain the cloud device code from the source device code when the device model of the source device is the same as that of the control cloud platform; the non-standard model mapping rule is to formulate a mapping rule to convert the source device code into the cloud device code when the device model of the source device is different from that of the control cloud platform.

[0018] Preferably, the operation data standardization conversion in step 5 is to convert the DMHS message into an operation data message according to the device model mapping rule, and the source device is converted into a control cloud device model through the device model mapping rule and sent to the message topic topic in the message bus B. B0 ; If no device model mapping rule is matched, it will be discarded.

[0019] Preferably, the classification processing of the operation data in step 7 is to classify the operation data according to the type of operation data, the type of equipment, and the sampling period.

[0020] The DMHS-based power grid operation data incremental collection system described in the present invention includes N source ends and a cloud end, the source end includes: a DMHS extraction module, a DMHS sending module, an operation data standardization conversion module and an operation data sending module; the cloud end includes: an operation data receiving module, an operation data verification module, an operation data processing module and an operation data storage module.

[0021] Preferably, the DMHS extraction module in the source end monitors and captures the incremental changes of the source end operation data table, encapsulates the change data of the table according to the DMHS message format, generates a DMHS message, and the DMHS sending module sends the DMHS message to the message topic topic in the message bus A. A0 , run the data standardization transformation module to convert the message topic topic from message bus A A0 Pull, parse and verify DMHS messages, convert DMHS messages into operation data messages that can be received by the cloud, and the operation data sending module sends the operation data message to the message topic topic in message bus B B0 ; The operation data receiving module in the cloud will receive the message topic topic from the message bus B B0 The operation data message will be pulled up and parsed. The operation data verification module will check the legality and validity of the operation data message. The operation data processing module will classify it according to the operation data type, device type, and sampling period. The operation data storage module will store the operation data to the control cloud operation data collection table according to the data type, device type, sampling period, etc.

[0022] Beneficial effects: Compared with the prior art, the present invention has the following advantages: Based on the DMHS real-time synchronization software, the incremental changes of the source system power grid operation data are monitored in real time, and the power grid operation data is quickly and efficiently collected and stored in the cloud within minutes, effectively improving the collection rate of operation data, providing highly reliable and real-time power grid operation data support for data analysis and big data applications, and ensuring the stable operation of business applications such as power grid operation analysis and early warning, dispatching intelligent decision-making, etc. The present invention does not rely on the source system architecture and operation mode, and has high scalability and portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a diagram of the architecture for the incremental collection of power grid operation data based on DMHS in the present invention;

[0024] Figure 2 It is the DMHS message structure of the present invention;

[0025] Figure 3 The data standardization conversion flow chart of the present invention is as follows;

[0026] Figure 4 This is a structural diagram of the power grid operation data incremental collection system based on DMHS in the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0028] DMHS monitors and captures incremental changes in the source operation data table by capturing the insertion, update, and deletion changes that occur in each source operation data table, and encapsulates these changes into DMHS messages in the DMHS message format and sends them to the message bus. Figure 2 As shown in the figure, the DMHS message consists of a message header and a message body. The message header contains metadata such as the monitoring table name, monitoring source, operation type, and data time. The operation types include insert, update, and delete; the message body contains the column name and column value of the monitoring table, and the column name and column value appear in pairs.

[0029] Since the device models and operation data structures of the source and cloud are different, the present invention formulates source and cloud device model mapping rules to realize operation data transmission between heterogeneous systems. Among them, the device model mapping rules include standard model mapping rules and non-standard model mapping rules. The standard model mapping rule refers to the situation where the source device and the cloud device model are the same, and the cloud device code can be directly obtained from the source device code. For example, the source device code is AA0001 and the cloud device code is also AA0001; the non-standard model mapping rule refers to the situation where the source device and the cloud device model are different, and a mapping rule needs to be formulated to convert the source device code into the cloud device code. For example, the source device code is AA0001, and after conversion to the cloud device code, it is BB1111.

[0030] The operation data standardization conversion in the present invention refers to converting the DMHS message into the operation data message received by the cloud according to the established device model mapping rules. The specific process is as follows: Figure 3 As shown, the following steps are included:

[0031] Step 1: From the message topic in message bus A A0 Receive, parse and verify DMHS messages;

[0032] Step 2: Obtain all source device codes according to the table name and column name carried in the DMHS message and store them in the source device set;

[0033] Step 3, start traversing the source device set and extract the source device code;

[0034] Step 4: Match the source device code with the standard model mapping rule. If the match is successful, the cloud device code can be obtained;

[0035] Step 5: If the match fails in step 4, the non-standard model mapping rule is matched. If the match succeeds, the cloud device code can be obtained. If the match fails, it is discarded.

[0036] Step 6: Add the cloud device code and the column value carried in the message to the running data message encapsulation queue;

[0037] Step 7, repeat steps 3 to 6 until the traversal of the source device set is completed;

[0038] Step 8: encapsulate the operation data message and output a single or multiple operation data message messages.

[0039] Among them, the operation data message packet encapsulation described in step 8 is a message format that the control cloud system can receive, including measurement data packets, alarm data packets and other types. After the operation data message packet is encapsulated, it is sent to the message topic topic in the message bus B. B0 Above: From the message topic topic in message bus B B0 Pull up and parse the running data message, and check the legitimacy and validity of the message.

[0040] Among them, the legality check refers to checking the message according to the operation data message format, including non-empty detection, length detection, data limit detection, etc.; the validity check includes message carrying time verification, data source verification, device object coding verification, sampling period verification and device collection verification, etc. After the legality and validity tests, the operation data is classified, processed and stored.

[0041] Among them, classification processing is to classify and organize the operation data according to the type of operation data, device type and operation data sampling period. After classification processing, large quantities of operation data of various types are organized into multiple batches of data, each of which belongs to the same operation data type, the same device type, the same sampling period and the same time. Then, the multiple batches of data are efficiently and quickly stored in the cloud operation data collection table of the corresponding year and the corresponding device. The classification processing process in this technical solution is: first, according to the types of measurement data, alarm data, faults and operation events, power, etc. contained in the operation data, the large batch of operation data is classified and organized to generate {D0,D1,D2,...,D N}, and then classified according to different equipment types, including lines, transformers, power plants and other equipment types. For example, D0 is divided into {D 0A ,D 0P ,D 0G,...}, finally, according to the operation data sampling period and the time to which the operation data belongs, a large number of discrete time points are regularized and submitted in batches to the database for storage.

[0042] The power grid operation data incremental collection system based on DMHS of the present invention includes N source terminals and 1 cloud terminal, such as Figure 4 As shown, the workflow of the source end is as follows: in the DMHS extraction module, DMHS monitors and captures the incremental changes of the source end operation data table, encapsulates the change data of the table according to the DMHS message format, generates a DMHS message, and sends the DMHS message to the message topic topic in the message bus A through the DMHS sending module. A0 , and then run the data standardization conversion module to convert the message topic topic from the message bus A A0 Pull, parse and verify the DMHS message, then convert the DMHS message into an operation data message that can be received by the cloud, and finally send the operation data message to the message topic topic in the message bus B through the operation data sending module B0 , each module on the source side supports single node, multiple nodes, dynamic addition and deletion of nodes, quickly receives DMHS messages and converts and sends operation data message packets to the cloud. Furthermore, the operation data standardization conversion module and the operation data message sending module support single node, multiple nodes, dynamic addition and deletion of nodes, and quickly convert and send operation data messages. Cloud processing process: The operation data message is sent from the message topic topic in the message bus B. B0 The data is pulled up and parsed, and then the legality and validity are checked, and then classified according to the operating data type, equipment type, sampling period, etc., and finally the operating data is stored in the control cloud operation data collection table. Each module in the cloud supports single node, multiple nodes, and dynamic addition and deletion of nodes, and can quickly receive and store various power grid operation data.

Claims

1. A method for incrementally collecting power grid operation data based on DMHS, characterized by: The method comprises the following steps: Step 1: Establish DMHS monitoring message rules; Step 2: DMHS captures the incremental changes of each source end operation data table in real time according to the monitoring message rules, generates DMHS messages and sends them to the message topic topic in message bus A. A0 ; Step 3: Formulate the mapping rules between the source end and the control cloud platform device model; Step 4: From the message topic in message bus A A0 Receive, parse and verify DMHS messages; Step 5: Run data standardization conversion, convert the DMHS message into an operation data message according to the device model mapping rules, and send it to the message topic topic in message bus B B0 ; Step 6: From the message topic in message bus B B0 Receive, parse and verify operation data message; Step 7: Classify the operation data and store it in the cloud operation data collection table.

2. The method for collecting incremental power grid operation data based on DMHS according to claim 1 is characterized in that: The DMHS monitoring message rules in step 1 include two monitoring modes: monitoring according to the name of the table and monitoring according to the operation type of the table.

3. The method for collecting incremental power grid operation data based on DMHS according to claim 2 is characterized in that: The monitoring according to the table name refers to setting the table name matching rules and only monitoring the incremental messages of the table that matches successfully; the monitoring according to the table operation type refers to setting the data of the specific operation type of the monitoring table, including "insert", "update" and "delete" operation types.

4. The method for collecting incremental power grid operation data based on DMHS according to claim 1 is characterized in that: The DMHS monitoring message rules include two monitoring modes. Monitoring according to the name of the table can be used alone, and monitoring according to the operation type of the table can be used in addition to monitoring according to the name of the table.

5. The method for collecting incremental power grid operation data based on DMHS according to claim 1 is characterized in that: The device model mapping rule in step 3 is a device code mapping rule between the source end and the control cloud platform. The device code mapping rule includes a standard model mapping rule and a non-standard model mapping rule.

6. The method for collecting incremental power grid operation data based on DMHS according to claim 5 is characterized in that: The standard model mapping rule is to directly obtain the cloud device code from the source device code when the device model of the source device is the same as that of the control cloud platform; the non-standard model mapping rule is to formulate a mapping rule to convert the source device code into the cloud device code when the device model of the source device is different from that of the control cloud platform.

7. The method for collecting incremental power grid operation data based on DMHS according to claim 1 is characterized in that: The operation data standardization conversion in step 5 is to convert the DMHS message into an operation data message according to the device model mapping rules. The source device is converted into a control cloud device model through the device model mapping rules and sent to the message topic topic in the message bus B. B0 ; If no device model mapping rule is matched, it will be discarded.

8. The method for incrementally collecting power grid operation data based on DMHS according to claim 1 is characterized in that: The operation data is classified in step 7, which is to classify the operation data according to the operation data type, device type and sampling period.

9. A DMHS-based power grid operation data incremental collection system, characterized by: It includes N source ends and a cloud end. The source end includes: a DMHS extraction module, a DMHS sending module, an operation data standardization conversion module and an operation data sending module; the cloud end includes: an operation data receiving module, an operation data verification module, an operation data processing module and an operation data storage module.

10. The DMHS-based power grid operation data incremental collection system according to claim 9, characterized in that: The DMHS extraction module in the source end monitors and captures the incremental changes of the source end operation data table, encapsulates the change data of the table according to the DMHS message format, generates a DMHS message, and the DMHS sending module sends the DMHS message to the message topic topic in the message bus A. A0 , run the data standardization transformation module to convert the message topic topic from message bus A A0 Pull, parse and verify DMHS messages, convert DMHS messages into operation data messages that can be received by the cloud, and the operation data sending module sends the operation data message to the message topic topic in message bus B B0 ; The operation data receiving module in the cloud will receive the message topic topic from the message bus B B0 The operation data message will be pulled up and parsed. The operation data verification module will check the legality and validity of the operation data message. The operation data processing module will classify it according to the operation data type, device type, and sampling period. The operation data storage module will store the operation data to the control cloud operation data collection table according to the data type, device type, sampling period, etc.

Images