Ammeter data VEE function implementation method and system based on big data framework

Abstract

The invention provides an ammeter data VEE function implementation method and system based on a big data framework. The method comprises the following steps: pre-configuring a VEE rule table comprising more than two VEE rules; associating an electric meter with a VEE rule selected from the VEE rule table; and the Spark distributed computing framework verifies and estimates the current ammeter data according to a VEE rule associated with an ammeter corresponding to the current ammeter data, stores a result in a VEE result table, and writes alarm information into an alarm information table if the alarm information exists. According to the method, a big data technical framework is fully utilized, and VEE verification, estimation and data editing are synchronously carried out by adopting a distributed calculation mode, so that the efficiency, accuracy and reliability of ammeter data processing are remarkably improved; particularly, the flexibility of VEE rule configuration can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric meter data processing, and in particular to a method and system for realizing electric meter data VEE functions based on a big data framework. Background Art

[0002] With the development of smart grid technology, the collection and processing of meter data has become increasingly complex. The meter MDM system is an important part of the smart grid, which is responsible for collecting, processing and analyzing meter data. In order to ensure data quality, the collected data needs to be verified, estimated and edited (collectively referred to as VEE). However, the traditional VEE implementation scheme often cannot efficiently process and store large-scale data sets, and can only use a fixed VEE rule, which is difficult to meet the needs of smart grid data processing.

[0003] The present invention realizes the VEE function of electric meter data in the MDM system based on big data technology frameworks such as Apache Kafka, Apache Spark, and Apache Doris. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for realizing VEE function of electric meter data based on a big data framework, which can improve the flexibility of VEE rule configuration and the efficiency and accuracy of electric meter data processing.

[0005] In order to solve the above technical problems, the first technical solution adopted by the present invention is:

[0006] A method for realizing VEE function of electric meter data based on big data framework, comprising:

[0007] A VEE rule table including two or more VEE rules is preconfigured;

[0008] associating an electricity meter with a VEE rule selected from the VEE rule table;

[0009] The Spark distributed computing framework verifies and estimates the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and stores the results in the VEE result table, and if there is alarm information, writes the alarm information into the alarm information table; wherein the results include verification passed, verification failed, and unverified status marks and their corresponding meter data.

[0010] Optionally, it also includes:

[0011] Displaying the meter data in the VEE result table with the status marked as failed verification and / or unverified through the front-end page of the MDM system;

[0012] The front-end page receives the modification result of the electric meter data corresponding to the state marked as verification failed and / or unverified by the user, and sends it to the Spark distributed computing framework;

[0013] The Spark distributed computing framework re-verifies and estimates the modification results according to the corresponding VEE rules, and updates the results to the VEE result table.

[0014] Optionally, it also includes:

[0015] The Spark distributed computing framework updates and stores the current electricity meter data into VEE historical data.

[0016] Optionally, the associating the electricity meter with a VEE rule selected from the VEE rule table comprises:

[0017] Selecting a VEE rule from the VEE rule table;

[0018] Select an electric meter and its data items to be verified;

[0019] The selected VEE rule is associated with the selected meter and its data item to be verified.

[0020] Optionally, the pre-configuration includes a VEE rule table including two or more VEE rules, including:

[0021] Through the VEE rule configuration canvas on the front-end page, customize the VEE rule name, validation and estimation rules, and execution logic to obtain a VEE rule;

[0022] The VEE rule is stored in the VEE rule table.

[0023] Optionally, the Spark distributed computing framework verifies and estimates the current meter data according to a VEE rule associated with the meter corresponding to the current meter data, before that, further comprising:

[0024] Collect and obtain the original data of the electric meter through the HTTP interface and / or parse the FTP file;

[0025] After parsing the original data, it is sent to the Kafka message queue;

[0026] The DorisDB database of the Spark distributed computing framework consumes the electric meter data in the Kafka message queue in real time.

[0027] Optionally, it also includes:

[0028] The asset information of each electricity meter is pre-entered into the MDM system, wherein the asset information includes basic information of the electricity meter, information of the customer to which it belongs, regional information, and power grid information.

[0029] Optionally, the Spark distributed computing framework verifies and estimates the current meter data according to a VEE rule associated with the meter corresponding to the current meter data, including:

[0030] For the first data without corresponding asset information, the Spark distributed computing framework stores the first data in a no-asset data table and performs alarm processing on the first data;

[0031] For the second data with corresponding asset information, no corresponding VEE rules and no meter data, the Spark distributed computing framework stores the second data in the VEE historical data;

[0032] For third data other than the first data and the second data, the Spark distributed computing framework verifies and estimates the third data according to a VEE rule associated with an electric meter corresponding to the third data.

[0033] Another technical solution provided by the present invention is:

[0034] A meter data VEE function implementation system based on a big data framework, including an MDM system and a Spark distributed computing framework;

[0035] The MDM system is used to preconfigure a VEE rule table including two or more VEE rules through a front-end page, and associate an electric meter with a VEE rule selected from the VEE rule table;

[0036] The Spark distributed computing framework is used to verify and estimate the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and store the results in the VEE result table, and if there is alarm information, write the alarm information into the alarm information table; wherein the results include verification passed, verification failed and unverified status marks and their corresponding meter data.

[0037] Optionally, the MDM system is further used to display the meter data in the VEE result table whose status is marked as failed verification and / or unverified through the front-end page; and receive the modification result of the user corresponding to the meter data whose status is marked as failed verification and / or unverified through the front-end page, and send it to the Spark distributed computing framework;

[0038] The Spark distributed computing framework is also used to re-verify and estimate the modification results according to the corresponding VEE rules, and update the results to the VEE result table.

[0039] The beneficial effects of the present invention are: supporting pre-configuration of multiple VEE rules, and by associating the electric meter with the selected VEE rules, the Spark distributed computing framework can perform personalized verification and estimation on the electric meter data according to the VEE rules associated with the electric meter, thereby achieving more targeted and flexible processing of the electric meter data, better meeting the processing requirements of different electric meter data, and improving the accuracy of VEE processing of the electric meter data; further, the electric meter data that fails the verification and estimation will be marked and displayed, and manual review and modification of this part of the data is supported, as well as re-verification and estimation of the modified electric meter data, thereby improving data quality and increasing the VEE pass rate of the electric meter data; further, the present invention also makes full use of the big data technology framework, and adopts a distributed computing method to synchronously perform VEE verification, estimation and editing of data, thereby significantly improving the efficiency, accuracy and reliability of electric meter data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a method for implementing the VEE function of electric meter data based on a big data framework provided by an embodiment of the present invention;

[0041] Figure 2 A flowchart of a method for implementing the VEE function of electric meter data based on a big data framework provided by a specific embodiment of the present invention;

[0042] Figure 3 An example diagram of the VEE rule configuration canvas interface provided for a specific example of the present invention;

[0043] Figure 4 A schematic diagram of the process of synchronizing the original data of the electric meter according to an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a flow chart of verification and estimation of electric meter data according to an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the data flow in the Kafka message queue consumed by the Spark distributed computing framework described in an embodiment of the present invention;

[0046] Figure 7 An example diagram of the front-end interface for manually editing and estimating VEE results provided for a specific example of the present invention;

[0047] Figure 8 This is a flowchart of the secondary processing of abnormal electric meter data provided by a specific example of the present invention. DETAILED DESCRIPTION

[0048] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in conjunction with the implementation modes and the accompanying drawings.

[0049] Explanation of technical terms involved in the present invention:

[0050]

[0051]

[0052] Please refer to Figure 1 , Figure 1 The present invention provides a flow chart of a method for realizing the VEE function of electric meter data based on a big data framework. Figure 1 As shown, the present embodiment provides a method for realizing the VEE function of electric meter data based on a big data framework, which includes at least the following steps S1 to S3.

[0053] In step S1, a VEE rule table including two or more VEE rules is preconfigured.

[0054] Among them, a VEE rule refers to a VEE process that includes complete VEE verification and estimation logic rules. Users can pre-customize and configure a variety of VEE rules for different meter data, and store them uniformly in the VEE rule table for subsequent use. In particular, the operation of pre-configuring VEE rules supports users to flexibly configure through the front-end page to better meet different meter data processing requirements.

[0055] In step S2, an electricity meter is associated with a VEE rule selected from the VEE rule table.

[0056] The association can also be presented and configured in a visual form, that is, the user is also supported to perform the association operation through the front-end page. Optionally, the data items that need to be verified by the electric meter can also be associated and configured at the same time.

[0057] In step S3, the Spark distributed computing framework verifies and estimates the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and stores the results in the VEE result table, and if there is alarm information, writes the alarm information into the alarm information table; wherein the results include verification passed, verification failed, and unverified status marks and their corresponding meter data.

[0058] In this embodiment, not only can the corresponding VEE rules be used to verify and estimate different meter data, to achieve personalized VEE processing of meter data and improve the accuracy of VEE results; but also the meter data that fails the verification and estimation will be marked and displayed for secondary processing; at the same time, the Spark distributed computing framework is used for VEE processing of meter data, which can significantly improve the efficiency and accuracy of data processing and achieve rapid response.

[0059] In some specific implementations of this embodiment, Figure 2 As shown, after the above step S3, the following steps S4 to S6 are also included.

[0060] In step S4, the meter data in the VEE result table whose status is marked as verification failed and / or unverified is displayed through the front-end page of the MDM system.

[0061] Here, users are supported to intuitively check abnormal meter data through the front-end page, which not only makes it convenient for users to easily grasp the VEE situation of meter data; it also makes it convenient for users to modify the number of abnormal meters more quickly and specifically in the subsequent secondary processing flow.

[0062] In step S5, the front-end page receives the modification result of the electric meter data corresponding to the status marked as verification failed and / or unverified by the user, and sends it to the Spark distributed computing framework;

[0063] In step S6, the Spark distributed computing framework re-verifies and estimates the modification results according to the corresponding VEE rules, and updates the results to the VEE result table.

[0064] The above specific implementation method not only supports manual review and modification of the abnormal meter data, but also can re-verify and estimate the modified meter data. Therefore, it is not only convenient for users to grasp the VEE situation of meter data, but also can improve the quality of meter data and increase the VEE pass rate of meter data.

[0065] In some specific implementations of this embodiment, the above step S4 also includes:

[0066] The Spark distributed computing framework updates and stores the current electricity meter data into VEE historical data.

[0067] That is, while the Spark distributed computing framework is performing VEE calculations, it will also synchronously update and store the meter data currently performing VEE calculations into the VEE historical data to support the retrieval and tracing of VEE historical data when necessary.

[0068] In some further specific implementations of this embodiment, the above step S1 specifically includes:

[0069] S11: Obtain a VEE rule by customizing the VEE rule name, validation and estimation rules, and execution logic through the VEE rule configuration canvas on the front-end page;

[0070] S12: Store the VEE rule into the VEE rule table.

[0071] The above is the specific process of configuring a VEE rule, and it supports configuring and storing multiple VEE rules at the same time for subsequent assignment to specific meters. It can be seen that the configuration of VEE rules is a visual presentation effect throughout the process, and user operation will be more convenient, efficient and flexible.

[0072] In some further specific implementations of this embodiment, the above step S2 specifically includes:

[0073] S21: Select a VEE rule from the VEE rule table;

[0074] S22: Select an electric meter and its data items to be verified;

[0075] S23: Associating the selected VEE rule with the selected electricity meter and the data item to be verified.

[0076] In particular, the above steps can be completed by operating in the rule configuration area of ​​the front-end page to achieve configuration visualization, and more efficiently and conveniently complete the designation of the meter and its corresponding VEE rules, as well as the designation of specific verification data items, thereby achieving flexible configuration of the meter data VEE function.

[0077] In some specific implementations of this embodiment, the above step S3 also includes obtaining VEE meter data. The specific process includes:

[0078] S31: collecting and obtaining the original data of the electric meter through the HTTP interface and / or parsing the FTP file;

[0079] That is, the original data collected by the electric meter is pulled by parsing the FTP file and / or receiving it through the HTTP interface.

[0080] S32: After standardizing and parsing the original data, the data is sent to the Kafka message queue;

[0081] S33: The DorisDB database of the Spark distributed computing framework consumes the meter data in the Kafka message queue in real time.

[0082] Here, the so-called "consumption" refers to whether the Spark distributed computing framework executes the verification and estimation process of the meter data according to the corresponding VEE rules, or other predetermined processes.

[0083] See also Figure 3 The embodiment of the present invention is further expanded based on the above embodiment. This embodiment provides a method for realizing the VEE function of electric meter data based on a big data framework, which specifically includes the following steps:

[0084] 1. Generate and store VEE rules.

[0085] The specific steps are as follows:

[0086] (1) Pre-enter the asset information of each meter device in the MDM system, wherein the asset information includes basic information of the meter, information of the customer to which it belongs, regional information, and grid information.

[0087] (2) In the rule configuration area of ​​the MDM system, select the meters that need to be configured with VEE rules and the data items that need to be verified by the meters.

[0088] (3) Enter the VEE rule configuration canvas of the MDM system, configure the verification and estimation rules according to the requirements, and connect them according to certain logic to form a VEE verification flow chart to complete the configuration of a VEE rule. Here, multiple VEE rules can be configured and stored in the VEE rule table. As a specific example of the VEE rule configuration canvas interface, please refer to Figure 3 .

[0089] (4) Associating the selected electric meter and its data items to be verified with the configured VEE rules. Optionally, the association relationship is stored in a relational database such as MySQL, Oracle, etc., which can ensure storage security and facilitate fast call.

[0090] 2. Synchronize the original data of the electric meter.

[0091] like Figure 4 As shown in the figure, the data synchronization system pulls the raw data collected by the meter through FTP file parsing and HTTP interface reception, and sends it to the Kafka message queue after standardized parsing; DorisDB consumes Kafka data in real time through Rountine Load mode and summarizes it into the raw data table. Here, the data synchronization system supports multiple HES sources and multiple data types.

[0092] 3. Verification and estimation of meter data.

[0093] like Figure 5 As shown in the figure, the original data table, VEE rule table and VEE historical database are aggregated through the Spark distributed computing framework; the meter data obtained from the original data table is verified and estimated according to the specified VEE rules, and the calculation results are stored in the VEE result table of DorisDB for subsequent use by the MDM data analysis module. At the same time, the currently calculated meter data will be updated and stored in the VEE historical data; and for information that needs to be alarmed, it will be written into the alarm information table for subsequent alarm information sending, alarm information review and tracing.

[0094] Among them, the VEE result table will be marked with status according to the VEE calculation results, which can be roughly divided into three types: verification passed, verification failed, and unverified. The front-end interface can filter, extract and display the corresponding meter data according to the status mark.

[0095] Specifically, the above process will be implemented by consuming data in the Kafka message queue through the Spark distributed computing framework, such as Figure 6 As shown, the specific process is as follows:

[0096] (1) Data in the Kafka message queue is imported into the original data table in real time;

[0097] (2) The data in the original data table will be retrieved and processed by the Spark distributed computing framework through scheduled tasks;

[0098] (3) The Spark distributed computing framework is connected to the MDM system and BI system. After retrieving the meter data,

[0099] For the first data without corresponding asset information and specified VEE rules but with specific meter data, it is stored in the asset-free data table, and alarm processing is performed on this part of the meter data, that is, it is written into the alarm information table of the MDM system;

[0100] For the second data with corresponding asset information and no specified VEE rules or specific meter data, store it in VEE historical data;

[0101] For the third data other than the first data and the second data, namely, "there is corresponding asset information, no specified VEE rules but specific meter data", "there is corresponding asset information, there is a specified VEE rule but no specific meter data" and "there is corresponding asset information, there is a specified VEE rule and there is specific meter data". The Spark distributed computing framework uses the VEE rules associated with the meter corresponding to the third data to verify and estimate the third data, that is, to perform VEE; then, the VEE results are stored in the VEE result table, and the meter data with "V success" will be stored in the "V success history data table"; at the same time, the meter data that needs to be alarmed in the VEE results are written into the alarm information table of the MDM system.

[0102] (4) Connect VEE historical data, VEE result table, VEE success historical data table and alarm information table through the BI system; at the same time, the MDM system will also establish a connection with the BI system through the JDBC Catalog database. In this way, it is possible to support the collection, analysis and display of the above data in a visual way, and support users to view and edit the above data.

[0103] It can be understood that the Spark distributed computing framework will be able to synchronously execute VEE functions, update data storage, and register alarms, significantly improving the efficiency, accuracy, and reliability of meter data processing.

[0104] 4. Secondary processing of abnormal meter data.

[0105] First, for the meter data marked as failed verification and unverified in the VEE result table, manual editing and estimation are supported on the front-end interface of the MDM system for secondary processing. Through this stage, the abnormal meter data can be reviewed and modified, and any errors or inconsistencies that have been found can be manually corrected. In this way, timely error correction can be achieved, improving the quality of meter data and the VEE pass rate of meter data. As a specific example, the front-end interface example of manual editing and estimation of VEE results is as follows Figure 7 shown.

[0106] Secondly, the manually modified meter data will be sent to the Kafka message queue again, consumed by Spark and calculated in real time, and the calculation results will be stored in the VEE result table of DorisDB. Here, by performing a second VEE on the corrected abnormal meter data, the processing rate of the meter data can be improved to a certain extent, so that all meter data can be analyzed by VEE as much as possible, and the management of meter data can be optimized. As a specific example, the process example of the secondary processing of abnormal meter data is shown in the figure below. Figure 8 shown.

[0107] 5. Storage and query of meter data after VEE.

[0108] For the meter data after VEE processing, that is, the VEE result table, Doris's Unique data model (an aggregation model) will be used, and the write-time merge technology will be adopted to achieve the best query performance. The data in the VEE result table is partitioned by time and bucketed by the unique serial number of the meter, so that the data storage is evenly distributed, improving the efficiency of storage and query. In addition, Doris data query supports MySQL syntax and provides JDBC Catalog to connect to other data sources for fast access to external data and multi-source data aggregation.

[0109] Based on the above embodiment, the embodiment of the present invention further provides a system for realizing VEE function of electric meter data based on a big data framework, including an MDM system and a Spark distributed computing framework;

[0110] The MDM system is used to preconfigure a VEE rule table including two or more VEE rules through a front-end page, and associate an electric meter with a VEE rule selected from the VEE rule table;

[0111] The Spark distributed computing framework is used to verify and estimate the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and store the results in the VEE result table, and if there is alarm information, write the alarm information into the alarm information table; wherein the results include verification passed, verification failed and unverified status marks and their corresponding meter data.

[0112] In some specific implementations, the MDM system is further used to display the meter data in the VEE result table whose status is marked as failed verification and / or unverified through the front-end page; and receive the modification result of the user corresponding to the meter data whose status is marked as failed verification and / or unverified through the front-end page, and send it to the Spark distributed computing framework;

[0113] The Spark distributed computing framework is also used to re-verify and estimate the modification results according to the corresponding VEE rules, and update the results to the VEE result table.

[0114] In some specific implementations, the Spark distributed computing framework is also used to update and store the current electricity meter data into VEE historical data.

[0115] In some specific implementations, the MDM system is further used to select a VEE rule from the VEE rule table; select an electricity meter and its data items to be verified; and associate the selected VEE rule with the selected electricity meter and its data items to be verified.

[0116] In some specific implementations, the MDM system is further used to customize the VEE rule name, validation and estimation rules, and execution logic through the VEE rule configuration canvas of the front-end page to obtain a VEE rule; and store the VEE rule in the VEE rule table.

[0117] In some specific implementations, the Spark distributed computing framework is also used to collect and obtain the original data of the electric meter through the HTTP interface and / or parse the FTP file; and after parsing the original data, send it to the Kafka message queue; and consume the electric meter data in the Kafka message queue in real time through the DorisDB database.

[0118] In some specific implementations, the MDM system is also used to pre-enter asset information of each electricity meter, wherein the asset information includes basic information of the electricity meter, information about the customer to which the meter belongs, regional information, and grid information.

[0119] In some specific implementations, the Spark distributed computing framework is further used to:

[0120] For the first data without corresponding asset information, the Spark distributed computing framework stores the first data in a no-asset data table and performs alarm processing on the first data;

[0121] For the second data with corresponding asset information, no corresponding VEE rules and no meter data, the Spark distributed computing framework stores the second data in the VEE historical data;

[0122] For third data other than the first data and the second data, the Spark distributed computing framework verifies and estimates the third data according to a VEE rule associated with an electric meter corresponding to the third data.

[0123] In summary, the present invention provides an efficient method and system for realizing the VEE (verification, estimation, editing) function of electric meter data based on the big data framework. Faced with the increasingly complex challenges of smart grid data, traditional technologies are unable to cope with the efficient processing and storage of large-scale data sets, while the present invention makes full use of the big data technology framework to significantly improve the efficiency, accuracy and reliability of data processing. Users can flexibly configure VEE rules through the front-end page to adapt to different electric meter data processing requirements; it can synchronize original data in real time, verify, estimate and edit data through a distributed computing framework, and use Doris's Unique data model to optimize storage and query; it supports multi-source data aggregation, has good scalability and fault tolerance, and can quickly respond to abnormal situations. In general, the present invention provides an advanced and efficient big data processing solution for the electric meter MDM system, significantly improves data quality and system reliability, and provides practical innovative technologies for the field of smart grids.

[0124] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for realizing VEE function of electric meter data based on big data framework, It is characterized in that include: A VEE rule table including two or more VEE rules is preconfigured; associating an electricity meter with a VEE rule selected from the VEE rule table; The Spark distributed computing framework verifies and estimates the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and stores the results in the VEE result table, and if there is alarm information, writes the alarm information into the alarm information table; wherein the results include verification passed, verification failed, and unverified status marks and their corresponding meter data.

2. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that Also includes: Displaying the meter data in the VEE result table with the status marked as failed verification and / or unverified through the front-end page of the MDM system; The front-end page receives the modification result of the electric meter data corresponding to the state marked as verification failed and / or unverified by the user, and sends it to the Spark distributed computing framework; The Spark distributed computing framework re-verifies and estimates the modification results according to the corresponding VEE rules, and updates the results to the VEE result table.

3. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that Also includes: The Spark distributed computing framework updates and stores the current electricity meter data into VEE historical data.

4. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that The associating an electric meter with a VEE rule selected from the VEE rule table comprises: Selecting a VEE rule from the VEE rule table; Select an electric meter and its data items to be verified; The selected VEE rule is associated with the selected meter and its data item to be verified.

5. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that The pre-configuration includes a VEE rule table of two or more VEE rules, including: Through the VEE rule configuration canvas on the front-end page, customize the VEE rule name, validation and estimation rules, and execution logic to obtain a VEE rule; The VEE rule is stored in the VEE rule table.

6. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that The Spark distributed computing framework verifies and estimates the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and before that, it also includes: Collect and obtain the original data of the electric meter through the HTTP interface and / or parse the FTP file; After parsing the original data, it is sent to the Kafka message queue; The DorisDB database of the Spark distributed computing framework consumes the electric meter data in the Kafka message queue in real time.

7. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 1, It is characterized in that Also includes: The asset information of each electricity meter is pre-entered into the MDM system, wherein the asset information includes basic information of the electricity meter, information of the customer to which it belongs, regional information, and power grid information.

8. A method for realizing the VEE function of electric meter data based on a big data framework as claimed in claim 7, It is characterized in that The Spark distributed computing framework verifies and estimates the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, including: For the first data without corresponding asset information, the Spark distributed computing framework stores the first data in a no-asset data table and performs alarm processing on the first data; For the second data with corresponding asset information, no corresponding VEE rules and no meter data, the Spark distributed computing framework stores the second data in the VEE historical data; For third data other than the first data and the second data, the Spark distributed computing framework verifies and estimates the third data according to a VEE rule associated with an electric meter corresponding to the third data.

9. A system for realizing VEE function of electric meter data based on big data framework, It is characterized in that Includes MDM system and Spark distributed computing framework; The MDM system is used to preconfigure a VEE rule table including two or more VEE rules through a front-end page, and associate an electric meter with a VEE rule selected from the VEE rule table; The Spark distributed computing framework is used to verify and estimate the current meter data according to the VEE rules associated with the meter corresponding to the current meter data, and store the results in the VEE result table, and if there is alarm information, write the alarm information into the alarm information table; wherein the results include verification passed, verification failed and unverified status marks and their corresponding meter data.

10. A system for realizing VEE function of electric meter data based on big data framework as claimed in claim 9, It is characterized in that The MDM system is further used to display the meter data in the VEE result table whose status is marked as verification failed and / or unverified through the front-end page; and receiving, through the front-end page, modification results of the electric meter data corresponding to the status marked as verification failed and / or unverified by the user, and sending the modification results to the Spark distributed computing framework; The Spark distributed computing framework is also used to re-verify and estimate the modification results according to the corresponding VEE rules, and update the results to the VEE result table.

Images