Financial power transmission right clearing data quality monitoring device and method based on cloud platform

Abstract

The invention discloses a financial power transmission right clearing data quality monitoring device and method based on a cloud platform. The device comprises a data integration monitoring module, a data quality verification module and a data model pushing module. The data integration monitoring module is used for performing data integration management on each source end system, developing, configuring and monitoring a data integration task through a Web page, and realizing dynamic interface configuration, data acquisition and difference comparison; the data quality verification module is used for classifying and setting corresponding service verification rules and technical verification rules according to the electric charge settlement service requirements, configuring verification operation, recording an operation execution log and a quality verification result, and visualizing a quality analysis result; and the data model pushing module is used for cleaning, converting and pushing the data meeting the electric charge settlement business requirements to an electric charge settlement model through a Web page according to a set rule or logic. According to the invention, the efficiency and accuracy of issuing the electricity fee settlement bill in the electricity market can be improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of clearing data quality monitoring, and particularly relates to a financial power transmission right clearing data quality monitoring device and method based on a cloud platform. BACKGROUND

[0002] With the further promotion of power market reform, especially the start of the power spot market, the market presents the characteristics of diversified transaction subjects, diversified transaction varieties, flexible transaction modes, and real-time transaction processes, and the data generated by the power market presents an explosive growth, which puts forward higher requirements for the financial power transmission right clearing data processing.

[0003] In order to improve the efficiency and accuracy of electricity settlement and ensure the efficient operation of the power market, it is necessary to build a perfect data integration monitoring and data quality management system, build an efficient data supervision platform, and strengthen the integration monitoring and quality management of the financial power transmission right clearing data, which has become an important part of the construction of the power market. SUMMARY

[0004] The purpose of the application is to provide a financial power transmission right clearing data quality monitoring device and method based on a cloud platform, which aims to take advantage of the cloud platform to assist in processing the cleaning work of the financial power transmission right clearing data, so as to improve the efficiency and accuracy of the electricity market electricity settlement bill.

[0005] To this end, the application adopts the following technical solutions.

[0006] In a first aspect, the application provides a financial power transmission right clearing data quality monitoring device based on a cloud platform, which comprises a data integration monitoring module, a data quality verification module, and a data model pushing module. The data integration monitoring module comprises a library table management function, an integration management function, and an integration monitoring function, is used for the data integration management of various source systems, develops, configures, and monitors the data integration task through a Web page, realizes dynamic interface configuration, data acquisition, and difference comparison, and realizes the dynamic interface configuration, data acquisition, and difference comparison. The data quality verification module comprises a rule type management function, a business rule management function, a technical rule management function, a verification job configuration function, and a verification job monitoring function, sets corresponding business verification rules and technical verification rules according to the electricity settlement business demand, configures the verification job, records the job execution log and the quality verification result, forms the quality analysis result visualization, and realizes the dynamic interface configuration, data acquisition, and difference comparison. The data model pushing module comprises a pushing management function and a pushing monitoring function, cleans, converts, and pushes the data meeting the electricity settlement business demand to the electricity settlement model according to the set rules or logic through a Web page.

[0007] Further, through the library table management function, the configuration of the specified table and table field in the background is dynamically viewed, including: configuring the data access and integration result function of the related business system, the data push and push result function of the cost table, and the keyword maintenance function of the business system integration table.

[0008] Further, through the integration management function, the data manual acquisition and data comparison function are realized, which is used to realize the query, comparison and access function of the statistical analysis e-file data and database data, and supports the e-file, database and comparison result export.

[0009] Further, through the integration monitoring function, the data integration situation is viewed, which is used to realize the real-time monitoring of whether the integration of each business system data is transmitted completely according to the predetermined frequency requirement, and to statistically analyze the data integration result information.

[0010] Further, through the rule type management function, the data quality characteristics are pre-set according to the related rules of the power market, the data quality characteristics are flexibly configurable through the Web page, and the data quality characteristics are added, modified, deleted and the like, which supports the adjustment of the business rule in the later stage; Through the business rule management function, the conditions that should be met by the front-end business data are proposed from the business logic point of view according to the related rules of the power market, the business rule is flexibly configurable through the Web page, and the business rule is added, modified, deleted and the like, which supports the development demand of the later stage; Through the technical rule management function, the technical implementation of the business rule is used according to the above business rule, one business rule involves one or more technical rules, the technical rule is flexibly configurable through the Web page, and the technical rule is added, modified, deleted and the like, which supports the development demand of the later stage.

[0011] Further, through the verification job configuration function, the data verification rule is managed, which meets the data item management and management demand; in order to improve the verification efficiency, the system allocates threads according to the number of methods involved in the verification job demand; the verification rule is allocated to multiple verification threads, the verification thread is started at the same time, and a companion thread is started at the same time to record the execution log of these verification methods.

[0012] Further, through the verification job monitoring function, the execution state and execution log of the data quality verification job and the associated technical rule are monitored and recorded.

[0013] Further, through the push management function, the integrated data of each business system is pushed to the electricity fee settlement model according to the settlement period and the library table attribute through the Web page by the authorized user.

[0014] Further, by pushing the monitoring function, the data information pushing result is displayed, the detailed information of the pushing can be inquired, so that the data is ensured to be timely, real and correct.

[0015] In a second aspect, the application provides a financial transmission right clearing data quality monitoring method based on a cloud platform, which is implemented by using the financial transmission right clearing data quality monitoring device, and includes the following steps: 1) The market rules are sorted out, and the data required by the electricity fee settlement is determined; 2) The source end data is mapped to the financial transmission right clearing database through the creation of an integrated instruction by a Web page; 3) The integrated instruction configuration is performed through a Web interface according to the integrated frequency requirement; 4) The integrated log is created to monitor the integrated instruction execution condition and the data acquisition condition; 5) The quality requirement is proposed according to the power market transaction and settlement related rules, and one or more data quality business rules are formed; 6) The business rule is converted into one or more data quality technical rules; 7) The business rule and the technical rule are classified, stored and managed; 8) The data quality technical rule is referenced, the quality task is created, the task basic information is defined, and the scheduling information is configured; 9) The data quality check log is created to monitor the check task execution condition; 10) The data quality check result information is counted, analyzed and formed into an information report; 11) The data meeting the electricity fee settlement business requirement is pushed to the electricity fee settlement model according to the set rule or logic through a Web page; 12) The pushing log is created to record the data pushing record.

[0016] The application has the following beneficial effects: the application can promote the whole process of the financial transmission right clearing data processing, and can promote the all-round control and improvement of the electricity fee calculation data quality; the application can improve the efficiency and accuracy of the electricity market electricity fee settlement bill, and provides a premise guarantee for the efficient operation of the electricity market. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a framework diagram of the financial transmission right clearing data quality monitoring device based on the cloud platform of the application; Figure 2 is a flowchart of the financial transmission right clearing data quality monitoring method based on the cloud platform of the application. DETAILED DESCRIPTION

[0018] The present application will be further clarified by the following description of the drawings and specific embodiments, which should be considered not limiting the scope of the application. Modifications of the application will occur to those skilled in the art upon reading the present application, which are intended to be within the scope of the claims.

[0019] Embodiment 1 The embodiment provides a cloud platform-based financial transmission right clearing data quality monitoring device, as shown in the figure, which comprises a data integration monitoring module, a data quality verification module and a data model pushing module. Figure 1 The data integration monitoring module comprises a library table management function, an integration management function and an integration monitoring function, is used for data integration management of each source system, develops, configures and monitors the data integration task through a Web page, and realizes dynamic interface configuration, data acquisition and difference comparison.

[0020] The data integration monitoring module comprises a library table management function, an integration management function and an integration monitoring function, is used for data integration management of each source system, develops, configures and monitors the data integration task through a Web page, and realizes dynamic interface configuration, data acquisition and difference comparison.

[0021] Through the library table management function, the configuration of the specified table and the table field in the background is dynamically viewed, including the functions of configuring the data access and integration result of the related business system, the data pushing and pushing result of the cost table, and the key word maintenance function of the business system integration table.

[0022] Through the integration management function, the data manual acquisition and data comparison function are realized, which is used for realizing the query, comparison and access function of the statistical analysis e-file data and the database data, and supporting the data information export of the e-file, the database and the comparison result.

[0023] Through the integration monitoring function, the data integration situation is viewed, which is used for realizing the real-time monitoring of whether the integration of each business system data is completely transmitted according to the predetermined frequency requirement, and statistically analyzing the data integration result information.

[0024] The data quality verification module comprises a rule type management function, a business rule management function, a technical rule management function, a verification job configuration function and a verification job monitoring function, sets the corresponding business verification rule and the technical verification rule according to the electricity fee settlement business demand, configures the verification job, records the job execution log and the quality verification result, forms the quality analysis result visualization.

[0025] Through the rule type management function, the data quality characteristics are set in advance according to the related rules of the power market, the data quality characteristics are flexibly configured through the Web page, the operations of adding, modifying and deleting the data quality characteristics are realized, and the later business rule adjustment is supported.

[0026] Through the business rule management function, based on the relevant rules of the power market, the conditions that the front-end business data should meet are proposed from the perspective of business logic. The business rules can be flexibly configured through the web page, and operations such as adding, modifying, and deleting business rules can be performed to support future business development needs.

[0027] Through the technical rule management function, based on the above business rules, it is used to technically implement business rules. One business rule involves one or more technical rules. The technical rules can be flexibly configured through the web page, and the addition, modification, and deletion of technical rules can be performed to support the needs of future business development.

[0028] The verification job configuration function is used to implement task management of data verification rules to meet the needs of data item governance and item management; in order to improve verification efficiency, the system allocates threads according to the number of methods involved in the verification job requirements; verification rules will be assigned to multiple verification threads, and the verification threads will be started at the same time, and a companion thread will be started at the same time to record the execution logs of these verification methods.

[0029] The verification job monitoring function is used to monitor and record the execution status and execution logs of data quality verification jobs and associated technical rules.

[0030] The data model push module includes a push management function and a push monitoring function. Through the Web page, the data that meets the electricity bill settlement business requirements is cleaned and converted according to the set rules or logic, and pushed to the electricity bill settlement model.

[0031] Through the push management function, authorized users can push the integrated data of each business system to the electricity bill settlement model according to the settlement cycle and database table attributes through the web page.

[0032] Through the push monitoring function, the data information push results can be displayed, and the detailed information pushed can be queried to ensure that the data is timely, true and correct.

[0033] Example 2 This embodiment provides a method for monitoring the quality of financial transmission rights clearing data based on a cloud platform. Figure 2 As shown, the financial transmission rights clearing data quality monitoring device described in Example 1 is used to implement the method, and the steps are as follows: 1) Sort out market rules and determine the data required for electricity bill settlement; 2) Create integration instructions through the web page to map the source data to the financial transmission rights clearing database; 3) Configure the integration instructions through the web interface according to the integration frequency requirements; 4) Create an integration log to monitor the execution of integration instructions and data acquisition; 5) According to the rules of power market transaction and settlement, quality requirements are proposed, and one or more data quality business rules are formed; 6) The business rules are converted into one or more data quality technical rules; 7) The business rules and technical rules are classified, stored and managed; 8) The data quality technical rules are cited, quality tasks are created, basic information of the tasks is defined, and scheduling information is configured; 9) Data quality check logs are created to monitor the execution of the check tasks; 10) The data quality check result information is counted, analyzed and information reports are formed; 11) Through a Web page, data meeting the business requirements of electricity settlement are cleaned, converted and pushed to an electricity settlement model according to the set rules or logic; 12) Push logs are created to record the data pushing records.

[0034] The above gives specific implementation manners, but the present application is not limited to the described implementation manners. The basic idea of the present application is the above basic scheme, and according to the teaching of the present application, various transformed models, formulas and parameters can be designed without creative labor for those skilled in the art. Changes, modifications, replacements and transformations of the implementation manners without departing from the principles and spirits of the present application still fall within the protection scope of the present application.

Claims

1. A cloud platform-based financial transmission right clearing data quality monitoring device, characterized in that, The data integration monitoring module, the data quality checking module and the data model pushing module are comprised; The data integration monitoring module comprises a library table management function, an integration management function and an integration monitoring function, is used for data integration management of each source system, develops, configures and monitors the data integration task through a Web page, realizes dynamic interface configuration, data acquisition and difference comparison, and realizes dynamic interface configuration, data acquisition and difference comparison. The data quality checking module comprises a rule type management function, a business rule management function, a technical rule management function, a checking job configuration function and a checking job monitoring function, sets corresponding business checking rules and technical checking rules according to the electricity fee settlement business demand, configures the checking job, records the job execution log and the quality checking result, forms the quality analysis result visualization, and forms the quality analysis result visualization. The data model pushing module comprises a pushing management function and a pushing monitoring function, cleans, converts and pushes the data meeting the electricity fee settlement business demand to the electricity fee settlement model according to the set rules or logic through a Web page.

2. The financial power transmission right clearing data quality monitoring device according to claim 1, characterized by, Through the library table management function, the configuration situation of the specified table and the table field in the background is dynamically viewed, including the configuration of the related business system data access and integration result function, the fee table data pushing and pushing result function, and the key word maintenance function of the business system integration table.

3. The financial transmission right settlement data quality monitoring apparatus of claim 1, wherein Through the integration management function, the data manual acquisition and data comparison function are realized, the statistical analysis e file data and the database data query, comparison and access function are realized, and the e file, the database and the comparison result export are supported.

4. The financial transaction clearing data quality monitoring apparatus of claim 1, wherein Through the integration monitoring function, the data integration situation is viewed, the real-time monitoring of whether the integration of each business system data is completely transmitted according to the predetermined frequency requirement is realized, and the data integration result information is counted and analyzed.

5. The financial transaction clearing data quality monitoring apparatus of claim 1, wherein, Through the rule type management function, the data quality characteristics are pre-set according to the related rules of the power market, the data quality characteristics are flexibly configured through a Web page, the data quality characteristics are added, modified and deleted, and the later business rule adjustment is supported. Through the business rule management function, the conditions that should be met by the front-end business data are proposed from the business logic point of view according to the related rules of the power market, the business rules are flexibly configured through a Web page, the business rules are added, modified and deleted, and the later business development demand is supported. Through the technical rule management function, the technical implementation of the business rules is realized according to the above business rules, one business rule involves one or more technical rules, the technical rules are flexibly configured through a Web page, the technical rules are added, modified and deleted, and the later business development demand is supported.

6. The device for monitoring the quality of financial transmission rights clearing data according to claim 1, characterized in that: Through the checking job configuration function, the data checking rule task management is realized, the data sub-item management and sub-item demand are met, the thread distribution is performed according to the number of methods involved in the checking job demand in order to improve the checking efficiency, the checking rules are distributed to multiple checking threads, the checking threads are started at the same time, and a companion thread is started at the same time to record the execution log of the checking methods.

7. The financial transaction clearing data quality monitoring apparatus of claim 1, wherein Through the check job monitoring function, the execution status and execution log of the data quality check job and the associated technical rules are monitored and recorded.

8. The financial transaction clearing data quality monitoring apparatus of claim 1, wherein Through the push management function, the integrated data of each business system is pushed to the electricity settlement model according to the settlement period and the library table attribute by the authorized user through the Web page.

9. The financial transaction clearing data quality monitoring apparatus of claim 1, wherein, Through the push monitoring function, the data information push result is displayed, and the detailed information of the push can be queried to ensure that the data is timely, true and correct.

10. A cloud platform-based financial transmission right clearing data quality monitoring method, implemented by using the financial transmission right clearing data quality monitoring device of any one of claims 1-9, characterized in that, The steps include: 1) Sort out market rules and determine the data required for electricity settlement; 2) Create integration instructions through a Web page to map the source data to the financial transmission right clearing database; 3) Configure integration instructions through a Web interface according to the integration frequency requirement; 4) Create integration log to monitor the execution status of the integration instructions and the data acquisition situation; 5) According to the rules related to electricity market trading and settlement, quality requirements are proposed to form one or more data quality business rules; 6) Convert business rules into one or more data quality technical rules; 7) Classify and store the business rules and technical rules for management; 8) Reference the data quality technical rules to create quality tasks, define the basic information of the tasks, and configure the scheduling information; 9) Create data quality check log to monitor the execution of the check tasks; 10) Statistically analyze the data quality check result information and form an information report; 11) Cleanse and convert the data that meets the business requirements of electricity settlement according to the set rules or logic through a Web page, and push it to the electricity settlement model; 12) Create push log to record the data push record.

Images