Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data quality governance system and method based on message bus

A technology of data quality and message bus, which is applied in the field of data quality management system based on message bus, and can solve problems such as poor credibility

Active Publication Date: 2021-02-05
WUHAN NARI LIABILITY OF STATE GRID ELECTRIC POWER RES INST +1
View PDF6 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a data quality management system and method based on a message bus, and the present invention can solve the problem of poor credibility of power system monitoring data quality assessment

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data quality governance system and method based on message bus
  • Data quality governance system and method based on message bus
  • Data quality governance system and method based on message bus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056] Below in conjunction with specific embodiment the present invention is described in further detail:

[0057] A kind of data quality management system based on the message bus designed by the present invention, such as figure 1 As shown, it includes a data access module 1, a verification rule building module 2, a clustering division module 3, a quality evaluation module 4 and a comprehensive evaluation module 5, wherein the data access module 1 is used to receive through the KAFKA message bus Power monitoring data sent by multiple data sending ends, and stored in categories;

[0058] The verification rule building module 2 is used to construct the data quality rule verification library according to the preset evaluation indicators. The preset evaluation indicators include accuracy, completeness, consistency, usability and real-time performance, and establish corresponding verification according to each indicator. Rules, such as integrity evaluation including field integ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a data quality governance system based on a message bus. The system performs the steps of receiving power monitoring data sent by a plurality of data sending ends through themessage bus and stored in a classified mode; determining a power monitoring data quality evaluation index, and constructing a data quality rule verification library; clustering the power monitoring data by adopting a k-means clustering method optimized on the basis of a grey wolf longhorn beard search algorithm, and dividing the power monitoring data into a plurality of data subsets; for each datasubset, carrying out accuracy evaluation through an LOF abnormal value detection algorithm, and carrying out integrity, consistency, availability and real-time evaluation through data quality rule verification; and calculating the weight of each evaluation index through an improved analytic hierarchy process, and calculating a comprehensive evaluation result of the power monitoring data quality.According to the system, the multi-dimensional quality evaluation of the power monitoring data is realized, and the reliability of the quality management of the power monitoring data is improved.

Description

[0001] technology neighborhood [0002] The invention relates to the technical field of automation information systems, in particular to a data quality management system and method based on a message bus. Background technique [0003] With the increasing number of automated information systems, the scale and capacity of power systems are increasing, and the amount of information is increasing. With the vigorous promotion of monitoring terminals, the characteristics of monitoring objects are different, and the early warning function of monitoring data correlation calculation is gradually highlighted. With the rapid growth of data, the data quality of different data sources is uneven, and data quality has different concepts and standards. Data quality generally includes quantitative descriptions such as data accuracy, completeness, timeliness, and consistency. [0004] For power systems in the era of big data, the amount of data that needs to be stored and processed is large, a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06Q10/06G06Q50/06G06K9/62G06N3/00
CPCG06Q10/06393G06Q10/06395G06Q50/06G06N3/006G06F18/23213Y04S10/50
Inventor 陶汉涛王海涛方玉河李健张磊林卿陈玥姜志博吴大伟李旺
Owner WUHAN NARI LIABILITY OF STATE GRID ELECTRIC POWER RES INST
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com