Data quality analysis system and method

A data quality and analysis system technology, applied in the field of big data, can solve problems such as multiple configurations and multiple records of abnormal data, and achieve the effects of improving execution performance, reducing resource overhead, and avoiding repeated reading.

Pending Publication Date: 2022-04-22
MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the embodiments of the present application is to provide a data quality analysis system and method, which can solve the problem of multiple configurations and repeated reading of data in related technologies when performing data quality analysis, resulting in abnormal data being recorded multiple times

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data quality analysis system and method
  • Data quality analysis system and method
  • Data quality analysis system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0015] The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

[0016] The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data quality analysis system and method.The system comprises a data warehouse, a metadata management module, a rule analysis module, a data collection module, a monitoring task transfer module and a data quality analysis module, and the metadata management module is used for storing rule metadata and data source metadata provided by the data warehouse; the rule metadata comprises a plurality of monitoring rules, each monitoring rule is configured with a corresponding rule attribute, and rule relations are established among the monitoring rules through the rule attributes; the rule analysis module is used for generating rule statements which can be recognized by the rule engine according to the rule metadata; the data acquisition module is used for data source metadata to acquire data source data from a data warehouse; the rule engine is used for registering a monitoring task according to a rule statement and executing the monitoring task on the data of the data source, and the monitoring task executes each monitoring rule on the data of the data source according to a rule relation; and the data quality analysis module is used for performing quality analysis on the data of the data source.

Description

technical field [0001] This application belongs to the field of big data technology, and specifically relates to a data quality analysis system and method. Background technique [0002] The data quality of a data set refers to the degree to which the data in the data set are suitable for use and meet the expectations of specific users. [0003] The data quality analysis framework in related technologies mainly provides the function of dynamically formulating and modifying data quality monitoring rules, but lacks the management of the relationship between the rules. For example, when a certain field needs to monitor multiple aspects such as non-emptiness, length, and size at the same time, because there is no connection between the monitoring rules, the current data quality analysis framework requires multiple configurations and repeated reading of data, resulting in An issue where exception data was logged multiple times. Contents of the invention [0004] The purpose of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/30G06F16/28
CPCG06F11/302G06F11/3051G06F11/3065G06F16/283
Inventor 全博楷刘楠
Owner MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap