Data quality analysis system and method

A data quality and analysis system technology, applied in the field of big data, can solve problems such as multiple configurations and multiple records of abnormal data, and achieve the effects of improving execution performance, reducing resource overhead, and avoiding repeated reading.
CN114385437APending Publication Date: 2022-04-22MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
MICRO DREAM TECHTRONIC NETWORK TECH CHINACO
Publication Date
2022-04-22

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a data quality analysis system and method.The system comprises a data warehouse, a metadata management module, a rule analysis module, a data collection module, a monitoring task transfer module and a data quality analysis module, and the metadata management module is used for storing rule metadata and data source metadata provided by the data warehouse; the rule metadata comprises a plurality of monitoring rules, each monitoring rule is configured with a corresponding rule attribute, and rule relations are established among the monitoring rules through the rule attributes; the rule analysis module is used for generating rule statements which can be recognized by the rule engine according to the rule metadata; the data acquisition module is used for data source metadata to acquire data source data from a data warehouse; the rule engine is used for registering a monitoring task according to a rule statement and executing the monitoring task on the data of the data source, and the monitoring task executes each monitoring rule on the data of the data source according to a rule relation; and the data quality analysis module is used for performing quality analysis on the data of the data source.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] This application belongs to the field of big data technology, and specifically relates to a data quality analysis system and method. Background technique

[0002] The data quality of a data set refers to the degree to which the data in the data set are suitable for use and meet the expectations of specific users.

[0003] The data quality analysis framework in related technologies mainly provides the function of dynamically formulating and modifying data quality monitoring rules, but lacks the management of the relationship between the rules. For example, when a certain field needs to monitor multiple aspects such as non-emptiness, length, and size at the same time, because there is no connection between the monitoring rules, the current data quality analysis framework requires multiple configurations and repeated reading of data, resulting in An issue where exception data was logged multiple times. Contents of the invention

[0004] The purpose of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More