A data quality problem reporting system

Abstract

This patent application discloses a data quality problem reporting system, including a metadata processing module, used to extract and define the logical data model and subject domain of the data source to be detected from the data source; a template processing module, used to process the data from the metadata The defined logical data model and subject domain obtained in the module are stored separately to form a logical data model template and a subject domain template; the report processing module is used to obtain quality problems from the data quality problem detection system and obtain them from the metadata processing module After the defined logical data model and subject domain are obtained, the logical data model template and the subject domain template are obtained from the template processing module; a data quality problem report is formed and displayed according to the quality problem defined by the logical data model and subject domain. This application can organize data quality problem reports that meet user needs in a variety of different ways.

Description

technical field [0001] The invention relates to the field of data quality monitoring, in particular to a data quality problem reporting system used in an ETL process. Background technique [0002] With the rapid development of information technology, data has gradually become one of the most important resources to realize the business value of enterprises. However, as the amount of data continues to increase, data quality issues also follow. Quality problems such as data loss, errors, and inconsistencies hinder the accurate application of data by enterprises, and even cause enterprises to make wrong decisions, lose important values ​​and cause a crisis of trust. [0003] We call these data with quality problems dirty data. For these dirty data, many data quality detection and data cleaning solutions have emerged as the times require. However, the presentation and reporting of data quality issues is a challenge that cannot be ignored, whether it is for a data quality inspec...

AI Technical Summary

Problems solved by technology

[0006] The existing commonly used data quality reporting system detects data quality problems in the form of workflow, and the exported problem report uses each component that detects different quality problems in the workflow to classify the problem categories. This classification method is too rigid and cannot be flexibly To deal with different business data, for the situation where the amount of data is too large and the types of data are complicated, the inherent component classification is not conducive to the segmentation and understanding of wrong data

Although other data quality reporting systems can number and annotate different types of data quality problems, they still have not solved the problem that the same data has been marked with data topics many times, resulting in a large workload and lengthy reports

In addition, the existing data quality problem reporting system cannot display the problem of specifying quality problems according to user needs

Images