Automatic data quality inspection method and device, equipment and storage medium

A data quality and data technology, applied in the field of big data, can solve problems such as the inability to achieve efficient and accurate quality inspection, and achieve the effects of quality inspection efficiency and quality improvement, strong adaptability to changes, and low quality inspection costs

Inactive Publication Date: 2021-07-16
数库(上海)科技有限公司
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The present invention aims at the technical problem that high-efficiency and accurate quality inspection cannot be realized through manual quality inspection or ETL script when a large amount of data needs quality inspection, and the purpose is to provide a method, device, equipment and storage medium for automatic data quality inspection

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic data quality inspection method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention will be further elaborated below in conjunction with specific diagrams.

[0041] refer to figure 1 , a method for automated data quality inspection, comprising:

[0042] S1. Obtain data and quality inspection rules: trigger the quality inspection service, obtain data characteristics, obtain multiple preset quality inspection rules according to data characteristics, and obtain data requiring quality inspection through preset data query methods.

[0043] The quality inspection rules in this step are the quality inspection rules pre-stored in the storage module, these quality inspection rules can adopt default rules, and users can also customize configurations according to needs.

[0044] The data features in this step include data features that can identify different types of data, such as data classification or data ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of big data, and particularly relates to an automatic data quality inspection method and device, equipment and a storage medium. The method comprises the steps of triggering a quality inspection service, obtaining data features, obtaining a plurality of preset quality inspection rules according to the data features, and obtaining data needing quality inspection through a preset data query mode; executing each quality inspection rule, respectively verifying the data, obtaining a quality inspection result and storing the quality inspection result; and displaying the quality inspection result. According to the method, quality inspection requirements of most production data are basically met through custom configuration of the quality inspection rules, and in the whole data production link, after the method is accessed, the quality inspection cost is lower, and the quality inspection efficiency and quality are greatly improved.

Description

technical field [0001] The invention belongs to the technical field of big data, and in particular relates to a method, device, equipment and storage medium for automatic data quality inspection. Background technique [0002] In the current context of big data, data accuracy must be checked before data analysis, and data quality is a key indicator to ensure data application. How to evaluate whether the data meets the expected quality requirements requires a way to test the data. At present, there are two types of quality inspection methods that are most commonly used: the first type is manual quality inspection, and the second type is to implement some quality inspection logic through ETL to achieve the purpose of data quality inspection. To sum up, both have their own shortcomings. [0003] When there is a large amount of data that needs quality inspection, the manual quality inspection in the traditional method obviously cannot control the efficiency and accuracy well. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215
CPCG06F16/215
Inventor 黄夫龙曹峰
Owner 数库(上海)科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap