Data quality detection method and apparatus

A technology of data quality and detection method, which is applied in the field of data processing, can solve the problems of low test coverage, low efficiency, and high cost, and achieve the effects of improving detection efficiency, wide coverage, and reducing labor and time costs

Active Publication Date: 2017-05-24
ALIBABA GRP HLDG LTD
View PDF4 Cites 45 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since this method is human participation, abnormal data may be missed, so the test coverage is low, and test cases need to be repeated for different fields and repaired data, so the efficiency is low and the cost is high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data quality detection method and apparatus
  • Data quality detection method and apparatus
  • Data quality detection method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] Embodiments of the present application are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary, and are only for explaining the present application, and should not be construed as limiting the present application.

[0021] The method and device for detecting data quality according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0022] figure 1 It is a flowchart of a data quality detection method according to an embodiment of the present application.

[0023] Such as figure 1 As shown, the data quality detection method according to the embodiment of the present application includes the following steps.

[0024] S101. Acquire metadata of the data to be detected, where...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data quality detection method and apparatus. The data quality detection method comprises the following steps of obtaining metadata of to-be-detected data, wherein the metadata comprises N fields of the to-be-detected data and data types of the fields, and N is a positive integer; searching for at least one detection rule corresponding to each field in a pre-established detection rule library according to a preset matching policy and the data type of each field; and performing quality detection on each field according to the at least one detection rule corresponding to each field, thereby obtaining detection results of the N fields and performing display. According to the data quality detection method, the coverage rate and universality of detection can be effectively improved, the detection efficiency can be improved, and the manpower and time costs are reduced.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a method and device for detecting data quality. Background technique [0002] With the advent of the data age, data information is developing towards mass and diversification. Abnormal data may lead to serious losses or accidents. Therefore, in the data research and development system, the detection of data quality is a very important link. [0003] At present, for the detection of data quality, testers usually need to construct a comprehensive test case for each field according to the business logic, that is, the tester writes the code and execution logic of the test case, and the tester judges whether there is an exception in the execution result , Whether to meet business needs. In addition, after the manual test is completed, if it involves the repair of abnormal data, it is necessary to manually test the regression again after the data repair, and repeate...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/215
Inventor 曲丹鹤
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap