Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for detecting data quality of data dependence

A data quality and detection method technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of inability to quickly detect data dependence errors, inability to accurately locate positions, etc., to improve detection speed and efficiency, The structure is simple and the effect of reducing the number of nodes

Active Publication Date: 2016-06-22
GUANGDONG POWER GRID CO LTD INFORMATION CENT
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to provide a data-dependent data quality detection method and device to overcome the above-mentioned technical defects and solve the problems that data-dependent errors cannot be quickly detected and the location of the error cannot be accurately located

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for detecting data quality of data dependence
  • Method and device for detecting data quality of data dependence
  • Method and device for detecting data quality of data dependence

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0071] As described above, the data-dependent data quality detection method, the difference of this embodiment is that, as image 3 As shown in the flow chart of Embodiment 1 of the data-dependent data quality detection method of the present invention, before step b, step a is also included to convert the external reference file to be input or the data to be detected into a required format.

[0072] In this step, the files in different formats are converted, which improves the adaptability of this method to different file formats.

Embodiment 2

[0074] In this example external references such as Figure 4 The data quality detection method of the data dependence of the present invention refers to the document table, wherein A, B, and C are three fields, A has 3 different values, B has 6 different values, and C has 9 different values. Organize the tree structure in the order of increasing value from top to bottom, such as Figure 5 The data-dependent data quality detection method of the present invention is shown in the tree structure of the reference file.

[0075] The data to be tested is as Figure 6 The data-dependent data quality detection method of the present invention is shown in the data table to be detected, where Col1, Col2, and Col3 are the fields to be detected, and the fields to be detected are mapped according to the received reference file fields and corresponding level information. The field Col1 to be detected corresponds to field A of the reference file, the field Col2 to be detected corresponds to ...

Embodiment 3

[0089] This embodiment is a data quality detection device corresponding to the above-mentioned data-dependent data quality detection method, such as Figure 9 As shown, it is a structural diagram of the data-dependent data quality detection device of the present invention; wherein, the data-dependent data quality detection device includes: a reference file analysis unit 2, a data dependence rule definition unit 3, and a data dependence rule inspection unit 4 and error message processing unit 5.

[0090] The reference file analysis unit 2 analyzes the external reference file, judges the level of each field according to the number of different values ​​of each field of the reference file, and organizes the values ​​of each field into a tree structure of the reference file. Send the field name and its corresponding level information to the data dependency rule definition unit 3 .

[0091] The external reference file is a file in the required format, which includes multiple field...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and device for detecting data quality of data dependence. The method comprises the following steps: b, analyzing a reference file, judging levels of fields of the reference file according to the numbers, with different values, of the fields, and organizing the values of the fields into a tree structure of the reference file; c, receiving to-be-detected data, determining mappings between to-be-detected fields of the to-be-detected data and a reference level according to the names of the fields of the reference file and corresponding level information, and organizing the mappings into a tree structure of the to-be-detected fields; and d, traversing the tree structure of the reference file, searching values at corresponding positions of the tree structure of the to-be-detected fields and performing marking. The method comprises a reference file analysis unit, a data dependence rule definition unit and a data dependence rule check unit corresponding to the steps, respectively. In such way, the occurring sources of the errors can be correctly positioned in the check process, and the detection speed and efficiency can be greatly improved.

Description

technical field [0001] The invention relates to the technical field of data quality monitoring, in particular to a data-dependent data quality detection method and device. Background technique [0002] With the rapid development of information technology, data has gradually become one of the most important resources to realize the business value of enterprises. However, as the amount of data continues to increase, data quality issues also follow. Missing data, errors, inconsistencies and other problems hinder the application of enterprises, and even cause enterprises to make wrong decisions, lose important value and cause a crisis of trust. [0003] For these dirty data, many data quality detection and cleaning schemes have emerged as the times require. Data dependence is a data quality problem that is more difficult to detect. Since the system often does not know the logical relationship between fields hidden in the data table, data dependency issues are generally checke...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/215
Inventor 彭泽武冯歆尧江疆杨秋勇张晓霞
Owner GUANGDONG POWER GRID CO LTD INFORMATION CENT
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com