Data quality judging and monitoring method, management system, storage medium and terminal

A data quality and data technology, applied in the field of data processing, can solve the problem of not considering the quality inspection of data repair processing, and achieve the effect of avoiding the loss of important data, improving efficiency and ensuring quality

Pending Publication Date: 2021-05-14
深圳市网联安瑞网络科技有限公司
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] (3) For the abnormal data, it is not advisable to directly filter or re-collect. The existing data quality management system does not consider repairing the data according to the abnormal type and checking the quality after repairing, so as to ensure the quality of the collected data to the greatest extent. integrity
[0011] (1) In the early stage of data management, various quality problems inevitably exist in data sources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data quality judging and monitoring method, management system, storage medium and terminal
  • Data quality judging and monitoring method, management system, storage medium and terminal
  • Data quality judging and monitoring method, management system, storage medium and terminal

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0072] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

[0073] Such as figure 1 As shown, the present invention provides a method for judging and monitoring data quality, including:

[0074] S101, for data from different sources, according to the quality judgment rules configured for each field, perform data quality judgment, and check each quality requirement one by one, and store all the inspection items after they are judged to be quali...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data quality judging and monitoring method, a management system, a storage medium and a terminal, and relates to the technical field of data processing. The method comprises the steps of performing data quality judgment on the data from different sources according to the quality judgment rule configured for each field, checking each item of quality requirement one by one, and warehousing nn after all detection items are judged to be qualified; for unqualified data found in the data quality judgment process, configuring a corresponding processing program according to the abnormal type of the unqualified data, reserving valuable data, and removing useless data; and continuously carrying out quality monitoring on the data, carrying out statistical analysis on related indexes such as repetition rate, missing rate and error rate, and finally outputting a data quality report. In the quality judgment process, the quality of each data source is continuously monitored, a data quality report is formed and fed back to the data source for rectification, and the efficiency of data quality management is effectively improved.

Description

technical field [0001] The disclosure of the present invention relates to the technical field of data processing, in particular to a method for judging and monitoring data quality, a management system, a storage medium, and a terminal. Background technique [0002] At present, with the development of the Internet, the network data generated by various industries has grown exponentially, and enterprises are paying more and more attention to the management of data assets, especially data governance. For the collected data, it is found that due to the large number of data sources and the uneven quality, it is easy to cause problems such as inaccurate data content, inconsistent format, irrational logic, missing data in some fields, repeated data, and data delay. Subsequent data processing and analysis work brought a lot of trouble. In this case, once the data is stored in batches, not only the utilization rate is not high, but also it will cause a waste of resources. Therefore,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06F16/23G06F16/2458G06F16/25
CPCG06F16/215G06F16/2365G06F16/2462G06F16/25
Inventor 黄静李雄峰丁建伟陈周国周伟中朱永川胡瑞雪
Owner 深圳市网联安瑞网络科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap