Data quality management system and method applied to big data system
A technology of data quality and management system, applied in the fields of electronic digital data processing, digital data information retrieval, special data processing applications, etc.
Active Publication Date: 2022-03-08
广州明动软件股份有限公司
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] The purpose of the present invention is to provide a data quality management system applied to big data systems. By establishing a layer of data supervision platform between the data sources of many application systems and the built big data center, it solves the problem of existing big data problems. Due to the wide range of data sources and various types of data in the center, the quality of data is uneven and the efficiency of data utilization is reduced.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0021] Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.
[0022] The present invention aims at supervising the data quality of the big data center. The main idea is to build a layer of supervision, screening and filtering between the data source and the big data center to check the source data and filter out abnormal data and dirty data to improve the quality of data collected by big data centers. Since the data of the current big data center mainly comes from the database of the application system, the embodiment of the present invention mainly takes the data source as the data source of the application domain as an example, constructs management indicators for the data in the data source of the application domain, and realizes Check the data information at the location.
[0023] figure 1A structural frame diagram of a data quality management system applied to a big data system according to an embodiment o...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a data quality management system applied to a big data system, which includes an application domain configuration module, which is used to configure the application domain to which data to be detected belongs, and generates application domain configuration information storage; a data source configuration module, which is used to configure the application domain to be detected The data source to which the data belongs generates data source configuration information storage; the management index configuration module is used to create management index storage for the data information in the data source; the task execution module can be used to call the management index to access the application domain according to the execution command Check the data of the data information to the data source. Also included is an operating method of a data quality management system applied to a big data system. For the integration of the environmental big data center in the big data environmental supervision platform and the collection of various business data from many systems of the provincial environmental protection department, the present invention can perform data inspection on the data of the environmental big data center, screen out dirty data or abnormal data, and improve the environment. Data quality in big data centers.
Description
technical field [0001] The invention relates to the technical field of big data processing, in particular to a data quality management system applied to a big data system and a method for using the system to manage the data quality of a big data system. Background technique [0002] Since the current big data center generally integrates and collects the business data of many systems, there are various types of data integrated and collected, and the data quality is uneven, including a large amount of dirty data or abnormal data, and the data quality is low, reducing Data utilization efficiency. Contents of the invention [0003] The purpose of the present invention is to provide a data quality management system applied to big data systems. By establishing a layer of data supervision platform between the data sources of many application systems and the built big data center, it solves the problem of existing big data problems. Due to the wide range of data sources and vario...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/215
CPCG06F16/215
Inventor 邹勇黄静刘明胡章平
Owner 广州明动软件股份有限公司