A configurable data cleaning system and method

A technology for configuring data and cleaning systems, applied in electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of difficult maintenance of data reading and storage cleaning steps, unified data recoding, and cumbersome configuration. Clear logic for fetching, storage and cleaning, effective and reasonable utilization, and ensure the effect of continuity

Inactive Publication Date: 2018-12-11
北京亚融方成科技有限公司
View PDF5 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present application provides a configurable data cleaning system, which solves the problems existing in the prior art such as cumbersome configuration, difficult maintenance of data reading and storage, and cleaning steps, and recoding of unified data cleaning in different environments.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A configurable data cleaning system and method
  • A configurable data cleaning system and method
  • A configurable data cleaning system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0034] The technical solutions provided by various embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0035] figure 1 A system diagram of a configurable data cleaning system provided for the embodiment of this application, such as figure 1 As shown, the configurable data cleaning...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A configurable data cleaning system and method are disclosed. The system includes a task controller, a cleaning tool, a first configuration table, a second configuration table, and a third configuration table. The first configuration table includes a task encoding and a cleaning tool. The second configuration table includes a task code, a data identifier, a source database, a target database, anda cleaning rule identifier. The source database includes the source field and the destination database includes the destination field. The third configuration table includes a data identifier, a fieldname, a source field, a destination field, and a translation rule identifier. The task controller reads the configuration table and invokes the cleaning tool. The cleaning tool reads the source datafrom the source database according to the data identifier corresponding to the task code, determines the cleaning rules, and filters the source data according to the cleaning rules. The conversion rule identifier corresponding to the field name is determined, the data of the source field is converted into the data of the target field, and the conversion rule identifier is sent to the target database. The system and the method ensure the consistency of data cleaning and effectively utilize time and resources.

Description

technical field [0001] The present application relates to the field of software, in particular to a configurable data cleaning system and method. Background technique [0002] With the rapid development of the Internet industry in recent years, more and more information can be read directly from the Internet. But at the same time, the data sources and contents on the Internet end are complex, and the amount of data is too large. Enterprises need a large amount of data in the process of project development, most of which come from the Internet. Secondary processing is required to clean the data before it can be used. In the process of data cleaning, data cleaning from different sources requires different configurations, resulting in cumbersome configuration. When there are many data cleaning tasks, data sources, targets, conversion methods and steps are difficult to maintain. The difference between the same data cleaning in the development, testing and formal environment w...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 陈鹏林郝东进沈惟冉王腾龙
Owner 北京亚融方成科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap