System and method for integrating data on basis of reverse clearing

A data integration and data technology, applied in the field of multi-source heterogeneous data integration system, to achieve efficient query, perfect ETL process, and improve quality

Inactive Publication Date: 2015-04-15
HUNAN UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a data integration system and method based on reverse cleaning that can reversely correct source data, so as to solve the technical problem of lack of quality improvement and correction of source data in the traditional ETL process

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System and method for integrating data on basis of reverse clearing
  • System and method for integrating data on basis of reverse clearing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways defined and covered by the claims.

[0041] see figure 1 , the data integration system based on reverse cleaning of the present invention includes the following two modules:

[0042] 1. The data integration module is used to extract, clean, convert and load the source data from the data source into valid data for the application platform to call;

[0043] 2. The reverse cleaning module is used to correct and update source data based on valid data.

[0044] In this preferred embodiment, see figure 1 , the data integration module includes:

[0045] (1) An adaptation unit, configured to adapt the parsed source data in the memory according to the adaptation rules, and retain the source data conforming to the adaptation rules as temporary data. Such as figure 1 As shown, the data struc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a system and a method for integrating data on the basis of reverse clearing. The system for integrating data on the basis of reverse clearing comprises a data integrating module and a reverse clearing module, wherein the data integrating module is used for extracting, clearing source data from data sources, transforming and loading the source data into effective data for being called by an application platform, and the reverse clearing module is used for modifying errors and updating the source data according to the effective data. The method includes two main steps: data integrating and reverse clearing. The data integrating step includes extracting, clearing the source data from the data sources, transforming and loading the source data into effective data for being called by the application platform. The reverse clearing step includes modifying errors and updating the source data according to the effective data. By the aid of the system and the method for integrating the data on the basis of reverse clearing, reverse modification of the source data can be realized while integrating the data.

Description

technical field [0001] The invention relates to the field of data integration, in particular to a multi-source heterogeneous data integration system and method. Background technique [0002] One of the current implementations of heterogeneous data integration technology is the data warehousing model. The data warehousing mode is to extract data from one or more data sources, perform necessary processing on the data, and finally store the data in the target data warehouse. The data storage mode generally adopts the form of ETL (Extract, Transform, Load) and data warehouse. The ETL process includes data extraction, data transformation, and data loading. ETL is responsible for extracting data from distributed and heterogeneous data sources to a temporary middle layer for cleaning and transformation, and finally loading it into a data warehouse or data mart. Although the current research on the ETL process at home and abroad is relatively mature, the heterogeneous data integr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 陈浩唐钰
Owner HUNAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap