CDC data distribution method and device thereof

A data distribution and data loading technology, applied in the field of data warehouse, can solve the problems of low extraction speed, does not support distributed extraction and loading, troublesome installation and maintenance, etc., and achieves the effect of improving extraction efficiency

Inactive Publication Date: 2012-09-12
北京英孚斯迈特信息技术有限公司
View PDF2 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The existing technology extracts data through the JDBC and ODBC interfaces of the database, and the extraction efficiency is not high; the processing logic is complex, the architecture is huge, not lightweight, and installation and maintenance are troublesome; the Oracle database can only be extracted by a single process, and the extraction speed is not high; Does not support distributed extract and load

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • CDC data distribution method and device thereof
  • CDC data distribution method and device thereof
  • CDC data distribution method and device thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0067] Such as figure 1 As shown, the overall implementation process of the present invention comprises the following steps:

[0068] ① Configure extraction information: configure the extraction information used to extract data in the database through the configuration interface. The configured information includes: the table to be extracted, the extracted file storage directory and file name format, the inspection file storage directory and inspection file name format, Data file retention days, extracted SQL, extracted pre-SQL, extraction time, empty data cut-off time, whether to enable, run host, configure host;

[0069] ②Extraction process: read the configured extraction information, extract the data in the source database from the database of the system related to the business to generate a text file, the system related to the business is: such as ERP system, financial system, business support system, OA system, EBS, logistics system, website shopping platform, customer s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a CDC (changed data capture) data distribution method and a CDC data distribution device, wherein the method comprises the following steps: step (1) of configuring extraction information, configuring the extraction information for extracting data in a database through a configuration interface; step (2) of extracting process, reading the configured extraction information, extracting the data in a source database from the database of a system related to businesses to generate a text file; step (3) of configuring loading information, configuring the loading information for loading the data in the database through the configuration interface; step (4) of loading process, reading the loading information, and loading the text file derived during the extraction process into a target database for storing the extracted text file. According to the CDC data distribution method and device provided by the invention, a configuration mode is graphically and flexibly opened, data is extracted fast, data extraction is fully implemented according to source data API, extraction and overloading are performed by the way of pipelining operation.

Description

technical field [0001] The invention relates to the field of data warehouses, in particular to data integration in the field of data warehouses. Background technique [0002] The CDC data distribution center system is a product specially designed for data integration in the data warehouse field. It is an ELT model rather than an ETL model. ELT means to extract first, then load, and finally clean and transform; ETL means to extract first, then clean, and finally load. At present, most of this field is still in ETL mode, while the IS / BI-CDC data distribution center system is in ELT mode. By quickly extracting and loading data, In order to carry out data transformation work in the data warehouse. It is mainly used for data extraction and loading. Extraction is to export data from the database of the business system into text files; loading is to import the extracted text files into the data warehouse for cleaning and conversion processing. [0003] The existing technology ext...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 官辉文彦峰齐科军李俊冯志强
Owner 北京英孚斯迈特信息技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap