Data synchronism ETL (Extract Transform Load) system

A data synchronization and database technology, applied in the fields of electronic digital data processing, structured data retrieval, special data processing applications, etc., can solve the problems of high error rate, data dispersion, low efficiency, etc., to reduce waiting time and improve work efficiency Effect

Active Publication Date: 2015-01-28
SHANGHAI HANDPAL INFORMATION TECH SERVICE
View PDF5 Cites 62 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] 1. Data dispersion caused by "data islands" is not conducive to data integration analysis and value-added services;
[0008] 2. Data files need to be landed and then copied to the target end, which requires manual intervention in multiple steps to complete;
[0010] 4. The workflow scheduling dependency betwee...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data synchronism ETL (Extract Transform Load) system
  • Data synchronism ETL (Extract Transform Load) system
  • Data synchronism ETL (Extract Transform Load) system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] The embodiments of the present invention aim to provide a data synchronization system with multiple data sources, which solves efficient and reliable data synchronization across multiple database environments, and extracts business data generated by different business systems for use by auxiliary systems. Implementation structure see figure 1 As shown, the specific implementation steps are as follows.

[0051] First, the data synchronization ETL system needs to connect to the source data through the context name, determine the table structure and field names that need to be synchronized, and the method adopted in the data synchronization process to synchronize to the table structure name corresponding to the target source. The following processing needs to be done through the present invention:

[0052] According to the context parameter configuration module, it is used to obtain connection information of the source database and the target database configured with different ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of data synchronism, and particularly discloses a data synchronism ETL (Extract Transform Load) system. The data synchronism ETL system comprises the following function modules: a context parameter configuration module, a synchronous interface definition module, a synchronous script generation module, a workflow configuration module, a data synchronism core module and a log and prewarning module, wherein the context parameter configuration module, the synchronous interface definition module and the synchronous script generation module are basic modules and are operated according to the sequence of the modules to generate a script; the workflow configuration module mainly plays the role of a dispatching center, and configures the script in a workflow to realize parallel or serial execution, parallel progress amount, and what time or condition a task is triggered; when anomaly or an error task occurs in the execution steps, the log and prewarning module captures an error reason, and informs system management personnel of the error reason. The data synchronism ETL system is provided with a bidirectional synchronism mechanism, and is capable of supporting data sources configured with multiple context names and accessing multiple environments, the coverage of the synchronism method is wide, meanwhile, a complicated user-defined synchronism method is supported, and the expansibility is stronger.

Description

Technical field [0001] The invention belongs to the technical field of data synchronization, and specifically relates to a data synchronization ETL system. Background technique [0002] The existing data synchronization system mainly provides production environment business data for R&D troubleshooting and simulation of problem recurrence, etc., as well as for data analysts to analyze business data, and provide daily, weekly, and monthly business operation reports for management Use or provide data support for the sales department’s business. Due to the existence of a cross-network in the two environments, the servers in the two environments cannot communicate with each other, and dial in through VPN to access the production environment and synchronize data files to the local environment. [0003] Existing method one, initialization is to complete the restoration of historical data by fully backing up database A in the production environment, copying the backup file to server B, a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/27
Inventor 王新亮
Owner SHANGHAI HANDPAL INFORMATION TECH SERVICE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap