Automatic base switching system and method for ETL operations

A technology of operation and database cutting, applied in the computer field, can solve the problems of inability to guarantee operation stability and reliability, labor-intensive, and low-efficiency implementation, and achieve humanized system maintenance, save labor costs, and improve work efficiency.

Active Publication Date: 2015-01-21
北京思特奇信息技术股份有限公司
View PDF2 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Most of the ETL tools avoid the peak period of data operation to obtain real-time photo data from the production database. However, in the era of highly developed information, the data operation of the production database is quite frequent, which will lead to the extraction of long-term ETL jobs due to

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic base switching system and method for ETL operations
  • Automatic base switching system and method for ETL operations

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0039] The principles and features of the present invention will be described below with reference to the accompanying drawings. The examples cited are only used to explain the present invention, and are not used to limit the scope of the present invention.

[0040] Such as figure 1 As shown, an ETL job automatic library cutting system includes a production library 100, a backup library 200, an ETL job execution module 300, an ETL job scanning module 400, a backup library status monitoring module 500, and an ETL library cutting processing module 600; The library 100 is used to provide data sources for ETL jobs; the backup library 200 is used to regularly back up data in the production library 100, and provide data for the ETL jobs that report errors when an error occurs in the operation of the ETL job in the production library 100 Source; the ETL job execution module 300, which is used to extract the required data from the production library 100 for ETL processing, when the operat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an automatic base switching system and method for ETL operations. The method comprises the steps of scanning the executing situation of the ETL operations at regular intervals; piecing up manual intervention information according to operation information of an ETL operation reported with errors when the ETL operation reported with errors is found; checking the data synchronization situation of a backup base at regular intervals, judging the working state of the backup base, generating a base switching trigger instrument if the backup base is in the data synchronization completing state, switching an operated object of the ETL operation reported with errors to the backup base from a production base, base switching is completed, modifying operation configuration information according the manual intervention information, and the ETL operation is executed in the backup base. After the errors of the operation are reported, the operated object of the operation is switched from the production base to the backup base, and normal extraction of the operation is ensured so that the stability of data extraction can be ensured. The data timeliness of a downstream system is supported, manpower cost is saved, and the working efficiency is obviously increased, so that the system maintenance is more humanized and intelligentized.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a system and method for automatically cutting databases for ETL operations. Background technique [0002] ETL job refers to the use of ETL tools to extract data from distributed and heterogeneous data sources (such as relational data, flat data files, etc.) to a temporary intermediate layer for cleaning, conversion, integration, and finally loading to a data warehouse or data mart It becomes the basis of online analytical processing and data mining. [0003] At present, the more important systems will have a production library and a backup library (BCV library). The backup library stores the photographic data at a certain moment of production for emergency use. Most of the ETL tools avoid the peak period of data operation to obtain real-time photo data from the production database. However, in the era of highly developed information, the data operation of the production databa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/254
Inventor 罗达志
Owner 北京思特奇信息技术股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap