ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting

A technology of automatic judgment and scheduling system, applied in the field of data warehouse, to achieve the effect of reducing maintenance cost, high efficiency and high degree of importance

Active Publication Date: 2013-04-10
FOCUS TECH
View PDF3 Cites 48 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] The purpose of the present invention is to provide an ETL task scheduling that can solve the problem that the subsequent subtasks are still executed due to errors in the preceding subtasks, improve task execution efficiency, and help reduce manual intervention and reduce operating loss. Error correction restart and automatic judgment Started ETL scheduling system and method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting
  • ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting
  • ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0170] For an ETL task process, such as Figure 8 shown.

[0171] This ETL task process is divided into 4 tasks, and the order of execution is task 1, task 2, task 3, and task 4.

[0172] Among them, task 1 includes 5 subtasks, and the order of execution is 1, 2, 3, 4, 5.

[0173] Task 2 includes 6 subtasks, and the order of execution is 6, 7, 8, 9, 10, and 11.

[0174] Task 3 includes 5 subtasks, and the order of execution is 12, 13, 14, 15, and 16.

[0175] Task 4 includes 6 subtasks, and the order of execution is 17, 18, 19, 20, 21, and 22.

[0176] Moreover, in the figure, the subtasks directly connected by lines have an association relationship, and the subtasks not directly connected by lines have no association relationship and are independent relations.

[0177] If an exception occurs in subtask 2, subtasks 3, 4, and 5 will not be executed.

[0178] If an exception occurs in subtask 7, subtasks 9 and 11 will not be executed.

[0179] If an exception occurs in sub...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting. The ETL dispatching system structurally comprises a business system database server, a data warehouse server, an ETL dispatching and monitoring server and an ETL dispatching platform server. The ETL dispatching method disclosed by the invention comprises the following steps of: dividing a data warehouse ETL task into a plurality of subject tasks, wherein each subject task comprises a plurality of sub-tasks; configuring information according to a preset executing sequence and automatically judging the end time for successfully executing the pre-posed associated sub-tasks; executing follow-up sub-tasks, stopping the execution of the associated follow-up sub-tasks according to the sub-task correlation after the execution of the sub-tasks has exception faults, and recording the exceptional sub-tasks; and only processing the exceptional sub-tasks and the associated follow-up sub-tasks in a way of automatically restarting for many times. According to the ETL dispatching system and method disclosed by the invention, the problem that the follow-up sub-tasks are continuously executed when the pre-posed sub-tasks have faults in the ETL task dispatching is solved; the task executing efficiency is improved; the manual intervention is reduced; and the operation loss is reduced.

Description

technical field [0001] The invention belongs to the technical field of data warehouses, in particular to an ETL scheduling system and method for error correction restart and automatic judgment startup. Background technique [0002] With the popularization and deepening of data warehouse technology, the core technology ETL tools for building data warehouses are gradually improved and diversified. ETL mainly assumes the role of data extraction, loading, and transformation in data warehouse construction, and is responsible for system program scheduling control. and resource allocation are directly related to the success or failure of data warehouse construction and implementation. [0003] At present, the value of information is increasingly reflected, especially in industries with a large number of data records. A successful data warehouse is very important to the development of an enterprise. The big data characteristics of banks, mobile telecommunications, insurance, and e-...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/07G06F9/48
Inventor 何昌桃黄建鹏徐晓冬陈静
Owner FOCUS TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap