ETL task dependence relationship detecting method and device and ETL tool

A technology of dependency relationship and detection method, applied in the detection of ETL task dependency relationship, ETL tool field, can solve the problem that ETL task dependency relationship error points and optimizable points cannot be automatically detected.

Active Publication Date: 2016-05-18
ALIBABA GRP HLDG LTD
View PDF5 Cites 30 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] The present invention provides a method and device for detecting ETL task dependencies, so as to solve the problem that the prior art cannot automatically detect error points and optimization points in ETL task dependencies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • ETL task dependence relationship detecting method and device and ETL tool
  • ETL task dependence relationship detecting method and device and ETL tool
  • ETL task dependence relationship detecting method and device and ETL tool

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0128] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described here, and those skilled in the art can make similar extensions without violating the connotation of the present invention, so the present invention is not limited by the specific implementations disclosed below.

[0129] In this application, a method and device for detecting ETL task dependencies, and an ETL tool are respectively provided. Each will be described in detail in the following examples.

[0130] Please refer to figure 1 , which is a flow chart of an embodiment of a method for detecting ETL task dependencies of the present application. The method comprises the steps of:

[0131] Step S101: For each task of the ETL, obtain the data operation instructions included in the task.

[0132] The detection method for ETL task d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an ETL task dependence relationship detecting method and device. The method comprises the following steps: for each task of ETL, obtaining a data operating order contained in the task; analyzing the data operating order to obtaining a source table and a target table related to the task; according to the target table, a task attribute table and a task dependence relationship configuration table, obtaining a source table of direct dependence by the target table and a source table of indirect dependence by the target table; and traversing the source table related to the task, the source table of direct dependence and the source table of indirect dependence and marking the error types and optimizable types of all task dependence relationships related to the task according to a preset rule. With adoption of the method provided by the invention, actual task dependence relationships and a preset task dependence relationship are compared according to a preset rule so as to automatically discover erroneous and optimizable task dependence relationships, thus reducing the occurrence frequency of faults caused by task dependence problem, saving time for manual task dependence problem check by testing personnel and further achieving the effect of improving testing efficiency.

Description

technical field [0001] The invention relates to the technical field of data warehouses, in particular to a detection method and device for ETL task dependencies. The present invention also relates to an ETL tool. Background technique [0002] ETL (Extract-Transform-Load, the process of data extraction, transformation, and loading), as the core and soul of BI / DW (Business Intelligence / Data Warehouse, business intelligence / data warehouse), can integrate and improve the value of data according to unified rules. Responsible for completing the process of data transformation from data source to target data warehouse, which is an important step in the implementation of data warehouse. The most difficult part in the entire project of the data warehouse is user requirement analysis and model design, while the design and implementation of ETL rules is the largest workload, accounting for about 60% to 80% of the entire project. [0003] After the ETL is released, the developer will c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 吴媛媛
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap