Method and device for generating data warehouse ETL (Extraction, Transformation and Loading) codes

A data warehouse and code technology, applied in the field of data warehouse ETL code generation, can solve the problems of high professional technical requirements of ETL developers, inability to guarantee complete consistency of code style, and slow response of technology to business support, so as to reduce manual development, Improve test efficiency and improve development efficiency

Active Publication Date: 2013-09-18
ADVANCED NEW TECH CO LTD
View PDF3 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Therefore, in the prior art, the amount of ETL code development is relatively large, the development efficiency is relatively low, and the technical support for the business is relatively slow; and the code development style of each developer cannot be guaranteed to be completely consistent, and the quality of the code is difficult to guarantee; at the same time The requirements for the professional skills of ETL developers are also very high
This will bring higher costs to code testing and management

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for generating data warehouse ETL (Extraction, Transformation and Loading) codes
  • Method and device for generating data warehouse ETL (Extraction, Transformation and Loading) codes
  • Method and device for generating data warehouse ETL (Extraction, Transformation and Loading) codes

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] In order to make the above objects, features and advantages of the present application more obvious and comprehensible, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0036] This application proposes a template-based efficient ETL code generation method, and designs a cross-platform and multi-language ETL development model by modularizing and abstracting the ETL code. According to various needs or projects, this method makes the common technical logic into an ETL development template to realize centralized automatic processing; and uses array variables to define each business logic that needs to be processed, and modularizes complex and changing business logic processing; and then use the ETL development template to automatically generate executable ETL code scripts.

[0037] This template-based ETL development can realize rapid ETL development, avoid repeated develop...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method for generating data warehouse ETL (Extraction, Transformation and Loading) codes and a device for generating the data warehouse ETL codes, so as to solve various problems caused by independent development of current data warehouse ETL. The method comprises the steps: by aiming at business logic needed to be processed, corresponding file header variables and explanatory note information thereof are added; the business logic needed to be processed is presented by SQL (Structured Query Language) statements, and then the SQL statements are used as variable values to be filled into array variables; general technological logic of various businesses is extracted, and is used for authoring codes of an ETL development template, wherein the ETL development template is used for calling the variable values to process the business logic; and the codes of the ETL development template, file headers and explanatory note information thereof and the array variables filled with the variable values are organized to form an executable ETL code script. With the device and method disclosed by the invention, the development efficiency is improved, the code style is uniform, the codes are convenient to manage, and the test efficiency is also effectively improved.

Description

technical field [0001] This application relates to computer technology, in particular to a method and device for generating data warehouse ETL codes. Background technique [0002] ETL is the process of data extraction (Extract), transformation (Transform), cleaning (Cleansing), and loading (Load). It is an important part of building a data warehouse. The user extracts the required data from the data source, after data cleaning, and finally loads the data into the data warehouse according to the pre-defined data warehouse model. [0003] At present, the enterprise data warehouse ETL basically adopts the method of independent development, mostly based on process language development. Every time a new requirement or project is received, ETL developers need to develop a large number of process language codes in addition to sorting out business logic. For example, some enterprises use perl language (Practical Extraction and Report Language, practical report extraction language) ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 王金龙周国尧杨树波
Owner ADVANCED NEW TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap