Unlock instant, AI-driven research and patent intelligence for your innovation.

Data verification combined ETL scheduling system and method

A data verification and scheduling system technology, applied in the field of data warehouses, can solve the problem of a lot of wrong data, achieve the effect of improving scheduling efficiency and saving server resources

Active Publication Date: 2015-03-25
SHANGHAI TOBACCO GRP CO LTD
View PDF5 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide an ETL scheduling system and method combined with data verification, so as to solve the problem of large error data caused by untimely verification during the establishment of the data warehouse in the prior art. many questions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data verification combined ETL scheduling system and method
  • Data verification combined ETL scheduling system and method
  • Data verification combined ETL scheduling system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0046] Such as figure 1As shown, in Embodiment 1, a kind of ETL scheduling system 1 combined with data checking provided by the present invention includes ETL scheduling device 11 and data checking device 12 connected with ETL scheduling device 11; wherein, ETL scheduling ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data verification combined ETL scheduling system and method. The method includes the steps of scheduling and executing ETL JOBs; obtaining verification data associated with the ETL JOBs in advance to be used for loading; before each ETL JOB is executed, verifying execution results of the ETL JOBs and the verification data loaded in the ETL JOBs, and further judging whether ETL JOBs having the dependency relationship with the ETL JOBs are executed or not. In this way, the data can be loaded more effectively, the valuable server resources are saved, and meanwhile the ETL scheduling efficiency is greatly improved.

Description

technical field [0001] The invention relates to the technical field of data warehouses, in particular to an ETL scheduling system and method combined with data verification. Background technique [0002] Traditional ETL (Extract-Transform-Load, data warehouse technology) refers to the process of extracting (Extract) data from data sources in the process of building a data warehouse, and loading (Load) into the data warehouse after data transformation (Transform). In general, ETL is responsible for the scheduling control of the system running program, but data verification is not involved in the scheduling process. [0003] Of course, there are also some ETL scheduling processes that involve data validation. Usually, it is implemented by deploying a data verification system around the ETL scheduling system, and when all the ETL JOBs (ETL JOBs) in all ETL scheduling systems are scheduled to run, then the data verification program is centrally scheduled, and then the data veri...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/254
Inventor 陈品栾晓宇陈德莉
Owner SHANGHAI TOBACCO GRP CO LTD