Data processing task relation setting method and system

A data processing and task relationship technology, applied in the field of cloud computing, can solve problems such as errors and affecting data processing results, and achieve the effect of improving accuracy and efficiency and improving the degree of automation

Active Publication Date: 2014-12-17
CHINA TELECOM CORP LTD
View PDF2 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This requires the operation and maintenance personnel to have a good understanding of the whole data processing before and after, otherwise the task dependencies will be wrong, which will directly affect the results of data processing
But in fa

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing task relation setting method and system
  • Data processing task relation setting method and system
  • Data processing task relation setting method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0061] The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

[0062] Such as figure 1 What is shown is a schematic flow chart of an embodiment of a method for setting data processing task relationships of the present invention. In this embodiment, the data processing task relationship setting method includes:

[0063] Step 101: Obtain at least one SQL script in the data processing task;

[0064] Step 102: Perform lexical analysis and syntax analysis on the SQL statements in each SQL script in at least one SQL script, respectively, to establish the data lineage relationship of the SQL statements;

[0065] Step 103: Establish the data lineage relationship of the SQL script to which it belongs through the data lineage relationship of the SQL statement;

[0066] Step 104: Establish the data lineage relationship of the data processing task according to the data lineage relationship of each SQL script in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a data processing task relation setting method and system. The method includes the steps of obtaining at least one SQL script in a data processing task, carrying out morphology analysis and semantic analysis on SQL sentences in each SQL script in the at least one SQL script to build a data lineage relation of the SQL sentences, building a data lineage relation of the SQL scripts according to the data lineage relation of the SQL sentences, building a data lineage relation of the data processing task according to the data lineage relation of the SQL scripts in the at least one SQL script, determining data input and output of a data level and a task level of the data processing task, and determining and setting the relation between the data processing task and another data processing task according to the data lineage relation and the data level of the data processing task. Intelligent analysis and setting of the relation of the SQL data processing tasks can be achieved, the automation degree of data task scheduling configuration is improved and accuracy and efficiency of data operation and maintenance are achieved.

Description

Technical field [0001] The invention relates to cloud computing technology, in particular to a method and system for setting data processing task relations. Background technique [0002] In the big data environment of the cloud computing era, data is growing rapidly, and the number of various data processing tasks is also increasing rapidly. Information processing technology places more emphasis on the ability to quickly obtain valuable information from massive amounts of data, which puts forward higher requirements for efficient scheduling and execution of data processing tasks. [0003] Data processing tasks include a variety of data conversion-summary and other processing. There are certain relationships between tasks (including dependencies and mutual exclusion relationships). Accurate task relationships are an important basis for efficient data scheduling. Take the data warehouse system as an example. At present, in the scheduling of data processing tasks, the relationship be...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/283
Inventor 陈翀向勇孙剑晖黄平陈康张青高智衡刘春
Owner CHINA TELECOM CORP LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap