Executive plan generation method for big data analysis process

A technology of execution plan and big data, applied in the field of big data, it can solve the problems of high technical threshold and low analysis efficiency, and achieve the effect of improving efficiency, eliminating cumbersomeness and reducing requirements.

Active Publication Date: 2018-04-06
TSINGHUA UNIV
View PDF6 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to provide a method for generating an execution plan of a big data analysis process, which solves the high technical threshold and low analysis efficiency of big data analysis products in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Executive plan generation method for big data analysis process
  • Executive plan generation method for big data analysis process
  • Executive plan generation method for big data analysis process

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] figure 1 A schematic diagram of a method for generating an execution plan of a big data analysis process according to an embodiment of the present invention, as shown in figure 1 As shown, the present invention provides a method for generating an execution plan of a big data analysis process, including:

[0049] Step S10, if it is determined that the task instance corresponding to the analysis task has not been executed and the task instance has dependent tasks, traverse the dependent task sequence, and execute the dependent task in the dependent task sequence;

[0050] Step S20, after the dependent task in the dependent task sequence is executed, if it is determined that the execution of the task instance is completed, a task result file is generated.

[0051] Further, the method also includes:

[0052]Judging whether the execution of the task instance is completed according to the execution status identifier of the task instance, the execution status identifier incl...

Embodiment 2

[0111] Figure 4 It is a schematic diagram of an execution plan generation device for a big data analysis process according to an embodiment of the present invention, as shown in Figure 4 As shown, the embodiment of the present invention provides an execution plan generation device for a big data analysis process, which is used to complete the methods described in the above embodiments. The device includes an execution module 10 and a generation module 20, wherein,

[0112] The execution module 10 is configured to traverse the dependent task sequence and execute the dependent task in the dependent task sequence if it is determined that the task instance corresponding to the analysis task has not been executed and the task instance has a dependent task;

[0113] The generating module 20 is configured to generate a task result file if it is determined that the execution of the task instance is completed after the dependent task in the dependent task sequence is executed.

[0...

Embodiment 3

[0126] Figure 5 A schematic structural diagram of an electronic device for generating an execution plan of a big data analysis process provided by an embodiment of the present invention, such as Figure 5 As shown, the device includes: a processor (processor) 801, a memory (memory) 802 and a bus 803;

[0127] Wherein, the processor 801 and the memory 802 complete mutual communication through the bus 803;

[0128] The processor 801 is used to call the program instructions in the memory 802 to execute the methods provided by the above method embodiments, including, for example:

[0129] If it is determined that the execution of the task instance corresponding to the analysis task has not been completed and the task instance has a dependent task, traverse the dependent task sequence, and execute the dependent task in the dependent task sequence;

[0130] After the dependent task in the dependent task sequence is executed, if it is determined that the execution of the task inst...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an executive plan generation method for a big data analysis process. The method comprises the steps of generating executable task instances according to analysis tasks; according to dependency relationships stored in the task instances, establishing a directed acyclic graph; according to the dependency relationships, traversing a dependency task sequence, and executing dependency instances, wherein the dependency instances are task instances of dependency tasks, and the dependency tasks are other analysis tasks depending on the analysis tasks in the dependency task sequence; and executing the task instances to generate a task result file. According to the executive plan generation method for the big data analysis process, provided by the invention, a data associationrelationship among the analysis tasks is enhanced in a rule form according to a characteristic that the analysis tasks in the big data analysis process have a data transmission relationship; the bigdata analysis efficiency is improved; the requirements of big data analysis on technician skills are lowered; the data is prevented from being revealed to big data analysis providers; and the securityrisk is lowered.

Description

technical field [0001] The invention relates to the field of big data technology, in particular to a method for generating an execution plan of a big data analysis process. Background technique [0002] With the continuous development of big data technology, many related technologies and products have appeared on the market, which have become effective means of big data acquisition, storage, processing analysis or visualization. However, there are high technical barriers to using these technologies and products. How to organize big data technologies in a standardized form and provide users with convenient and efficient usage methods has become a huge challenge. As a part of the application system in various fields, the traditional workflow engine has good versatility and completeness, but the process definition is complicated, and the solution that emphasizes roles, division of labor, information transfer routing, content level, etc. to execute business processes is not suit...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06Q10/10
CPCG06F9/4843G06F9/4881G06Q10/103
Inventor 王建民龙明盛孙家广裴忠一黄向东
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap