Task management method and system based on Hadoop

A task management and task technology, applied in the field of cloud computing, can solve problems such as the inability to manage tasks in a unified manner, and achieve the effects of avoiding idle time, improving execution efficiency, and saving running time

Inactive Publication Date: 2015-03-25
广州时代数据服务有限公司
View PDF5 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Based on this, it is necessary to provide a Hadoop-based task management method and system for the problem that Hadoop cannot perform unified management of tasks in the prior art, which can perform unified management of tasks with dependencies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task management method and system based on Hadoop
  • Task management method and system based on Hadoop
  • Task management method and system based on Hadoop

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] see figure 1 , which is a flowchart of an embodiment of the Hadoop-based task management method of the present invention.

[0020] A Hadoop-based task management method, comprising steps:

[0021] S101: Obtain a task dependency configured by a user, load a current task, and determine a superior task and a subordinate task of the current task according to the task dependency;

[0022] The user configures the task dependencies of each task through the console interface, and the dependencies include the starting point task and the execution conditions of other tasks except the starting point task, such as figure 2 As shown, it is a schematic diagram of the dependency relationship of an embodiment of the Hadoop-based task management method of the present ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a task management method and system based on Hadoop. The method comprises the steps that a task dependency relationship configured by a user is acquired, a current task is loaded, and upper-level tasks and lower-level tasks of the current task are determined according to the task dependency relationship; the execution results of the upper-level tasks are queried in a task status manager, and if all the upper-level tasks are successfully executed, the current task is triggered to be executed; the execution result of the current task is monitored, and if the current task is successfully executed, the execution result is sent to the task status manager, and the lower-level tasks are triggered to be loaded. By means of the task management method and system, the situations that the tasks are executed independently, and consequently concurrent execution of the tasks is caused are avoided, and the tasks can be uniformly managed. Once the current task is successfully executed, the lower-level tasks are loaded; under the condition that the upper-level tasks are successfully executed, the lower-level tasks can be immediately triggered to be executed. Thus, the running time needed for executing all the tasks can be shortened, and accordingly the execution efficiency of Hadoop can be improved.

Description

【Technical field】 [0001] The invention relates to the technical field of cloud computing, in particular to a Hadoop-based task management method and system. 【Background technique】 [0002] In recent years, the application of big data platform Hadoop (distributed computing) technology has become more and more mature. Hadoop provides a distributed cluster system for carrying and computing big data. Hadoop is an open source technology, and the technology has been continuously matured and improved in the open source community. At present, it has reached commercial requirements and has been applied in various big data platforms. [0003] Hadoop is a basic framework platform that provides a set of underlying API (Application Programming Interface, application programming interface) for Hadoop interface calls. Users can call the Hadoop interface in the console, in the form of applications and hiveql (data warehouse) scripts, to complete tasks. The invocation of the Hadoop interf...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/44
Inventor 王海山李天旺
Owner 广州时代数据服务有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap