Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system

A multi-priority, task management technology, applied in the field of big data, can solve problems such as poor results, and achieve the effect of improving utilization and speeding up execution

Inactive Publication Date: 2015-05-27
ECDATA INFORMATION TECHNOLOGY CO LTD
View PDF4 Cites 82 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, these methods will have different adaptation scenarios in different business scenarios, and the effect is not good in the distributed processing scenario of big data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system
  • Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system
  • Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] based on the following Figure 1 ~ Figure 4 , specifically explain the preferred embodiment of the present invention.

[0032] like figure 1 As shown, the big data task management system based on multiple queues and multiple priorities provided by the present invention includes: multiple clients 1 for initiating task requests; each client 1 includes a setting module, which uses To set the task queue label and priority for the task that initiates the request; the resource manager 2, which is connected to a plurality of said clients 1, is used to receive the task request initiated by each client 1, and assign it to Corresponding to the task queue, and unified management and distribution of all resources; a plurality of node resource managers 3 are respectively set on each distributed node and connected to the resource manager 2, and the resources owned by the distributed node The resource is reported to the resource manager 2 for unified management and distribution, and...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing the system. The method comprises the following steps: S1. a client side launches a task request and sets a task queue tag and priority for a task; S2. a resource manager receives the task request launched by the client side, and allocates the task request into the corresponding task queue; S3. each node resource manager reports a resource owned by a distributive node where on which node resource manager is positioned to the corresponding resource manager so as to form a resource pool, and each resource manager allocates the resources for each task queue; and S4. each resource manager sequentially allocates the current task needing to be executed in a plurality of task queues to the node resource managers, and after a task command is received, each node resource manager dispatches a container unit to execute the task. According to the multi-queue multi-priority big data task management system and the method utilizing the system, the unified management and dispatching of the resources can be conducted, the task dispatch execution can be conducted based on multi-queue and multi-priority conditions, the utilization rate of resources can be increased, and the task execution speed can be increased.

Description

technical field [0001] The present invention relates to a big data task scheduling control management system and method, specifically a multi-queue and multi-priority based big data task management system and method, belonging to the field of big data technology. Background technique [0002] In recent years, with the rapid development and popularization of computer and information technology, the scale of industrial application systems has expanded rapidly, and the data generated by industrial applications has grown explosively. Industry / enterprise big data that can easily reach hundreds of terabytes or even tens to hundreds of petabytes has far exceeded the processing capabilities of existing traditional computing technologies and information systems. Therefore, seeking effective big data processing technologies, methods and means has become an urgent need in the real world. [0003] McKinsey, a world-renowned consulting firm, was the first to propose the arrival of the e...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/50
Inventor 倪巍
Owner ECDATA INFORMATION TECHNOLOGY CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap