Multitask dispatching method and system based on capacity

A scheduling method and task scheduling technology, applied in the capacity-based multi-task scheduling method and system field, can solve the problems of task execution node overload, load and insufficient memory, etc., and achieve the effect of improving utilization and avoiding overload

Inactive Publication Date: 2013-08-14
SHENZHEN TENCENT COMP SYST CO LTD
View PDF3 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The invention provides a multi-task scheduling method to solve the problems that the existing task scheduling system based on the MapReduce framework easily causes task execution node overload, load and insufficient memory

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multitask dispatching method and system based on capacity
  • Multitask dispatching method and system based on capacity
  • Multitask dispatching method and system based on capacity

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0029] figure 2 The composition structure of the multi-task scheduling system provided by Embodiment 1 of the present invention is shown, and for the convenience of description, only the parts related to the embodiment of the present invention are shown.

[0030] The multi-task scheduling system 1 includes a task scheduling device 11 and at least one task execution device 12 . The multi-task scheduling system is based on the MapReduce architecture.

[0031] The task scheduling device 11 connects and communicates with the task execution device 12 through a wired or wireless method, and is used to receive a task request sent by the task execution device 12 that carries information such as the current load value and available memory space, and according to the carried load value, Information such as memory space can be used to perform task scheduling on the task execution device 12 .

[0032] The task execution device 12 is configured to send a task request carrying informatio...

Embodiment 2

[0034] image 3 The implementation flow of the multi-task scheduling method provided by Embodiment 2 of the present invention is shown, and the process of the method is described in detail as follows:

[0035] In step S301, the task execution node sends a task acquisition request to the scheduling node, and the request carries the current load value and available memory space of the task execution node.

[0036] In this embodiment, when the task execution node triggers sending a heartbeat message, it sends a request to the scheduling node to obtain a task through the heartbeat information, and the request carries the current load value and available memory space of the task execution node.

[0037] Wherein, the current load value of the task execution node refers to the current processing performance of the task execution node, such as the usage rate of the CPU of the task execution node. The calculation formula of the current available memory space of the task execution node...

Embodiment 3

[0048] Figure 4 The composition structure of the task scheduling device provided by the third embodiment of the present invention is shown, and for the convenience of description, only the parts related to the embodiment of the present invention are shown.

[0049] The task scheduling device can be a software unit, a hardware unit or a combination of software and hardware running in the multi-task scheduling system, or it can be integrated into the multi-task scheduling system or run in the application system of the multi-task scheduling system as an independent pendant middle.

[0050] The task scheduling device 4 includes a request information receiving unit 41, a first judging unit 42, a second judging unit 43 and an allocating unit 44, and its specific functions are as follows:

[0051] The request information receiving unit 41 is configured to receive a request for obtaining a task sent by the task execution node, the request carrying the current load value and availabl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention is suitable for the technical field of data processing and provides a multitask dispatching method and system based on capacity. The multitask dispatching method comprises the steps: a task execution node sends a task obtaining request to a dispatching node, wherein the request carries a current load value and a current available memory space of the task execution node; and the dispatching node judges whether the load value is smaller than a threshold value or not, if so, task dispatching is performed to the task execution node according to the current available memory space of the task execution node. By means of the multitask dispatching method and system based on the capacity, the problems including overload, loading, insufficient memory and the like of the task execution node can be solved, the resource utilization rate of the task execution node is improved, and the task dispatching and executing efficiency is improved.

Description

technical field [0001] The invention belongs to the technical field of data processing, and in particular relates to a capacity-based multi-task scheduling method and system. Background technique [0002] MapReduce (MapReduce) is a distributed parallel programming model or general framework for processing large-scale data sets. The function of distributed data processing is realized by defining corresponding mapping (Map) and simplification (Reduce) functions. [0003] In the existing task scheduling system based on the MapReduce framework, it includes a scheduling node (JobTracker) and several task execution nodes (TaskTracker), and its network architecture is as follows figure 1 As shown, the client is used to submit the parallel processing job arranged by the user to the scheduling node, and the scheduling node decomposes the job submitted by the client into multiple Map tasks with the same processing function (but the input data may be different) and multiple Reduce ta...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/46
CPCG06F9/505G06F9/48G06F9/5088G06F2209/5022
Inventor 徐钊赵彦荣郭玮李均赵伟张书彬洪坤乾
Owner SHENZHEN TENCENT COMP SYST CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap