Task scheduling algorithm based on MapReduce

A technology of task scheduling and scheduling algorithm, applied in the direction of program startup/switching, multiprogramming device, etc.

Active Publication Date: 2014-03-12

LANGCHAO ELECTRONIC INFORMATION IND CO LTD

5 Cites 43 Cited by

AI-Extracted Technical Summary

Problems solved by technology

It can overcome many problems existing in the existing scheduling algorithm, effectively solve the problems of local computing and small job processing, and ...

Abstract

The invention relates to task scheduling algorithms in a very important programming computing framework MapReduce in the current field of large data, and discloses a task scheduling algorithm based on MapReduce. According to the task scheduling algorithm, under the heterogeneous cluster environment, tasks are assigned to all computing nodes on the basis of a multitask scheduling algorithm of an ant colony algorithm through computing node processing capacity measuring according to a new task object transition function, a new node updating rule and a local computing rule. According to the task scheduling algorithm, on the basis of the classical ant colony algorithm, large-scale optimization is performed, the multitask scheduling algorithm under the heterogeneous cluster environment is provided, testing and performance analysis of scenes such as small operation, load and local are performed on an open source Hadoop platform, and results show that the execution efficiency and the task balance are greatly improved.

Application Domain

Program initiation/switching

Technology Topic

Ant colonyDistributed computing +3

Image

Examples

Experimental program(9)

Example Embodiment

[0068] Example 1:

[0069] A task scheduling algorithm based on MapReduce, in a heterogeneous cluster environment, a multi-task scheduling algorithm based on ant colony algorithm, by measuring the processing performance of computing nodes, according to the new task goal transfer function and new node update rules, according to The local computing principle allocates tasks to each computing node.

Example Embodiment

[0070] Example 2:

[0071] On the basis of embodiment 1, the measurement of the processing performance of the computing node in this embodiment mainly measures the initial processing capacity of the node and the target transition probability of tasks assigned to the node. The initial processing capacity of the node is based on the processing speed, The four metrics of memory capacity, number of CPUs, and network transmission bandwidth are comprehensively measured, and thresholds are set for these four metric parameters respectively. If the threshold is exceeded, the threshold is calculated uniformly; when task scheduling, a scheduler is set up to be responsible for Calculate the initial transition probability of the task assigned to the requesting node.

Example Embodiment

[0072] Example 3:

[0073] On the basis of Embodiment 2, the initial processing capability of the node depends on the initial pheromone of the node, which is calculated and determined by formula 1.1.

PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.