Supercharge Your Innovation With Domain-Expert AI Agents!

Method and device for task scheduling

A task scheduling and task technology, applied in the computer field, can solve problems such as long time required for task execution and poor overall system performance, and achieve the effect of improving overall performance and reducing time

Active Publication Date: 2013-09-04
BEIJING MFOX SCI & TECH CO LTD
View PDF4 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Existing task scheduling methods only consider the computing resources of the execution nodes when scheduling tasks. However, when the execution nodes execute Map tasks and Reduce tasks, they need to transmit data through the network. If only the computing resources of the execution nodes are considered, the It will lead to a longer time required for task execution and poorer overall performance of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for task scheduling
  • Method and device for task scheduling
  • Method and device for task scheduling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0059] The embodiment of the present invention provides a method for task scheduling, see figure 1 , the method flow includes:

[0060] 101: Obtain multiple tasks after the job to be processed is decomposed, and the task type of each task is a Map task or a Reduce task.

[0061] 102: Select a task to be scheduled from multiple tasks, and after receiving the task request sent by the working node, select and execute it from the working node that sent the task request according to the task type of the task to be scheduled and the network available bandwidth information obtained in advance node.

[0062] Further, before selecting the execution node from the working nodes sending the task request according to the task type of the task to be scheduled and the network available bandwidth information obtained in advance, it also includes:

[0063] Obtain network topology information in advance, and the network topology information includes at least the connection relationship betwee...

Embodiment 2

[0080] In order to reduce the time required for executing tasks in the MapReduce system, an embodiment of the present invention provides a task scheduling method, which is applied to the MapReduce system, and the MapReduce system includes a terminal, a scheduling node, and multiple working nodes. Wherein, the scheduling node can be a server, which is used to manage the task scheduling of the entire MapReduce system; the working node is a server or other equipment, and is used to execute the tasks sent by the scheduling node. In combination with the content of the first embodiment above, see figure 2 , the method flow includes:

[0081] 201: The scheduling node obtains multiple tasks after the job to be processed is decomposed, and the task type of each task is a Map task or a Reduce task.

[0082] Wherein, the jobs to be processed may be jobs processed in parallel. The scheduling node can directly obtain multiple tasks after the pending job is decomposed; it can also obtain...

Embodiment 3

[0119] An embodiment of the present invention provides a task scheduling device, and the device is used to execute the task scheduling method provided in the first or second embodiment above. see Figure 4 , the device consists of:

[0120] The first acquiring module 401 is configured to acquire multiple tasks after the job to be processed is decomposed, and decompose the job to be processed into multiple tasks, and the task type of each task is a Map task or a Reduce task;

[0121] The first selection module 402 is configured to select a task to be scheduled from the multiple tasks acquired by the first acquisition module 401;

[0122] The second selection module 403 is configured to, after receiving the task request sent by the working node, select from the working node sending the task request according to the task type of the task to be scheduled selected by the first selection module 402 and the network available bandwidth information obtained in advance Select the exec...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a device for task scheduling, and belongs to the field of computers. The method includes: acquiring multiple tasks after a to-be-processed job is decomposed; selecting to-be-scheduled tasks from the tasks, and after task requests sent by working nodes are received, selecting execution nodes from the working nodes sending the task requests according to task types of to-be-processed tasks and available network bandwidth information acquired in advance; and sending the to-be-processed tasks to the execution nodes, and executing the to-be-processed tasks by the execution nodes. The execution nodes are selected from the working nodes sending the task requests according to the task types of the to-be-processed tasks and the available network bandwidth information acquired in advance, and then the to-be-processed tasks are sent to the execution nodes for execution; a large amount of data need to be transmitted in a network when the tasks are executed, and the execution data are selected according to the available network bandwidth information, so that data transmission rate in the network can be improved, further needed time for task execution is shortened, and overall system performance is improved.

Description

technical field [0001] The invention relates to the field of computers, in particular to a task scheduling method and device. Background technique [0002] The MapReduce (mapping simplification) system is widely used in the field of data processing and analysis. The biggest advantage of this system is that it realizes parallel data processing, and can automatically decompose pending jobs into multiple subtasks in parallel and schedule them on server clusters. Execute in parallel. The MapReduce system includes a terminal (Client), a scheduling node (Master) and multiple working nodes (Worker). Among them, the client is used to send the pending job to the scheduling node; the scheduling node is used to decompose the pending job into multiple tasks, and the task type of each task can be Map (map) task or Reduce (simplify) task; The scheduling node also needs to perform task scheduling, select the task to be scheduled from the generated tasks, and select the execution node to ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/44
Inventor 张鹏金晨
Owner BEIJING MFOX SCI & TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com