Unlock instant, AI-driven research and patent intelligence for your innovation.

A task scheduling method and device for a computing node

A computing node and task scheduling technology, applied in the computer field, can solve problems such as low resource utilization, idle clusters, and inability to realize multiple tasks in parallel, and achieve the effect of improving resource utilization and improving completion speed

Active Publication Date: 2019-03-01
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Among them, FIFO Scheduler (queue task scheduler) can perform tasks in a simple and sequential manner, but it is not suitable for clusters
Only a single task can be executed at a time, and multiple tasks cannot be parallelized; CapacityScheduler (capacity task scheduler) can support multi-task parallel execution, but the execution order of subtasks lacks priority sorting, which may lead to an increase in task completion time; Fair Scheduler (fair Task scheduler) can support priority preemption, but often a queue resource is full, but the cluster as a whole is still idle
Overall resource utilization is not high

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A task scheduling method and device for a computing node
  • A task scheduling method and device for a computing node
  • A task scheduling method and device for a computing node

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0041] This embodiment provides a task scheduling method for computing nodes, such as figure 1 As shown, the method includes S11-S13:

[0042] S11. Setting the priority weight of each subtask of the task;

[0043] S12. Regularly sort the subtasks to be executed according to the priority weights and required resources of the subtasks;

[0044] S13. Select subtasks to be executed in parallel according to the ranking.

[0045] In the embodiment of the present invention, under the condition of limited computing node resources, the resource utilization rate of the computing nodes is improved, and the task completion speed is improved.

[0046] In an exemplary embodiment, the task may include one or more tasks. The periodicity may be "timing", or "when the execution completion rate of the task reaches a preset value", etc. Required resource amount = initially marked required resource amount * priority weight.

[0047] In an exemplary embodiment, the setting of the priority weig...

Embodiment 2

[0071] This embodiment specifically describes the methods in the above embodiments.

[0072] The resources of computing nodes are known, such as the number of CPU cores and the maximum capacity of memory. Multiple computationally intensive data processing tasks have been divided into multiple subtasks by other algorithms according to logical relationships. According to the complexity of the data processing method and the size of the data, the subtasks are correspondingly marked with the required computing resources, such as the number of CPU cores and memory capacity required by the subtasks. Each subtask is executed in parallel in the computing node, and the sum of the computing resources required by the parallelly executed subtasks should be as close as possible to the resource amount of the computing node. In addition, for subtasks that need to be executed first, the original required resource amount will be multiplied by a priority weight greater than 1, so that they can ...

Embodiment 3

[0099] This embodiment provides a task scheduling device for computing nodes, and the descriptions in the above method embodiments are also applicable to this embodiment, for example image 3 As shown, the device includes a setting module 31, a sorting module 32 and an execution module 33, wherein:

[0100] The setting module 31 is used to set the priority weight of each subtask of the task;

[0101] The sorting module 32 is configured to regularly sort the subtasks to be executed according to the priority weights and required resources of the subtasks;

[0102] The executing module 33 is configured to select subtasks to be executed in parallel according to the ranking.

[0103] In an exemplary embodiment, the setting module 31 sets the priority weight of each subtask of the task, including:

[0104] When the setting module 31 receives multiple subtasks of a task, it sets the priority weight of the subtasks to an initial value.

[0105] In an exemplary embodiment, the setti...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a task scheduling method and device for a computing node, which can improve the resource utilization rate of the computing node. The method comprises the following steps: setting a priority weight of each sub-task of a task; according to the priority weights of the subtasks and required resources, the subtasks to be executed are ranked regularly; and selecting sub-tasks according to the sequence and executing the sub-tasks in parallel. By adopting the embodiment of the invention, the calculation task sequence is dynamically scheduled and arranged, so that the calculation time is not too long, and the resource utilization rate of the calculation node is improved as much as possible.

Description

technical field [0001] The invention relates to computer technology, in particular to a task scheduling method and device for computing nodes. Background technique [0002] In the field of big data, the amount of data is huge, the data processing process is complex, and the computing tasks are huge and time-consuming. The cost of hardware (mainly CPU and memory) for computing resources continues to increase. Improving the utilization of computing resources can not only reduce hardware costs, but also save computing time. [0003] Usually, when a computing node is faced with multiple large-scale data processing tasks, it will divide the original data processing task into multiple sub-tasks of moderate scale, and adopt a certain scheduling strategy for the sub-tasks before the computing node performs the calculation. deal with. For example, there are three schedulers in Hadoop Yarn: FIFO Scheduler (queue task scheduler), Capacity Scheduler (capacity task scheduler), Fair Sc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48
CPCG06F9/4881
Inventor 王振
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD