Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Task scheduling method and device

A job scheduling and job technology, applied in the field of distributed computing, can solve the problems of inability to wait for resources, distributed system consumption, reduce distributed system throughput and resource utilization, etc., to improve rationality, throughput and resources. Utilization, the effect of improving processing efficiency and resource utilization

Active Publication Date: 2018-08-24
ALIBABA GRP HLDG LTD
View PDF11 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But in fact, a job is likely to get enough resources within a short period of time after the upper limit of the waiting time, and switching to offline mode may consume more resources than waiting for a short period of time in online mode. time and resources
Even if the same job is submitted at different time points, if there are many jobs in the waiting queue at that time, it is very likely that the resources will not be available after waiting for a period of time.
[0005] Therefore, since the running conditions of each job are different and the running conditions of other jobs in its waiting state are also different, it is unreasonable to schedule each job in the waiting queue by setting a fixed upper limit of waiting time, which may cause A distributed system consumes more time and resources on a job, thereby reducing the throughput and resource utilization of the distributed system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task scheduling method and device
  • Task scheduling method and device
  • Task scheduling method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0058] This embodiment provides a job scheduling method, such as figure 1 shown, including the following steps:

[0059] Step 101, for the first job in the waiting queue in the online mode, according to the task information of all jobs before the first job, estimate the first time point when the available resources meet the requirements of the first job;

[0060] Step 102, comparing the first time point with the upper limit of the waiting time of the first job;

[0061] Step 103, when the first time point is greater than the upper limit of the waiting time of the first job, turn the first job to run in an offline mode.

[0062] In this embodiment, when the scheduled job is in the waiting state, the first time point at which the available resources can meet the demand of the scheduled job can be estimated at any time according to the own situation of the scheduled job and the current running status of other jobs, if the If the first time point is greater than the upper limit ...

example 1

[0086] The process of estimating the end time of the task and the corresponding amount of releasable resources and the way of saving the histogram are described in detail below in the form of an example.

[0087] Before executing the scheduling process on the jobs in the waiting queue, you can pre-estimate the task end time and the number of releasable resources at the end of the unfinished tasks in the jobs before the job, and convert them into a histogram for easy storage.

[0088] Such as figure 2 Shown is an example of a histogram. Each bar in the histogram corresponds to an unfinished task. The width of the bar indicates the amount of releasable resources of the unfinished task. The time point corresponding to the bar is the end of the task of the unfinished task. time.

[0089] Such as image 3 As shown, the end time of the task and the corresponding amount of releasable resources can be estimated and converted into histogram elements through the following process:

...

example 2

[0100] The method of determining the lower limit of the waiting time and the upper limit of the waiting time of the job in this embodiment will be described in detail below with specific examples.

[0101] In practical applications, the lower limit of waiting time and the upper limit of waiting time can be determined by the following formula:

[0102] Lower limit of waiting time = job submission time + waiting time threshold;

[0103] Upper limit of waiting time = job submission time + max (waiting time threshold, Tx2-Tx1);

[0104] Among them, Tx2 represents the running time of the job in offline mode, Tx1 represents the running time of the job in online mode, and the waiting time threshold can be an empirical value, which can be calculated by historical running data of each job in the online mode According to the analysis, for example, the waiting time threshold may be set as 10 seconds.

[0105] In practical applications, when a job is submitted to the online mode, if the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Provided are a task scheduling method and device. The method comprises the following steps: estimating a first time point when available resources meet a first operation requirement according to taskinformation of all tasks before first operation as for the first operation in a waiting queue under an online mode; comparing waiting time limit of the first time point and the first operation; and shifting the first operation to be operated in an off-line mode when the waiting time limit of the first time point is larger than that of the first operation. Tasks can be reasonably scheduled according to operation conditions of the tasks in the waiting queue and operation conditions of other operations in the waiting state, thereby fulfilling the purposes of increasing the throughput rate and theresources utilization rate of a distributed system.

Description

technical field [0001] The present invention relates to the technical field of distributed computing, in particular to a job scheduling method and device. Background technique [0002] The distributed system is divided into offline mode and online mode. Among them, the offline mode can use the batch processing (Batch) scheduling mode to process the job (job), and the online mode can use the first-in-first-out (AON, All-or-Nothing) scheduling mode to process the job. . Among them, the processing speed of the online mode is fast, but it needs to consume system memory, and the processing speed of the offline mode is relatively slow, but it does not rely too much on the memory and mainly uses the disk. Since the price of disks is higher than that of memory, in order to improve the overall utilization of cluster resources and reduce the running time of jobs as much as possible, many distributed systems currently use online and offline mixed running methods to process jobs. [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5011G06F9/5022G06F2209/503
Inventor 陆一峰陶阳宇王博李超吴永军李治吕志强刘耀莉付迪
Owner ALIBABA GRP HLDG LTD
Features
  • Generate Ideas
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com