Dedicated heterogeneous node scheduling including backfill scheduling

Abstract

A method and system for job backfill scheduling dedicated heterogeneous nodes in a multi-node computing environment. Heterogeneous nodes are grouped into homogeneous node sub-pools. For each sub-pool, a free node schedule (FNS) is created so that the number of to chart the free nodes over time. For each prioritized job, using the FNS of sub-pools having nodes useable by a particular job, to determine the earliest time range (ETR) capable of running the job. Once determined for a particular job, scheduling the job to run in that ETR. If the ETR determined for a lower priority job (LPJ) has a start time earlier than a higher priority job (HPJ), then the LPJ is scheduled in that ETR if it would not disturb the anticipated start times of any HPJ previously scheduled for a future time. Thus, efficient utilization and throughput of such computing environments may be increased by utilizing resources otherwise remaining idle.

Description

CLAIM OF PRIORITY IN PROVISIONAL APPLICATION[0001]This application claims priority in provisional application filed on May 1, 2001, entitled “Dedicated Heterogeneous Node Backfill Scheduling” serial No. 60 / 288,255, by inventor(s) Robert R. Wood, Philip D. Eckert, and Gregg Hommes.[0002]The United States Government has rights in this invention pursuant to Contract No. W-7405-ENG-46 between the United States Department of Energy and the University of California for the operation of Lawrence Livermore National Laboratory.FIELD OF THE INVENTION[0003]The present invention relates to job scheduling in dedicated multi-node computer systems and networks. More particularly, the present invention relates to job scheduling and backfill scheduling in dedicated heterogeneous multi-node computing environments where nodes of varying capacities may be efficiently and optimally utilized without degrading utilization or throughput of higher priority jobs.BACKGROUND OF THE INVENTION[0004]Scheduling jo...

AI Technical Summary

Benefits of technology

[0012]One advantage of the method and system of the present invention is that a job scheduler can backfill schedule dedicated heterogeneous multi-node computers without causing any delay in the execution of higher priority jobs, regardless of the resources required by the higher priority jobs. In this manner, the overall efficient utilization and throughput of dedicated heterogeneous multi-node computers may be increased without degrading utilization or throughput of higher priority jobs by utilizing resources that would otherwise remain idle.

Problems solved by technology

Scheduling jobs on parallel computer systems is challenging due to high job submission rates, high utilization of system processors and memory, i.e. scarcity of free resources at any given time, and unpredictable arrival of jobs having various system resource requirements.

Backfill scheduling technology, however, is essentially limited to scheduling dedicated homogenous nodes of a multi-node computer system or network, i.e. where all the nodes have identical capacities.

This limitation effectively prevents current backfill scheduling technology from recognizing or distinguishing the capacities (e.g., CPUs per node and memory) of nodes in the scheduling set, or the differing resource requirements of jobs to be scheduled.

Consequently, current backfill scheduling technology does not work properly when the node set contains nodes not all having identical or equal capacities, i.e. heterogeneous nodes.

When used in a heterogeneous environment, these deficiencies may frequently result in: (1) erroneous priority scheduling when the resources required by the priority job are not available causing it to not start at the intended schedule time, (2) erroneous backfill scheduling when the resources required by the backfill job are not available causing it to not start at the intended schedule time and consequently causing the delayed start of higher priority jobs, or (3) erroneously not backfill scheduling as a result of computing the start time of a higher priority job to be sooner than it could really start.

While both methods permit user jobs to run, they do not fully utilize all of a system resources in an efficient manner.

Images