Method, system and program products for a dynamic, hierarchical reporting framework in a network job scheduler

a job scheduler and hierarchical reporting technology, applied in the field of job scheduling systems, can solve the problems of scheduling agents that are easily damaged by heavy communication load, so as to improve the scalability and performance of scheduling agents

US20090094605A1Inactive Publication Date: 2009-04-09IBM CORP
19 Cites 32 Cited by

Patent Information

Authority / Receiving Office
US · United States
Patent Type
Applications(United States)
Current Assignee / Owner
Publication Date
2009-04-09
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The present invention employs a master node for each job to be scheduled and in turn the master node distributes job start information and executable tasks to a plurality of nodes configured in a hierarchical node tree of a multinode job scheduling system. The status of the various tasks executing at the leaf nodes and other nodes of the tree report status back up the same hierarchical tree structure used to start the job, not to a scheduling agent but rather to the master node which has been established by the scheduling agent as the focal point, not only for job starting, but also for the reporting of status information from the leaf and other nodes in the tree.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention is generally directed to a job scheduling system in an environment which includes a large plurality of data processing nodes. More particularly, the present invention is directed to providing a hierarchical structure for the return of job or task status information so as to relieve bottlenecks created at a scheduling agent especially when there are a large plurality of nodes carrying out the job.BACKGROUND OF THE INVENTION

[0002] Scheduling parallel workloads in a High Performance Computing (HPC) cluster is an increasingly complex task, especially when it concerns scalability and performance of the scheduling agent. This is because clusters are being used to solve extremely large and complicated problems. This has led to an increase in the number of nodes required to execute a parallel job by an order of magnitude or more. By implication, the total number of nodes in a typical HPC cluster has gone up by an order of magnitude as well.

[0003] Whe...

Examples

Embodiment Construction

[0013]When a job is submitted for execution to the, scheduling agent determines a set of nodes on which this job can run based on the requirements of the job as well as the availability of necessary resources on these compute nodes. Each compute node runs an agent that is capable of reporting the status of jobs dispatched to them for execution.

[0014]To start the job, the scheduling agent sets up a master compute node and sends the job object to it in a JOB_START transaction. This master compute node forwards this “job start order” to a predetermined number of slave nodes while initiating the hierarchical job launch tree. The master node informs each slave node who its child nodes are. Each slave node further forwards the job to its children until all the nodes on which the job runs have received the job start transaction (See FIG. 1).

[0015]Every node in the tree now communicates that it is ready to receive the executable or task(s) to be run. In most existing schemes, all of the age...