Task-centric Job Scheduling for Heterogeneous Accelerator Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed schedulers for compute clusters are not well-suited for fine-grained execution patterns, leading to inefficient resource utilization, idle resources, and increased costs due to over-reservation and lack of automation in adapting to varying workload demands.
Innovation Solution
A computer-implemented scheduling method that uses a computational graph to split jobs into atomic compute tasks, allowing individual scheduling and migration of tasks across different machines and accelerators at runtime, with a scheduler and daemon for memory allocation and kernel execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If current distributed schedulers reserve nodes as a whole with their accelerators, then resource allocation is simplified, but resource utilization deteriorates due to idle accelerators when applications use only one accelerator per node
Solution Approach 1:
The patent segments the scheduling unit from the node level to the accelerator level. Instead of reserving entire nodes with their accelerators as a single unit, the system allows individual accelerators to be scheduled and allocated independently. This segmentation enables finer-grained resource allocation where each accelerator can be assigned to different applications based on actual demand, preventing idle accelerators when only one is needed per node.
2Ease of operation
If users specify resource requirements upfront, then resource reservation is straightforward, but execution time is wasted when users over-reserve resources due to lack of understanding
Solution Approach 1:
The patent implements dynamic resource allocation that adjusts resource assignment during job execution rather than relying solely on static upfront specifications. The scheduler can dynamically allocate accelerators to tasks based on runtime conditions, allowing users to specify initial requirements while the system adapts to actual needs, preventing over-reservation and reducing wasted execution time.
Solution Approach 2:
The system enables self-service scheduling where the scheduler automatically manages resource allocation without requiring users to precisely specify all requirements. The scheduler monitors job execution and dynamically adjusts accelerator assignment, allowing users to submit jobs with minimal specification while the system optimizes resource usage automatically.
3Device complexity
If nodes are reserved for the entire job duration, then resource allocation is simplified, but cost increases due to idle but reserved time when workload changes at runtime
Solution Approach 1:
The patent segments the job execution into independent tasks that can be scheduled on different accelerators at different times. Instead of reserving a node for the entire job duration, the system allows tasks to be migrated between accelerators and nodes, enabling other jobs to use idle accelerators when original jobs are not utilizing them fully, thereby reducing energy waste and cost.
Solution Approach 2:
The system implements dynamic task migration capabilities that allow tasks to be moved between different accelerators and nodes during execution. This dynamic approach enables the scheduler to reassign idle accelerators to other jobs based on runtime workload changes, preventing paid-but-unused resource time and reducing overall energy consumption and cost.
Data Source
AI summary
A computer-implemented scheduling method for accelerator hardware includes using a computational graph that splits jobs to be executed by the accelerator hardware into atomic compute tasks, using a scheduler to individually schedule and/or migrate each task for execution to different machines and/or accelerators at job runtime, and using a computer daemon to allocate memory and execute kernels for each task on the respective machines and/or accelerators.