GPU Pool Scheduling for Mixed-Size Job Fragmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU resource scheduling technologies face issues of inter-job interference and resource fragmentation, leading to reduced efficiency when handling jobs of varying sizes on computing systems with multiple GPUs.
Innovation Solution
Implementing multiple GPU pools based on the number of GPUs required for each job, with strategic assignment and release of GPUs to minimize interference and fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple jobs of varying sizes are scheduled on shared GPU resources, then resource utilization increases, but inter-job interference and resource fragmentation occur leading to reduced efficiency
Solution Approach 1:
The patent divides the GPU resource pool into multiple GPU pools based on job size categories (small, medium, large jobs). Each GPU pool is dedicated to handling specific job size ranges, preventing inter-job interference by isolating different job types. This segmentation allows simultaneous execution of multiple jobs without resource conflicts while maintaining high overall utilization.
Solution Approach 2:
The patent assigns different scheduling strategies and resource allocation policies to different GPU pools based on their specific characteristics. Small job pools use different allocation rules compared to large job pools, optimizing performance for each job category locally rather than applying a uniform scheduling approach across all jobs.
2Adaptability or versatility
If GPUs are dynamically allocated to different jobs, then flexibility and adaptability improve, but resource fragmentation increases reducing scheduling efficiency
Solution Approach 1:
By segmenting GPUs into dedicated pools based on job size, the system maintains flexibility within each pool while avoiding fragmentation across the entire resource pool. Each segmented pool manages resources independently, preventing the fragmentation that would occur with fully dynamic allocation across heterogeneous job types.
Solution Approach 2:
Each GPU pool serves multiple purposes: handling various jobs within its size category, providing isolation to prevent interference, and maintaining a reserved capacity for future allocations. This multi-functionality allows the segmented structure to provide both flexibility and fragmentation prevention simultaneously.
Data Source
AI summary
The present disclosure provides a method and apparatus for scheduling a plurality of available graphics processing units (GPUs). Multiple GPU pools may be set, wherein each GPU pool is configured to serve one or more jobs requiring the same number of GPUs. Available GPUs may be assigned to each GPU pool. A job and job information related to the job may be received, wherein the job information indicates a number of GPUs required for performing the job. A corresponding GPU pool may be selected from the multiple GPU pools based at least on the job information. Available GPUs to be scheduled to the job in the selected GPU pool may be determined based at least on the job information. In addition, the determined available GPUs may be scheduled to the job.