Host Scheduler for Heterogeneous GPU Resource Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to fairly allocate GPU resources among virtual machines (VMs) due to disparity in graphics command complexity and GPU processing capabilities, leading to monopolization and sub-optimal throughput.
Innovation Solution
A computing device assigns shares to users with VMs, generates composite scores for GPUs based on processing capabilities, adjusts shares based on graphics command characteristics, and allocates quantum to VMs, ensuring fair distribution and scheduling of graphics commands across GPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU resources are allocated using first-in, first-out manner without considering command complexity or GPU capabilities, then implementation is simple, but resource allocation fairness deteriorates and throughput decreases
Solution Approach 1:
The patent changes the allocation parameters from simple FIFO ordering to a composite score system that incorporates command complexity metrics and GPU capability metrics. This allows the system to evaluate and allocate resources based on multiple parameters simultaneously, improving throughput by matching appropriate workloads to suitable GPUs while maintaining fairness among users.
Solution Approach 2:
The patent introduces a host-based scheduler as an intermediary between the GPU drivers and the physical GPUs. This mediator evaluates command characteristics, determines fairness metrics, and makes intelligent allocation decisions, thereby resolving the contradiction between simple allocation and optimal throughput without requiring complex changes at the GPU driver level.
2Ease of operation
If all graphics commands are treated as originating from a single user, then driver implementation is simplified, but resource allocation fairness among multiple users deteriorates
Solution Approach 1:
The host-based scheduler acts as an intermediary layer that maintains the simple single-user interface expected by GPU drivers while simultaneously implementing multi-user fairness mechanisms. The scheduler tracks user-specific allocations and command characteristics, ensuring fair resource distribution without complicating the driver implementation.
Solution Approach 2:
The patent segments the resource allocation function into two distinct layers: the GPU driver layer that handles simple command submission, and the host-based scheduler layer that handles complex multi-user fairness management. This segmentation allows each layer to operate independently with appropriate complexity, maintaining driver simplicity while achieving fairness.
3Adaptability or versatility
If heterogeneous GPUs with different processing capabilities are allocated without considering their specific characteristics, then hardware utilization flexibility is improved, but allocation optimization deteriorates
Solution Approach 1:
The patent incorporates GPU capability parameters (such as processing power, memory bandwidth, and architectural features) into the composite score calculation. This allows the system to differentiate between heterogeneous GPUs and make optimized allocation decisions that match workload characteristics to appropriate GPU capabilities, thereby maximizing throughput while maintaining the flexibility to use diverse hardware configurations.
Data Source
AI summary
Examples of the disclosure assign a plurality of graphics processing units (GPUs) to a plurality of virtual machines (VMs) or processes. A composite score is generated for each GPU. The composite score represents the normalized processing capabilities of the multiple GPUs. Based on a comparison between the composite scores and allocated quantum corresponding to a proportional amount of GPU resources to which each VM is entitled, each VM is assigned to at least one of the GPUs. Graphics commands from the VMs are scheduled for execution by the assigned GPUs.