GPU Workload Scheduling via Extended IOMMU
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU virtualization technologies face challenges in efficiently managing and scheduling graphics processing resources across multiple virtual machines, leading to suboptimal performance and resource utilization, especially in cloud environments where flexible resource management is crucial for revenue generation and workload balancing.
Innovation Solution
The proposed solution involves a method and apparatus for scheduling graphics processing resources that utilize a virtualization architecture allowing direct access to performance-critical resources by virtual machines, with mediated pass-through and trap-and-emulation for privileged operations, enabling secure isolation and efficient sharing of physical GPUs among multiple virtual machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If shadow page tables are used for GPU memory translation to support multiple VMs, then multiple virtual machines can share a physical GPU, but the system memory consumption increases and translation overhead is introduced
Solution Approach 1:
The patent merges the GPU address translation functionality into the existing IOMMU (I/O Memory Management Unit) hardware structure. Instead of implementing separate shadow page tables for GPU translation, the system reuses the IOMMU's address translation capabilities to handle GPU virtual address to physical address translation, thereby consolidating translation functions and eliminating duplicate memory structures.
Solution Approach 2:
The IOMMU is extended to perform multiple functions: it handles both traditional I/O device address translation and GPU-specific virtual address translation. By making the IOMMU universal for both purposes, the system eliminates the need for separate translation hardware or software structures specifically for GPUs, reducing overall system complexity and memory usage.
2Speed
If hardware address translation is implemented for GPU to support multiple VMs, then translation speed improves, but the GPU can only be passed through to one VM at a time
Solution Approach 1:
The system implements dynamic address translation by extending the IOMMU to handle GPU virtual addresses with page table entries that can be dynamically configured. The IOMMU maintains translation tables that map GPU virtual addresses from different VMs to physical addresses, allowing the translation hardware to adaptively handle multiple VMs' address spaces without requiring static assignment of GPUs to single VMs.
Solution Approach 2:
The extended IOMMU acts as an intermediary between GPU virtual addresses from multiple VMs and the physical GPU resources. It mediates the address translation process by intercepting GPU memory access requests, translating virtual addresses to physical addresses through its translation tables, and forwarding the translated addresses to the GPU, thereby enabling multiple VMs to share the GPU with hardware-accelerated translation speeds.
3Productivity
If direct DMA access is allowed from VMs to GPU memory, then device performance improves, but security isolation between VMs is compromised
Solution Approach 1:
The extended IOMMU serves as a security intermediary that sits between VMs and GPU memory. It allows direct DMA access from VMs to GPU memory for performance, but enforces security isolation by validating address translations against configured page table entries. The IOMMU monitors and controls all GPU memory access requests from multiple VMs, ensuring that each VM can only access its allocated memory regions while maintaining high-performance direct access.
4Reliability
If trap-and-emulation is used for privileged GPU operations in virtualized environment, then security and control are maintained, but performance overhead increases
Solution Approach 1:
The patent extracts privileged GPU operation handling from the software trap-and-emulation path and implements it in hardware within the extended IOMMU. The IOMMU hardware directly handles privileged operations such as memory management and device control by intercepting and translating their addresses, eliminating the need for software traps and emulations while maintaining security and control.
Solution Approach 2:
The system replaces the software-based trap-and-emulation mechanism with a hardware-based solution in the extended IOMMU. Instead of using software interrupts and emulation loops to handle privileged GPU operations, the IOMMU hardware directly processes these operations through its address translation and control logic, significantly reducing performance overhead while maintaining the necessary security and control mechanisms.
Data Source
AI summary
An apparatus and method for scheduling workloads across virtualized graphics processors. For example, one embodiment of a graphics processing apparatus comprises first graphics processing resources to process graphics commands and execute graphics data; workload scheduling circuitry to schedule workloads for execution on the first graphics processing resources; and workload queuing circuitry to implement a local queue to store local workload entries, each local workload entry associated with a locally-submitted workload and an external workload queue to store external workload entries, each external workload entry associated with an externally-submitted workload submitted for execution by an external graphics processing apparatus, in one embodiment, the workload scheduling circuitry schedules the locally-submitted workloads identified in the local queue and externally-submitted workloads identified in the external workload queue for processing by specified portions of the first graphics processing resources.