GPU Virtualization via Segmented Pass-Through and Mediated Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU virtualization technologies face challenges in efficiently sharing graphics memory among multiple virtual machines while maintaining performance and secure isolation, particularly in scenarios where multiple virtual machines need to access performance-critical resources without hypervisor intervention.
Innovation Solution
The implementation of a full GPU virtualization environment with mediated pass-through, where each virtual machine is presented with a virtual full-fledged GPU, allowing direct access to performance-critical resources, and privileged operations are trap-and-emulated for secure isolation, using a split CPU/GPU scheduling mechanism to minimize hypervisor intervention and optimize resource partitioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU is passed-through to one VM only, then device performance is improved, but device sharing capability deteriorates
Solution Approach 1:
The physical GPU is segmented into multiple virtual GPU instances (vGPUs), each assigned to a different VM. The GPU driver divides the physical GPU resources into separate address spaces, allowing each VM to have its own dedicated virtual GPU while sharing the underlying physical hardware. This segmentation enables both high performance (each VM gets dedicated resources) and sharing capability (multiple VMs access the same physical GPU).
Solution Approach 2:
The GPU driver acts as an intermediary between VMs and the physical GPU. It implements address space virtualization that translates guest physical addresses (GPAs) to host physical addresses (HPAs), enabling multiple VMs to access the GPU simultaneously without direct conflicts. The driver mediates resource allocation and manages address translation, allowing both performance optimization and multi-VM sharing.
2Adaptability or versatility
If shadow page tables are implemented, then multi-VM sharing is improved, but memory overhead increases
Solution Approach 1:
Multiple VMs share a single shadow page table structure instead of each VM having its own separate shadow page table. The address translation mechanism combines address spaces from multiple VMs into one unified shadow page table, reducing memory overhead while still enabling multi-VM sharing. This merging approach maintains the isolation and translation capabilities of shadow page tables while eliminating redundant structures.
Solution Approach 2:
The shadow page table structure is designed to serve multiple VMs simultaneously through a universal address translation mechanism. A single shadow page table can handle address translations for multiple VMs by incorporating their respective address spaces, making the translation structure multi-functional. This universality reduces the total memory required compared to having dedicated shadow page tables for each VM.
3Productivity
If direct DMA is used, then device performance is improved, but security and isolation deteriorate
Solution Approach 1:
The GPU driver serves as a security intermediary that sits between VMs and the physical GPU DMA operations. While VMs can perform direct DMA to the GPU for high performance, the driver intercepts and validates address translations through its shadow page table mechanism, ensuring that VMs can only access authorized memory regions. This intermediary approach maintains both direct DMA performance and security isolation.
Solution Approach 2:
Each VM is configured with its own address space and DMA permissions, allowing it to perform self-service DMA operations to the GPU without requiring hypervisor intervention for each transaction. The VM's own address translation mechanism ensures security and isolation while enabling high-performance direct access to GPU resources.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus and method are described for efficiently rendering an transmitting to a remote display. For example, one embodiment of a remote display apparatus comprises: a display engine to render a sequence of video images; an encoder to compress the sequence of video images to generate a sequence of compressed video images; a network interface controller to transmit the compressed video images over a network link to a remote display; a plurality of buffer pointer registers to store read pointers and write pointers identifying read locations and write locations, respectively, in a frame buffer and a compressed stream buffer; a central processing unit (CPU) to initialize the read pointers and write pointers for processing one or more of the video images; and the display engine to access a first write pointer to write to a specified location in the frame buffer, the encoder to begin reading from the frame buffer based on a first read pointer value, the encoder to write to the compressed stream buffer based on a second write pointer value, and the network interface controller to read from the compressed stream buffer based on a second read pointer value, the first and second write and read pointer values to be updated without intervention from the CPU as the display engine writes to the frame buffer, the encoder reads from the frame buffer and writes to the compressed stream buffer, and the network interface controller reads from the compressed stream buffer.