VM Checkpointing for GPU Pass-Through State Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtualization techniques fail to provide effective fault-tolerance and high availability for Graphics Processing Units (GPUs) due to challenges in capturing GPU state during virtual machine replication, especially in pass-through configurations where memory access is non-deterministic, leading to performance bottlenecks and incomplete state snapshots.
Innovation Solution
Implementing a command-based state tracker that intercepts GPU commands and uses MMU virtualization to predict and snapshot dirty memory pages and register changes, allowing for efficient tracking and periodic checkpointing of GPU state, even in dedicated or shared pass-through scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If VM checkpointing is implemented for GPU pass-through devices, then fault-tolerance and high availability are improved, but device complexity and difficulty of detecting and measuring GPU state increase
Solution Approach 1:
The patent introduces a VMM (Virtual Machine Monitor) as an intermediary layer between the GPU pass-through device and the VM. The VMM intercepts GPU commands, tracks state changes, and manages checkpointing operations, thereby enabling fault-tolerance without requiring direct modification of the GPU device itself. This mediator approach resolves the contradiction by adding virtualization-layer complexity rather than device-layer complexity.
Solution Approach 2:
The patent implements checkpointing by creating snapshots (copies) of the GPU state at specific points in time. These snapshots include register states, memory states, and command buffer states. By copying the GPU state rather than modifying the original device, the system achieves fault-tolerance while maintaining the original GPU's operational integrity, thus reducing the effective complexity burden.
2Reliability
If complete GPU state snapshots are taken for checkpointing, then reliability is improved, but loss of time and productivity decrease due to performance bottlenecks
Solution Approach 1:
The patent extracts only the essential and changed GPU state information for checkpointing, rather than snapshotting the entire GPU state. Specifically, it tracks and snapshots only dirty memory pages, modified register states, and relevant command buffer contents. This extraction approach significantly reduces checkpointing time and overhead while maintaining sufficient state information for effective fault-tolerance and failover.
Solution Approach 2:
The patent applies partial action by selectively checkpointing only the necessary portions of GPU state that are critical for fault-tolerance. Instead of taking complete snapshots of all GPU resources, it focuses on tracking changes to specific registers, memory pages, and command buffers that are most relevant for state restoration after failures, thereby reducing time loss.
3Stability of the object's composition
If lock-stepping is used for VM replication, then deterministic execution is improved, but productivity decreases due to performance bottlenecks in MP guest situations
Solution Approach 1:
The patent replaces continuous lock-stepping with periodic checkpointing. Instead of maintaining deterministic execution at every instruction cycle, the system takes periodic snapshots of the GPU state and allows the VM to execute independently between checkpoints. This periodic approach maintains sufficient determinism for fault-tolerance while eliminating the continuous performance bottleneck of lock-stepping in multi-processor guest environments.
Data Source
AI summary
Methods, software, and apparatus for application transparent, high available GPU computing with VM checkpointing. The guest access of certain GPU resources, such as MMIO resources, are trapped to keep a copy of guest context per semantics, and/or emulate the guest access of the resources prior to submission to the GPU, while other commands relating to certain graphics memory address regions are trapped before being passed through to the GPU. The trapped commands are scanned before submission to predict: a) potential to-be-dirtied graphics memory pages, and b) the execution time of intercepted commands, so the next checkpointing can be aligned to a predicted execution time. The GPU internal states are drained by flushing internal context/tlb/cache, at the completion of submitted commands, and then a snapshot of the vGPU state is taken, based on tracked GPU state, GPU context (through GPU-specific commands), detected dirty graphics memory pages and predicted to-be dirtied graphics memory pages.