GPU Microcontroller Scheduling for Virtual Machine Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional GPU workload scheduling in virtualized environments is limited by software managers, leading to latency and complexity, especially when handling graphics-intensive applications and non-graphics compute workloads, as it relies on a single software stack and introduces challenges in translating APIs between virtual machines and the host operating system.
Innovation Solution
The implementation of a graphics microcontroller within the GPU that schedules workloads concurrently across multiple virtual machines, maintains unique GPU states for each VM, and uses aperture memory for storage and transmission, allowing for flexible and efficient access to GPU capabilities by managing workload execution and configuration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a software manager is used to manage GPU workloads from multiple VMs, then shared access to the GPU is enabled, but latency increases and system complexity increases
Solution Approach 1:
The patent replaces the software-based workload management approach with a hardware microcontroller integrated into the GPU. This microcontroller directly schedules and manages workloads from multiple VMs without requiring complex software translation layers, thereby reducing latency while maintaining shared access capability.
Solution Approach 2:
The microcontroller acts as an intermediary between the VMs and the GPU execution units. It receives workload commands from multiple VMs, translates them into appropriate execution formats, and schedules them for concurrent execution, thereby eliminating the need for complex software managers while reducing latency.
2Adaptability or versatility
If a software manager translates APIs between VMs and host OS, then shared access is enabled, but device complexity increases
Solution Approach 1:
The patent replaces complex software API translation layers with a hardware microcontroller that natively handles workload scheduling. This eliminates the need for software-based translation between VM APIs and host OS APIs, significantly reducing device complexity while maintaining versatile shared access.
Solution Approach 2:
The patent extracts the workload scheduling and translation functionality from the software stack and embeds it directly into the GPU as a microcontroller. This removes the complex software translation layer while preserving the ability to support multiple VMs and their respective APIs.
3Productivity
If pass-through is used to provide full GPU capability to a single VM, then performance is optimized, but versatility is limited to one-to-one mapping
Solution Approach 1:
The patent segments the GPU into multiple execution units or contexts that can be independently scheduled for different VMs. The microcontroller manages these segments, allowing multiple VMs to concurrently access different portions of the GPU or share execution units in a time-sliced manner, thereby maintaining high performance while enabling many-to-many mapping.
Solution Approach 2:
The patent introduces dynamic workload scheduling where the microcontroller can reallocate GPU resources based on current workload demands from different VMs. This dynamic allocation allows the system to optimize performance for each active VM while maintaining the ability to serve multiple VMs simultaneously, transitioning from static one-to-one mapping to dynamic many-to-many mapping.
Data Source
AI summary
In an embodiment, a system includes a graphics processing unit (GPU) that includes one or more GPU engines, and a microcontroller. The microcontroller is to assign a respective schedule slot for each of a plurality of virtual machines (VMs). When a particular VM is scheduled to access a first GPU engine, the particular VM has exclusive access to the first GPU engine. Other embodiments are described and claimed.