Best Effort QoS Scheduling for GPU Command Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU architectures do not support preemption of execution, leading to challenges in scheduling multiple GPU workloads in virtual machine environments, resulting in delayed scheduling decisions and impact on quality of service (QoS) for video/audio streams and 3D/GPGPU processing.
Innovation Solution
Implementing a best effort quality of service (BEQOS) scheduling approach that estimates the execution time of GPU commands and uses fine-grain controlled delivery to ensure timely completion within allocated time slices, balancing accuracy and performance by reserving a margin for execution and employing temporary co-delivering to manage dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If the GPU executes commands without preemption support, then the execution continuity and hardware simplicity are improved, but the scheduling flexibility and QoS guarantee deteriorate
Solution Approach 1:
The system performs preliminary estimation of GPU command execution time before scheduling. The scheduler estimates the time required to execute a batch of GPU commands and uses this prediction to make scheduling decisions in advance, allowing the GPU to execute commands continuously while still enabling flexible multi-tenant scheduling through predicted time slices.
2Device complexity
If the OS scheduler waits for GPU execution completion before making scheduling decisions, then the scheduling simplicity is improved, but the scheduling response time and QoS deteriorate
Solution Approach 1:
The scheduler performs preliminary estimation of GPU command execution time before actually executing the commands. By predicting how long the GPU will take to process a batch of commands, the scheduler can make timely scheduling decisions without waiting for completion, thus reducing scheduling response time while maintaining relative simplicity.
Solution Approach 2:
The system uses feedback from actual GPU execution performance to refine future time estimates. By monitoring the actual time taken to execute command batches and comparing it with predictions, the scheduler continuously improves its estimation accuracy, enabling better scheduling decisions without requiring complex real-time monitoring during execution.
3Productivity
If the GPU executes large batches of commands without interruption, then the processing throughput is improved, but the QoS guarantee for time-sensitive workloads deteriorates
Solution Approach 1:
The scheduler estimates the execution time of command batches in advance and uses this information to allocate time slices to different virtual machines. By predicting execution time before scheduling, the system can guarantee that each VM receives its allocated time slice while still allowing efficient batch processing, thus maintaining both throughput and QoS reliability.
Solution Approach 2:
The system schedules command batches that may slightly exceed the allocated time slice based on estimated execution time, with the understanding that the scheduler will make adjustments. This allows the GPU to process commands in larger, more efficient batches while still maintaining QoS guarantees through the scheduler's ability to manage time slice allocations and make corrective scheduling decisions.
Data Source
AI summary
An apparatus and method for best effort quality of service scheduling in a graphics processing architecture. For example, one embodiment of an apparatus comprises: a graphics processing unit (GPU) to perform graphics processing operations for a plurality of guests; a plurality of buffers to store one or more graphics commands associated with each guest to be executed by the GPU; and a scheduler to evaluate commands in the buffers of a first guest to estimate a cost of executing the commands, the scheduler to select all or a subset of the buffers of the first guest for execution on the GPU based on a determination that the selected buffers can be executed by the GPU within a remaining time slice allocated to the first guest.