Hybrid vGPU Co-Scheduling with Shared Workload Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vGPU scheduling technologies face challenges in achieving flexible programmability and optimal GPU utilization, leading to reduced scalability and increased CPU usage peaks, which affect system responsiveness and compliance with industry standards in environments like cloud service providers and in-vehicle infotainment systems.
Innovation Solution
A hybrid vGPU scheduling approach that combines hardware and software scheduling schemes, allowing workload queues and scheduling data to be shared between CPU and GPU, with GPU executing scheduling policies when idle and CPU handling new workloads, enabling flexible user-defined policies and improved GPU utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional vGPU scheduling is used, then system simplicity is maintained, but GPU utilization is suboptimal and CPU usage peaks increase
Solution Approach 1:
The scheduling system is segmented into two independent schedulers: a CPU-side scheduler that handles workload submission and a GPU-side scheduler that handles execution. This segmentation allows each scheduler to specialize in its function, improving GPU utilization without overwhelming the CPU, while maintaining manageable system complexity through clear separation of concerns.
Solution Approach 2:
A hybrid scheduling mechanism acts as an intermediary between the CPU and GPU, coordinating their operations. The CPU scheduler submits workloads to a queue, and the GPU scheduler picks up and executes them, with synchronization mechanisms ensuring proper coordination. This intermediary layer optimizes resource utilization while distributing the scheduling burden.
2Reliability
If CPU-only scheduling is used, then implementation simplicity is maintained, but system responsiveness decreases due to increased CPU load
Solution Approach 1:
The GPU is empowered to schedule its own workloads through a dedicated GPU-side scheduler. This self-service approach allows the GPU to autonomously manage its execution queue, reducing the scheduling burden on the CPU and improving system responsiveness without requiring continuous CPU intervention.
Solution Approach 2:
The CPU scheduler performs preliminary actions by submitting workloads to a shared queue in advance, without immediately executing them. The GPU scheduler then picks up these pre-submitted workloads and executes them when resources are available. This preliminary action separates workload submission from execution, reducing CPU load and improving responsiveness.
3Adaptability or versatility
If flexible user-defined scheduling policies are implemented, then adaptability improves, but implementation complexity increases
Solution Approach 1:
The scheduling system implements dynamic, configurable parameters that can be adjusted at runtime. User-defined policies are represented as configurable structures with parameters such as priority levels, weight factors, and scheduling algorithms that can be modified without changing the underlying system architecture. This dynamic approach provides flexibility while maintaining implementation simplicity through a unified configuration interface.
Data Source
AI summary
An embodiment of a semiconductor package apparatus may include technology to manage one or more virtual graphic processor units, and co-schedule the one or more virtual graphic processor units based on both general processor instructions and graphics processor instructions. Other embodiments are disclosed and claimed.