Dynamic vGPU Allocation for Disaggregated GPU Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face inefficiencies in GPU resource allocation due to over-allocation and inability to adapt to changing resource utilization during process execution, leading to underutilization and suboptimal performance in GPU-intensive workloads like machine learning and AI applications.
Innovation Solution
A method for dynamically optimizing and modifying GPU allocation by using virtual graphical processing units (vGPUs) that are disaggregated and linked to physical GPUs, allowing seamless sharing and reallocation based on workload demands, with a scheduler component managing resource migrations and allocations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional systems over-allocate virtualized resources to minimize the likelihood of process needing more resources, then resource availability to processes is improved, but resource utilization efficiency deteriorates
Solution Approach 1:
The system dynamically adjusts virtual GPU allocations based on real-time workload demands. The resource manager continuously monitors process resource usage and reallocates vGPUs accordingly, transitioning from static over-allocation to dynamic adaptation. This resolves the contradiction by maintaining reliability through available resources while improving utilization efficiency through demand-based allocation.
Solution Approach 2:
The system implements feedback mechanisms where the resource manager monitors process resource utilization and adjusts allocations based on this feedback. The system observes actual resource usage patterns and modifies vGPU allocations to match demand, preventing both over-allocation waste and under-allocation shortages. This feedback loop resolves the contradiction between maintaining availability and improving efficiency.
2Stability of the object's composition
If conventional systems do not modify allocated virtual resource amounts during process execution, then allocation stability is improved, but adaptability to changing resource utilization deteriorates
Solution Approach 1:
The system enables dynamic modification of virtual GPU allocations during process execution. The resource manager can add or remove vGPUs from running processes based on monitored workload demands, breaking the conventional static allocation model. This dynamic capability simultaneously maintains stability through controlled modifications and achieves adaptability to changing resource needs.
Solution Approach 2:
The system changes the allocation parameters (number of vGPUs) during process execution based on workload conditions. Rather than fixing allocations at process start, the system continuously adjusts the quantity of virtual resources allocated, allowing adaptation to varying computational demands while maintaining system stability through managed transitions.
3Productivity
If systems allocate dedicated GPUs to processes, then process performance is improved, but overall GPU utilization across multiple processes deteriorates
Solution Approach 1:
The system merges multiple physical GPUs into a shared pool that serves multiple processes through virtualization. Instead of dedicating entire physical GPUs to single processes, the resource manager allocates virtual GPU slices from the combined pool to different processes, enabling both processes to benefit from aggregated GPU resources while maintaining isolation and dedicated performance characteristics.
Solution Approach 2:
The system makes GPU resources universal by creating vGPUs that can be allocated to any process needing GPU acceleration. A single physical GPU can serve multiple processes through virtualization, making the resource multi-functional. The resource manager dynamically assigns vGPUs to different processes based on demand, allowing GPU resources to fulfill multiple roles and serve diverse workloads efficiently.
Data Source
AI summary
A method for dynamically optimizing and modifying allocation of at least one disaggregated graphical processing unit (GPU) to at least one process via at least one virtual graphical processing units (vGPU) includes receiving, by a resource manager, from a runtime component executing on a first computing device, a request for allocation, to a process executing on the first computing device, of access to a GPU in the first computing device. The method includes allocating, by the resource manager, to the process, access to at least one vGPU associated with a second GPU on a second computing device. The method includes transmitting, by the resource manager, to the runtime component, an identification of the at least one vGPU identifying the at least one vGPU as located on the first computing device. The method includes instantiating, by the runtime component, the process with access to the at least one vGPU.