VM Assignment in Virtual GPU Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual machine (VM) placement decisions in virtual GPU-enabled systems are often sub-optimal due to insufficient information about resource demands and availability, leading to host overload, network saturation, and inefficient resource utilization.
Innovation Solution
A scheduler that uses machine learning techniques, such as simulated annealing, bin packing, and particle swarm optimization to optimize VM assignment based on a cost function that considers GPU utilization and VM execution/wait times, dynamically assigning VMs to GPUs and creating new run queues to match resource profiles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional scheduling algorithms are used for VM placement, then the system is simple to implement, but resource utilization is inefficient and hosts become overloaded
Solution Approach 1:
The patent introduces a scheduler as an intermediary component that sits between VMs and hosts, making intelligent placement decisions based on resource demands and availability. This mediator analyzes multiple factors including GPU utilization, network bandwidth, and host capacity to optimize VM placement without requiring direct complex interactions between VMs and hardware resources.
Solution Approach 2:
The scheduling system implements feedback mechanisms by continuously monitoring resource utilization metrics, VM performance data, and host status. This feedback loop enables the scheduler to dynamically adjust placement decisions, relocate VMs from overloaded hosts to underutilized ones, and optimize resource allocation in response to changing system conditions.
2Reliability
If VMs are relocated frequently to balance load, then host overload is reduced, but network saturation and link overloading occur
Solution Approach 1:
The scheduler applies partial relocation strategies by selectively moving only certain VMs to optimal hosts rather than frequently relocating all VMs. It balances load by making targeted placement decisions that achieve host load equilibrium while minimizing unnecessary network traffic associated with excessive VM migrations.
Solution Approach 2:
The system performs preliminary analysis of resource demands and host capacity before making placement decisions. By pre-evaluating the impact of potential VM relocations on network bandwidth and host capacity, the scheduler can choose placement strategies that balance load without causing network saturation or link overloading.
3Productivity
If placement decisions are made with limited information, then the decision-making process is fast, but placement is sub-optimal leading to host overload
Solution Approach 1:
The scheduler performs preliminary gathering and analysis of resource demand information from VMs and resource availability data from hosts before making placement decisions. This advance information collection includes GPU utilization metrics, network bandwidth requirements, and host capacity status, enabling optimized placement decisions without causing delays in the scheduling process.
Data Source
AI summary
Disclosed are aspects of task assignment for systems that include graphics processing units (GPUs) that are virtual GPU (vGPU) enabled. In some examples, an algorithm is determined based on predetermined virtual machine assignment algorithms. The algorithm optimizes for a predetermined cost function. A virtual machine is queued in an arrival queue for assignment. A graphics configuration of a system is determined. The graphics configuration specifies a number of graphics processing units (GPUs) in the system. The system includes a vGPU enabled GPU. The algorithm is selected based on a correlation between the algorithm and the graphics configuration of the system. The virtual machine is assigned to a run queue based on the selected algorithm.