Interference-Aware vGPU Scheduling for Cloud Hosts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scheduling services for virtual GPU-enabled systems often result in sub-optimal workload placements due to insufficient information about resource demands and availability, leading to unbalanced hosts, network saturation, and inefficient resource utilization, particularly because they do not account for interference between workloads sharing GPUs.
Innovation Solution
An interference-aware scheduling service that profiles workloads to collect baseline parameters, measures interference between co-located workloads, and uses machine learning models to predict and minimize interference by optimally assigning workloads to GPUs with the least interference, thereby improving resource allocation and performance in cloud and data center environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing scheduling services place virtual machines on hosts based on limited resource information, then placement decisions can be made quickly, but the placement is sub-optimal leading to unbalanced hosts and inefficient resource utilization
Solution Approach 1:
The scheduling service performs preliminary actions by collecting comprehensive resource demand information from workload profiles and host capability data before making placement decisions. This advance information gathering enables more accurate and optimal placement decisions, resolving the contradiction between quick decision-making and optimal resource utilization.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring resource utilization metrics, workload performance data, and host status. This feedback loop allows the scheduling service to learn from past placements and adjust future decisions, improving resource utilization efficiency while maintaining manageable complexity through iterative optimization.
2Productivity
If virtual machines are placed without considering workload interference on shared GPUs, then placement is simpler and faster, but interference causes performance degradation and network saturation
Solution Approach 1:
The scheduling service applies preliminary anti-action by proactively identifying and preventing workload placements that would cause interference on shared GPUs. By analyzing workload profiles and predicting potential interference scenarios before placement, the system avoids performance degradation and network saturation caused by incompatible workload co-location.
Solution Approach 2:
The system introduces an intermediary analysis layer that evaluates workload compatibility and interference potential between co-located virtual machines. This intermediary assessment mechanism mediates placement decisions to prevent harmful interference while maintaining efficient resource utilization, resolving the contradiction between placement simplicity and performance protection.
3Measurement precision
If comprehensive resource demand and availability information is collected for all workloads and hosts, then optimal placement decisions can be made, but the information gathering process becomes more complex and time-consuming
Solution Approach 1:
The system performs preliminary information gathering by collecting and storing workload profile data and host capability information in advance of placement decisions. This pre-collected comprehensive data enables accurate placement decisions to be made quickly without real-time information gathering delays, resolving the contradiction between measurement precision and decision time.
Solution Approach 2:
The scheduling service implements dynamic information collection strategies, gathering detailed resource information only when needed for placement decisions rather than continuously monitoring all parameters. This dynamic approach maintains high measurement precision for critical placement factors while reducing overall information gathering time and system complexity.
Data Source
AI summary
Disclosed are aspects of interference-aware virtual machine assignment for systems that include graphics processing units (GPUs) that are virtual GPU (vGPU) enabled. In some examples, an interference function is used to predict interference for assignment of a workload to a graphics processing unit (GPU). The interference function outputs a predicted interference to place the workload on the GPU. The workload is assigned to the GPU based on a comparison of the predicted interference to a plurality of predicted interferences for the workload on various GPUs.