NUCA-Aware VM Scheduling via LLC Group Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hypervisors are not NUCA aware, leading to sub-optimal virtual machine performance due to ineffective leveraging of LLC locality in systems with non-uniform cache access processors, as they incorrectly assume or ignore LLC boundaries, resulting in varying memory access times and CPU contention.
Innovation Solution
Implementing a NUCA-aware scheduler that partitions virtual CPUs into LLC groups matching the LLC domain size and places them to maximize LLC locality, using a 'group leader' and 'group follower' approach to balance compute load and cache contention, ensuring virtual CPUs remain within the same LLC domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing hypervisors schedule virtual CPUs without NUCA awareness, then the scheduling process is simple and compatible with traditional NUMA systems, but LLC locality is not effectively leveraged resulting in sub-optimal VM performance
Solution Approach 1:
The patent segments virtual CPUs into LLC groups, where each group is assigned to a specific LLC domain. This segmentation allows the scheduler to manage VMs at the LLC domain level rather than treating all cores uniformly, thereby improving LLC locality and VM performance while maintaining manageable scheduling complexity through structured group management.
Solution Approach 2:
The patent applies local quality by assigning different scheduling policies to different LLC domains based on their specific characteristics. Each LLC domain can have optimized placement strategies tailored to its capacity and workload patterns, allowing the system to leverage local cache properties for improved performance without requiring complete redesign of the scheduling mechanism.
2Speed
If virtual CPUs are placed without considering LLC boundaries, then placement is simpler and more flexible, but cache access times increase due to remote LLC access
Solution Approach 1:
The patent performs preliminary action by pre-assigning virtual CPUs to specific LLC domains during VM placement, before actual workload execution begins. This advance assignment ensures that virtual CPUs are positioned in optimal locations regarding LLC locality, reducing cache access times during runtime without requiring complex dynamic remapping mechanisms.
Solution Approach 2:
The patent introduces LLC domain assignment as an intermediary layer between the hypervisor and physical CPU placement. This intermediary structure simplifies the placement problem by adding one level of abstraction (LLC domain assignment) that mediates between high-level VM scheduling requirements and low-level physical core placement constraints, making the overall system more manageable.
3Productivity
If LLC domains are overpacked with too many virtual CPUs, then more VMs can be consolidated on fewer domains, but compute capacity is exceeded and performance degrades
Solution Approach 1:
The patent applies partial action by filling LLC domains to an optimal capacity level rather than maximizing consolidation at any cost. It recognizes that exceeding a certain threshold of virtual CPUs per LLC domain leads to diminishing returns and performance degradation, so it deliberately limits placement to maintain compute capacity headroom and ensure reliable performance.
Solution Approach 2:
The patent implements feedback mechanisms that monitor compute capacity utilization in each LLC domain and adjust placement decisions accordingly. When an LLC domain approaches its optimal capacity threshold, the scheduler receives feedback and redirects subsequent virtual CPU placements to underutilized domains, thereby maintaining balanced load distribution and preventing performance degradation from overpacking.
Data Source
AI summary
Techniques for optimizing virtual machine (VM) scheduling on a non-uniform cache access (NUCA) system are provided. In one set of embodiments, a hypervisor of the NUCA system can partition the virtual CPUs of each VM running on the system into logical constructs referred to as last level cache (LLC) groups, where each LLC group is sized to match (or at least not exceed) the LLC domain size of the system. The hypervisor can then place/load balance the virtual CPUs of each VM on the system's cores in a manner that attempts to keep virtual CPUs which are part of the same LLC group within the same LLC domain, subject to various factors such as compute load, cache contention, and so on.