NUMA-Aware VM Placement in Heterogeneous Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software-defined datacenters (SDDCs), the placement of virtual machines (VMs) on hosts with non-uniform memory access (NUMA) architectures is challenging due to misalignment of virtual and physical CPU and memory topologies, leading to performance issues, especially in heterogeneous clusters with varying CPU core counts and memory capacities, where current cluster services are not NUMA-aware.
Innovation Solution
A NUMA-aware clustering service is introduced within the virtualization management server to compare virtual CPU and memory configurations with physical NUMA topologies, selecting hosts that maximize locality for VM placement and providing a list of optimal hosts to the distributed resource scheduler (DRS) for placement decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If current cluster services (HA, DRS) are used for VM placement, then ease of operation is improved, but performance deteriorates due to NUMA topology misalignment
Solution Approach 1:
The system automatically performs NUMA-aware placement decisions without requiring manual intervention from infrastructure experts. The NUMA-aware clustering service autonomously compares VM configurations with host NUMA topologies and makes optimal placement decisions, enabling self-service deployment while maintaining performance optimization.
Solution Approach 2:
The system changes the placement decision parameters by considering NUMA topology characteristics (socket count, memory capacity, memory bandwidth) alongside traditional placement factors. This parameter expansion allows the DRS to make informed decisions that align virtual topology with physical NUMA architecture, resolving the performance issue.
2Adaptability or versatility
If heterogeneous host configurations are used in clusters, then adaptability is improved, but device complexity increases due to varying CPU core counts and memory capacities
Solution Approach 1:
The system applies local quality by tailoring placement decisions to the specific NUMA characteristics of each host. Instead of treating all hosts uniformly, the NUMA-aware clustering service evaluates individual host properties (socket count, memory capacity per socket, memory bandwidth) and makes localized optimization decisions for each VM placement scenario.
Solution Approach 2:
The system performs preliminary analysis of VM configurations and host NUMA topologies before making placement decisions. The NUMA-aware clustering service pre-compares VM vCPU and memory requirements against host capabilities to identify suitable hosts in advance, simplifying the overall placement process despite hardware heterogeneity.
3Ease of operation
If virtual CPU and memory topology are misaligned with physical NUMA topology, then ease of operation is improved, but performance deteriorates by 60-70%
Solution Approach 1:
The system implements feedback by continuously monitoring and comparing VM placement configurations against actual host NUMA topologies. The NUMA-aware clustering service uses this feedback to adjust placement decisions, ensuring that virtual topology aligns with physical architecture to prevent performance degradation.
Solution Approach 2:
The system replaces manual topology alignment mechanisms with automated software-based comparison and decision-making. The NUMA-aware clustering service uses software algorithms to analyze VM configurations and host NUMA characteristics, substituting the need for manual infrastructure expert intervention with automated intelligent decision-making.
Data Source
AI summary
An example method of placing a virtual machine (VM) in a cluster of hosts is described. Each of the hosts having a hypervisor managed by a virtualization management server for the cluster, the hosts separated into a plurality of nonuniform memory access (NUMA) domains. The method including: comparing a virtual central processing unit (vCPU) and memory configuration of the VM with physical NUMA topologies of the hosts; selecting a set of the hosts spanning at least one of the NUMA domains, each host in the set of hosts having a physical NUMA topology that maximizes locality for vCPU and memory resources of the VM as specified in the vCPU and memory configuration; and providing the set of hosts to a distributed resource scheduler (DRS) executing in the virtualization management server, the DRS configured to place the VM in a host selected from the set of hosts.