Hypervisor vCPU Throttling for VM Memory Starvation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing, shared system memory among virtual machines can lead to memory access starvation when one virtual machine increases its memory access rate, undermining the principle of isolation between virtual machines.
Innovation Solution
A software solution within a hypervisor dynamically throttles virtual CPUs assigned to a virtual machine by allocating timeslices and forcing the vCPU to be idle when memory access rates exceed a threshold, ensuring fair access to shared memory resources across CPU families without requiring specialized server-class CPU capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If system memory is shared by multiple VMs to improve resource utilization, then resource efficiency is improved, but memory access starvation occurs when one VM increases its memory access rate
Solution Approach 1:
The patent implements a feedback mechanism by monitoring memory access rates of individual VMs and dynamically adjusting their vCPU memory access bandwidth accordingly. When a VM's memory access rate exceeds a threshold, the system automatically throttles its vCPUs to prevent memory access starvation of other VMs, thereby maintaining VM isolation while allowing efficient resource sharing.
Solution Approach 2:
The patent changes the parameter of memory access bandwidth allocation dynamically based on VM behavior. Instead of static allocation, the system adjusts memory access bandwidth for each vCPU in real-time based on monitored memory access patterns, allowing the system to adapt to changing workload conditions while maintaining isolation guarantees.
2Reliability
If hardware-based memory bandwidth allocation at CPU core level is used, then memory access control is improved, but specialized server-class CPU capabilities are required
Solution Approach 1:
The patent introduces a software intermediary layer (the hypervisor with the scheduler) that mediates between the physical CPU and VMs to enforce memory bandwidth allocation. This intermediary implements memory access rate monitoring and vCPU throttling logic in software, eliminating the need for specialized hardware-based memory bandwidth allocation capabilities in the CPU while still providing reliable memory access control.
Solution Approach 2:
The patent replaces the mechanical/hardware-based memory bandwidth allocation system with a software-based solution. Instead of relying on hardware counters and throttling mechanisms at the CPU level, the system uses software monitoring of memory access rates and software-controlled vCPU scheduling to achieve the same memory bandwidth management goals, thereby improving CPU compatibility.
3Measurement precision
If vCPUs are configured to execute tasks for a single vCPU per CPU core, then correlation between HW counters and vCPU behavior is established, but system configuration is overly restrictive
Solution Approach 1:
The patent enables self-service by allowing multiple vCPUs to share a single physical CPU core while the software layer automatically tracks and attributes memory access behavior to the appropriate vCPUs. The system uses data structures to maintain mappings between vCPUs and their memory access patterns, enabling precise monitoring without requiring restrictive one-vCPU-per-core configuration.
Data Source
AI summary
Systems and methods are disclosed for throttling memory bandwidth accessed by virtual machines (VMs). A technique for dynamically throttling the virtual computer processing units (vCPUs) assigned to a VM (tenant) controls the memory access rate of the VM. When the memory is shared by multiple VMs in a cloud-computing environment, one VM increasing its memory access rate may cause another VM to suffer memory access starvation. This behavior violates the principle of VM isolation in cloud computing. In contrast to conventional systems, a software solution for dynamically throttling the vCPUs may be implemented within a hypervisor and is therefore portable across CPU families and doesn't require specialized server-class CPU capabilities or limit the system configuration.