Virtual Machine Memory Latency Control Under Bandwidth Saturation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud service providers face significant performance variations in virtual machine instances due to uncontrolled memory access latency, leading to adverse conditions and heavy performance regression, especially when memory bandwidth saturation occurs, which existing workload characterization methods fail to address efficiently.

Innovation Solution

Implementing a dynamic resource controller with hardware circuitry to monitor and manage memory bandwidth usage, setting a memory bandwidth threshold, and applying throttling algorithms to ensure fair allocation among virtual machines, thereby maintaining maximum memory access latency as per service level agreements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If memory bandwidth is shared among virtual machines without control, then resource utilization is improved, but memory access latency varies significantly

Engineering Contradiction:

Improveresource utilizationVSAvoidmemory access latency consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent implements dynamic memory bandwidth allocation that adjusts allocation ratios in real-time based on current system conditions and service level agreement requirements, transforming static sharing into adaptive dynamic control to maintain latency consistency while preserving resource utilization

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system continuously monitors memory access latency and bandwidth usage, using feedback loops to detect when latency thresholds are approached and automatically adjusts allocation to prevent violations, ensuring reliable latency performance while maintaining high resource utilization

Inventive Principle:

Principle #23Feedback

2Ease of operation

If workload characterization methods are used to manage memory, then memory management is simplified, but they fail to address memory bandwidth saturation efficiently

Engineering Contradiction:

Improvememory management simplicityVSAvoidmemory bandwidth saturation handling

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent introduces a memory bandwidth allocation manager as an intermediary layer between the workload characterization system and physical memory, which translates simple workload descriptions into detailed bandwidth allocation decisions, maintaining ease of operation while efficiently handling saturation through intermediate control mechanisms

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If memory bandwidth is allocated without throttling, then virtual machine performance is improved, but service level agreements on maximum memory latency cannot be guaranteed

Engineering Contradiction:

Improvevirtual machine performanceVSAvoidservice level agreement compliance

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary allocation of memory bandwidth based on service level agreement requirements before workloads can cause saturation, pre-establishing allocation ratios that guarantee maximum latency compliance while allowing virtual machines to achieve high performance within their allocated budgets

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250251961A1Enforcement of maximum memory access latency for virtual machine instances

Publication Date: 2025.08.07 INTEL CORP

AI summary

Enforcement of maximum memory access latency for virtual machine instances is described. An example of a computer-readable storage medium includes instructions to implement operation of multiple virtual machines (VMs) in a cloud computing system; monitor operation of the VMs in processing a set of active workloads, including monitoring of memory access latency for the VMs using a dynamic resource controller, the dynamic resource controller comprising hardware circuitry to monitor memory bandwidth usage; and, upon detecting memory access in the cloud computing system reaching a memory bandwidth setpoint, implementing memory access throttling of one or more of the set of active workloads for the plurality of VMs, and allocating memory bandwidth to the active workloads according to a distribution algorithm.