Resource Bandwidth Enforcement Through Credit-Based Memory Throttling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The 'noisy neighbor' problem in cloud computing environments causes significant performance variability due to memory bandwidth contention among virtual machines, leading to degradation in cloud service provider perception and efficiency.
Innovation Solution
Implementing a system with resource bandwidth enforcement (RBE) through credit-based request throttling and programmable delay elements to control memory bandwidth usage, using Resource Monitoring IDs (RMIDs) and Model Specific Registers (MSRs) to manage and throttle aggressive applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory bandwidth is shared among multiple virtual machines, then resource utilization and scalability are improved, but performance variability and contention increase
Solution Approach 1:
The patent segments memory bandwidth allocation by introducing separate credit counters and delay elements for different virtual machines. Each VM gets dedicated credit allocation that is independently tracked and enforced, preventing one VM from monopolizing memory bandwidth while maintaining shared resource access. This segmentation resolves the contradiction by enabling fair sharing without performance variability.
Solution Approach 2:
The patent implements feedback mechanisms through credit counters that monitor and track memory bandwidth usage by each virtual machine. When a VM exhausts its allocated credits, the system provides feedback by blocking further memory requests until credits are replenished. This feedback loop ensures that no single VM can exceed its fair share, maintaining performance stability while utilizing shared memory bandwidth.
2Device complexity
If memory bandwidth is allocated without enforcement, then system simplicity is maintained, but noisy neighbor interference and performance degradation occur
Solution Approach 1:
The patent introduces credit counters and delay elements as intermediary components between virtual machines and the memory subsystem. These intermediaries enforce bandwidth limits by tracking credit consumption and introducing controlled delays, preventing noisy neighbor interference without requiring complex redesign of the entire memory architecture. The intermediaries provide a manageable layer of control that maintains relative system simplicity.
Solution Approach 2:
The patent changes the operational parameters of memory access by introducing programmable delay elements that can dynamically adjust timing characteristics. By modifying the time parameter of memory requests through controlled delays, the system prevents bandwidth monopolization while maintaining flexible and manageable control mechanisms. This parameter-based approach avoids complex structural changes.
3Measurement precision
If credit-based throttling is implemented, then memory bandwidth control precision is improved, but device complexity increases
Solution Approach 1:
The patent extracts the bandwidth control function into separate, dedicated components (credit counters and delay elements) that operate independently from the main memory subsystem. This extraction allows for precise bandwidth measurement and control through specialized hardware that doesn't complicate the core memory architecture. The control precision is achieved through dedicated counting and timing circuits rather than complex integrated control logic.
Data Source
AI summary
Systems, methods, and apparatuses for resource bandwidth monitoring and control are described. For example, in some embodiments, an apparatus comprising a requestor device to send a credit based request, a receiver device to receive and consume the credit based request, and a delay element in a return path between the requestor and receiver devices, the delay element to delay a credit based response from the receiver to the requestor are detailed.