Direct Swap Caching with Noisy Neighbor Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-tenant cloud systems, conflicting cache lines in near memory can lead to reduced memory bandwidth and capacity for one tenant due to shared resources, causing noisy neighbor issues where one tenant's activities impact another.
Innovation Solution
A system and method that provisions compute nodes with both near memory and far memory, allowing the host operating system to allocate memory in a granular fashion using swappable and non-swappable address ranges, with conflict sets split between near and far memory to prevent conflicts, and dynamically adjust address ranges without reprogramming hardware registers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple tenants share near memory resources, then resource utilization improves, but memory bandwidth and capacity for individual tenants deteriorates due to noisy neighbor issues
Solution Approach 1:
The patent segments the address space into conflict sets that are evenly distributed between near memory and far memory. Each tenant is allocated specific conflict sets, ensuring that conflicting cache lines are separated across different memory types. This segmentation prevents noisy neighbor issues while maintaining high resource utilization by allowing multiple tenants to share the overall memory system without interfering with each other's performance.
2Ease of operation
If memory is allocated in coarse granularity, then allocation simplicity improves, but memory isolation and prevention of noisy neighbor issues deteriorates
Solution Approach 1:
The patent introduces dynamic address range assignment that allows the system to adaptively allocate memory addresses to tenants based on their needs. The host OS can dynamically adjust which conflict sets are assigned to which tenants, providing fine-grained isolation when needed while maintaining allocation simplicity through automated management. This dynamic approach enables the system to prevent noisy neighbor issues without requiring complex manual configuration.
3Stability of the object's composition
If address ranges are fixed at hardware level, then system stability improves, but runtime flexibility in memory allocation deteriorates
Solution Approach 1:
The patent adds a software layer (host OS) that manages address space allocation in the virtualization layer, separating hardware register configuration from runtime address allocation. The hardware registers maintain fixed, stable address ranges for reliability, while the host OS provides runtime flexibility by dynamically assigning specific conflict sets to tenants. This dimensional separation allows both system stability and runtime adaptability to coexist.
Data Source
AI summary
Systems and methods related to direct swap caching with noisy neighbor mitigation and dynamic address range assignment are described. A system includes a host operating system (OS), configured to support a first set of tenants associated with a compute node, where the host OS has access to: (1) a first swappable range of memory addresses associated with a near memory and (2) a second swappable range of memory addresses associated with a far memory. The host OS is configured to allocate memory in a granular fashion such that each allocation of memory to a tenant includes memory addresses corresponding to a conflict set having a conflict set size. The conflict set includes a first conflicting region associated with the first swappable range of memory addresses with the near memory and a second conflicting region associated with the second swappable range of memory addresses with the far memory.