Fabric Memory Interleaving for Pooled Bandwidth Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud-based deployments, memory pooling architectures face challenges with memory capacity and bandwidth bottlenecks during seasonal or temporal events, such as end-of-quarter spikes in memory-intensive queries, leading to reduced performance and resource overburdening.
Innovation Solution
Implementing network-aware, transparent memory interleaving methods through a switch that disaggregates memory requests across multiple pools, using hints for bandwidth requirements and dynamically allocating memory from interleaved or non-interleaved spaces, with the switch managing the allocation and aggregation of responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory pooling is used to share memory resources across multiple servers, then memory utilization efficiency is improved, but memory bandwidth and capacity bottlenecks occur during temporal spikes
Solution Approach 1:
The patent segments memory access requests into multiple parallel paths through the network fabric. Instead of handling all memory requests through a single pooled memory server, the system divides requests across multiple servers in the pool, each handling a portion of the total memory capacity. This segmentation prevents bandwidth bottlenecks during temporal spikes by distributing the load across multiple independent memory servers, thereby maintaining both high utilization efficiency and sufficient bandwidth capacity.
2Productivity
If dedicated hardware is used for on-premises in-memory databases, then performance is improved, but adaptability to cloud-based multi-tenant environments is reduced
Solution Approach 1:
The patent creates a universal memory pooling architecture that can serve multiple functions and deployment scenarios. The same pooled memory infrastructure supports on-premises single-tenant deployments with dedicated resources while also accommodating cloud-based multi-tenant environments with shared resources. The system adapts to different performance requirements by dynamically allocating memory capacity from the pool, allowing high-performance dedicated access when needed and efficient shared access when appropriate, thus achieving both performance and adaptability.
3Device complexity
If memory requests are handled through a single pooled memory server, then architecture simplicity is improved, but load balancing during temporal spikes is reduced
Solution Approach 1:
The patent merges multiple memory servers into a unified pooled memory infrastructure while maintaining logical simplicity. The system presents a single pooled memory namespace to applications but physically distributes memory capacity across multiple servers. This merging approach combines the architectural simplicity of a single memory server with the load balancing capabilities of multiple servers, as the pooled memory controller automatically distributes requests across the merged resource pool, achieving both simplicity and effective load balancing during temporal spikes.
Data Source
AI summary
Methods and apparatus for advanced interleaving techniques for fabric based pooling architectures. The method implemented in an environment including a switch connected to host servers and to pooled memory nodes or memory servers hosting memory pools. Memory is interleaved across the memory pools using interleaving units, with the interleaved memory mapped into a global memory address space. Applications running on the host servers are enabled to access data stored in the memory pools via memory read and write requests issued by the applications specifying address endpoints within the global memory space. The switch generates multi-cast or multiple unicast messages associated with the memory read and write requests that are sent to the pooled memory nodes or memory servers. For memory reads, the data returned from multiple memory pools is aggregated at the switch and returned to the application using one or more packets as a single response.