Integrated Memory Pooling With Direct Swap Cache Cleanup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face inefficiencies in memory resource allocation due to uneven usage by compute entities, leading to suboptimal allocation and utilization of memory resources across multiple tenants in cloud environments.
Innovation Solution
Implementing a system with integrated memory pooling and direct swap caching, where local memory (near memory) is used for high-performance tasks and pooled memory (far memory) is used for less critical operations, utilizing a data-mover offload engine to manage memory content swapping and cleanup operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory resources are allocated to multiple compute entities using a host operating system, then multiple users or tenants can share computing systems, but memory resources are not allocated efficiently due to uneven usage
Solution Approach 1:
The pooled memory system serves multiple compute entities simultaneously, allowing a single memory resource to be shared across multiple tenants. The memory pool acts as a universal resource that can be dynamically allocated to any compute entity that needs it, replacing the dedicated per-tenant memory allocation model.
Solution Approach 2:
The system dynamically allocates and deallocates memory from the pool based on real-time usage patterns of compute entities. Memory that is not currently being used by one compute entity can be automatically made available to another compute entity that requires it, optimizing overall utilization.
2Speed
If local memory is used for high-performance tasks, then system performance is improved, but memory costs increase
Solution Approach 1:
The system provides different memory access characteristics for different workloads. Frequently accessed data and performance-critical operations use fast local memory, while less frequently accessed data uses slower pooled memory. This creates a heterogeneous memory system where each location has properties optimized for its specific use case.
Solution Approach 2:
The pooled memory system is nested within the local memory system. Local memory acts as a cache layer in front of the pooled memory, with the hierarchy organized so that fast local memory is embedded within the slower pooled memory infrastructure, creating a nested memory architecture.
3Quantity of substance
If pooled memory is used for cost-effective storage, then memory costs are reduced, but access speed decreases
Solution Approach 1:
Local memory acts as an intermediary between the compute entities and the pooled memory system. It buffers and caches data from the pooled memory, so that frequently accessed pooled memory data can be served from the faster local memory, effectively mediating the speed difference between the two memory types.
Solution Approach 2:
Data is pre-loaded into local memory from pooled memory before it is needed for processing. The system anticipates memory access patterns and proactively moves data from the pooled memory to local memory caches, so that when actual access occurs, the data is already in the faster memory location.
4Productivity
If memory content is swapped between local and pooled memory, then memory utilization is optimized, but processing overhead increases
Solution Approach 1:
The memory management system operates autonomously to swap content between local and pooled memory based on usage patterns. The system self-regulates by automatically identifying which data to move, when to move it, and where to place it, without requiring manual intervention or complex external control mechanisms.
Solution Approach 2:
The system uses feedback from memory access patterns to drive swap decisions. By monitoring which data is frequently accessed and which is rarely used, the system dynamically adjusts its swap strategy, moving data that will be needed soon into local memory and evicting data that is unlikely to be accessed again to pooled memory.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods related to integrated memory pooling and direct swap caching are described. A system includes a compute node comprising a local memory and a pooled memory. The system further includes a host operating system (OS) having initial access to: (1) a first swappable range of memory addresses associated with the local memory and a non-swappable range of memory addresses associated with the local memory, and (2) a second swappable range of memory addresses associated with the pooled memory. The system further includes a data-mover offload engine configured to perform a cleanup operation, including: (1) restore a state of any memory content swapped-out from a memory location within the first swappable range of memory addresses to the pooled memory, and (2) move from the local memory any memory content swapped-in from a memory location within the second swappable range of memory addresses back out to the pooled memory.