Multi-level caching for volatile and persistent memory allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory management systems face challenges in efficiently allocating memory between volatile DRAM and persistent memory (PMEM) in computer systems, particularly when PMEM is used in volatile mode, as it may not be as fast as DRAM and can introduce security issues and performance bottlenecks, especially in applications like machine learning where memory buffers need to be optimized.
Innovation Solution
A memory allocation capture library is implemented to intercept memory calls from applications and apply an allocation policy, determining whether to allocate memory from DRAM or PMEM based on capture criteria, using multi-level caching and distributed memory objects to optimize resource usage and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory is allocated from DRAM to ensure fast access speed, then access latency is reduced, but DRAM capacity is limited and cannot handle large working data sets
Solution Approach 1:
The memory system is segmented into multiple tiers: fast DRAM for frequently accessed data and slower but larger capacity PMEM for less frequently accessed data. The memory manager divides the address space and dynamically assigns pages to appropriate tiers based on access patterns, resolving the contradiction between speed and capacity by spatial segmentation of memory resources.
Solution Approach 2:
The system transitions from a single-dimension memory hierarchy to a multi-dimensional memory management approach, incorporating both local and remote memory tiers. By adding the dimension of remote direct memory access (RDMA) over networks, the system can access distant memory resources with latency comparable to local memory, effectively expanding the capacity dimension while maintaining speed performance.
2Quantity of substance
If PMEM is used in volatile mode to increase memory capacity, then available memory space is expanded, but access speed decreases compared to DRAM
Solution Approach 1:
The memory manager continuously monitors access patterns of memory pages and dynamically migrates data between DRAM and PMEM tiers to maintain optimal performance. Frequently accessed pages are kept in or migrated to DRAM, while less frequently accessed pages reside in PMEM. This continuous optimization ensures that the system maintains high access speeds for active data while utilizing PMEM capacity for inactive data, resolving the speed-capacity tradeoff.
3Speed
If DRAM is used for memory allocation, then fast data access is achieved, but security issues and performance bottlenecks arise when PMEM is used in volatile mode
Solution Approach 1:
A user-space memory manager library acts as an intermediary between applications and the underlying memory hardware. This library intercepts memory allocation calls and manages the complexity of multi-tier memory allocation, migration, and fault handling. By providing a unified interface that abstracts the heterogeneous memory subsystem, the intermediary ensures consistent security policies and performance characteristics regardless of which memory tier is used, resolving the reliability concerns.
4Productivity
If multi-level caching is implemented to optimize memory usage, then system efficiency is improved, but device complexity increases
Solution Approach 1:
The memory manager implements self-service mechanisms including automatic monitoring of access patterns, autonomous decision-making about page migration between tiers, and automated handling of memory faults. The system uses simple heuristics and policies that require minimal configuration or intervention, allowing the complex multi-level caching system to manage itself efficiently. This self-service approach masks the underlying complexity while maintaining high system efficiency.
Data Source
AI summary
A technique is introduced for applying multi-level caching to deploy various types of physical memory to service captured memory calls from an application. The various types of physical memory can include local volatile memory (e.g., dynamic random-access memory), local persistent memory, and/or remote persistent memory. In an example embodiment, a user-space page fault notification mechanism is used to defer assignment of actual physical memory resources until a memory buffer is accessed by the application. After populating a selected physical memory in response to an initial user-space page fault notification, page access information can be monitored to determine which pages continues to be accessed and which pages are inactive to identify candidates for eviction.