Heterogeneous Memory Page Allocation with Near Far Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory management systems fail to optimize data placement across heterogeneous memory technologies, leading to suboptimal performance and wastage of low-latency memory due to fixed migration granularities and unawareness of memory heterogeneity by the operating system.
Innovation Solution
Implementing a multi-component cooperative memory management scheme that includes an asymmetric heterogeneous memory layout, where the operating system is aware of different memory regions with varying capacity ratios, and custom hardware performs fine-grained data remapping to optimize data placement based on spatial and temporal locality characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed migration granularity (cache line size) is used for data movement between heterogeneous memories, then hardware overhead is reduced, but data placement optimization capability deteriorates because the operating system cannot optimize data placement at finer granularity
Solution Approach 1:
The patent segments the data management function into two levels: the operating system handles page-level management (4KB granularity) while the memory controller handles cache line-level management (64B granularity). This segmentation allows each component to operate at its optimal granularity, with the memory controller providing fine-grained data placement optimization without requiring the OS to be aware of memory heterogeneity details.
Solution Approach 2:
The memory controller acts as an intermediary between the operating system and the heterogeneous memory system. It receives page-level management decisions from the OS and automatically performs cache line-level data movement between DRAM and NVM, bridging the granularity gap without exposing complexity to the OS.
2Adaptability or versatility
If the operating system manages data placement at page granularity (4KB or 8KB), then data placement flexibility is improved, but low-latency memory utilization deteriorates because large memory chunks cannot be efficiently utilized and many cache lines remain untouched
Solution Approach 1:
The patent segments the 4KB page into smaller 64B cache line units for management purposes. The OS continues to manage pages at 4KB granularity for flexibility, while the memory controller further segments each page into cache lines to identify and migrate only the hot data portions to DRAM, improving low-latency memory utilization without sacrificing OS-level placement flexibility.
Solution Approach 2:
The patent applies local quality by treating different cache lines within the same page differently based on their access patterns. Hot cache lines are migrated to DRAM while cold cache lines remain in NVM, allowing selective optimization of specific regions within each page rather than treating the entire page uniformly.
3Ease of manufacture
If symmetric memory layout is used where data is laid out identically within one memory technology, then implementation simplicity is improved, but performance deteriorates because different memory rows have different access characteristics that are not optimized
Solution Approach 1:
The patent implements an asymmetric memory layout where different regions of the memory address space are mapped to different memory technologies based on access patterns. The memory controller maintains separate data structures (e.g., page tables, translation lookaside buffers) that track which cache lines are located in DRAM versus NVM, enabling asymmetric treatment of memory regions to optimize for different access characteristics while maintaining manageable complexity.
Data Source
AI summary
A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.