NUMA Memory Replication for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-uniform memory architectures in portable computing devices lead to higher latency due to the need for processors to access data in both near and far DRAM, resulting from the use of multi-die or multi-SoC products, where data access via high-performance buses and lower-performance interchip interfaces creates inefficiencies.
Innovation Solution
The system allocates memory by determining if available physical pages on local volatile memory devices exceed a minimum threshold, allowing for the replication of memory data between devices, enabling processors to access data through a single virtual page address, thereby reducing latency through the use of adaptive and non-blocking NUMA latency optimization modules and modified page table entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If multi-die or multi-SoC products are used to increase transistor density, then device complexity and processing power are improved, but memory access latency increases due to far DRAM access via lower-performance interchip interfaces
Solution Approach 1:
The system proactively identifies frequently accessed far DRAM pages and replicates them to local DRAM before they are needed, using page table entry flags and accessibility information to predict future access patterns. This preliminary replication action eliminates future latency by ensuring hot data is already in fast local memory.
Solution Approach 2:
The system creates duplicate copies of memory pages from far DRAM and stores them in local DRAM. By maintaining both original and replicated copies with appropriate page table entries, the system enables processors to access data from fast local memory instead of slow interchip interfaces, directly addressing the latency problem.
2Loss of time
If memory data is replicated between local volatile memory devices, then memory access latency is reduced, but available physical memory capacity is consumed
Solution Approach 1:
Instead of replicating all memory pages, the system selectively replicates only those pages that meet specific criteria (frequently accessed, small size, high accessibility). This partial replication approach provides latency benefits for critical data while consuming minimal physical memory capacity.
Solution Approach 2:
The system dynamically changes parameters such as page size (preferring smaller pages for replication), accessibility thresholds, and replication priorities based on workload characteristics. By adjusting these parameters, the system optimizes the balance between latency reduction and memory capacity consumption.
3Productivity
If page table entries are modified to support replication, then memory access efficiency is improved, but system complexity increases
Solution Approach 1:
The page table entry structure is designed to serve multiple functions: standard virtual-to-physical address translation, replication status tracking, accessibility information storage, and future access prediction. By making the page table entry multi-functional, the system avoids creating separate data structures that would increase complexity.
Solution Approach 2:
The system merges replication management functionality directly into the existing page table structure rather than creating separate replication control mechanisms. By combining address translation and replication management in a single unified structure, the system improves memory access efficiency while minimizing the increase in system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and computer programs are disclosed for allocating memory in a portable computing device having a non-uniform memory architecture. One embodiment of a method comprises: receiving from a process executing on a first system on chip (SoC) a request for a virtual memory page, the first SoC electrically coupled to a second SoC via an interchip interface, the first SoC electrically coupled to a first local volatile memory device via a first high-performance bus and the second SoC electrically coupled to a second local volatile memory device via a second high-performance bus; determining whether a number of available physical pages on the first and second local volatile memory devices exceeds a minimum threshold for initiating replication of memory data between the first and second local volatile memory devices; and if the minimum threshold is exceeded, allocating a first physical address on the first local volatile memory device and a second physical address on the second local volatile memory device to a single virtual page address.