Hierarchical Memory Architecture Reducing Cross-Device Migration Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for aggregating memory resources across computing devices, particularly between small and large form factor devices, result in high migration latencies and user-perceptible delays due to software implementations, which are not seamless and are constrained by hardware differences and power issues.
Innovation Solution
A hierarchical memory architecture with a two-level memory (2LM) system, comprising a fast near memory and a slower far memory, is used to migrate memory contents and computational states between devices, allowing for transparent execution of applications between devices, with periodic data copying from dirty blocks to reduce latency during undocking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If software implementation is used to aggregate memory resources across devices, then memory resources can be shared between devices, but high migration latencies and user-perceptible delays occur
Solution Approach 1:
The patent segments the memory system into device-local memory (fast) and remote memory (slower), with the memory management unit acting as an intermediary layer. This segmentation allows memory operations to be performed locally without triggering full migrations, reducing latency while maintaining resource sharing capability.
Solution Approach 2:
The memory management unit serves as an intermediary between the processor and memory resources, managing the complexity of cross-device memory access. It handles page table updates, translation lookaside buffer management, and coordination with the far memory controller, abstracting the complexity from the processor and reducing migration overhead.
2Adaptability or versatility
If memory resources are aggregated across small and large form factor devices, then computational capabilities are enhanced, but hardware differences and power issues constrain performance
Solution Approach 1:
The system dynamically changes memory access parameters based on the state of the interconnect. When devices are docked, memory access utilizes the high-speed wired interconnect with different timing and protocol parameters compared to wireless or undocked states. The memory management unit adjusts page table configurations and buffer management strategies based on connection status, ensuring reliable performance across varying hardware conditions.
3Reliability
If full memory migration is performed when devices dock and undock, then memory resources are properly transferred, but user-perceptible delays occur
Solution Approach 1:
The system performs preliminary actions by maintaining device-local memory and caching mechanisms that prepare for potential migrations. The translation lookaside buffer pre-loads and caches translation entries, and the memory management unit pre-configures page tables for anticipated memory access patterns, reducing the need for full migrations during docking/undocking events.
Solution Approach 2:
Instead of continuous or full memory migration, the system uses periodic updates and selective synchronization. The memory management unit periodically updates the translation lookaside buffer and synchronizes only the necessary memory pages between devices, rather than performing complete memory transfers, thereby reducing transitional delays while maintaining data consistency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Examples include composing memory resources across devices and reducing transitional latency. In some examples, memory resources associated with executing one or more applications by circuitry at two separate devices may be composed across the two devices via use of a midstream buffer. The circuitry may be capable of executing the one or more applications using a hierarchical memory architecture including a near memory and a far memory. In some examples, near memories may be separately located at first and second devices and a far memory may be located at the first device. The near memory of the first device may be used as a midstream buffer to facilitate movement of data over a wired or wireless interconnect to or from the near memory of the second device.