Dynamic HBM–DDR Routing for Memory Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems with heterogeneous architectures face challenges in optimizing data routing to minimize latency due to varying access latencies and usage thresholds of different memory types, leading to inefficient performance and energy consumption.
Innovation Solution
A memory controller dynamically monitors and adjusts data routing based on latency criteria and usage thresholds, routing requests to DDR memory when usage is below a threshold and to HBM memory when usage exceeds the threshold, thereby optimizing access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If read requests are always routed to the same memory type, then routing simplicity is maintained, but access latency increases when usage exceeds thresholds
Solution Approach 1:
The patent implements dynamic routing that adapts memory access paths based on real-time usage conditions. The system monitors DDR memory usage and dynamically switches between DDR and HBM routing paths, transforming a static routing system into a dynamic one that responds to changing workload conditions to optimize access latency.
Solution Approach 2:
The system changes routing parameters based on usage thresholds. When DDR usage exceeds a predetermined threshold, the routing parameter switches from DDR to HBM memory path. This parameter change approach allows the system to adapt to varying workload conditions and maintain optimal performance across different usage scenarios.
2Power
If HBM memory is used as cache for DDR memory, then bandwidth is improved, but access latency increases when DDR usage is below threshold
Solution Approach 1:
The patent applies different memory access paths based on local usage conditions. Instead of using a uniform caching strategy, the system evaluates local DDR usage conditions and selectively routes requests through HBM or DDR paths. This local quality approach ensures that HBM caching is applied only where and when it provides benefit, avoiding unnecessary latency overhead.
Solution Approach 2:
The system implements partial caching action by conditionally routing only certain requests through HBM based on usage thresholds. Rather than always using HBM as cache, the system applies caching selectively when DDR usage exceeds thresholds, avoiding the latency penalty of HBM access when DDR has available capacity.
3Loss of time
If DDR memory usage is monitored and routing is adjusted dynamically, then access latency is optimized, but system complexity increases
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors DDR memory usage and uses this information to dynamically adjust routing decisions. The usage threshold monitoring and conditional routing create a closed-loop feedback system that automatically optimizes performance without requiring complex manual intervention or sophisticated control algorithms.
Data Source
AI summary
Systems and methods related to memory status based traffic routing on heterogeneous memory subsystem are disclosed herein. A high bandwidth memory (HBM) may act as a cache for a double data rate (DDR) memory. HBM may have a higher access latency than DDR memory in some situations, such as low usage. Access latency for a read request via a DDR memory may increase substantially when the usage exceeds one or more thresholds. Accordingly, the routing of the read requests may be tailored to reduce access latency. For example, the usage of the DDR memory may be monitored. When the usage is below a percentage threshold, the access request may be routed to DDR memory rather than HBM. When the usage is above a percentage threshold, the access request may be routed to HBM or DDR. Routing read requests in this manner may minimize overall latency of read access requests.