Dynamic Cache Allocation for Tiered-Memory Latency Gaps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In tiered-memory systems, the difference in latency between different memory types causes performance degradation due to non-uniform memory access (NUMA) interleaving.
Innovation Solution
A processor and system that dynamically allocate cache ways into partitions based on latency information, using masking values to adjust cache allocation between local and remote memories, thereby reducing average latency differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory interleaving technique is used in tiered-memory system, then memory bandwidth and memory capacity are expanded, but latency difference between local and remote memories causes performance degradation
Solution Approach 1:
The cache is divided into multiple cache ways that are partitioned into first and second partitions. Each partition is selectively allocated to cache data from either local memory or remote memory, creating segmented caching paths that address different latency characteristics of memory types.
Solution Approach 2:
The cache allocation is dynamically adjusted based on latency information. The system monitors access latencies to local and remote memories and reconfigures the cache partition allocation accordingly, making the cache structure adaptive to changing memory performance conditions.
2Device complexity
If fixed cache allocation is used, then cache management is simple, but it cannot adapt to latency differences between different memory types
Solution Approach 1:
The system implements a feedback mechanism where latency information from memory accesses is monitored and used to adjust cache allocation. This closed-loop control enables the cache to adapt to latency differences between memory types while maintaining automated management.
Solution Approach 2:
The cache allocation parameters (partition sizes and assignments) are changed based on monitored latency conditions. When latency to a particular memory type increases, the system adjusts the cache partition allocation to favor that memory type, optimizing performance under varying conditions.
Data Source
AI summary
A processor includes a processing core configured to process each of a plurality of requests by accessing a corresponding one of a first memory and a second memory, a latency monitor configured to generate first latency information and second latency information, the first latency information comprising a first access latency to the first memory, and the second latency information comprising a second access latency to the second memory, a plurality of cache ways divided into a first partition and a second partition, and a decision engine configured to allocate each of the plurality of cache ways to one of the first partition and the second partition, based on the first latency information and the second latency information.