Memory-Side Cache With Extended Line Width
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing systems face inefficiencies due to architectural limitations in data communication between processing and memory sub-systems, leading to higher data retrieval latency and operational inefficiency, as data communication within the processing sub-system is faster than between sub-systems.
Innovation Solution
Implementing hierarchical caches with different cache line sizes in both the processing and memory sub-systems, where processor-side caches use smaller cache lines for efficient data retrieval and memory-side caches use larger cache lines to reduce cache misses, leveraging temporal and spatio-temporal data access patterns to predict and manage data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If processor-side caches use smaller cache lines for efficient data retrieval, then data retrieval speed is improved, but cache misses increase
Solution Approach 1:
The cache system is segmented into multiple levels with different cache line sizes. Processor-side caches (L1, L2) use smaller cache lines (e.g., 64 bytes) for fast retrieval, while memory-side caches use larger cache lines (e.g., 512 bytes or 1 KB) to capture more data per access, reducing the frequency of cache misses.
Solution Approach 2:
The patent introduces an additional dimension to the cache hierarchy by adding memory-side caches with larger cache line sizes. This creates a multi-dimensional cache structure where data can be accessed at different granularity levels, allowing the system to balance between retrieval speed and hit rate by selecting the appropriate cache level.
2Reliability
If memory-side caches use larger cache lines to reduce cache misses, then cache hit rate is improved, but data communication overhead increases
Solution Approach 1:
Different parts of the cache hierarchy are assigned different cache line sizes based on their specific functions. Memory-side caches use larger cache lines optimized for capturing spatially related data, while processor-side caches use smaller cache lines optimized for quick access. This local optimization reduces overall data communication overhead by minimizing the number of transfers needed.
Solution Approach 2:
Memory-side caches perform preliminary action by pre-fetching and storing larger blocks of data before the processor actually needs them. This anticipatory caching reduces the frequency of data communication between the memory sub-system and processing sub-system, thereby reducing energy overhead despite the larger cache line size.
3Productivity
If hierarchical caches with different cache line sizes are implemented, then operational efficiency is improved, but device complexity increases
Solution Approach 1:
The memory-side caches serve multiple functions: they act as an extended cache for the processor, a buffer for memory operations, and a spatial prefetching mechanism. This multi-functionality justifies the added complexity by providing multiple benefits from a single structural enhancement.
Solution Approach 2:
Memory-side caches act as an intermediary layer between the processor-side caches and the main memory. This intermediary structure manages the complexity by providing a buffer zone that handles data translation and transfer protocols, simplifying the interaction between the processor and main memory while enabling the complex hierarchical structure to function efficiently.
Data Source
AI summary
The present disclosure techniques for implementing an apparatus, which includes processing circuitry that performs an operation based a target data block, a processor-side cache that implements a first cache line, memory-side cache that implements a second cache line having line width greater than the first cache line, and a memory array. The apparatus includes one or more memory controllers that, when the target data block results in a cache miss, determine a row address that identifies a memory cell row as storing the target data block, instruct the memory array to successively output multiple data blocks from the memory cell row to enable the memory-side cache to store each of the multiple of data blocks in the second cache line, and instruct the memory-side cache to output the target data block to a coherency bus to enable the processing circuitry to perform the operation based on the target data block.