Tag Accelerator Cache for Low Latency DRAM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-node computer systems, maintaining cache coherency across processors while minimizing latency and bandwidth costs is challenging due to the high expense of large static random-access memory (SRAM) tag arrays, which are often co-located with data in large caches, increasing lookup latency and bandwidth usage.
Innovation Solution
Implementing a tag accelerator cache that caches tag blocks fetched from the data cache, allowing for simultaneous retrieval of multiple tags and reducing latency by filtering out tag accesses through spatial locality, thereby improving cache bandwidth and reducing the need for expensive SRAM arrays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large SRAM tag array is used to maintain cache coherency, then cache coherency is maintained, but the cost and area increase significantly
Solution Approach 1:
The patent segments the tag array into multiple smaller DRAM banks organized in a grid structure, where each bank stores a portion of the tag data. This segmentation allows the system to use cheaper DRAM technology instead of expensive SRAM, while maintaining the required cache coherency functionality through distributed tag storage across multiple segments.
Solution Approach 2:
The patent creates multiple copies of tag data distributed across different DRAM banks and uses replication strategies to ensure data redundancy. By copying tag information across multiple locations in the DRAM array, the system maintains reliability and coherency while using cost-effective DRAM technology instead of SRAM.
2Adaptability or versatility
If tags and data are co-located in the large cache, then the implementation becomes scalable, but lookup latency and cache bandwidth increase
Solution Approach 1:
The patent segments the tag lookup process by organizing tags into separate DRAM banks that can be accessed independently. This segmentation allows parallel access to multiple tag banks simultaneously, reducing the overall lookup latency while maintaining the scalable co-located tag-data structure.
Solution Approach 2:
The patent introduces a new dimensional organization of tags using a grid-like structure with rows and columns in DRAM banks, adding spatial dimensionality to the tag storage. This dimensional organization enables more efficient addressing and parallel access patterns, reducing lookup latency while maintaining scalability.
3Adaptability or versatility
If tags and data are co-located in the large cache, then the implementation becomes scalable, but cache bandwidth increases
Solution Approach 1:
The patent segments the tag storage into multiple DRAM banks that can be accessed in parallel, distributing the bandwidth requirement across multiple independent access paths. This segmentation reduces the bandwidth burden on any single access path while maintaining the scalable architecture.
Solution Approach 2:
The patent implements continuous tag validation and cache coherency checking through the distributed DRAM bank structure, allowing overlapping and pipelined operations. This continuity enables efficient utilization of cache bandwidth by keeping the system continuously productive rather than experiencing idle waiting periods.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a tag accelerator cache are disclosed. A system includes at least a data cache and a control unit coupled to the data cache via a memory controller. The control unit includes a tag accelerator cache (TAC) for caching tag blocks fetched from the data cache. The data cache is organized such that multiple tags are retrieved in a single access. This allows hiding the tag latency penalty for future accesses to neighboring tags and improves cache bandwidth. When a tag block is fetched from the data cache, the tag block is cached in the TAC. Memory requests received by the control unit first lookup the TAC before being forwarded to the data cache. Due to the presence of spatial locality in applications, the TAC can filter out a large percentage of tag accesses to the data cache, resulting in latency and bandwidth savings.