Tile Snoop Filter Cache Reduces GCU Lookup Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-socket/node systems, the Global Coherence Unit (GCU) becomes a performance bottleneck due to numerous requests from tiles for snoop filter lookups, leading to communication bottlenecks and high latency in memory access, as each tile must check the GCU for remote cache hits, resulting in slow responses and increased clock cycles.
Innovation Solution
Implementing a snoop filter cache in each tile to locally track cache information from the GCU, reducing the number of lookups to the GCU by checking the snoop filter cache first for remote cache hits or misses, and only sending requests to the GCU when necessary, thereby avoiding unnecessary communication and clock cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If every tile sends a request to the GCU for snoop filter lookup, then memory coherency is maintained, but communication bottleneck and latency increase
Solution Approach 1:
The patent divides the centralized snoop filter functionality into distributed snoop filter caches at each tile. Instead of one central GCU handling all lookups, each tile maintains its own local snoop filter cache, segmenting the lookup function across multiple independent units. This eliminates the communication bottleneck while maintaining coherency through periodic synchronization with the GCU.
Solution Approach 2:
The patent implements preliminary action by having tiles proactively populate their local snoop filter caches with information from the GCU before actual memory access requests occur. The GCU periodically pushes snoop filter entries to local caches, so when a tile needs to perform a lookup, the information is already available locally, avoiding real-time communication delays.
2Reliability
If tiles check the GCU for every cache access, then remote cache hits are detected, but clock cycles are consumed
Solution Approach 1:
The patent introduces local snoop filter caches as intermediary structures between tiles and the GCU. These local caches store copies of snoop filter information, allowing tiles to perform lookups locally without directly communicating with the GCU for every access. The intermediary layer filters out unnecessary GCU communications while maintaining accurate remote cache hit detection.
Solution Approach 2:
The patent creates local copies of the snoop filter data structure at each tile's L3 cache. Instead of querying the original centralized snoop filter at the GCU for every operation, tiles use these local copies to perform rapid lookups. The copies are kept synchronized with the GCU through periodic updates, enabling fast local decision-making about remote cache hits without consuming clock cycles for frequent GCU communications.
3Reliability
If a centralized GCU handles all snoop filter lookups, then coherency is maintained, but communication overhead increases
Solution Approach 1:
The patent segments the centralized snoop filter into distributed local caches at each tile, eliminating the single-point communication bottleneck. Each tile independently manages its local snoop filter cache, reducing inter-communication overhead while maintaining system-wide coherency through periodic synchronization with the GCU.
Solution Approach 2:
The patent enables each tile to serve its own snoop filter lookup needs through local caches, rather than relying on the GCU for every query. Tiles autonomously manage their local cache contents and perform lookups independently, reducing communication overhead. The GCU maintains system-wide coherency by periodically updating local caches, creating a self-service architecture that minimizes communication requirements.
Data Source
AI summary
In a multi-node system, each node includes tiles. Each tile includes a cache controller, a local cache, and a snoop filter cache (SFC). The cache controller responsive to a memory access request by the tile checks the local cache to determine whether the data associated with the request has been cached by the local cache of the tile. The cached data from the local cache is returned responsive to a cache-hit. The SFC is checked to determine whether any other tile of a remote node has cached the data associated with the memory access request. If it is determined that the data has been cached by another tile of a remote node and if there is a cache-miss by the local cache, then the memory access request is transmitted to the global coherency unit (GCU) and the snoop filter to fetch the cached data. Otherwise an interconnected memory is accessed.