Directory-Based Cache Coherency for Scalable Agent Interconnects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of agents accessing memory increases, the number of cache coherency messages also increases, overwhelming the interconnect bandwidth and leading to increased memory latency and reduced system performance.
Innovation Solution
A scalable cache coherency protocol that minimizes interconnect traffic by using a 3-hop protocol for cache block transfers, where snoops are targeted only to affected agents based on directory updates, and incorporates snoop forward and snoop back messages to manage cache states and resolve race conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional cache coherency protocols are used with multiple agents, then cache coherency is maintained, but interconnect bandwidth is overwhelmed and memory latency increases
Solution Approach 1:
The patent extracts and eliminates unnecessary snoop messages from the cache coherency protocol. By using directory-based tracking of cache block locations, the system only sends snoop messages when actually needed (on conflicts), rather than broadcasting to all agents. This extraction of redundant communication maintains coherency while reducing interconnect traffic and improving system performance.
2Adaptability or versatility
If the number of agents increases, then system capacity increases, but the number of cache coherency messages increases and overwhelms the interconnect
Solution Approach 1:
The patent introduces a directory structure as an intermediary between agents and the cache coherency mechanism. The directory tracks the location of each cache block and determines which agents hold copies. This intermediary eliminates the need for broad snoop broadcasting, as the directory can directly identify and communicate only with relevant agents, enabling scalable system capacity without proportional increases in interconnect traffic.
3Productivity
If cache coherency messages are reduced, then interconnect bandwidth is preserved, but cache coherency management becomes more complex
Solution Approach 1:
The patent adds a directory dimension to the cache coherency system, moving from a flat agent-to-agent messaging model to a hierarchical model with directory management. The directory maintains metadata about cache block locations and ownership, enabling selective snoop messaging. This dimensional addition (the directory layer) simplifies the overall messaging complexity by providing centralized tracking, even though it adds storage requirements.
Data Source
AI summary
In some embodiments, multiple coherent agent circuits include respective caches configured to cache memory data. Control circuitry may implement a directory that tracks which of the multiple coherent agent circuits is caching copies of a cache block and a state of one or more cached copies. Based on a first request for the cache block by a first agent circuit, the control circuitry may: read an entry corresponding to the cache block from the directory, issue a message to a second agent circuit that has a first cached copy of the cache block according to the entry, where the second agent circuit updates a coherency state of the first cached copy based on the message, and update the entry in the directory, based on issuance of the message and prior to the message being processed by the second agent circuit, to reflect completion of the first request.