Multi-Level Cache Request Tracking for Processor Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor cache hierarchies face inefficiencies due to coarse-grained encoding of core-valid bits, leading to spurious messages, performance loss, and energy waste, particularly as the number of tiles and caches increases, causing CV bit aliasing and stale entries.
Innovation Solution
Implementing a multi-level tracker system with a primary CAM-based tracker for global observation and a secondary SRAM-based tracker for state storage, allowing for increased outstanding requests without significant power or area impact, thereby improving memory bandwidth and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If coarse-grained encoding is used for core-valid bits to reduce tracker area, then area is reduced, but spurious messages are generated causing performance loss and energy waste
Solution Approach 1:
The tracker is divided into two separate structures: a primary tracker using coarse-grained encoding for area efficiency, and a secondary tracker using fine-grained encoding for precision. This segmentation allows each tracker to specialize - the primary tracker reduces area while the secondary tracker eliminates spurious messages by providing accurate cache validity information.
Solution Approach 2:
The secondary tracker acts as an intermediary between the primary tracker and the cache coherence protocol. It receives requests from the primary tracker, resolves ambiguity about cache validity using its fine-grained encoding, and provides precise responses. This intermediary structure eliminates the need for spurious invalidation messages while keeping the overall system area-efficient.
2Area of stationary object
If coarse-grained encoding is used for core-valid bits, then tracker area is reduced, but spurious invalidation messages are sent increasing energy consumption
Solution Approach 1:
The tracker is divided into two separate structures: a primary tracker using coarse-grained encoding for area efficiency, and a secondary tracker using fine-grained encoding for precision. This segmentation allows each tracker to specialize - the primary tracker reduces area while the secondary tracker eliminates spurious messages by providing accurate cache validity information.
Solution Approach 2:
The secondary tracker acts as an intermediary between the primary tracker and the cache coherence protocol. It receives requests from the primary tracker, resolves ambiguity about cache validity using its fine-grained encoding, and provides precise responses. This intermediary structure eliminates the need for spurious invalidation messages while keeping the overall system area-efficient.
3Measurement precision
If tracker size is increased to reduce CV bit aliasing, then measurement precision is improved, but device complexity and area increase
Solution Approach 1:
The tracker is divided into two separate structures: a primary tracker using coarse-grained encoding for area efficiency, and a secondary tracker using fine-grained encoding for precision. This segmentation allows each tracker to specialize - the primary tracker reduces area while the secondary tracker eliminates spurious messages by providing accurate cache validity information.
Solution Approach 2:
Instead of increasing the size of a single tracker structure, the solution adds a new dimension by introducing a second tracker with different encoding granularity. This dimensional approach allows the system to achieve high precision without proportionally increasing overall complexity, as each tracker operates in its optimal encoding regime.
4Productivity
If the number of outstanding memory requests is increased to improve bandwidth, then productivity is improved, but power consumption increases
Solution Approach 1:
The system dynamically manages outstanding memory requests by implementing separate allocation and deallocation paths for the primary and secondary trackers. This dynamic management allows the system to maintain a high number of outstanding requests for improved bandwidth while efficiently releasing tracker entries when cache coherence is resolved, optimizing power usage based on actual workload demands.
Solution Approach 2:
The system performs preliminary actions by allocating tracker entries in advance for outstanding memory requests. The primary tracker pre-allocates entries using coarse-grained encoding, and the secondary tracker is pre-configured to resolve validity. This preliminary preparation enables high bandwidth utilization while avoiding the need for complex real-time tracking, thus managing power consumption efficiently.
Data Source
AI summary
An apparatus and method for multi-level cache request tracking. For example, one embodiment of a processor comprises: one or more cores to execute instructions and process data; a memory subsystem comprising a system memory and a multi-level cache hierarchy; a primary tracker to store a first entry associated with a memory request to transfer a cache line from the system memory or a first cache within the cache hierarchy to a second cache; primary tracker allocation circuitry to allocate and deallocate entries within the primary tracker; a secondary tracker to store a second entry associated with the memory request; secondary tracker allocation circuitry to allocate and deallocate entries within the secondary tracker; the primary tracker allocation circuitry to deallocate the first entry in response to a first indication that one or more cache coherence requirements associated with the cache line have been resolved, the secondary tracker allocation circuitry to deallocate the second entry in response to a second indication related to transmission of the cache line to the second cache.