Cache-to-Cache Bypass Mechanism for Producer-Consumer Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In shared memory multiprocessor systems, cache-to-cache latency increases due to growing cache sizes, leading to inefficiencies in communication between caching agents that do not share a last-level cache, necessitating improved techniques for accelerating data transfers between these agents.
Innovation Solution
A communication bypass mechanism that uses a shadow tag to record communication information and bypasses the last-level cache by evicting cache lines with a modified state from the penultimate-level cache, allowing direct access to the consumer caching agent and reducing latency by prefetching the cache line into the consumer's last-level cache and writing it to main memory if necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cache sizes are grown larger to improve local data storage capacity, then the proportion of communication misses increases and cache-to-cache latency increases, but data storage capacity is improved
Solution Approach 1:
The memory hierarchy is segmented into multiple levels: penultimate-level cache, last-level cache, and main memory. The invention further segments the eviction path by introducing a bypass mechanism that divides the traditional sequential eviction path into two parallel paths: one through the last-level cache and another direct path to main memory for communication lines, allowing selective optimization of different data types
Solution Approach 2:
A shadow tag entry is introduced as an intermediary data structure that stores communication state information alongside cache line data. This shadow tag acts as a mediator that enables the cache controller to identify communication lines and route them appropriately without interfering with the main cache eviction logic
2Reliability
If cache-to-cache transfers are performed through the last-level cache, then cache coherence is maintained, but transfer latency increases
Solution Approach 1:
The system performs preliminary actions by setting the communication state bit in the shadow tag when a cache line is identified as a communication line. This preliminary marking enables the bypass mechanism to recognize and handle communication lines differently during eviction, preparing the system in advance to optimize future transfer operations
Solution Approach 2:
The cache eviction mechanism is made dynamic by conditionally selecting between two different eviction paths based on the communication state bit. Normal cache lines follow the traditional path through the last-level cache, while communication lines dynamically switch to the direct bypass path, allowing the system to adapt its behavior based on data type
Data Source
AI summary
A communication bypass mechanism accelerates cache-to-cache data transfers for communication traffic between caching agents that have separate last-level caches. A method includes bypassing a last-level cache of a first caching agent in response to a cache line having a modified state being evicted from a penultimate-level cache of the first caching agent and a communication attribute of a shadow tag entry associated with the cache line being set. The communication attribute indicates prior communication of the cache line with a second caching agent having a second last-level cache.