Dynamic Inclusive Last Level Cache for Bandwidth and Power Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor cache technologies face inefficiencies in bandwidth and power consumption due to the exclusive LLC policy, which requires additional data transfer and power consumption for clean evictions, while the inclusive LLC policy lacks the combined capacity of core caches and LLC, leading to performance losses for workloads sensitive to additional capacity.
Innovation Solution
Implementing a dynamic inclusive LLC (DIL) that identifies workloads with high re-use patterns, sending shared data copies to cores and maintaining them in the LLC, thereby reducing data transfer and power consumption by silently dropping evicted data if a shared copy exists in the LLC, and optimizing cache behavior through counter-based mechanisms and snoop filters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If an exclusive LLC policy is used, then the combined capacity of core caches and LLC is effectively utilized, but additional on-chip bandwidth is required to support more frequent evictions from core caches
Solution Approach 1:
The patent implements a dynamic LLC policy that can switch between inclusive and exclusive behaviors based on workload characteristics. The system monitors eviction patterns and automatically adjusts the LLC inclusion policy, allowing it to provide combined capacity benefits when needed while reducing bandwidth consumption during workloads that generate frequent evictions.
2Productivity
If an inclusive LLC policy is used, then memory accesses are reduced by utilizing combined capacity, but the LLC capacity determines overall capacity because blocks are duplicated between core caches and LLC
Solution Approach 1:
The system dynamically adjusts the LLC policy based on workload characteristics, switching between inclusive and exclusive behaviors. This allows the system to provide combined capacity benefits for workloads that benefit from additional space while reducing bandwidth consumption for workloads that generate frequent evictions.
Solution Approach 2:
The patent changes the inclusion policy parameter of the LLC based on detected workload patterns. When workloads exhibit high reuse patterns, the system transitions to an inclusive policy to maximize combined capacity utilization. When eviction patterns indicate otherwise, it switches to an exclusive policy to minimize bandwidth overhead.
3Quantity of substance
If an exclusive LLC policy is used, then combined capacity is utilized effectively, but power consumption increases due to additional data transfer for clean evictions
Solution Approach 1:
The dynamic policy adjustment mechanism allows the system to switch between inclusive and exclusive LLC behaviors based on workload characteristics. By detecting when workloads generate frequent clean evictions, the system can transition to an inclusive policy that eliminates the need for data transfer during clean evictions, thereby reducing power consumption while maintaining combined capacity benefits.
Data Source
AI summary
An embodiment of an integrated circuit may comprise a core, and a cache controller coupled to the core, the cache controller including circuitry to identify data from a working set for dynamic inclusion in a next level cache based on an amount of re-use of the next level cache, send a shared copy of the identified data to a requesting core of one or more processor cores, and maintain a copy of the identified data in the next level cache. Other embodiments are disclosed and claimed.