Shared Cache Fill Partitioning for Multi-Threaded Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-threaded processors, shared cache resources lead to conflicts where one thread filling the cache causes eviction of data used by other threads, resulting in inefficiencies and cache thrashing, and partitioning the cache reduces available space and leads to data replication.
Innovation Solution
Implementing a fill partitioning policy that restricts each thread to storing data in specific, non-overlapping portions of the shared cache, using a modified replacement algorithm and dynamic mask selection based on thread ID and cache conditions to manage cache misses and evictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the cache is shared between threads, then cache capacity is maximized and data sharing is enabled, but cache thrashing occurs where one thread fills the cache causing eviction of data used by other threads
Solution Approach 1:
The patent divides the shared cache into multiple partitions, each associated with a specific thread. When a thread experiences cache misses beyond a threshold, fill partitioning is enabled for that thread, restricting its cache fills to a specific partition. This segmentation prevents any single thread from filling the entire cache and causing thrashing, while still allowing all threads to access the full cache capacity through shared partitions.
2Reliability
If the cache is partitioned between threads, then cache thrashing is prevented, but cache space is reduced and data replication occurs
Solution Approach 1:
The patent implements dynamic fill partitioning where threads can transition between shared and partitioned modes based on their cache miss rates. Threads start in shared mode with full cache access. When a thread's cache miss rate exceeds a threshold, fill partitioning is enabled for that thread, restricting it to a specific partition. When the miss rate falls below the threshold, partitioning is disabled and the thread returns to shared mode. This dynamic approach prevents static partitioning from reducing overall cache capacity while still preventing thrashing when needed.
3Reliability
If fill partitioning is applied to all cache miss events, then cache thrashing is completely prevented, but cache performance degrades due to restricted access
Solution Approach 1:
The patent applies fill partitioning selectively rather than universally. Threads start in shared mode with full cache access for all cache misses. Only when a thread's cache miss rate exceeds a specific threshold does fill partitioning become active for that thread. This partial application of partitioning ensures that only threads causing or experiencing thrashing are restricted, while other threads maintain full cache access efficiency. The threshold-based approach prevents unnecessary performance degradation from blanket partitioning.
Data Source
AI summary
Fill partitioning of a shared cache is described. In an embodiment, all threads running in a processor are able to access any data stored in the shared cache; however, in the event of a cache miss, a thread may be restricted such that it can only store data in a portion of the shared cache. The restrictions to storing data may be implemented for all cache miss events or for only a subset of those events. For example, the restrictions may be implemented only when the shared cache is full and/or only for particular threads. The restrictions may also be applied dynamically, for example, based on conditions associated with the cache. Different portions may be defined for different threads (e.g. in a multi-threaded processor) and these different portions may, for example, be separate and non-overlapping. Fill partitioning may be applied to any on-chip cache, for example, a L1 cache.