Transient Cache Line Prefetching to Reduce Memory Bandwidth Overshoot
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multiprocessor systems, aggressive hardware-based data prefetching can lead to overshoot, wasting memory bandwidth and inefficiently using available memory bandwidth, which affects the performance of data-intensive applications and individual threads.
Innovation Solution
A system with a processor core, cache, transience register, length register, prefetch unit, and cache controller that sets a transient property for cache lines and limits prefetch requests based on stream length, allowing for efficient memory bandwidth usage by victimizing cache lines with a transient property earlier than standard cache replacement policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If aggressive hardware-based data prefetching is employed to improve thread performance, then thread performance is improved, but memory bandwidth is wasted due to overshoot
Solution Approach 1:
The patent changes the state/parameter of cache lines by marking them as transient, which alters their replacement behavior. This allows the system to prefetch more aggressively for threads needing performance while controlling bandwidth waste by enabling early replacement of transient cache lines that are likely to be unused, thus resolving the contradiction between thread performance improvement and memory bandwidth conservation.
2Loss of energy
If cache lines are replaced sooner to conserve memory bandwidth, then memory bandwidth is conserved, but thread performance may be affected
Solution Approach 1:
The patent applies local quality by treating different cache lines differently based on their transient status. Transient cache lines (which are likely to be used only once) are replaced sooner to conserve bandwidth, while non-transient cache lines maintain normal replacement behavior to support thread performance. This differentiated approach allows bandwidth conservation without negatively impacting thread performance.
3Loss of energy
If prefetching is limited to a predetermined number of cache lines to reduce overshoot, then memory bandwidth waste is reduced, but thread performance improvement is limited
Solution Approach 1:
The patent performs preliminary action by marking cache lines as transient before prefetching occurs. This pre-marking allows the prefetcher to know in advance which cache lines should be replaced early, enabling more aggressive prefetching without the risk of significant overshoot waste. The preliminary classification of cache lines facilitates both bandwidth efficiency and performance improvement.
Data Source
AI summary
A prefetch unit includes a transience register and a length register. The transience register hosts an indication of transient for data stream prefetching. The length register hosts an indication of a stream length for data stream prefetching. The prefetch unit monitors the transience register and the length register. The prefetch unit generates prefetch requests of data streams with a transient property up to the stream length limit when the transience register indicates transient and the length register indicates the stream length limit for data stream prefetching. A cache controller coupled with the prefetch unit implements a cache replacement policy and cache coherence protocols. The cache controller writes data supplied from memory responsive to the prefetch requests into cache with an indication of transient. The cache controller victimizes cache lines with an indication of transient independent of the cache replacement policy.