Correlation-Based Instruction Prefetching for Discontinuous Accesses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing instruction prefetching mechanisms are ineffective in reducing instruction cache misses caused by large discontinuities in instruction fetch addresses, such as taken branches and function calls, as they fail to accurately predict and prefetch cache lines in processors with high instruction working sets.
Innovation Solution
A correlation-based instruction prefetcher that uses an outer-level cache (L2 or L3) to store correlation keys, predicting and prefetching cache lines based on discontinuous instruction accesses by updating a next line table (NLT) associated with the cache tags, thereby reducing storage requirements and improving prefetch accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential prefetching is used to remove instruction cache misses from sequential instruction fetches, then sequential instruction cache misses are reduced, but discontinuous instruction cache misses (caused by branches, jumps, function calls) cannot be effectively removed
Solution Approach 1:
The system performs preliminary action by detecting discontinuous instruction cache accesses and prefetching the target cache lines before they are actually needed. When a discontinuous access is detected (address difference exceeds threshold), the system immediately prefetches the predicted cache line, ensuring it is available in the instruction cache before the processor needs it, thereby eliminating stalls caused by function calls and branches.
Solution Approach 2:
The patent introduces an intermediary mechanism - a threshold-based detection system that mediates between sequential prefetching and demand fetching. This intermediary detects address discontinuities and triggers appropriate prefetching actions, bridging the gap between simple sequential prefetching and complex correlation-based prefetching methods.
2Measurement precision
If traditional correlation-based prefetching is used to predict discontinuous instruction accesses, then prefetch accuracy for branches and jumps improves, but storage requirements increase significantly
Solution Approach 1:
The system extracts only the essential correlation information needed for discontinuous prefetching - specifically, the threshold-based address difference detection and the identified target cache line addresses. By extracting only this critical information rather than storing complete correlation keys, the system achieves effective prefetching with minimal storage overhead in the instruction cache.
Solution Approach 2:
The patent employs a lightweight, temporary correlation structure that exists only momentarily during the prefetch decision process. The threshold-based detection creates temporary prefetch requests without maintaining persistent correlation key storage, effectively using disposable prefetch information that is generated, used, and discarded without requiring long-term storage resources.
Data Source
AI summary
The disclosed embodiments provide a system that facilitates prefetching an instruction cache line in a processor. During execution of the processor, the system performs a current instruction cache access which is directed to a current cache line. If the current instruction cache access causes a cache miss or is a first demand fetch for a previously prefetched cache line, the system determines whether the current instruction cache access is discontinuous with a preceding instruction cache access. If so, the system completes the current instruction cache access by performing a cache access to service the cache miss or the first demand fetch, and also prefetching a predicted cache line associated with a discontinuous instruction cache access which is predicted to follow the current instruction cache access.