Cache Miss Prediction via Pattern History Table
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The latency of first-level cache misses in processors leads to pipeline stalls and performance degradation, as the scheduler assumes data availability in the L1 cache, resulting in mis-predictions and power overhead due to cache misses.
Innovation Solution
A method is introduced to predict cache misses using data access behavior and instruction address by decoding instructions, accessing a pattern history table and global history buffer to determine if a memory access will result in a cache miss, allowing for re-scheduling of instructions and updating the prediction tables based on actual execution results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the scheduler assumes data availability in L1 cache and issues dependent instructions, then instruction throughput is improved, but pipeline stalls occur when cache misses happen
Solution Approach 1:
The patent applies preliminary action by performing cache hit/miss prediction during the decode stage, before the instruction reaches the execution stage. The pattern history table and global history buffer are accessed early to predict cache behavior, allowing the scheduler to make informed decisions about issuing dependent instructions before the actual cache access occurs. This resolves the contradiction by enabling accurate predictions (improving reliability) while maintaining instruction throughput (improving productivity).
Solution Approach 2:
The patent implements feedback mechanisms where the pattern history table and global history buffer are continuously updated with actual cache access outcomes. The prediction accuracy is improved by feeding back real cache hit/miss results to refine future predictions. This feedback loop allows the system to adapt to actual cache behavior patterns, resolving the contradiction between maintaining high instruction throughput and ensuring accurate cache behavior predictions.
2Reliability
If a reject-and-replay mechanism is triggered on cache miss, then prediction accuracy is maintained, but performance penalty and power overhead increase
Solution Approach 1:
The patent performs cache prediction in the decode stage, before the instruction is issued to execution. By predicting cache behavior in advance, the system can identify likely cache misses early and avoid issuing dependent instructions that would subsequently need to be rejected and replayed. This preliminary prediction action maintains high prediction accuracy while preventing the performance penalty and power overhead associated with reject-and-replay mechanisms.
Solution Approach 2:
The patent converts the potentially harmful effect of cache misses into a benefit by using the pattern history table and global history buffer to predict cache behavior. Instead of suffering performance penalties from unexpected cache misses, the system uses historical cache access patterns to anticipate misses and adjust instruction scheduling accordingly. This transforms the harmful cache miss behavior into useful predictive information, maintaining performance while preserving prediction accuracy.
3Device complexity
If simple predictors are used for cache prediction, then device complexity is reduced, but prediction accuracy deteriorates
Solution Approach 1:
The patent segments the prediction system into two distinct components: a pattern history table for capturing local cache access patterns and a global history buffer for tracking overall cache behavior. This segmentation allows each component to specialize in different aspects of cache prediction, improving overall accuracy without requiring a single complex predictor. The segmented structure maintains relatively simple device complexity while achieving high prediction accuracy through coordinated operation of multiple specialized units.
Solution Approach 2:
The patent merges the pattern history table and global history buffer into a unified prediction system. The pattern history table captures fine-grained local access patterns while the global history buffer provides broader context about cache behavior. By combining these two complementary prediction mechanisms, the system achieves high prediction accuracy without the complexity of a single monolithic predictor. The merged structure leverages the strengths of both components to resolve the contradiction between simplicity and accuracy.
Data Source
AI summary
In a decode stage of hardware processor pipeline, one particular instruction of a plurality of instructions is decoded. It is determined that the particular instruction requires a memory access. Responsive to such determination, it is predicted whether the memory access will result in a cache miss. The predicting in turn includes accessing one of a plurality of entries in a pattern history table stored as a hardware table in the decode stage. The accessing is based, at least in part, upon at least a most recent entry in a global history buffer. The pattern history table stores a plurality of predictions. The global history buffer stores actual results of previous memory accesses as one of cache hits and cache misses. Additional steps include scheduling at least one additional one of the plurality of instructions in accordance with the predicting; and updating the pattern history table and the global history buffer subsequent to actual execution of the particular instruction in an execution stage of the hardware processor pipeline, to reflect whether the predicting was accurate.