Cache Replacement States for Prefetch and Non-Temporal Loads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache replacement policies, such as LRU and PLRU, lack flexibility in handling different types of memory accesses, including demand, prefetch, and non-temporal loads, leading to inefficient data management in caches.
Innovation Solution
Implementing a cache replacement policy that tracks more cache states than the number of ways in a cache, using additional states for prefetch and non-temporal loads, with a 3-bit field to manage 8 states per way, including MRU, MRU-1, LRU+1, MRP, LRP, MRNT, and LRNT states, and enforcing specific promotion and eviction rules for these states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional cache replacement policies (LRU, PLRU) are used, then the cache structure is simple and easy to implement, but the cache cannot differentiate between different types of memory accesses (demand, prefetch, non-temporal loads)
Solution Approach 1:
The cache state space is segmented into multiple distinct states: demand states (MRU, MRU-1, LRU+1, LRU) and non-temporal states (MRNT, LRNT). This segmentation allows the cache to differentiate between demand loads and non-temporal loads, enabling specialized replacement policies for each access type while maintaining a manageable state structure through systematic organization of the 3-bit state field.
Solution Approach 2:
The 3-bit cache state field serves multiple functions simultaneously: it tracks LRU ordering for demand loads, identifies non-temporal load states, and provides replacement policy information. This multi-functionality allows the cache to handle different access types with enhanced flexibility without proportionally increasing hardware complexity, as the same state bits encode multiple pieces of information.
2Adaptability or versatility
If more cache states are tracked (8 states per way), then prefetch and non-temporal loads can be handled differently, but the cache state field requires more bits (3 bits per way)
Solution Approach 1:
The cache state field is extended from 2 bits to 3 bits, adding another dimension to the state space. This additional bit enables the cache to represent 8 distinct states instead of 4, allowing differentiation between demand and non-temporal loads while maintaining a compact representation. The extra dimension in state space provides the necessary granularity without excessive overhead.
3Productivity
If traditional LRU replacement is used, then the replacement logic is simple, but prefetch data and non-temporal data are evicted based on the same criteria as demand data, reducing cache efficiency
Solution Approach 1:
Different replacement policies are applied to different cache states based on their local characteristics. Demand loads use LRU replacement among demand states (MRU, MRU-1, LRU+1, LRU), while non-temporal loads use a separate replacement mechanism among non-temporal states (MRNT, LRNT). This local differentiation ensures that each access type is managed according to its specific access patterns and importance, improving overall cache efficiency.
Solution Approach 2:
The extended cache state field acts as an intermediary that carries information about access type and recency. By encoding both the LRU ordering and the access type (demand vs. non-temporal) in the state bits, the system enables specialized replacement logic without requiring separate complex tracking structures. The state field mediates between the diverse access types and the replacement decision logic.
Data Source
AI summary
Described are methods, data structures, logic, and circuitry which enable a cache replacement policy to track a number of cache states greater than a number of ways in a cache. A cache replacement policy state structure provides that the number of cache states is greater than the N ways for each way per cache index in the cache The additional states can enable different handling and prioritization of data that has been prefetched (prefetch cache states) or loaded with a non-temporal hint from software (non-temporal cache states). The number of prefetcher states and non-temporal states are each less than the number of ways. The non-temporal states can only be promoted to certain of the cache states.