Coarse Grain Tag Cache Replacement Policy for Memory Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems face inefficiencies in tag cache replacement policies, particularly in two-level memory systems, where per-line replacement policies require large tag cache capacities and do not effectively improve hit rates, leading to increased latency and storage overhead.
Innovation Solution
Implementing a coarse grain tag cache replacement policy that groups tag and metadata information, using a SRAM structure to store group IDs and LRU states, reducing on-chip storage overhead and increasing the number of entries, while prefetching adjacent lines to reduce misses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If per-line replacement policy is used in tag cache, then tag cache capacity can be increased, but storage overhead increases and hit rate does not effectively improve
Solution Approach 1:
The patent merges multiple tag cache lines into a single group entry. Instead of managing individual line tags separately, the system combines tags for multiple memory lines into one unified tag cache entry, reducing the number of tag storage locations needed while maintaining effective cache management.
Solution Approach 2:
The group tag cache entry serves multiple functions simultaneously: it stores tags for multiple memory lines, tracks LRU state for the entire group, and enables collective replacement decisions. This multi-functionality reduces the overall storage overhead compared to dedicated per-line tag structures.
2Quantity of substance
If per-line replacement policy is used, then tag cache capacity increases, but latency increases due to larger cache structure
Solution Approach 1:
By merging multiple tag lines into single group entries, the patent reduces the total number of tag cache entries required. This consolidation decreases the time needed to search through tag entries during cache access, thereby reducing latency while maintaining adequate cache capacity.
3Loss of substance
If coarse grain group replacement is implemented, then storage overhead is reduced, but tag cache hit rate may decrease
Solution Approach 1:
The system performs preliminary actions by prefetching adjacent memory lines into the same group before they are actually needed. When a memory access occurs, the adjacent lines that are likely to be accessed soon are already in the cache group, increasing the probability of cache hits without requiring additional tag storage.
Solution Approach 2:
The patent implements LRU (Least Recently Used) feedback mechanisms at the group level to dynamically adjust which groups are retained or replaced. This feedback ensures that frequently accessed groups remain in the tag cache while less frequently accessed groups are replaced, maintaining high hit rates despite the coarse-grained approach.
Data Source
AI summary
An embodiment of a memory apparatus may include a tag cache to cache tag information, and a memory controller communicatively coupled to the tag cache to determine if a request for a memory line results in a tag cache miss, bring tag information for the missed memory line into the tag cache if the request results in a cache miss, and bring tag information for at least one additional memory line adjacent to the missed memory line into the tag cache if the request results in a cache miss. Additional embodiments are disclosed and claimed.