Weighted LRU Cache Replacement for Poor Locality Programs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current CPU cache replacement methods, such as LRU, are inefficient for programs with poor locality, leading to lower cache hit rates and increased processing time due to their inability to effectively manage cache lines with varying access frequencies.
Innovation Solution
The Weighted LRU (WLRU) method assigns weights to each cache line based on access frequency, incrementing weights for hits and decrementing for misses, allowing for dynamic replacement decisions to prioritize cache lines with lower weights, thereby improving cache management for programs with poor locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LRU cache replacement method is used, then recently referenced addresses are kept in cache, but cache hit rate decreases for programs with poor locality
Solution Approach 1:
The patent changes the parameter used for cache replacement from simple recency (LRU) to a composite parameter that includes both recency and frequency of access. Each cache line is assigned a weight that combines these two dimensions, allowing the system to adapt to different access patterns and improve hit rates for programs with poor locality.
Solution Approach 2:
The patent introduces dynamic weight adjustment for cache lines based on their access patterns. The weight of each cache line is updated dynamically - incremented on hits and decremented on misses - allowing the cache replacement policy to adapt to changing program behavior and improve performance for various types of workloads.
2Reliability
If cache size is increased, then cache hit rate increases, but CPU circuit technology limits cache size
Solution Approach 1:
The patent changes the replacement parameter from simple recency to a weighted metric that incorporates both recency and frequency. This allows more efficient utilization of the existing cache size by prioritizing cache lines that are both recently and frequently accessed, effectively increasing hit rate without increasing physical cache size.
3Productivity
If LRU keeps recently referenced addresses, then processing efficiency improves for programs with good locality, but performance decreases for programs with poor locality
Solution Approach 1:
The patent modifies the replacement parameter to include both recency and frequency components. This composite parameter allows the cache to efficiently serve programs with good locality (by maintaining recent accesses) while also improving performance for programs with poor locality (by prioritizing frequently accessed lines), thus enhancing overall adaptability.
Solution Approach 2:
The patent implements dynamic weight adjustment that adapts to different program types. For programs with good locality, the dynamic updates reinforce recent accesses. For programs with poor locality, the frequency component becomes more significant, allowing the cache to identify and retain valuable frequently accessed lines, thereby improving performance across diverse workloads.
Data Source
AI summary
Aspects of the invention relate to improvements to the Least Recently Used (LRU) cache replacement method. Weighted LRU (WLRU) and Compact Weighted LRU (CWLRU) are CPU cache replacement methods that have superior hit rates to LRU replacement for programs with poor locality, such as network protocols and applications. WLRU assigns weights to cache lines and makes replacement decision by comparing weights. When a cache line is first brought into the cache, it is assigned an initial weight. Weights of cache lines in WLRU increase when hit and decrease when not hit. Weights in WLRU also have upper limits, and the weight of a cache line never increases beyond the upper limit. CWLRU is a more space-efficient implementation of WLRU. Compared to WLRU, CWLRU uses fewer bits per cache line to store the weight.