Low-Latency Cache with Staged LRU Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The implementation of the least recently used (LRU) replacement policy in cache systems is hindered by the time-consuming process of reading LRU data, leading to increased latency in cache transactions.
Innovation Solution
A cache system is designed with multiple stages for LRU replacement policy implementation, including separate storage of tag and LRU data in different memory devices, allowing early determination of HIT/MISS results and reducing latency by initiating cache data retrieval before completing LRU replacement computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the cache system reads LRU data to implement the LRU replacement policy, then the replacement policy can be accurately executed, but the cache transaction latency increases due to the time-consuming reading process
Solution Approach 1:
The patent segments the cache transaction process into multiple independent stages: tag data reading stage, LRU data reading stage, and replacement computation stage. By separating these operations into distinct stages that can execute in parallel, the system reduces the total time required for cache transactions while maintaining accurate LRU replacement policy execution.
Solution Approach 2:
The patent implements preliminary action by reading tag data and making hit/miss determinations before completing the LRU data reading and replacement computation stages. This allows the cache system to prepare results in advance and reduce overall transaction latency while ensuring accurate replacement decisions are made after all necessary data is read.
2Reliability
If the cache system waits for LRU data reading to complete before providing cache data, then the replacement computation can be accurate, but the time efficiency of cache transactions decreases
Solution Approach 1:
The patent divides the cache transaction into segmented stages where tag data reading and hit/miss determination occur in parallel with LRU data reading. This segmentation enables the system to provide cache data earlier without sacrificing replacement computation accuracy, as each stage completes its necessary operations independently.
Solution Approach 2:
The patent maintains continuity of useful action by overlapping multiple operations that would traditionally execute sequentially. The tag data reading, hit/miss determination, and LRU data reading occur in parallel, keeping the cache system productive throughout the transaction process rather than having idle waiting periods.
3Device complexity
If tag data and LRU data are stored in the same memory device, then the device complexity is reduced, but the cache transaction latency increases due to sequential access requirements
Solution Approach 1:
The patent segments the storage of tag data and LRU data into separate memory devices, allowing independent and parallel access to both data types. This segmentation eliminates the sequential access constraint and reduces cache transaction latency, while the overall system complexity remains manageable through modular memory device design.
Data Source
AI summary
A cache includes multiple sets with each set having multiple respective ways, and replacement logic configured to implement an LRU replacement policy based on an LRU replacement computation in multiple stages for a transaction. The multiple stages include: a first stage in which the cache reads tag data for the transaction and makes a hit determination based on the tag data, a second stage in which the cache reads LRU data for the transaction, and a third stage in which the cache performs an LRU replacement computation. If the hit determination is a hit, the cache is configured to provide the resulting cache data before the third stage is complete.