Cache Way Pre-Allocation for Pipelined Miss Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance cache designs face pipeline stalls due to limitations in the number of outstanding allocate requests to the same set in a cache, which restricts the processing of multiple requests and introduces unnecessary delays.
Innovation Solution
The solution involves pre-allocating destination ways for read allocate requests based on the least recently used (LRU) indication, allowing multiple outstanding requests to the same set and updating the LRU values immediately to facilitate continuous processing without stalls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of outstanding allocate requests to the same set is limited to the number of ways in the set, then the cache structure remains simple and manageable, but pipeline stalls occur when the limit is reached, reducing processing throughput
Solution Approach 1:
The patent pre-allocates destination ways for allocate requests before the actual allocation occurs. When an allocate request is received, a destination way is pre-selected and reserved, allowing the request to proceed without waiting for other requests to complete. This preliminary action eliminates pipeline stalls while maintaining manageable complexity through structured pre-allocation logic.
Solution Approach 2:
The patent segments the cache allocation process into distinct phases: pre-allocation of destination way, actual allocation when data returns, and LRU tracking updates. This segmentation allows multiple requests to be in different phases simultaneously, enabling parallel processing without contention. Each phase is independently managed, reducing overall control complexity while improving throughput.
2Loss of time
If multiple outstanding allocate requests to the same set are allowed, then pipeline stalls are eliminated and processing continues uninterrupted, but tracking and managing these requests becomes more complex
Solution Approach 1:
The patent implements LRU (Least Recently Used) indication tracking for each way in every set. This feedback mechanism continuously monitors which ways are least recently used and uses this information for pre-allocation decisions. The LRU feedback allows the system to intelligently manage multiple outstanding requests by always allocating to the least recently used way, preventing conflicts while maintaining simple tracking logic.
Solution Approach 2:
The patent changes the state parameter of each way by maintaining LRU indications that are updated as requests are pre-allocated and completed. This parameter change (LRU state) dynamically guides the pre-allocation process, allowing the cache to adapt to access patterns. By tracking LRU parameters rather than maintaining complex request queues, the system manages multiple outstanding requests efficiently.
3Productivity
If pre-allocation of ways is implemented for read allocate requests, then unnecessary pipeline stalls are removed and performance improves, but the cache control logic becomes more complex
Solution Approach 1:
The patent performs pre-allocation of destination ways as a preliminary action when allocate requests are received. By selecting and reserving destination ways in advance, the system eliminates the need for complex runtime arbitration and prevents pipeline stalls. The pre-allocation logic, while adding some complexity, uses simple LRU-based selection that keeps the overall control logic manageable.
Solution Approach 2:
The patent introduces dynamic pre-allocation where destination ways are selected based on current LRU states rather than fixed allocation schemes. This dynamic approach adapts to changing access patterns and optimizes performance. The dynamic nature is managed through simple LRU bit updates rather than complex control logic, achieving high performance with minimal added complexity.
Data Source
AI summary
This invention is a data processing system with a data cache. The cache controller responds to a cache miss requiring allocation by pre-allocating a way in the set to an allocation request according to said least recently used indication of said ways and then update the least recently used indication of remaining ways of the set. This permits read allocate requests to the same set to proceed without introducing processing stalls due to way contention. This also allows multiple outstanding allocate requests to the same set and way combination. The cache also compares the address of a newly received allocation request to stall this allocation request if the address matches an address of any pending allocation request.