Variable Distance Bypass for Cache Data Array Lookup Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cache systems experience delays in data array lookups due to sequential execution with tag array lookups, leading to reduced throughput, especially when data array pipeline resources are not available immediately.
Innovation Solution
Implementing a variable distance bypass mechanism that allows data array lookup requests to be picked for execution opportunistically based on resource availability in the data array pipeline, reducing the fixed delay between tag array and data array lookups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data array lookup is performed sequentially after tag array lookup, then resource conflicts are avoided, but lookup latency increases significantly
Solution Approach 1:
The patent prepares data array lookup requests in advance by capturing tag array lookup results and placing them in a request buffer before the data array pipeline is ready, so that when resources become available, the lookup can immediately proceed without waiting for tag array completion
Solution Approach 2:
The patent introduces a request buffer as an intermediary component between the tag array pipeline and data array pipeline, which stores prepared lookup requests and coordinates their execution timing to resolve resource conflicts while minimizing latency
2Productivity
If data array lookup is performed in parallel with tag array lookup, then throughput improves, but resource conflicts occur when data array pipeline resources are unavailable
Solution Approach 1:
The patent implements dynamic parallel execution by allowing data array lookups to be initiated at variable distances behind tag array lookups based on real-time resource availability, rather than using a fixed sequential or parallel timing scheme
Solution Approach 2:
The patent maintains continuous pipeline utilization by keeping the data array pipeline busy with prepared requests from the request buffer, ensuring that resources are continuously productive without idle cycles while managing resource conflicts
3Device complexity
If fixed delay is used between tag array and data array lookups, then pipeline timing is simplified, but throughput is reduced due to unnecessary waiting
Solution Approach 1:
The patent performs data array lookup preparation in advance and stores it in a request buffer, allowing the actual execution to start immediately when resources are available rather than waiting for a fixed delay period to elapse
Solution Approach 2:
The patent changes the timing parameter from a fixed delay to a variable delay that adapts based on data array pipeline resource availability, optimizing throughput while maintaining manageable pipeline complexity through the request buffer mechanism
Data Source
AI summary
The present application describes embodiments of techniques for picking a data array lookup request for execution in a data array pipeline a variable number of cycles behind a corresponding tag array lookup request that is concurrently executing in a tag array pipeline. Some embodiments of a method for picking the data array lookup request include picking the data array lookup request for execution in a data array pipeline of a cache concurrently with execution of a tag array lookup request in a tag array pipeline of the cache. The data array lookup request is picked for execution in response to resources of the data array pipeline becoming available after picking the tag array lookup request for execution. Some embodiments of the method may be implemented in a cache.