Dual Request and Transaction Buffers for Low-Latency Cache Misses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache management systems face latency issues due to inefficient handling of cache misses, whether using Miss Status Handling Registers (MSHRs) or cache tags, as they either incur high latency or storage limitations during heavy miss events.
Innovation Solution
Implement a dual data structure system comprising a request buffer and a transaction buffer, where the request buffer handles cache misses with minimal latency and the transaction buffer indirectly tracks cache misses using pointers, allowing efficient management of multiple cache misses without the drawbacks of traditional MSHRs or cache-based tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If MSHR is used to handle cache misses, then cache miss handling efficiency is improved, but hardware cost and storage capacity are limited
Solution Approach 1:
The patent divides the cache miss tracking system into two separate data structures: a request buffer that stores incoming cache miss requests, and a transaction buffer that stores pointers to these requests. This segmentation allows the system to handle more cache misses than a single unified buffer could accommodate, resolving the contradiction between handling efficiency and storage capacity.
Solution Approach 2:
The transaction buffer acts as an intermediary between the request buffer and the cache memory. Instead of directly storing all cache miss information in one structure, the transaction buffer stores pointers that reference the request buffer, enabling indirect tracking of cache misses. This intermediary approach allows the system to track more misses without proportionally increasing hardware storage capacity.
2Device complexity
If cache tags are used to track cache misses, then separate data structures are eliminated, but latency increases during heavy miss events
Solution Approach 1:
The patent segments the cache miss tracking function across two specialized buffers rather than using a single general-purpose cache tag structure. The request buffer is optimized for receiving and holding miss requests, while the transaction buffer is optimized for pointer-based indirect tracking. This segmentation allows each buffer to be optimized for its specific function, reducing overall latency compared to using cache tags for all tracking purposes.
Solution Approach 2:
The request buffer pre-stores incoming cache miss requests before they are processed by the transaction buffer. This preliminary action allows the system to have requests ready and organized before the actual cache memory access occurs, reducing the overall latency of cache miss handling compared to using cache tags which would require sequential processing.
3Quantity of substance
If indirect tracking with pointers is used, then number of tracked misses is maximized, but device complexity increases
Solution Approach 1:
The transaction buffer serves as an intermediary layer that uses pointers to indirectly track cache misses in the request buffer. This indirect tracking mechanism allows the system to track a large number of misses without requiring proportionally large storage capacity in each buffer, as the transaction buffer entries are relatively small pointer structures. The complexity is localized to the pointer management in the transaction buffer, while the request buffer remains relatively simple.
Solution Approach 2:
Instead of duplicating full cache miss information in both buffers, the system uses pointer copying in the transaction buffer that references the request buffer. This copying approach allows the system to track many misses with minimal additional storage overhead, as each transaction buffer entry only needs to store a pointer rather than complete miss information. The complexity of managing these pointers is offset by the significant reduction in total storage requirements.
Data Source
AI summary
Methods and systems for cache miss monitoring and fulfillment are disclosed. A disclosed method comprises receiving, from a processing unit, a cache access request, determining, based on a cache failing to fulfill the cache access request, that a cache miss has occurred, populating a request buffer based on the cache miss and the cache access request, populating a transaction buffer with a transaction entry for the request buffer based on the cache miss request and the cache access request, determining by the request buffer, that information requested in the cache access request should be retrieved from main memory, determining, by the transaction buffer if information was not retrieved, that the cache access request satisfies criteria for creating a cache miss tag, creating the cache miss tag based on the cache access request, and storing the cache miss tag in a portion of the cache.