Vector Prefetch Unit for Cache Miss Consolidation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Out-of-order processors experience frequent data cache misses due to aggressive instruction issuance, leading to increased memory latency, which is exacerbated by deep prefetchers producing numerous outstanding cache misses that overwhelm the interconnect and memory systems.
Innovation Solution
Implement vector prefetching, where a missing address storage unit identifies relationships between prefetch requests and generates a single vector prefetch request that includes a starting cache block and the number of related blocks, reducing the number of requests sent over the interconnection network and improving processor performance by merging non-issued related prefetch requests into a single command.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If deep prefetchers are used to reduce memory latency, then memory access speed is improved, but the number of outstanding cache misses increases overwhelming the interconnect and memory systems
Solution Approach 1:
The patent merges multiple individual prefetch requests into a single vector prefetch request that can retrieve multiple cache blocks simultaneously. The vector prefetch unit combines several outstanding cache misses into one consolidated request sent to the interconnect, reducing the total number of requests while maintaining the ability to fetch multiple data blocks needed by the processor.
2Reliability
If multiple individual prefetch requests are sent to handle outstanding cache misses, then cache hit rate is improved, but command bandwidth utilization and interconnect congestion increase
Solution Approach 1:
The vector prefetch unit consolidates multiple individual prefetch requests into a single vector request that traverses the interconnect. This merging approach maintains the ability to service multiple cache misses and improve cache hit rates while significantly reducing interconnect congestion and command bandwidth utilization by sending one request instead of many separate requests.
3Productivity
If numerous prefetch requests are issued to reduce memory latency, then processor throughput is improved, but the memory system's command handling capacity is exceeded
Solution Approach 1:
The patent merges multiple prefetch requests into a single vector prefetch request that can service multiple outstanding cache misses. This approach maintains processor throughput by ensuring all needed data is fetched while reducing command bandwidth overload by decreasing the total number of requests sent to the memory system.
Solution Approach 2:
The vector prefetch request serves multiple functions simultaneously - it addresses multiple outstanding cache misses, fetches multiple cache blocks, and maintains processor throughput all in a single request. This multi-functionality allows the system to maintain high productivity without exceeding memory system capacity.
Data Source
AI summary
Described is a computing system for vector prefetching which includes a hierarchical memory including multiple caches, a missing address storage unit (MASU) associated with each cache which stores prefetch requests suffering a cache miss, a prefetcher which sends prefetch requests towards the hierarchical memory, and a vector prefetch unit. The vector prefetch unit determines existence of at least one of a relationship between a cache block associated with the prefetch request and cache blocks associated with one or more entries in a MASU, or a relationship between cache blocks associated with different entries in a MASU, and sends a vector prefetch request based on related prefetch requests including indicators indicating a starting cache block and a number of related cache blocks to a higher memory level to obtain data associated with each cache block. The hierarchical memory stores the data received in at least one response message from the higher memory level if available.