Adaptive Prefetch Engine Page Boundary Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data prefetch engines experience performance impairments and increased latency when data streams cross page boundaries, leading to cache misses and re-ramping of prefetches, as they typically use conservative startup profiles that only prefetch within a real page and do not aggressively anticipate subsequent cache lines across page boundaries.
Innovation Solution
Implementing an aggressive startup profile for the prefetch engine when a program accesses a cache line at a new page boundary, tracking historical page boundary percentages to set an aggressive profile flag, and generating prefetch requests accordingly, which minimizes latency for long data streams while switching to a normal profile for short streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a conservative startup profile is used for prefetching at page boundaries, then implementation complexity is reduced and safety is improved, but prefetching performance deteriorates and latency increases
Solution Approach 1:
The patent applies dynamics by making the prefetch startup profile adaptive rather than static. The system dynamically switches between conservative and aggressive profiles based on runtime detection of stream continuity across page boundaries. When a stream is detected to continue across a page boundary, the system transitions to an aggressive profile that immediately prefetches multiple cache lines, otherwise it maintains a conservative profile. This dynamic adaptation resolves the contradiction by selecting the appropriate profile based on actual workload characteristics.
Solution Approach 2:
The patent changes the parameter of prefetch aggressiveness based on detected patterns. By monitoring whether data streams continue across page boundaries, the system adjusts the prefetch behavior parameter - using conservative prefetching when streams are short or discontinuous, and aggressive prefetching when streams are long and continuous. This parameter change allows the system to optimize for either safety or performance depending on the actual data access patterns.
2Device complexity
If prefetching terminates at page boundaries, then implementation simplicity is improved, but productivity deteriorates due to cache misses and re-ramping
Solution Approach 1:
The patent applies preliminary action by detecting page boundary crossings in advance and proactively continuing prefetch operations into the next page. Instead of terminating prefetching at page boundaries, the system identifies when a data stream is likely to continue across a boundary and preemptively prefetches the subsequent cache lines in the new page. This preliminary action prevents cache misses and eliminates the need for re-ramping, thereby maintaining high productivity without significantly increasing complexity.
3Productivity
If aggressive prefetching is used across page boundaries, then productivity is improved by reducing cache misses, but device complexity increases due to tracking requirements
Solution Approach 1:
The patent applies local quality by applying aggressive prefetching only locally at specific locations where data streams cross page boundaries, rather than universally throughout the prefetch engine. The system maintains simple conservative prefetching for most cases but activates aggressive prefetching with full tracking capabilities only when a page boundary crossing is detected. This localized application of complexity resolves the contradiction by limiting the increased device complexity to only the necessary contexts where it provides productivity benefits.
Data Source
AI summary
A system and method for improving the page crossing performance of a data prefetcher is presented. A prefetch engine tracks times at which a data stream terminates due to a page boundary. When a certain percentage of data streams terminate at page boundaries, the prefetch engine sets an aggressive profile flag. In turn, when the data prefetch engine receives a real address that corresponds to the beginning/end of a new page, and the aggressive profile flag is set, the prefetch engine uses an aggressive startup profile to generate and schedule prefetches on the assumption that the real address is highly likely to be the continuation of a long data stream. As a result, the system and method minimize latency when crossing real page boundaries when a program is predominately accessing long streams.