Candidate Prediction Buffer for High-Bandwidth Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processing apparatuses face challenges in achieving high bandwidth due to the trade-off between the size and speed of prediction caches, leading to increased latency and reduced throughput, as smaller prediction caches result in higher miss rates and lower hit rates.
Innovation Solution
A data processing apparatus comprising a prediction cache, prediction circuitry, and a candidate prediction buffer, where the candidate prediction buffer stores candidate predictions that can be used to supplement the prediction cache, allowing for more accurate predictions by including entries not present in the cache due to size constraints, thereby reducing latency and increasing hit rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the prediction cache size is reduced to decrease latency, then the speed of prediction is improved, but the hit rate decreases due to fewer stored entries
Solution Approach 1:
The prediction cache is divided into two segments: a smaller primary prediction cache that provides fast access with low latency, and a larger secondary prediction cache that stores more entries to achieve higher hit rates. The system segments the cache functionality to simultaneously achieve both speed and reliability requirements.
Solution Approach 2:
The smaller prediction cache is nested within or alongside the larger prediction cache structure. When a miss occurs in the smaller cache, the system automatically accesses the larger cache, creating a nested hierarchy where the smaller cache provides fast path access while the larger cache provides comprehensive coverage.
2Reliability
If the prediction cache size is increased to store more entries and improve hit rate, then the reliability is improved, but the latency increases due to larger cache access time
Solution Approach 1:
The prediction cache is divided into two segments: a smaller primary prediction cache that provides fast access with low latency, and a larger secondary prediction cache that stores more entries to achieve higher hit rates. The system segments the cache functionality to simultaneously achieve both speed and reliability requirements.
Solution Approach 2:
The system performs preliminary action by maintaining a smaller, faster prediction cache that is pre-filled with the most frequently accessed prediction entries. This allows the system to handle common prediction cases with minimal latency, while the larger cache serves as a backup for less frequent cases.
3Reliability
If a hierarchical prediction cache structure is implemented with a main prediction cache to handle misses, then the hit rate is improved, but the throughput decreases due to multiple clock cycles required for miss handling
Solution Approach 1:
The system performs preliminary action by pre-loading candidate prediction entries into the smaller prediction cache before they are needed. This proactive approach ensures that entries likely to be accessed are already in the fast cache, reducing the frequency of misses and the need to access the main cache, thereby maintaining high throughput.
Solution Approach 2:
The system implements a fast path mechanism that allows the prediction unit to skip the main cache access entirely when the smaller cache contains the required entry. This skipping mechanism rushes through the prediction process in a single cycle for hit cases, avoiding the multi-cycle penalty that would otherwise occur with hierarchical cache access.
Data Source
AI summary
A data processing apparatus and a method are disclosed. The data processing apparatus comprising: a prediction cache to store a plurality of prediction entries, each defining an association between a prediction cache lookup address and a predicted behaviour; prediction circuitry to select a prediction entry based on a prediction cache lookup of the prediction cache based on a given prediction cache lookup address and to determine the predicted behaviour associated with the given prediction cache lookup address based on the selected prediction entry; and a candidate prediction buffer to store a plurality of candidate predictions each indicative of a candidate prediction entry to be selected for inclusion in a subsequent prediction cache lookup, wherein the candidate prediction entry is selected in response to a candidate prediction lookup based on a candidate lookup address different to a candidate prediction cache lookup address indicated as associated with a candidate predicted behaviour in the candidate prediction entry.