Instruction Prefetch Throttling for Cache-Stable Fetch Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face inefficiencies due to high bandwidth usage and wasted power from unconstrained instruction fetching, leading to cache thrashing and redundant fetching of instructions, especially in the presence of branch mispredictions.
Innovation Solution
Implementing an instruction fetch queue with prefetching circuitry that issues requests based on instruction storage availability and a variable throttling threshold derived from properties of previously fetched blocks, limiting outstanding requests to optimize fetching behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If prefetching circuitry issues requests for all instruction fetch entries without constraint, then instruction availability is improved, but bandwidth usage and power consumption increase excessively
Solution Approach 1:
The patent implements a dynamic throttling threshold that adjusts the number of outstanding prefetch requests based on system conditions. The control circuitry modifies the threshold value to balance between fetching instructions proactively (reducing latency) and limiting bandwidth/power usage, resolving the contradiction between instruction availability and energy consumption
Solution Approach 2:
The control circuitry monitors the number of outstanding prefetch requests and uses this feedback to dynamically adjust the throttling threshold. When too many requests are outstanding, the threshold is reduced to limit bandwidth usage; when fewer requests are outstanding, the threshold can be increased to improve instruction availability, creating a self-regulating system that balances both objectives
2Loss of time
If prefetching circuitry issues requests for all instruction fetch entries without constraint, then instruction availability is improved, but cache thrashing occurs
Solution Approach 1:
The dynamic throttling threshold prevents cache thrashing by adaptively limiting the number of outstanding prefetch requests. When the threshold is reduced, it prevents too many instructions from being fetched simultaneously, which would cause cache evictions and thrashing. When increased, it allows sufficient prefetching to maintain instruction availability without crossing into thrashing territory
Solution Approach 2:
The control circuitry proactively limits prefetch requests before cache thrashing can occur by monitoring outstanding requests and adjusting the threshold preemptively. This preliminary anti-action prevents the harmful effect of cache thrashing before it manifests, while still allowing beneficial prefetching to occur
3Productivity
If variable throttling threshold is derived from properties of previously fetched blocks, then fetching efficiency is improved, but control circuitry complexity increases
Solution Approach 1:
The control circuitry derives the throttling threshold automatically from properties of previously fetched instruction blocks without requiring external intervention or complex configuration. The system self-regulates by analyzing its own prefetch history and adjusting the threshold accordingly, improving fetching efficiency while keeping the control logic self-contained and relatively simple
Data Source
AI summary
There is provided an apparatus, a method, a chip containing product, a system and a medium. The apparatus is provided with an instruction fetch queue for storage of instruction fetch entries, and prefetching circuitry configured, for each instruction fetch entry and in response to a determination that a block of instructions indicated by that instruction fetch entry is not present in instruction storage circuitry, to issue a request to fetch the block and to cause the block to be allocated to the instruction storage circuitry. The apparatus is also provided with control circuitry configured to suppress issuing the request in response to a number of outstanding requests meeting or exceeding a variable throttling threshold. The control circuitry is configured to derive the variable throttling threshold from properties of previously fetched blocks, the properties determined before or during storage of the previously fetched blocks in the instruction storage circuitry.