Instruction Fetch Queue Throttling for Cache Pollution Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in efficiently managing instruction fetch queues to optimize performance and reduce power consumption, particularly in scenarios where prediction accuracy is uncertain.
Innovation Solution
The implementation of an instruction fetch queue with prefetching circuitry and control circuitry that suppresses allocation of new entries when the queue reaches a predetermined limit, dynamically adjusting this limit based on prediction accuracy to mitigate misprediction risks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the instruction fetch queue is fully populated with predicted instructions, then the latency of accessing instructions is decreased, but the power consumption and cache/memory traffic increase due to unnecessary prefetching
Solution Approach 1:
The patent implements a dynamic limit value that adjusts the maximum number of instruction fetch entries based on prediction accuracy. When prediction accuracy is low, the limit value is reduced to prevent unnecessary prefetching and save power. When prediction accuracy is high, the limit value is increased to maximize performance benefits. This dynamic adjustment resolves the contradiction by adapting the prefetching behavior to current prediction reliability.
Solution Approach 2:
The patent changes the parameter of instruction fetch queue size by introducing a limit value that is dynamically adjusted based on prediction accuracy metrics. This parameter change allows the system to optimize between full queue utilization for performance and reduced queue size for power savings, directly addressing the contradiction between instruction access latency and power consumption.
2Loss of time
If the instruction fetch queue is fully populated with predicted instructions, then the latency of accessing instructions is decreased, but unnecessary instructions are cached causing increased cache/memory traffic
Solution Approach 1:
The dynamic limit value adjusts the instruction fetch queue size based on prediction accuracy, preventing the queue from being fully populated when predictions are unreliable. This reduces unnecessary cache/memory traffic while maintaining performance benefits when predictions are accurate, resolving the contradiction between reduced access latency and reduced energy loss.
3Productivity
If the limit value is set to the maximum number of entries, then the instruction cache is fully utilized, but misprediction risks increase causing performance degradation
Solution Approach 1:
The patent applies preliminary anti-action by proactively limiting the instruction fetch queue size based on predicted prediction accuracy before mispredictions occur. When prediction accuracy is low, the limit value is reduced in advance to prevent mispredictions and their harmful effects, such as pipeline flushes and performance degradation. This resolves the contradiction by prioritizing reliability over maximum throughput when predictions are unreliable.
4Loss of time
If prefetching is continuously performed without limits, then the instruction cache is pre-populated with predicted instructions, but power consumption increases
Solution Approach 1:
The patent implements periodic action by continuously monitoring prediction accuracy and dynamically adjusting the limit value accordingly. Prefetching is performed periodically with adjusted intensity based on current prediction reliability, rather than continuously at full capacity. This resolves the contradiction by enabling prefetching to occur in periods when prediction accuracy is high while reducing or suspending prefetching when accuracy is low, optimizing both performance and power consumption.
Data Source
AI summary
An apparatus is provided for limiting the effective utilisation of an instruction fetch queue. The instruction fetch entries are used to control the prefetching of instructions from memory, such that those instructions are stored in an instruction cache prior to being required by execution circuitry while executing a program. By limiting the effective utilisation of the instruction fetch queue, fewer instructions will be prefetched and fewer instructions will be allocated to the instruction cache, thus causing fewer evictions from the instruction cache. In the event that the instruction fetch entries are for instructions that are unnecessary to the program, the pollution of the instruction cache with these unnecessary instructions can be mitigated.