Speculative Code Fetch Throttling to Reduce Cache Eviction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Modern processors face performance losses due to the inefficient fetching of speculative code paths, leading to increased cache evictions and energy consumption, especially in large code footprint workloads.

Innovation Solution

Implementing a mechanism to throttle code fetches for speculative code paths by using Bloom filters to track code accesses and control the number of ongoing speculative fetches, disallowing further fetching when a threshold is reached, and allowing only non-speculative or less speculative code lines to proceed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If speculative code paths are fetched a priori to hide latency, then code fetch performance is improved, but cache eviction rate increases and energy consumption increases

Engineering Contradiction:

Improvecode fetch performanceVSAvoidenergy consumption

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The patent applies partial action by selectively fetching only non-speculative code paths while limiting speculative code fetches to a threshold number. The front-end processor identifies speculative versus non-speculative paths and restricts speculative fetching, thereby performing the fetch operation partially rather than universally, reducing unnecessary energy consumption while maintaining performance benefits for critical paths.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent changes the parameter of speculative code fetch limit by introducing a configurable threshold value. When the number of outstanding speculative code fetches exceeds this threshold, additional speculative fetches are disallowed. This parameter adjustment allows the system to optimize between performance and energy consumption based on workload characteristics.

Inventive Principle:

Principle #35Parameter changes

2Speed

If speculative code paths are fetched a priori to hide latency, then code fetch performance is improved, but cache eviction rate increases

Engineering Contradiction:

Improvecode fetch performanceVSAvoidcache eviction rate

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent applies partial action by selectively fetching only non-speculative code paths while limiting speculative code fetches to a threshold number. The front-end processor identifies speculative versus non-speculative paths and restricts speculative fetching, thereby performing the fetch operation partially rather than universally, reducing unnecessary cache pollution while maintaining performance benefits for critical paths.

Inventive Principle:

Principle #16Partial or excessive action

3Loss of energy

If code fetch threshold is enforced to reduce speculative fetching, then energy consumption is reduced, but code fetch performance may deteriorate

Engineering Contradiction:

Improveenergy consumptionVSAvoidcode fetch performance

Core Design Contradiction:

Loss of energyVSProductivity

Solution Approach 1:

The patent applies local quality by differentiating between speculative and non-speculative code paths and applying different fetch policies to each. Non-speculative paths are fetched without restriction to maintain performance, while speculative paths are subject to threshold limits to reduce energy consumption. This localized differentiation ensures that performance-critical code is fetched aggressively while energy-intensive speculative fetching is curtailed.

Inventive Principle:

Principle #3Local quality

4Quantity of substance

If code fetch threshold is enforced to reduce speculative fetching, then cache eviction rate is reduced, but code fetch performance may deteriorate

Engineering Contradiction:

Improvecache eviction rateVSAvoidcode fetch performance

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The patent applies local quality by differentiating between speculative and non-speculative code paths and applying different fetch policies to each. Non-speculative paths are fetched without restriction to maintain performance, while speculative paths are subject to threshold limits to reduce cache pollution. This localized differentiation ensures that performance-critical code is fetched aggressively while cache-polluting speculative fetching is curtailed.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12360768B2Throttling code fetch for speculative code paths

Publication Date: 2025.07.15 INTEL CORP

AI summary

Methods and apparatus relating to throttling a code fetch for speculative code paths are described. In an embodiment, a first storage structure stores a reference to a code line in response to a request to be received from a cache. A second storage structure to store a reference to the code line in response to an update to an Instruction Dispatch Queue (IDQ). Logic circuitry controls additional code line fetch operations based at least in part on a comparison of a number of ongoing speculative code fetches and a determination that the code line is speculative. Other embodiments are also disclosed and claimed.