Hardware Prefetch Boundary Hints to Limit Cache Pollution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional prefetching techniques result in cache pollution and inefficient memory access due to unnecessary prefetches, leading to performance degradation and bandwidth saturation, as they fail to accurately determine when to stop prefetching based on stride access patterns.

Innovation Solution

Implement throttling logic in hardware prefetchers that uses software-provided boundary hints to control prefetch requests, setting access boundaries based on operation code information to minimize unnecessary prefetches.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If hardware prefetchers continuously prefetch data based on stride access patterns, then memory access speed is improved, but cache pollution increases and bandwidth is saturated

Engineering Contradiction:

Improvememory access speedVSAvoidcache pollution

Core Design Contradiction:

SpeedVSObject-generated harmful factors

Solution Approach 1:

The system uses feedback from boundary condition instructions to dynamically adjust prefetch behavior. The throttling logic monitors whether accessed memory addresses satisfy the boundary conditions specified in software-provided hints, and uses this feedback to enable or disable throttling of prefetch requests, thereby preventing cache pollution while maintaining performance

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The prefetch throttling mechanism is made dynamic by allowing software to provide boundary hints that can change during program execution. The system transitions from static prefetching to adaptive prefetching where the throttling behavior adjusts based on runtime boundary condition satisfaction, enabling precise control over when prefetching should occur

Inventive Principle:

Principle #15Dynamics

2Speed

If hardware prefetchers continuously prefetch data based on stride access patterns, then memory access speed is improved, but memory bandwidth is saturated

Engineering Contradiction:

Improvememory access speedVSAvoidmemory bandwidth

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The boundary condition feedback mechanism allows the system to monitor actual memory access patterns against software-defined boundaries. When addresses satisfy the boundary conditions, throttling is disabled and prefetching proceeds normally; when boundaries are exceeded, throttling enables and reduces prefetch traffic, thereby preventing bandwidth saturation

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system applies partial throttling rather than complete shutdown of prefetching. By selectively enabling throttling only when boundary conditions are violated, the system maintains sufficient prefetching for valid accesses while reducing excessive prefetches that would saturate bandwidth

Inventive Principle:

Principle #16Partial or excessive action

3Object-generated harmful factors

If conventional prefetchers use confidence counters to throttle prefetch requests, then some cache pollution is reduced, but incorrect prefetches still occur in loops with varying locations or counts

Engineering Contradiction:

Improvecache pollutionVSAvoidprefetch accuracy

Core Design Contradiction:

Object-generated harmful factorsVSReliability

Solution Approach 1:

The system introduces software-provided boundary condition hints as an intermediary mechanism between the hardware prefetcher and the workload. These hints act as precise mediators that convey semantic information about memory access boundaries, enabling more accurate throttling decisions compared to generic confidence counters

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the parameter used for throttling decisions from confidence counter values to boundary condition satisfaction status. This parameter change allows the system to distinguish between legitimate prefetches within boundaries and excessive prefetches beyond boundaries, improving both accuracy and pollution reduction

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20260003792A1Software-Guided Prefetch Throttling based on Memory Region Boundaries

Publication Date: 2026.01.01 ADVANCED MICRO DEVICES INC

AI summary

Systems and techniques for software-guided prefetch throttling based on memory region boundaries are described. In one example, a processor includes a cache system having a cache system and a hardware prefetcher associated with a cache level of the cache system. The hardware prefetcher receives a boundary hint from a workload of an execution unit that accesses the cache level. The hardware prefetcher throttles prefetch requests based on the memory hint being satisfied. The described techniques overcome cache pollution from conventional prefetchers without limiting a prefetcher's ability to identify and respond to stride access patterns.