Merging Demand and Prefetch Load Requests in Cache Queues

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In computing systems, demand load requests often experience increased latency due to being queued behind prefetch load requests, as they must wait for cache line fills to complete, leading to inefficiencies in data retrieval.

Innovation Solution

A central request queue merges prefetch and demand load requests, promoting them as a single request to prioritize and expedite data delivery to processing units, thereby reducing latency and optimizing cache operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If prefetch load requests are issued to pre-load cache lines, then cache hit rate is improved, but demand load requests experience increased latency due to being queued behind prefetch requests

Engineering Contradiction:

Improvecache hit rateVSAvoiddemand load request latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent merges a demand load request with a pending prefetch load request for the same cache line into a single promoted load request. This combination allows the system to maintain the prefetch optimization while ensuring the demand request receives priority service, eliminating the queuing delay that would otherwise occur when demand requests are processed after prefetch requests.

Inventive Principle:

Principle #5Merging (Combining)

2Loss of time

If demand load requests are processed before prefetch load requests, then demand request latency is reduced, but prefetch efficiency is degraded and cache lines may not be loaded in advance

Engineering Contradiction:

Improvedemand load request latencyVSAvoidprefetch efficiency

Core Design Contradiction:

Loss of timeVSProductivity

Solution Approach 1:

The system performs preliminary identification of demand load requests that coincide with pending prefetch requests. By detecting this overlap in advance and promoting the demand request before the prefetch completes, the system ensures demand requests are serviced with minimal delay while the prefetch mechanism continues to operate efficiently for other cache lines.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If prefetch load requests are issued, then future cache misses are reduced, but power consumption increases due to additional cache lookups and data re-retrievals

Engineering Contradiction:

Improvecache miss reductionVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts and eliminates redundant operations by merging demand and prefetch requests. When a demand request is promoted from a prefetch request, the system avoids issuing separate load operations, thereby reducing the number of cache lookups and data re-retrievals. This directly reduces power consumption while maintaining the cache hit rate benefits of prefetching.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS9286223B2Merging demand load requests with prefetch load requests

Publication Date: 2016.03.15 ADVANCED MICRO DEVICES INC

AI summary

A processor includes a processing unit, a cache memory, and a central request queue. The central request queue is operable to receive a prefetch load request for a cache line to be loaded into the cache memory, receive a demand load request for the cache line from the processing unit, merge the prefetch load request and the demand load request to generate a promoted load request specifying the processing unit as a requestor, receive the cache line associated with the promoted load request, and forward the cache line to the processing unit.