Probabilistic Cache Tag Lookup Using Home Directory Entry

Solution Overview

Problem

Probabilistic-2way (P2w) caches in High Bandwidth Memory (HBM) configurations face significant bandwidth and miss-latency overheads, making them unattractive for productization due to increased conflicts in direct-mapped caches.

Innovation Solution

Incorporating an extra tag as a 'home directory entry' to determine immediately whether a cache line is present in the correct or wrong way, eliminating the need for additional lookups and dynamically configuring between direct-mapped and P2w caches at runtime.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If P2w cache is used to alleviate conflicts in direct-mapped cache, then cache conflict resolution is improved, but bandwidth overhead and miss-latency increase significantly

Engineering Contradiction:

Improvecache conflict resolutionVSAvoidbandwidth overhead

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent segments the tag comparison process into two independent parts: primary tag comparison and secondary tag comparison. This segmentation allows the cache system to quickly determine cache hits using the primary tag without always requiring the full P2w lookup process, thereby reducing bandwidth overhead while maintaining conflict resolution capability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary tag comparison in the primary tag array before committing to the full P2w lookup process. By performing this preliminary action, the system can quickly identify and resolve cache hits without incurring the full bandwidth and latency costs of a complete P2w search, thus reducing overall overhead.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If P2w cache is used to alleviate conflicts in direct-mapped cache, then cache conflict resolution is improved, but miss-latency increases significantly

Engineering Contradiction:

Improvecache conflict resolutionVSAvoidmiss-latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the tag comparison process into two independent parts: primary tag comparison and secondary tag comparison. This segmentation allows the cache system to quickly determine cache hits using the primary tag without always requiring the full P2w lookup process, thereby reducing bandwidth overhead while maintaining conflict resolution capability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary tag comparison in the primary tag array before committing to the full P2w lookup process. By performing this preliminary action, the system can quickly identify and resolve cache hits without incurring the full bandwidth and latency costs of a complete P2w search, thus reducing overall overhead.

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If separate tag arrays are used in HBM cache, then cache management flexibility is improved, but device complexity becomes impractical

Engineering Contradiction:

Improvecache management flexibilityVSAvoidtag array structure

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent merges the primary and secondary tag arrays into a unified tag storage structure within the HBM cache. This merging approach maintains the functional benefits of having separate tag comparison stages while avoiding the practical complexity of completely separate tag arrays, thus achieving a balance between flexibility and implementability.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent designs the tag array structure to serve multiple functions: it supports both direct-mapped and P2w cache modes, handles both primary and secondary tag comparisons, and provides flexible cache management. This multi-functionality achieves adaptability without proportionally increasing device complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12124371B2Apparatus and method to reduce bandwidth and latency overheads of probabilistic caches

Publication Date: 2024.10.22 INTEL CORP

AI summary

An apparatus and method to reduce bandwidth and latency associated with probabilistic caches. For example, one embodiment of a processor comprises: a plurality of cores to execute instructions and process data, one or more of the cores to generate a request for a first cache line; a cache controller comprising cache lookup logic to determine a first way of a cache in which to search for the first cache line based on a first set of tag bits comprising one or more bits associated with the first cache line; the cache lookup logic to compare a second set of tag bits of the first cache line with a third set of tag bits of an existing cache line stored in the first way, wherein if the second set of tag bits and the third set of tag bits to not match, then the cache lookup logic to determine that the first cache line is not in the first way and to compare a fourth set of tag bits of the first cache line with a fifth set of tag bits of the existing cache line, wherein responsive to a match between the fourth set of tag bits and the fifth set of tag bits, the cache lookup logic to determine that the first cache line is stored in a second way and to responsively read the first cache line from the second way.