Partial Local Cache Replacement Policy for Set-Associative Caches

Solution Overview

Problem

Set-associative caches face a trade-off between performance and hardware cost due to the need for either maintaining local replacement policies for each set, which is expensive in terms of hardware, or using a shared replacement policy that is less accurate and prone to cache thrashing.

Innovation Solution

Implementing a partial local replacement policy scheme where local replacement policy entries are stored for a proper subset of sets, with the remaining sets using a shared replacement policy, allowing for smarter replacement decisions while reducing storage overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If local replacement policy entries are maintained for each set of the cache, then cache replacement accuracy is improved and cache thrashing is reduced, but hardware cost and complexity increase

Engineering Contradiction:

Improvecache replacement accuracyVSAvoidhardware cost

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies local quality by providing local replacement policy entries for only certain sets of the cache (identified by a subset mask) rather than uniformly for all sets. This allows different parts of the cache to have different replacement policy characteristics - sets in the subset get personalized attention while others share policies, optimizing the balance between accuracy and hardware cost.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent implements partial action by maintaining local replacement policy entries for only a proper subset of cache sets rather than all sets. The subset mask selectively enables local policies for specific sets where it provides the most benefit, while other sets use shared replacement policies, thus achieving improved performance without the full hardware overhead.

Inventive Principle:

Principle #16Partial or excessive action

2Productivity

If local replacement policy entries are maintained for all sets, then cache thrashing is reduced, but use of energy and hardware resources increases

Engineering Contradiction:

Improvecache performanceVSAvoidenergy consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent applies local quality by providing local replacement policy entries for only certain sets of the cache (identified by a subset mask) rather than uniformly for all sets. This allows different parts of the cache to have different replacement policy characteristics - sets in the subset get personalized attention while others share policies, optimizing the balance between accuracy and hardware cost.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent changes the parameter of replacement policy granularity from uniform (all sets have local policies) to selective (only some sets have local policies) using the subset mask. This parameter change allows the system to adjust the level of localization based on performance needs and hardware constraints, reducing energy consumption while maintaining effectiveness.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If a shared replacement policy is used for all sets, then hardware cost is reduced, but cache thrashing increases and performance decreases

Engineering Contradiction:

Improvehardware costVSAvoidcache performance

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The patent applies local quality by providing local replacement policy entries for only certain sets of the cache (identified by a subset mask) rather than uniformly for all sets. This allows different parts of the cache to have different replacement policy characteristics - sets in the subset get personalized attention while others share policies, optimizing the balance between accuracy and hardware cost.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent segments the cache sets into two groups: those in the subset that have local replacement policy entries and those outside the subset that use shared replacement policies. This segmentation allows the system to apply different replacement strategies to different segments, improving overall performance while controlling hardware costs.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11086777B2Replacement of cache entries in a set-associative cache

Publication Date: 2021.08.10 ARM LTD

AI summary

An apparatus comprises a set-associative cache comprising a plurality of sets of cache entries, and cache replacement policy storage circuitry to store a plurality of local replacement policy entries. Each local replacement policy entry comprises local replacement policy information specific to a corresponding set of the set-associative cache. Cache control circuitry controls replacement of cache entries of the set-associative cache based on the local replacement policy information stored in the cache replacement policy storage circuitry. The cache replacement policy storage circuitry stores local replacement policy entries for a proper subset of sets of the set-associative cache.