Cache Set-Dueling with Oldest-Operation Wait-Time Feedback

Solution Overview

Problem

Existing cache management systems in computing devices struggle to optimize cache policies dynamically, leading to inefficient execution core wait times and suboptimal performance.

Innovation Solution

A cache subsystem is partitioned into multiple portions, with different policies applied to each, and the wait times are monitored to determine a performance-optimal policy for the remainder of the cache by comparing aggregate wait times across these portions, allowing dynamic adjustment of cache policies based on real-time conditions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single cache policy is applied to the entire cache, then the cache management is simple, but the execution core wait time is suboptimal

Engineering Contradiction:

Improvecache policy management complexityVSAvoidexecution core wait time

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The cache is divided into multiple portions (first portion, second portion, and remainder) with different policies applied to each. The first and second portions use different cache policies while the remainder uses a policy determined by comparing wait times from the first two portions. This segmentation allows optimized policies for different cache regions without managing a single complex policy across the entire cache.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different cache policies are applied to different portions of the cache based on their specific access patterns and performance characteristics. The first portion uses one policy, the second portion uses another policy, and the remainder uses a policy selected based on comparative wait time analysis. This local quality approach optimizes each region's performance independently.

Inventive Principle:

Principle #3Local quality

2Loss of time

If multiple cache policies are applied to different cache portions, then the execution core wait time is reduced, but the cache management complexity increases

Engineering Contradiction:

Improveexecution core wait timeVSAvoidcache policy management complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The cache is divided into multiple portions (first portion, second portion, and remainder) with different policies applied to each. The first and second portions use different cache policies while the remainder uses a policy determined by comparing wait times from the first two portions. This segmentation allows optimized policies for different cache regions without managing a single complex policy across the entire cache.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system monitors execution core wait times for the first and second portions, compares these wait times, and uses the comparison result to determine the policy for the remainder portion. This feedback mechanism automatically adjusts policies based on real-time performance data without requiring manual intervention.

Inventive Principle:

Principle #23Feedback

3Productivity

If cache policies are dynamically adjusted based on real-time conditions, then system performance is enhanced, but the measurement and control complexity increases

Engineering Contradiction:

Improvesystem performanceVSAvoidmeasurement and control complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system monitors execution core wait times for the first and second portions, compares these wait times, and uses the comparison result to determine the policy for the remainder portion. This feedback mechanism automatically adjusts policies based on real-time performance data without requiring manual intervention.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The cache subsystem automatically monitors its own performance metrics (wait times), compares different policies, and self-adjusts the remainder portion's policy without external intervention. The system serves itself by using its own performance data to optimize its operation.

Inventive Principle:

Principle #25Self-service

Data Source

PatentEP4026006B1Oldest operation wait time indication input into set-dueling

Publication Date: 2025.11.05 ADVANCED MICRO DEVICES INC

Figure 1
Figure 2
Figure 3

AI summary

Systems, apparatuses, and methods for dynamically adjusting cache policies to reduce execution core wait time are disclosed. A processor includes a cache subsystem. The cache subsystem includes one or more cache levels and one or more cache controllers. A cache controller partitions a cache level into two test portions and a remainder portion. The cache controller applies a first policy to the first test portion and applies a second policy to the second test portion. The cache controller determines the amount of time the execution core spends waiting on accesses to the first and second test portions. If the measured wait time is less for the first test portion than for the second test portion, then the cache controller applies the first policy to the remainder portion. Otherwise, the cache controller applies the second policy to the remainder portion.