Unified Cache Directory for Multi-Level Hierarchy Latency Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing multi-level cache hierarchies face inefficiencies due to the need for multiple directories and cache tag lookups, which increase latency and energy consumption, especially in inclusive cache hierarchies where data is replicated across levels.

Innovation Solution

Implementing a unified cache directory that tracks the presence and state of data across all levels of the cache hierarchy, allowing for parallel searches of the second-level cache and the unified directory, and reducing the need for multiple directory lookups.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If multiple directories and cache tag lookups are used in existing multi-level cache hierarchies, then data tracking accuracy is improved, but latency and energy consumption increase

Engineering Contradiction:

Improvedata tracking accuracyVSAvoidlatency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent merges multiple cache directories into a single unified cache directory that tracks data presence and state across all cache levels (L1, L2, L3). This consolidation eliminates the need for separate directory lookups at each cache level, thereby reducing latency while maintaining accurate data tracking through the unified directory's comprehensive mapping of data locations and states throughout the entire cache hierarchy.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The unified cache directory serves multiple functions simultaneously: it tracks data presence in inclusive caches, manages exclusive caches, records data state (shared, private, invalid), and provides navigation information across all cache levels. This multi-functional design replaces multiple specialized directories with a single universal structure, reducing the number of operations required while maintaining comprehensive data tracking capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Measurement precision

If multiple directories and cache tag lookups are used in existing multi-level cache hierarchies, then data tracking accuracy is improved, but energy consumption increases

Engineering Contradiction:

Improvedata tracking accuracyVSAvoidenergy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by stationary object

Solution Approach 1:

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The unified cache directory performs preliminary tracking of data presence and state information in advance, maintaining up-to-date records of where data is located and its state across all cache levels. This preliminary action eliminates the need for multiple subsequent directory lookups and cache tag comparisons, thereby reducing energy consumption while maintaining comprehensive data tracking accuracy.

Inventive Principle:

Principle #10Preliminary action

3Loss of time

If a unified cache directory is implemented, then latency and energy consumption are reduced, but device complexity changes

Engineering Contradiction:

ImprovelatencyVSAvoidcache directory structure

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The unified cache directory is segmented into multiple entries, each representing a different data location or state in the cache hierarchy. Each entry contains structured information about data presence, state (shared, private, invalid), and location details. This segmentation allows the complex unified directory to be organized into manageable units while maintaining the benefits of a single consolidated structure for reduced latency.

Inventive Principle:

Principle #1Segmentation

4Reliability

If multiple cache levels search for data sequentially, then search completeness is improved, but productivity decreases

Engineering Contradiction:

Improvesearch completenessVSAvoiddata retrieval speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The unified cache directory performs preliminary identification of data locations and states across all cache levels before the actual data retrieval process begins. By pre-determining where data is located and its state through the unified directory's comprehensive mapping, the system can directly navigate to the correct cache level without performing sequential searches, thereby maintaining search completeness while significantly improving data retrieval speed.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250139006A1Systems, methods, and apparatus for a cache directory for a multi-level cache hierarchy

Publication Date: 2025.05.01 SAMSUNG ELECTRONICS CO LTD

AI summary

In some aspects, the techniques described herein relate to a device including a storage media and a processor including a cache hierarchy including a first cache, a second cache, and a third cache, wherein the first cache and the third cache are organized in an inclusive cache hierarchy, and wherein the second cache is an exclusive cache to the inclusive cache hierarchy; and a cache directory, wherein the cache directory corresponds to the first cache, second cache, and third cache. In some aspects, the processor performs operations including searching the first cache for data, searching the second cache for the data, and searching the cache directory for the data. In some aspects, searching the cache directory includes determining that the data is located in the cache directory and determining a location of the data in the cache hierarchy based on an entry in the cache directory.