Cache Way Prediction Serial Lookup for Power and Speed

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Multi-core processing systems face inefficiencies in cache memory access, leading to increased power consumption and reduced access/snoop bandwidth due to parallel tag way lookups, which can be exacerbated by fixed or random access orders.

Innovation Solution

An N-way set associative cache memory system with a configurable cache controller and way predictor that allows for parallel or serial tag lookups, selecting initial subsets based on access requests and history to optimize access efficiency and reduce power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If parallel tag way lookups are used in cache memory, then access speed is improved, but power consumption increases

Engineering Contradiction:

Improvecache access speedVSAvoidpower consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The cache controller dynamically switches between parallel and serial lookup modes based on operational requirements. The system can configure the cache controller to perform cache lookups using N-ways in parallel for high-speed operations, or serially using sets of M ways in parallel for power-efficient operations, making the lookup strategy adaptive rather than fixed

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the operational parameters of the cache lookup process by adjusting the degree of parallelism (from N-way parallel to M-way parallel where M<N). This parameter adjustment allows the system to optimize between speed and power consumption based on current workload and performance requirements

Inventive Principle:

Principle #35Parameter changes

2Productivity

If fixed or random access orders are used for tag lookups, then device complexity is reduced, but access efficiency and bandwidth are reduced

Engineering Contradiction:

Improveaccess efficiencyVSAvoidlookup control complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by pre-configuring the cache controller with lookup strategies (parallel or serial modes) before actual cache operations. The cache controller is pre-configurable to perform cache lookups using N-ways in parallel or by serially using sets of M ways in parallel, allowing optimized access patterns without real-time decision complexity

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The tag lookup process is segmented into different operational modes (parallel mode with N-ways, serial mode with sets of M ways). This segmentation allows the system to divide the lookup operation into manageable configurations, selecting appropriate segments based on performance and power requirements without requiring complex real-time control logic

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10565122B2Serial tag lookup with way-prediction

Publication Date: 2020.02.18 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

The lookup of accesses (including snoops) to cache tag ways is serialized to perform one (or less than all) tag way access per clock (or even slower). Thus, for a N-way set associative cache, instead of performing lookup/comparison on the N tag ways in parallel, the lookups are performed one tag way a time. Way prediction is utilized to select an order to look in the N ways. This can include selecting which tag way will be looked in first. This helps to reduce the average number of cycles and lookups required.