Tag Orientation Predictor for Hybrid Branch Prediction Override

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Hybrid branch prediction schemes struggle to accurately track long-term prediction characteristics for branches that are frequently mis-predicted, leading to increased processing time and resource wastage due to the limited size of counters and destructive interference.

Innovation Solution

The implementation of a Tag Orientation Predictor (TOP) that tracks the long-term behavior of individual branch instructions by using counters to record statistics and categorize branches based on their prediction accuracy, overriding hybrid predictor decisions when confident predictions are made.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If hybrid branch prediction is used, then prediction speed is improved, but prediction accuracy for frequently mis-predicted branches deteriorates due to limited counter size and destructive interference

Engineering Contradiction:

Improveprediction speedVSAvoidprediction accuracy

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The branch prediction system is segmented into multiple independent components: a hybrid predictor for fast general predictions and a separate long-term tracking structure for accurately tracking frequently mis-predicted branches. This segmentation allows each component to specialize - the hybrid predictor maintains speed while the long-term tracker improves accuracy for difficult cases without being affected by counter size limitations in the hybrid predictor.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A long-term tracking structure acts as an intermediary between the hybrid predictor and the branch execution unit. When the hybrid predictor makes a prediction, the long-term tracker monitors outcomes over many iterations and can override or supplement the hybrid predictor's decisions for branches that exhibit persistent mis-prediction patterns, thereby improving overall accuracy without sacrificing the speed of the hybrid predictor.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If counter size in hybrid predictor is increased to track long-term characteristics, then prediction accuracy improves, but device complexity and resource usage increase

Engineering Contradiction:

Improveprediction accuracyVSAvoidcounter structure complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Instead of increasing counter size throughout the entire hybrid predictor structure, the system segments the tracking function into a separate long-term tracking component. This allows the main hybrid predictor to maintain its compact counter structure for speed, while the dedicated long-term tracker uses larger counters only where needed to track persistent patterns, reducing overall device complexity while improving accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies different counter sizes and tracking depths locally - the hybrid predictor uses smaller counters for fast general predictions, while the long-term tracking structure uses larger counters specifically for branches that require long-term pattern recognition. This local differentiation optimizes the balance between accuracy and complexity by applying enhanced tracking only where it is most beneficial.

Inventive Principle:

Principle #3Local quality

3Reliability

If long-term tracking is implemented for all branches, then prediction accuracy improves, but processing time increases due to additional tracking operations

Engineering Contradiction:

Improveprediction accuracyVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system implements long-term tracking partially - only for branches that are identified as frequently mis-predicted or 'hotly mis-predicted' branches. Rather than applying long-term tracking to all branches uniformly, the system selectively applies it where needed, reducing the overall processing overhead while still improving accuracy for the critical cases that benefit most from long-term pattern recognition.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The branch prediction process is segmented into fast-path and slow-path handling. Most branches are handled quickly by the hybrid predictor without long-term tracking overhead, while only branches exhibiting persistent mis-prediction patterns are subjected to additional long-term tracking analysis. This segmentation minimizes processing time for the majority of branches while improving accuracy for the difficult minority.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11416257B2Hybrid and aggregrate branch prediction system with a tagged branch orientation predictor for prediction override or pass-through

Publication Date: 2022.08.16 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Branch prediction in an instruction using a tag orientation predictor (TOP) is described. When a branch instruction is hotly mis-predicted by a hybrid branch predictor, the branch is tracked over a longer time period using the TOP. Once the TOP has collected enough data to confidently predict a branch prediction, the TOP is used to override a branch prediction from the hybrid predictor when the TOP branch prediction.