Dual-Level Branch Target Buffer Entry Transfer

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In processor architecture, reducing pipeline bubbles caused by branch instructions is challenging, as existing branch target buffer structures either lead to performance losses due to slow access times or low prediction accuracy when decoupled micro buffers are used.

Innovation Solution

Implementing a dual-level branch target buffer system where a first-level buffer with faster read/write performance stores entries for frequently executed branch instructions, and a second-level buffer with a multi-way set-associative memory structure is used for less frequent ones, with the instruction fetch unit dynamically transferring entries between the two based on execution results and performance analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If a single-level branch target buffer is used, then the device complexity is low, but the access speed is slow causing pipeline bubbles

Engineering Contradiction:

Improvebranch target buffer access speedVSAvoidbranch target buffer structure

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The branch target buffer is segmented into two hierarchical levels: a first-level buffer with fast access for frequently executed branches and a second-level buffer with larger capacity for less frequent branches. This segmentation allows the system to achieve fast access speeds for critical branches while maintaining comprehensive coverage, resolving the contradiction between speed and complexity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to the branch target buffer structure, organizing buffers in levels rather than a single flat structure. This dimensional change enables differentiated access speeds and capacities across levels, allowing fast access for frequently accessed entries while storing less frequent entries in larger-capacity lower-level buffers.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Speed

If a decoupled micro buffer is used, then the access speed improves, but the prediction accuracy decreases leading to performance losses

Engineering Contradiction:

Improvebranch target buffer access speedVSAvoidbranch prediction accuracy

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

Different regions of the branch target buffer system are assigned different qualities: the first-level buffer provides fast access for frequently executed branches that require high prediction accuracy, while the second-level buffer handles less frequent branches. This local quality differentiation ensures that critical branches receive the attention needed for accurate prediction while maintaining overall system speed.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system employs feedback mechanisms where branch execution results and performance metrics are used to dynamically manage buffer contents. Entries are added to or transferred between buffer levels based on their execution frequency and prediction accuracy, ensuring that the most reliable predictions are maintained in the fastest access locations.

Inventive Principle:

Principle #23Feedback

3Reliability

If more branch instruction entries are stored in the buffer, then the prediction accuracy improves, but the access time increases causing pipeline bubbles

Engineering Contradiction:

Improvebranch prediction accuracyVSAvoidpipeline bubble time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The large number of branch entries are segmented across two buffer levels, with the first-level buffer containing a smaller subset of frequently accessed entries for rapid access and the second-level buffer containing the remaining entries. This segmentation allows the system to maintain high prediction accuracy by storing all necessary entries while avoiding the time penalty of searching through a single large buffer by organizing entries hierarchically.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11704131B2Moving entries between multiple levels of a branch predictor based on a performance loss resulting from fewer than a pre-set number of instructions being stored in an instruction cache register

Publication Date: 2023.07.18 ALIBABA GROUP HOLDING LTD

AI summary

An instruction processing device and an instruction processing method are provided. The instruction processing device includes: a first-level branch target buffer, configured to store entries of a first plurality of branch instructions; a second-level branch target buffer, configured to store entries of a second plurality of branch instructions, wherein the entries in the first-level branch target buffer are accessed faster than the entries in the second-level branch target buffer; an instruction fetch unit coupled to the first-level branch target buffer and the second-level branch target buffer, the instruction fetch unit including circuitry configured to add, for a first branch instruction, one or more entries corresponding to the first branch instruction into the first-level branch target buffer when the one or more entries corresponding to the first branch instruction are identified in the second-level branch target buffer; and an execution unit including circuitry configured to execute the first branch instruction.