Out-of-Order Processor Load Replay Reduction via Reservation Station Stalling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In out-of-order processors, the power wastage due to frequent replays of instructions waiting for operands from off-core resources, such as shared L2 caches and fuse arrays, is significant, leading to increased power utilization and thermal issues despite minimal performance impact.

Innovation Solution

Implementing a mechanism with reservation stations that detect load micro instructions directed to off-core resources and stall younger dependent micro instructions until the operand is retrieved, reducing unnecessary replays and conserving power by managing execution unit power states.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If younger dependent micro instructions are dispatched immediately after a load micro instruction, then instruction throughput is improved, but power consumption increases due to frequent replays

Engineering Contradiction:

Improveinstruction throughputVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The mechanism performs preliminary detection of load micro instructions directed to off-core resources before dispatching younger dependent instructions. By identifying these long-latency operations in advance and stalling dependent instructions accordingly, the system avoids unnecessary dispatches that would lead to replays, thus reducing power consumption while maintaining throughput for instructions that can safely proceed

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The mechanism implements feedback by monitoring the type of resource accessed by load micro instructions and using this information to control the dispatch of younger dependent instructions. When a load is detected to access off-core resources, feedback signals stall the dispatch of dependent instructions, creating a closed-loop control system that adapts instruction dispatch to actual memory access patterns and reduces power-wasting replays

Inventive Principle:

Principle #23Feedback

2Reliability

If younger dependent micro instructions are stalled until operand retrieval from off-core resources, then replay frequency is reduced, but instruction latency increases

Engineering Contradiction:

Improvereplay frequencyVSAvoidinstruction latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The mechanism applies different dispatch strategies to different instructions based on their dependency characteristics. Only younger dependent instructions that specifically require the operand from the off-core load are stalled, while independent instructions and instructions depending on other operands continue to dispatch normally. This localized stalling minimizes the impact on overall instruction latency while effectively reducing replays for the specific problematic case

Inventive Principle:

Principle #3Local quality

3Device complexity

If all load micro instructions are treated uniformly, then dispatch simplicity is maintained, but power efficiency decreases due to unnecessary replays of instructions waiting for off-core operands

Engineering Contradiction:

Improvedispatch logic simplicityVSAvoidenergy wasted on replays

Core Design Contradiction:

Device complexityVSLoss of energy

Solution Approach 1:

The mechanism changes the dispatch parameter based on the type of resource accessed by the load micro instruction. By detecting whether a load accesses on-core or off-core resources, the system dynamically adjusts the dispatch behavior of younger dependent instructions - allowing immediate dispatch for on-core loads and stalling for off-core loads. This parameter-based differentiation maintains relatively simple dispatch logic while dramatically improving power efficiency by preventing replays

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10108420B2Mechanism to preclude load replays dependent on long load cycles in an out-of-order processor

Publication Date: 2018.10.23 VIA ALLIANCE SEMICON CO LTD

AI summary

An apparatus including first and second reservation stations. The first reservation station dispatches a load micro instruction, and indicates on a hold bus if the load micro instruction is a specified load micro instruction directed to retrieve an operand from a prescribed resource other than on-core cache memory, where the specified load instruction requires more than a first number of clock cycles to retrieve the operand. The second reservation station is coupled to the hold bus, and dispatches one or more younger micro instructions therein that depend on the load micro instruction for execution after a number of clock cycles following dispatch of the first load micro instruction, and if it is indicated on the hold bus that the load micro instruction is the specified load micro instruction, the second reservation station is configured to stall dispatch of the one or more younger micro instructions until the load micro instruction has retrieved the operand.