GPU Out-of-Order Workload Execution to Reduce Pipeline Stalls

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional graphics processing unit (GPU) hardware experiences inefficiencies due to concurrent execution of workloads with different execution profiles, leading to stalls and idle periods, particularly when workloads with higher memory access latencies are executed in-order, reducing hardware efficiency and increasing processing times.

Innovation Solution

Implementing out-of-order execution techniques for workloads within a time-shared GPU processing pipeline, allowing workloads to be executed based on data availability rather than a predetermined sequence, thereby avoiding stalls and optimizing hardware efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If workloads are executed in a predetermined sequence (in-order execution), then workload submission order is maintained, but GPU hardware efficiency decreases due to stalls and idle periods

Engineering Contradiction:

Improveworkload execution orderVSAvoidGPU hardware efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent inverts the traditional in-order execution approach by implementing out-of-order execution, where workloads are executed based on data availability rather than submission order. The execution component dynamically selects and executes workload sub-chunks in an optimized sequence, reversing the conventional constraint to improve hardware utilization and eliminate stalls.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent introduces dynamic workload scheduling where the execution component adapts the execution sequence based on real-time data availability. Instead of a static predetermined order, the system dynamically adjusts which workload sub-chunks to execute next, optimizing resource utilization and eliminating idle periods while maintaining eventual workload completion order.

Inventive Principle:

Principle #15Dynamics

2Quantity of substance

If workloads with higher memory access latencies are executed first, then memory access patterns are followed, but processing time increases due to stalls

Engineering Contradiction:

Improvememory access throughputVSAvoidprocessing time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent divides workloads into smaller sub-chunks and performs preliminary organization of these sub-chunks in storage structures before execution. This allows the execution component to select and execute ready sub-chunks immediately without waiting for memory operations to complete, overlapping memory access with computation and reducing overall processing time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent ensures continuous GPU operation by maintaining a pool of ready-to-execute workload sub-chunks. When one sub-chunk completes or data becomes available, another is immediately selected and executed, eliminating idle periods and ensuring continuous useful action throughout the processing pipeline.

Inventive Principle:

Principle #20Continuity of useful action

3Productivity

If workloads are divided into sub-workloads and executed out-of-order, then hardware efficiency improves, but execution control complexity increases

Engineering Contradiction:

Improveworkload execution efficiencyVSAvoidexecution control mechanism

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments workloads into smaller sub-chunks that can be independently executed and tracked. This segmentation enables the execution component to make finer-grained scheduling decisions, improving hardware utilization while keeping individual sub-chunk management simpler and more tractable.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces storage structures (such as buffers or queues) as intermediaries between workload submission and execution. These intermediaries hold divided sub-chunks and provide the execution component with a manageable interface for selecting and tracking workloads, reducing control complexity while enabling out-of-order execution.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250245041A1Out-of-order execution for graphics processing unit hardware

Publication Date: 2025.07.31 QUALCOMM INC

AI summary

Aspects presented herein relate to methods and devices for data or graphics processing including an apparatus, e.g., a GPU. The apparatus may obtain an indication of a plurality of workloads for graphics processing. The apparatus may also perform a binning process for a first workload of the plurality of workloads. Further, the apparatus may divide the first workload into a plurality of first sub-workloads and a second workload of the plurality of workloads into a plurality of second sub-workloads. The apparatus may also perform a rendering process for at least one first sub-workload of the plurality of first sub-workloads and a binning process for at least one second sub-workload of the plurality of second sub-workloads. The apparatus may also perform a rendering process for the at least one second sub-workload prior to a completion of the rendering process for the at least one first sub-workload.