Workload Reordering via Dependency Graphs for GPU Concurrency

Solution Overview

Problem

Cross-thread resource dependencies in multi-core GPUs limit concurrency and lead to underutilization of processing resources and increased execution time during the rendering of graphics and video data, as separate cores may stall waiting for results from other cores.

Innovation Solution

A processor-based device constructs a weighted dependency graph to represent workloads and their dependencies, performs a topological sort to reorder workloads, and schedules independent workloads to execute during idle times between dependent workloads, maximizing concurrency by efficiently using processor resources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If separate processor cores execute workloads in parallel to improve GPU performance, then processing speed is improved, but cross-thread resource dependencies cause stalls and underutilization of processing resources

Engineering Contradiction:

Improveprocessing speedVSAvoidresource utilization

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

The system performs preliminary analysis of workload dependencies by constructing a dependency graph before execution. This allows the scheduler to identify independent workloads in advance and schedule them for concurrent execution, preventing stalls before they occur and maximizing resource utilization from the outset.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The workload scheduler dynamically reorders and selects workloads based on real-time dependency analysis and resource availability. By continuously monitoring the dependency graph and adjusting execution order, the system adapts to changing resource states to maintain optimal concurrency and minimize stalls.

Inventive Principle:

Principle #15Dynamics

2Speed

If separate processor cores execute workloads in parallel, then processing speed is improved, but execution time increases due to stalls waiting for results from other cores

Engineering Contradiction:

Improveprocessing speedVSAvoidexecution time

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of workload dependencies by constructing a dependency graph before execution. This allows the scheduler to identify independent workloads in advance and schedule them for concurrent execution, preventing stalls before they occur and minimizing execution time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The scheduler identifies workloads that can be executed independently and rushes them through during idle periods of dependent workloads. By skipping over dependent workloads and executing independent ones in parallel, the system reduces overall execution time by eliminating unnecessary waiting periods.

Inventive Principle:

Principle #21Skipping (Rushing through)

3Productivity

If workloads are executed sequentially to avoid cross-thread dependencies, then resource utilization is improved, but processing speed decreases

Engineering Contradiction:

Improveresource utilizationVSAvoidprocessing speed

Core Design Contradiction:

ProductivityVSSpeed

Solution Approach 1:

The system segments workloads into independent units based on dependency analysis. By dividing the workload graph into separable tasks, the scheduler can execute segments in parallel without resource conflicts, achieving both high resource utilization and processing speed simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The workload scheduler dynamically reorders and selects workloads based on real-time dependency analysis and resource availability. By continuously monitoring the dependency graph and adjusting execution order, the system adapts to changing resource states to maintain optimal concurrency and minimize stalls.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20240045736A1Reordering workloads to improve concurrency across threads in processor-based devices

Publication Date: 2024.02.08 QUALCOMM INC

AI summary

Reordering workloads to improve concurrency across threads in processor-based devices is disclosed herein. In this regard, in some exemplary aspects, a processor-based device receives a plurality of workloads from a requestor, and constructs a weighted dependency graph based on the plurality of workloads. The weighted dependency graph comprises a plurality of vertices that each correspond to a workload of the plurality of workloads, and further comprises one or more directed edges that each connects two vertices of the plurality of vertices and indicates a dependency between a corresponding two workloads of the plurality of workloads. After generating the weighted dependency graph, the processor-based device performs a topological sort of the weighted dependency graph, and generates a workload execution order based on the topological sort. By scheduling workload execution according to the workload execution order, concurrency among threads may be maximized and processor resources may be more efficiently utilized.