Tokenized Streams for Concurrent GPU Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing programming languages are sequential, preventing developers from leveraging the parallel execution capabilities of processors like GPUs, resulting in inefficient data processing.

Innovation Solution

The introduction of a method using streams, where a device driver parses commands to identify stream tokens and semaphores, allowing parallel execution of commands across multiple streams without dependencies, thereby exploiting the parallel nature of GPUs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If sequential programming languages are used, then code simplicity is maintained, but parallel processing capability is lost

Engineering Contradiction:

Improvedata processing efficiencyVSAvoidprogramming model complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the execution model into independent streams that can be executed in parallel. Each stream represents an independent sequence of operations that can run concurrently without dependency, allowing the system to leverage parallel processing capabilities while maintaining a relatively simple programming interface through the stream abstraction layer.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary abstraction layer (the stream API and device driver) between the sequential programming language and the parallel GPU architecture. This intermediary translates sequential commands into parallel stream executions, hiding the complexity of parallel processing from the developer while enabling efficient data processing on the GPU.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If parallel execution is implemented, then processing speed increases, but synchronization complexity increases

Engineering Contradiction:

Improvecommand execution speedVSAvoidsynchronization mechanism complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The patent implements self-service synchronization where the GPU automatically manages stream execution and synchronization without requiring complex manual coordination. The device driver handles semaphore signaling and stream dependency management internally, allowing developers to simply submit commands to different streams without worrying about synchronization details.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent uses feedback mechanisms through semaphores that automatically signal when streams have completed execution. The device driver monitors stream completion status and provides feedback to the CPU, enabling automatic synchronization and dependency resolution without requiring complex polling or waiting mechanisms from the developer.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9542192B1Tokenized streams for concurrent execution between asymmetric multiprocessors

Publication Date: 2017.01.10 NVIDIA CORP

AI summary

A method for executing an application program using streams. A device driver receives a first command within an application program and parses the first command to identify a first stream token that is associated with a first stream. The device driver checks a memory location associated with the first stream for a first semaphore, and determines whether the first semaphore has been released. Once the first semaphore has been released, a second command within the application program is executed. Advantageously, embodiments of the invention provide a technique for developers to take advantage of the parallel execution capabilities of a GPU.