GPU Thread Cancellation API for Duplicate Work Prevention

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

GPU thread scheduling can result in wasted resources and duplicate processing due to inefficient management of thread operations.

Innovation Solution

Implementing an application programming interface (API) that allows running threads to request the cancellation of pending threads, confirm the cancellation, and acquire starting dimensions for new threads, thereby reducing resource wastage and preventing duplicate processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If GPU threads are scheduled independently without coordination, then thread processing can proceed in parallel, but resource wastage and duplicate processing occur

Engineering Contradiction:

Improvethread processing throughputVSAvoidresource wastage

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent implements a feedback mechanism where running threads send cancellation requests to the scheduler when they need to prevent duplicate processing. The scheduler receives these requests and coordinates thread scheduling accordingly, allowing parallel processing while preventing resource wastage through coordinated feedback from executing threads.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The scheduler acts as an intermediary between running threads and pending threads. It receives cancellation requests from running threads and uses this information to make informed scheduling decisions, preventing duplicate processing while maintaining parallel execution efficiency.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If threads are cancelled without coordination, then resource allocation can be adjusted, but latency increases due to transition overhead

Engineering Contradiction:

Improvethread resource allocation flexibilityVSAvoidtransition latency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent implements preliminary action by allowing running threads to send cancellation requests before actually completing their current tasks. This enables the system to prepare for resource reallocation in advance, reducing the latency impact when thread transitions occur by overlapping the cancellation decision-making process with ongoing computations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system maintains continuity of useful action by allowing threads to send cancellation requests during their execution, enabling resource reallocation to occur without complete task termination. The scheduler can begin preparing new thread assignments while running threads are still executing, reducing transition latency.

Inventive Principle:

Principle #20Continuity of useful action

3Ease of operation

If the scheduler manages all thread scheduling centrally, then coordination is simplified, but device complexity increases

Engineering Contradiction:

Improvethread scheduling coordinationVSAvoidscheduler management overhead

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent implements self-service by enabling running threads to actively participate in their own scheduling coordination by sending cancellation requests when needed. This distributes the scheduling decision-making burden from a purely central scheduler to the threads themselves, simplifying coordination while reducing scheduler management overhead.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20250390339A1Application programming interface to prevent thread performance

Publication Date: 2025.12.25 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to cancel pending GPU thread work to allow said work to be assumed by running thread clusters. In at least one embodiment, processors comprising one or more circuits to perform an application programming interface (API) to cause one or more software threads identified by the API to be prevented from being performed by one or more processors.