GPU Thread Cancellation API for Pending Work Handover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
GPU thread scheduling often results in wasted resources and duplicate processing due to inefficient management of pending and running threads.
Innovation Solution
Implementing an application programming interface (API) that allows running threads to request cancellation of pending threads and acquire their dimensions for seamless transition, thereby reducing resource wastage and preventing duplicate processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU thread scheduling allows pending threads to be processed independently, then thread parallelism is improved, but resource wastage and duplicate processing increase
Solution Approach 1:
The patent implements a feedback mechanism where running threads provide status information to the scheduler, which then adjusts pending thread execution accordingly. The scheduler receives feedback about completed work units and uses this information to prevent duplicate processing by other threads, thereby maintaining parallelism while reducing resource wastage.
Solution Approach 2:
The system dynamically changes the execution parameters of threads based on real-time scheduling decisions. When a running thread completes a work unit, the scheduler modifies the state of related pending threads to prevent their execution, effectively changing the parameter of thread execution from independent to conditional based on scheduler decisions.
2Loss of energy
If GPU thread scheduling cancels pending threads to avoid duplicate processing, then resource wastage is reduced, but scheduling complexity increases
Solution Approach 1:
The scheduler performs preliminary actions by pre-identifying and marking pending threads that should be cancelled before they are executed. This preliminary cancellation marking simplifies the overall scheduling process by preventing the need for complex runtime coordination and conflict resolution mechanisms.
Solution Approach 2:
The patent introduces an intermediary scheduling layer that mediates between thread execution and resource allocation. This intermediary scheduler manages the complexity of thread coordination by centralizing the decision-making process for thread cancellation and work unit allocation, thereby reducing the scheduling burden on individual threads.
3Loss of time
If running threads can assume work from cancelled threads, then latency is reduced, but thread management complexity increases
Solution Approach 1:
The patent merges the work units from cancelled threads into the execution queue of running threads. This combining of work units allows running threads to seamlessly continue processing without interruption, reducing latency by eliminating the need to wait for new work unit allocation while maintaining manageable thread complexity through centralized scheduler control.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Apparatuses, systems, and techniques to cancel pending GPU thread work to allow said work to be assumed by running thread clusters. In at least one embodiment, processors comprising one or more circuits to perform an application programming interface (API) to cause one or more software threads identified by the API to be prevented from being performed by one or more processors.