Non-blocking API for GPU Thread Reallocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for configuring applications to utilize multiple processing resources in parallel often result in underutilized resources due to idle threads, highlighting the need for improved parallelism to enhance performance.
Innovation Solution
The implementation of a non-blocking application programming interface (API) that initiates reduce or scan operations by allocating multiple threads, which continue to perform tasks even after the calling thread returns, allowing for thread reallocation between operations and efficient use of processing resources through algorithms like binomial tree and pipeline algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If blocking API calls are used to perform reduce or scan operations, then each operation completes before the next begins, but processing resources remain idle between operations reducing productivity
Solution Approach 1:
The system performs preliminary actions by initiating reduce or scan operations in advance using non-blocking API calls. When a first operation is initiated, threads are allocated and begin execution immediately, allowing the system to prepare and queue subsequent operations without waiting for completion of previous ones. This preliminary initiation of operations eliminates idle time between completions.
Solution Approach 2:
The system maintains continuity of useful action by implementing a queue mechanism that continuously feeds operations to available processing threads. As threads complete one operation, they are immediately reallocated to the next operation from the queue, ensuring that processing resources are constantly engaged in productive work without interruption or idle periods between operations.
2Productivity
If threads are allocated for each operation until completion, then operations can be tracked and managed, but processing resources cannot be reallocated to other operations, reducing productivity
Solution Approach 1:
The system implements dynamic thread allocation where thread assignments are not fixed for the duration of an operation. Instead, threads are dynamically reallocated from completed operations to pending operations in the queue. This dynamic approach allows the same pool of threads to serve multiple operations sequentially, maximizing utilization without requiring dedicated threads for each operation.
Solution Approach 2:
The system introduces an intermediary queue mechanism that mediates between operation requests and thread execution. The queue acts as a buffer that manages the flow of operations to available threads, handling the complexity of thread management and allocation decisions. This intermediary structure simplifies the overall system by centralizing the management logic and enabling flexible thread reallocation without direct complex coordination between operations and threads.
3Reliability
If multiple operations are initiated simultaneously with dedicated threads, then each operation has guaranteed resources, but resource overhead increases and utilization decreases
Solution Approach 1:
The system implements universality by using a single pool of processing threads to perform multiple different operations. Instead of dedicating specific threads to specific operations, the same threads are universally applicable to any operation in the queue. This multi-functional approach ensures that operations are completed reliably while minimizing the total number of threads required, as each thread can serve multiple operations sequentially.
Data Source
AI summary
Apparatuses, systems, and techniques to perform collective operations using parallel processing. In at least one embodiment, a non-blocking application programming interface allow programs to improve performance of one or more collective operations on a GPU.