Multicasting Synchronization Barriers for Asynchronous GPU Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional synchronization barrier methods in parallel computing suffer from delays as processing elements must wait for flags to be set across multiple GPUs, leading to inefficiencies in compute task progression.
Innovation Solution
The implementation of multicasting notifications allows processing elements to locally track and determine if each element has reached the synchronization barrier, enabling asynchronous continuation of compute tasks once synchronization is confirmed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional synchronization barrier methods are used with distributed SMs across multiple GPUs, then synchronization can be achieved, but delays occur as processing elements must wait for flags to be set across multiple GPUs
Solution Approach 1:
The patent divides the centralized flag-setting operation into distributed counter increments at each GPU. Instead of one GPU setting a flag that all others must poll, each GPU independently increments its own counter when an SM arrives at the barrier, allowing parallel progress tracking without centralized coordination delays.
Solution Approach 2:
The patent combines the flag-setting function and the counter-tracking function into a unified counter-based synchronization mechanism. Each GPU maintains and updates its own counter, merging the roles of synchronization coordinator and status tracker into a distributed system where all GPUs operate independently but consistently.
2Measurement precision
If one GPU polls memory to detect when the counter indicates all SMs have arrived, then synchronization status can be determined, but write operations must be sent to each other GPU to change the flag value
Solution Approach 1:
Each GPU serves itself by maintaining and updating its own counter locally. When an SM arrives at the synchronization barrier, the corresponding GPU increments its own counter without needing to communicate with other GPUs. This self-service approach eliminates the need for inter-GPU write operations to propagate flag changes.
Solution Approach 2:
The patent extracts the flag-setting operation from the synchronization mechanism and replaces it with local counter increments. By removing the centralized flag management, the system eliminates the communication overhead associated with propagating flag changes across multiple GPUs while maintaining synchronization detection capability.
3Reliability
If other GPUs poll memory to detect whether the flag is set, then they can determine when the synchronization barrier has been reached, but they must wait for the flag to be set prior to proceeding with compute tasks
Solution Approach 1:
Each GPU performs preliminary action by pre-initializing its counter and continuously monitoring it for the synchronization condition. Instead of waiting for a flag to be set, GPUs are already in position to detect when all SMs have arrived, allowing them to proceed with compute tasks immediately upon detecting the synchronization condition without waiting for flag propagation.
Data Source
AI summary
In various examples, a single notification (e.g., a request for a memory access operation) that a processing element (PE) has reached a synchronization barrier may be propagated to multiple physical addresses (PAs) and/or devices associated with multiple processing elements. Thus, the notification may allow an indication that the processing element has reached the synchronization barrier to be recoded at multiple targets. Each notification may access the PAs of each PE and/or device of a barrier group to update a corresponding counter. The PEs and/or devices may poll or otherwise use the counter to determine when each PE of the group has reached the synchronization barrier. When a corresponding counter indicates synchronization at the synchronization barrier, a PE may proceed with performing a compute task asynchronously with one or more other PEs until a subsequent synchronization barrier may be reached.