Transaction Barrier Synchronization for Asynchronous GPU Workloads

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing hardware-implemented synchronization barriers in GPUs are limited in flexibility and programmability, leading to performance losses when additional software barriers are implemented, and do not efficiently support cross-processor synchronization and asynchronous data exchanges.

Innovation Solution

A hardware-accelerated synchronization primitive, referred to as a 'transaction barrier', which tracks both thread and transaction arrivals, and is integrated with a synchronization unit to provide efficient cross-processor synchronization and asynchronous data exchange, minimizing bandwidth and latency costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If hardware-implemented synchronization barriers are used in GPUs, then synchronization speed is improved, but flexibility and programmability deteriorate

Engineering Contradiction:

Improvesynchronization speedVSAvoidflexibility and programmability

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The patent merges hardware acceleration with software programmability by implementing a transaction barrier system where hardware tracks transaction arrivals while software maintains barrier state and controls synchronization logic. This combination allows the system to achieve hardware-speed synchronization while retaining software flexibility for configuring barrier behavior, handling multiple processors, and supporting asynchronous operations.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The transaction barrier mechanism serves multiple functions: it synchronizes threads within a processor, coordinates across multiple processors, tracks asynchronous transaction arrivals, and provides both hardware-accelerated performance and software-configurable behavior. This multi-functionality resolves the contradiction by making a single mechanism adaptable to diverse synchronization scenarios.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If additional software barriers are implemented to improve flexibility, then adaptability is improved, but performance deteriorates

Engineering Contradiction:

ImproveflexibilityVSAvoidperformance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The hardware transaction tracker acts as an intermediary between software barriers and the actual synchronization point. Software can configure flexible barrier behavior while the hardware intermediary handles the time-critical counting and arrival tracking, offloading performance-critical operations from software to hardware and thus maintaining flexibility without sacrificing performance.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If cross-processor synchronization is implemented, then coordination capability is improved, but bandwidth and latency costs increase

Engineering Contradiction:

Improvecoordination capabilityVSAvoidbandwidth and latency costs

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The synchronization system is segmented into processor-local hardware tracking (which operates with minimal bandwidth consumption) and selective software coordination (which handles cross-processor communication only when necessary). This segmentation reduces bandwidth costs by keeping most synchronization operations local while maintaining cross-processor coordination capability through the transaction barrier mechanism.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12536056B2Hardware accelerated synchronization with asynchronous transaction support

Publication Date: 2026.01.27 NVIDIA CORP

AI summary

A new transaction barrier synchronization primitive enables executing threads and asynchronous transactions to synchronize across parallel processors. The asynchronous transactions may include transactions resulting from, for example, hardware data movement units such as direct memory units, etc. A hardware synchronization circuit may provide for the synchronization primitive to be stored in a cache memory so that barrier operations may be accelerated by the circuit. A new wait mechanism reduces software overhead associated with waiting on a barrier.