Multi-Die Barrier Synchronization for Stall-Free ML Thread Groups

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional barrier and synchronization techniques in machine learning are inefficient due to software dependency, limited access across multiple thread groups, and lack of universal functionalities, leading to suboptimal resource utilization in terms of time, memory, power, and bandwidth.

Innovation Solution

A novel multi-die barrier technique is introduced, enabling per-thread synchronization within thread groups and facilitating synchronization across thread groups without shared local memory, along with thread group preemption based on barriers, to optimize scheduling and resource allocation across streaming multiprocessors and graphics processors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional barrier and synchronization techniques are used, then software implementation is flexible, but system resource efficiency deteriorates due to lack of hardware support

Engineering Contradiction:

Improvesoftware flexibilityVSAvoidsystem resource efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent merges barrier synchronization functionality into the hardware architecture of the parallel processing system, specifically implementing barrier logic within the streaming multiprocessors and memory interconnect. This integration allows software barrier instructions to be directly executed by hardware circuits, combining the flexibility of software control with the efficiency of hardware execution, thereby resolving the contradiction between software adaptability and system resource efficiency

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces hardware-mediated barrier synchronization mechanisms that act as intermediaries between software threads and the parallel processing architecture. The hardware barrier logic receives synchronization requests from software, manages the coordination of multiple thread groups across different streaming multiprocessors, and provides feedback to the software system, thereby enabling efficient resource utilization while maintaining software flexibility

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If conventional synchronization techniques are used, then implementation is simple within single thread groups, but access across multiple thread groups deteriorates due to limited functionality

Engineering Contradiction:

Improveimplementation simplicityVSAvoidcross-thread-group access

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent implements universal barrier synchronization functionality that can operate at multiple levels: within individual thread groups, across multiple thread groups, and even across different streaming multiprocessors. The hardware barrier logic is designed to handle diverse synchronization scenarios through a unified interface, allowing the same barrier mechanism to serve both simple intra-group synchronization and complex inter-group coordination, thereby providing ease of operation while enabling versatile cross-thread-group access

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Productivity

If more compute elements are used for machine learning operations, then processing capacity increases, but synchronization overhead increases due to conventional techniques

Engineering Contradiction:

Improveprocessing capacityVSAvoidsynchronization overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent replaces software-based barrier synchronization mechanisms with dedicated hardware circuits implemented within the parallel processing architecture. The hardware barrier logic uses specialized circuits to track thread completion status, generate synchronization signals, and coordinate thread groups, substituting the time-consuming software polling and waiting mechanisms with efficient hardware-level coordination. This mechanical substitution significantly reduces synchronization overhead while enabling the system to utilize a larger number of compute elements for machine learning operations

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11353868B2Barriers and synchronization for machine learning at autonomous machines

Publication Date: 2022.06.07 INTEL CORP

AI summary

One or more examples include an apparatus having a hardware barrier logic to detect thread groups relating to machine learning operations and facilitate barrier synchronization of the thread groups across multiple dies representing multiple processors, such that data processing using the threads groups across the multiple processors is synchronized and stall-free.