Multi-Die Barrier Synchronization for ML Thread Group Scheduling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional barrier and synchronization techniques in parallel graphics processing are inefficient due to software dependency, limited access across multiple thread groups, and lack of universal functionalities, leading to suboptimal resource utilization in terms of time, memory, power, and bandwidth.

Innovation Solution

A novel multi-die barrier technique is introduced, enabling per-thread synchronization within thread groups and facilitating synchronization across thread groups without shared local memory, along with thread group preemption based on barriers, to optimize scheduling and data sharing across graphics processors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional barrier and synchronization techniques are used in parallel graphics processing, then software implementation flexibility is maintained, but system resource utilization (time, memory, power, bandwidth) deteriorates due to inefficiency

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidsystem resource utilization

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent replaces software-based barrier and synchronization techniques with hardware-based mechanisms implemented in the graphics processor. This substitution eliminates software dependency and enables direct hardware control over thread group synchronization, significantly improving processing efficiency while reducing the overhead of software intervention and optimizing system resource utilization including time, memory, power, and bandwidth.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces new parameters and control mechanisms for barrier synchronization at the hardware level, including per-thread synchronization within thread groups and cross-thread-group synchronization. These parameter changes enable more granular and efficient control over parallel processing operations, allowing the system to optimize resource allocation and reduce waste in time, memory, power, and bandwidth consumption.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If conventional synchronization techniques are used, then implementation simplicity is maintained, but access across multiple thread groups is limited

Engineering Contradiction:

Improveaccess across thread groupsVSAvoidsynchronization mechanism complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements a universal barrier synchronization mechanism that can operate across multiple thread groups simultaneously. The hardware-based barrier unit provides multi-functional capability to synchronize not only within single thread groups but also across multiple thread groups, enabling versatile access patterns while maintaining a unified synchronization interface that does not significantly increase overall system complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent segments the synchronization mechanism into hierarchical levels: per-thread synchronization within thread groups, and cross-thread-group synchronization. This segmentation allows independent optimization of each level while maintaining overall system coherence, enabling flexible access across multiple thread groups without requiring a monolithic complex synchronization structure.

Inventive Principle:

Principle #1Segmentation

3Productivity

If thread groups are restricted to specific compute elements, then local memory access is optimized, but scheduling flexibility across compute elements deteriorates

Engineering Contradiction:

ImprovethroughputVSAvoidscheduling flexibility

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements dynamic scheduling of thread groups across compute elements based on barrier synchronization completion status. The hardware barrier mechanism enables the scheduler to dynamically allocate thread groups to compute elements as they become ready, optimizing throughput by keeping compute elements busy while maintaining flexibility to adapt to varying workload conditions and resource availability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent uses preliminary barrier synchronization to prepare thread groups for execution before they are scheduled to compute elements. By establishing synchronization points in advance, the system can pre-coordinate data sharing and state synchronization, enabling more flexible scheduling decisions without compromising local memory access optimization or overall throughput.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3901774B1Barriers and synchronization for machine learning at autonomous machines

Publication Date: 2024.10.30 INTEL CORP

Figure 1
Figure 2A
Figure 2B

AI summary

A mechanism is described for facilitating barriers and synchronization for machine learning at autonomous machines. A method of embodiments, as described herein, includes detecting thread groups relating to machine learning associated with one or more processing devices. The method may further include facilitating barrier synchronization of the thread groups across multiple dies such that each thread in a thread group is scheduled across a set of compute elements associated with the multiple dies, where each die represents a processing device of the one or more processing devices, the processing device including a graphics processor.