Multi-Tile Processor Synchronization via Domain Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Bulk synchronous parallel (BSP) communication schemes in multi-tile processors face challenges in synchronizing workloads across different tiles, leading to latency and throughput issues due to the need for global synchronization and queuing, which contaminates time-deterministic exchanges with non-time-deterministic communications.

Innovation Solution

The method involves separating BSP processes into time-deterministic and non-time-deterministic stages, using a time-deterministic interconnect within domains and a non-time-deterministic interconnect between domains, with internal and external barrier synchronizations to maintain time-determinism and avoid queuing within domains, while allowing non-time-deterministic exchanges between domains.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If global synchronization is used in BSP communication schemes, then synchronization across tiles is achieved, but latency increases and time-determinism is lost

Engineering Contradiction:

ImprovesynchronizationVSAvoidlatency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system segments the multi-tile processor into multiple domains, each with its own time-deterministic interconnect. BSP operations are then segmented into internal exchanges (within domains) and external exchanges (between domains), allowing time-deterministic processing within domains while isolating non-time-deterministic synchronization to boundary points between domains.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different quality characteristics to different parts of the system: time-deterministic interconnects and queuing mechanisms are used within domains for high-performance local exchanges, while non-time-deterministic interconnects are used between domains for synchronization. This local differentiation allows each domain to maintain time-determinism while achieving global synchronization.

Inventive Principle:

Principle #3Local quality

2Ease of operation

If queuing is used for synchronization, then tiles can proceed asynchronously, but throughput decreases due to waiting

Engineering Contradiction:

Improveasynchronous processingVSAvoidthroughput

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The queuing mechanism is segmented and localized to domain boundaries rather than being globally applied. Internal exchanges within domains proceed without queuing, maintaining high throughput. Queuing is only applied at external exchange points between domains, minimizing its impact on overall system throughput while still enabling asynchronous operation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The domain boundary acts as an intermediary layer between internal and external exchanges. This intermediary manages the transition from time-deterministic internal processing to non-time-deterministic external synchronization, allowing tiles to proceed asynchronously within domains while coordinating with other domains through controlled synchronization points.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Speed

If time-deterministic interconnect is used within domains, then exchange speed is improved, but system complexity increases when integrating multiple domains

Engineering Contradiction:

Improveexchange speedVSAvoidsystem integration

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

Multiple domains with identical time-deterministic interconnect architectures are merged into a larger system. Each domain maintains its own time-deterministic interconnect for internal exchanges, while external exchanges between domains use a standardized interface protocol. This merging approach allows high-speed internal processing while managing external complexity through standardization.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system transitions from a flat single-domain architecture to a hierarchical multi-domain structure. This dimensional change allows time-deterministic interconnects to operate efficiently within each domain while introducing a higher-level domain management layer that handles inter-domain synchronization, effectively distributing and managing complexity across multiple levels.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS11023413B2Synchronization in a multi-tile, multi-chip processing arrangement

Publication Date: 2021.06.01 GRAPHCORE LTD

AI summary

A method of operating a system comprising multiple processor tiles divided into a plurality of domains wherein within each domain the tiles are connected to one another via a respective instance of a time-deterministic interconnect and between domains the tiles are connected to one another via a non-time-deterministic interconnect. The method comprises: performing a compute stage, then performing a respective internal barrier synchronization within each domain, then performing an internal exchange phase within each domain, then performing an external barrier synchronization to synchronize between different domains, then performing an external exchange phase between the domains.