Hierarchical Topology for Heterogeneous Accelerator Data Sync

Solution Overview

Problem

Existing deep learning model training methods require homogeneous acceleration devices for synchronous data parallelism, leading to inefficient use of heterogeneous devices like GPUs and FPGAs in data centers.

Innovation Solution

The method constructs hierarchical ring topologies among heterogeneous acceleration devices, utilizing the cache coherence protocol for communication, and performs scatter_reduce and all_gather operations to synchronize data across devices, optimizing bandwidth and latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If homogeneous acceleration devices are used for synchronous data parallelism, then data communication efficiency is improved, but hardware resource utilization deteriorates

Engineering Contradiction:

Improvedata communication efficiencyVSAvoidhardware resource utilization

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent segments the communication topology into two levels: first-level ring topologies for homogeneous devices and second-level ring topologies for heterogeneous devices. This segmentation allows each level to optimize for its specific device type while maintaining overall system efficiency and resource utilization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a cache coherence protocol as an intermediary mechanism to enable efficient communication between heterogeneous acceleration devices. This protocol acts as a mediator that allows different device types to communicate directly without CPU intervention, resolving the contradiction between communication efficiency and hardware utilization.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If heterogeneous acceleration devices are used for synchronous data parallelism, then hardware resource utilization is improved, but data communication efficiency deteriorates

Engineering Contradiction:

Improvehardware resource utilizationVSAvoiddata communication efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent segments the communication topology into two levels: first-level ring topologies for homogeneous devices and second-level ring topologies for heterogeneous devices. This segmentation allows each level to optimize for its specific device type while maintaining overall system efficiency and resource utilization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a cache coherence protocol as an intermediary mechanism to enable efficient communication between heterogeneous acceleration devices. This protocol acts as a mediator that allows different device types to communicate directly without CPU intervention, resolving the contradiction between communication efficiency and hardware utilization.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If CPU is used as intermediate medium for communication between heterogeneous devices, then adaptability is improved, but communication speed deteriorates

Engineering Contradiction:

Improvedevice compatibilityVSAvoidcommunication speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The patent replaces the CPU intermediary with a cache coherence protocol intermediary for heterogeneous device communication. This new intermediary maintains device compatibility through standardized cache protocols while enabling direct high-speed communication between acceleration devices, eliminating the speed bottleneck of CPU-mediated communication.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent substitutes the CPU-based communication mechanism with a cache coherence protocol-based mechanism. This replacement transitions from a general-purpose processing intermediary to a specialized cache-based communication path, achieving both compatibility and high performance.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12289213B2Data synchronization method and apparatus, and device and storage medium

Publication Date: 2025.04.29 LANGCHAO ELECTRONIC INFORMATION IND CO LTD

AI summary

A data synchronization method and apparatus, and a device and a storage medium are disclosed. The method comprises: constructing first-level physical topologies between acceleration devices of the same type, and constructing second-level physical topologies between acceleration devices of different types, the acceleration devices in the second-level physical topology are connected by using a cache coherence protocol; performing first processing on to-be-synchronized data among the acceleration devices through scatter reduce communication based on the first-level physical topologies, and performing second processing on data after the first processing among the acceleration devices through scatter reduce communication based on the second-level physical topologies; and performing third processing on data after the second processing among the acceleration devices through all gather communication based on the second-level physical topologies, and performing fourth processing on data after the third processing among the acceleration devices through all gather communication based on the first-level physical topologies.