Gateway Sync Networks for Asynchronous Accelerator Groups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high-performance computing systems, particularly in machine intelligence applications, there is a challenge in synchronizing data exchange between multiple accelerators and gateways efficiently, especially when scaling subsystems, as existing methods often require global synchronization, which can lead to performance bottlenecks and inefficiencies.
Innovation Solution
Implementing multiple independent sync networks within a gateway that allows for asynchronous and simultaneous synchronization between different synchronization groups, enabling each group to synchronize independently during overlapping exchange phases without the need for a global sync mechanism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If global synchronization is used to coordinate data exchange between multiple accelerators and gateways, then data consistency is ensured, but system performance deteriorates due to synchronization bottlenecks and idle waiting time
Solution Approach 1:
The patent divides the synchronization system into multiple independent sync networks, each handling synchronization for specific accelerator groups. This segmentation allows different groups to synchronize independently without blocking others, resolving the contradiction by maintaining data consistency within each segment while enabling parallel progress across segments, thus improving overall system performance.
Solution Approach 2:
The patent introduces a hierarchical dimension to synchronization by organizing sync networks at different levels (gateway level, accelerator group level, individual accelerator level). This multi-dimensional approach allows simultaneous synchronization operations at different levels, ensuring data consistency where needed while avoiding global bottlenecks, thereby resolving the performance contradiction.
2Reliability
If global synchronization is implemented to coordinate exchange phases, then data exchange correctness is maintained, but system scalability worsens due to increased synchronization overhead
Solution Approach 1:
By segmenting the sync system into independent networks, the patent enables the addition of more accelerators and gateways without proportionally increasing global synchronization overhead. Each segment manages its own synchronization, allowing the system to scale while maintaining data exchange correctness through localized sync protocols.
Solution Approach 2:
The patent introduces gateway-level synchronization mediators that coordinate between independent sync networks without requiring global synchronization. These intermediaries maintain data exchange correctness by managing cross-network dependencies while allowing each network to operate autonomously, thus enabling scalability.
3Reliability
If frequent synchronization is performed to maintain data consistency, then data accuracy is improved, but system efficiency deteriorates due to increased synchronization frequency
Solution Approach 1:
The patent applies different synchronization frequencies and protocols to different sync networks based on their specific requirements. Critical data paths maintain high synchronization frequency for accuracy, while less critical paths use lower frequency to preserve efficiency. This local differentiation resolves the contradiction by optimizing each path independently.
Solution Approach 2:
The patent implements dynamic synchronization where the sync frequency and depth are adjusted based on real-time system state, data criticality, and workload characteristics. This dynamic approach maintains data accuracy when needed while reducing synchronization overhead during routine operations, resolving the efficiency-accuracy tradeoff.
Data Source
AI summary
A gateway implementing multiple independent sync networks. The independent sync networks can be used to allow for synchronisation between different synchronisation groups of accelerators. The independent sync networks allow synchronisations to be carried out asynchronously and simultaneously. The gateway has sync propagation circuitry that receives a first synchronisation request for an upcoming exchange phase and propagates this sync request through a first sync network. The first synchronisation request is a request for synchronisation between subsystems of a first synchronisation group. The sync propagation circuitry of the gateway also receives a second synchronisation request for a different exchange phase and propagates this sync request through the second sync network. The second synchronisation request is a request for synchronisation between subsystems of a second synchronisation group. The two exchange phases overlap in time. Therefore, the syncs are simultaneous and asynchronous.