Topology-Aware Parallel Reduction for Accelerator Bandwidth Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for communication among accelerators are inefficient due to idle computing capacity during the reduction phase, as they do not optimize synchronization and fail to utilize the full-duplex PCIe bandwidth effectively, leading to wasted resources.
Innovation Solution
A topology-aware parallel reduction system that partitions data across accelerators based on their connection topology, utilizing intra-root, intra-node, and inter-node reduction methods to maximize full-duplex PCIe bandwidth and parallelize data transfer, ensuring simultaneous data transfer in both directions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional reduction techniques are used where accelerators work independently and then reduce results sequentially, then implementation simplicity is maintained, but accelerator utilization efficiency deteriorates due to idle computing capacity during the reduction phase
Solution Approach 1:
The reduction operation is segmented into multiple phases (first reduction phase, second reduction phase) with different parallelization strategies. The first phase uses all accelerators in parallel to reduce data to intermediate results, while the second phase uses a subset of accelerators to reduce intermediate results to final results. This segmentation allows continuous utilization of accelerators throughout the reduction process.
Solution Approach 2:
The system dynamically adjusts the number of accelerators involved in different reduction phases based on data size and available resources. The control device determines whether to use all accelerators or a subset for the second reduction phase, optimizing accelerator utilization dynamically rather than using a fixed sequential approach.
2Loss of time
If accelerators transfer data sequentially in the reduction phase, then PCIe bandwidth management is simplified, but synchronization time increases due to accelerators waiting for data to arrive
Solution Approach 1:
Data transfer is segmented into multiple independent transfer operations occurring in parallel. Different accelerators transfer different portions of data simultaneously through the PCIe bus, rather than sequential transfers. This parallel transfer approach significantly reduces the total synchronization time.
Solution Approach 2:
The PCIe bus maintains continuous data transfer activity throughout the reduction phase by coordinating multiple parallel transfer operations. Accelerators continuously send and receive data without idle waiting periods, maximizing the utilization of PCIe bandwidth and eliminating synchronization bottlenecks.
3Productivity
If only one direction of PCIe bandwidth is used for data transfer, then full-duplex PCIe capability is not utilized, but system complexity increases if bidirectional simultaneous transfer is implemented
Solution Approach 1:
The full-duplex PCIe bandwidth is segmented into multiple directional transfer channels. Different accelerators utilize different directions (send/receive) simultaneously, effectively dividing the bandwidth into parallel pathways. This segmentation allows the system to leverage the full dual-directional capacity of the PCIe bus.
Solution Approach 2:
Different accelerators are assigned different roles (sender or receiver) in the data transfer process, optimizing local utilization of PCIe bandwidth. Each accelerator operates at full capacity in its assigned direction, and the collective system achieves full-duplex bandwidth utilization through this localized role assignment.
Data Source
AI summary
A topology-aware parallel reduction method, system, and recording medium including a partitioning device configured to partition data in each accelerator of a plurality of accelerators into partitions based on a topology of connections between the plurality of accelerators and a control device configured to control, based on a topology of connections between the plurality of accelerators, a type of parallel reduction of data to use.