In-Switch Collective Processing to Reduce HPC Operation Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing high-performance computing (HPC) systems face performance bottlenecks and latency issues in collective operations due to centralized data traffic management, particularly in collective operations like all-reduce and barrier operations.
Innovation Solution
Implementing in-switch collective primitive processing by configuring a group of collective engines as a distributed collective engine, where a master collective engine communicates and distributes collective commands and results across multiple switches, reducing the need for single-point congestion through a distributed network architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If centralized data traffic management is used in HPC systems, then data flow control is simplified, but performance bottlenecks and latency increase
Solution Approach 1:
The patent divides the centralized data traffic management into distributed collective engines across multiple switches. Each switch becomes an independent collective engine that can process collective operations locally, segmenting the monolithic management structure into modular units that operate in parallel, thereby reducing bottlenecks and latency while maintaining manageable complexity through standardized interfaces
Solution Approach 2:
The patent introduces a new dimension of parallelism by distributing collective engines across multiple network switches rather than concentrating them in a single point. This spatial distribution across the network fabric creates additional processing paths and parallel execution opportunities, transforming the single-point bottleneck into a multi-dimensional parallel processing architecture that improves throughput and reduces latency
2Productivity
If distributed collective engines are implemented across multiple switches, then latency and bottlenecks are reduced, but system complexity increases
Solution Approach 1:
The patent implements universal collective engine functionality across multiple switches, where each switch can act as a collective engine capable of performing the same set of collective operations. This multi-functionality allows any switch to take on the role of a collective engine, providing flexibility and load distribution while maintaining a consistent interface and behavior model that simplifies the overall system architecture despite the distributed nature
Solution Approach 2:
The patent introduces master collective engines as intermediary components that coordinate between the distributed collective engines and the source node. These master engines act as mediators that manage the complexity of distributed operations by handling configuration, coordination, and result aggregation, thereby shielding the underlying complexity from both the source node and the subordinate collective engines while enabling efficient parallel execution
3Productivity
If single-point congestion is avoided through distributed architecture, then performance improves, but communication overhead increases
Solution Approach 1:
The patent implements preliminary action by having master collective engines pre-process and prepare collective commands before distributing them to subordinate engines. Configuration information and command structures are prepared in advance at the master engine level, and the distributed architecture is pre-configured with knowledge of the collective operation parameters, thereby reducing the actual execution time and overhead during runtime while maintaining the performance benefits of distribution
Data Source
AI summary
Provided are systems, methods, and apparatuses for distributed, in-switch collective processing. In one or more examples, the systems, devices, and methods include configuring the group of collective engines as one or more logical collective engines. The systems, devices, and methods include distributing a collective message from the source node to a master engine of all, or some portion of, the group of collective engines, and the collective engines exchanging one or more collective messages with participating engines and end nodes of the collective to satisfy the collective operation. Multiple collective contexts may be supported simultaneously with either the same or a different collective engine instance serving as the master instance for a given collective context.