NIC-Based Collective Acceleration Through Parallel Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional in-network reduction systems for complex collective operations like all-reduce increase latency and networking complexity, leading to poor performance in high-performance computing applications.
Innovation Solution
An accelerated compute fabric (ACF) system with multiple reduction processors that parallelizes data reduction operations by distributing input data across multiple processors, using scatter-gather lists to manage data movement and perform operations like summation, maximum, or minimum, thereby reducing data volume and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional in-network reduction is used for complex collective operations, then data reduction is achieved, but latency increases and networking complexity escalates
Solution Approach 1:
The patent divides the data reduction process into multiple segments by distributing data across multiple reduction processors (RP0-RP3) in parallel. Each processor handles a portion of the collective operation simultaneously, breaking down the single-threaded reduction bottleneck into concurrent parallel operations that reduce overall latency while maintaining data volume reduction.
Solution Approach 2:
The patent transitions from a single-dimension sequential reduction approach to a multi-dimensional parallel processing architecture. By introducing multiple reduction processors operating in parallel and using scatter-gather lists for coordinated data distribution, the system adds a temporal parallelism dimension that reduces latency without sacrificing the data reduction capability.
2Quantity of substance
If traditional in-network reduction is used for complex collective operations, then data reduction is achieved, but networking complexity increases
Solution Approach 1:
The patent introduces scatter-gather lists as intermediary data structures that mediate between the network interface controller and multiple reduction processors. These lists organize and coordinate data flow to multiple RPs systematically, reducing the networking complexity by providing a structured intermediary layer that manages the parallel data distribution without requiring complex ad-hoc routing logic.
Solution Approach 2:
The reduction processors are designed as universal multi-functional units that can handle various collective operations (all-reduce, reduce, broadcast) through the same parallel processing architecture. This universality reduces networking complexity by providing a single versatile processing path rather than requiring specialized handling for each operation type.
3Productivity
If data is moved to multiple reduction processors in parallel, then processing speed increases, but buffering requirements increase
Solution Approach 1:
The patent employs preliminary action by pre-allocating and pre-configuring scatter-gather lists before the actual reduction operation begins. The NIC prepares the data distribution plan and allocates buffer resources in advance, allowing parallel data movement to multiple reduction processors to proceed efficiently without requiring excessive dynamic buffering during the operation itself.
Data Source
AI summary
A system and method for producing and transmitting reduced data is disclosed. In some embodiments, the system comprises an ACF and reduction processors. The reduction processors are configured to perform a data reduction process. The ACF is configured to obtain access to input data from multiple flows, the input data identified by SGLEs included in input SGLs, and move a portion of the input data from each flow of the multiple flows to a respective reduction processor of the multiple reduction processors, such that each reduction processor receives a respective portion of the input data from each flow. The ACF is further configured to obtain access to reduced data produced from the input data using the data reduction process performed by the multiple reduction processors and move the reduced data to one or more destinations, where the reduced data identified by an output SGL.