TOR Switch Flow Pinning to Reduce Collective Traffic Congestion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for directing communication flows in network collectives often result in poor performance due to network congestion, out-of-order packet arrival, and the need for specialized NIC hardware, as they fail to consider the overall network congestion and path correlations.
Innovation Solution
A network switch determines flow groups with common sources and destinations, pinning each flow to a corresponding link to evenly distribute them across links, ensuring balanced traffic and preventing congestion by leveraging the temporal correlation of collective flows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If flows are distributed across shared links using hashing functions, then flows are spread across multiple paths, but network congestion occurs and performance deteriorates
Solution Approach 1:
The system performs preliminary identification of collective communication flows and their temporal correlations before routing decisions are made. By recognizing flows that belong to the same collective operation in advance, the system can proactively distribute them across different links to avoid congestion, rather than reacting to congestion after it occurs.
Solution Approach 2:
The system monitors network link utilization and dynamically adjusts flow distribution based on observed congestion patterns. When certain links become congested, the system receives feedback about this condition and redirects subsequent flows from the same collective to less congested links, maintaining optimal performance.
2Productivity
If packets of a single flow are spread across many links, then link utilization improves, but packets arrive out of order requiring specialized NIC hardware
Solution Approach 1:
Instead of splitting a single flow across multiple links, the system segments at the collective level by identifying and grouping flows that belong to the same collective operation. Each grouped flow maintains its packet order integrity while the collection of grouped flows achieves balanced link utilization through distribution across multiple paths.
Solution Approach 2:
The system introduces an intermediary layer of flow grouping and management between the network interface card and the actual data transmission. This intermediary structure allows packets to be distributed across multiple links while maintaining order within each flow group, eliminating the need for specialized NIC hardware to handle out-of-order packet reassembly.
3Ease of operation
If flows are assigned to least congested links based on first-hop information, then initial routing optimizes for current congestion, but subsequent link congestion cannot be predicted
Solution Approach 1:
The system performs preliminary identification and grouping of flows that belong to the same collective operation before routing decisions are made. By recognizing these temporal correlations in advance, the system can proactively distribute entire flow groups across different links, ensuring that subsequent links are not overwhelmed by concentrated traffic from the same collective, thus predicting and preventing future congestion points.
Data Source
AI summary
A network device such as a top-of-rack (TOR) switch receives communication flows for communication collectives from source devices and determines, by importing from the source devices or snooping from the communication traffic, topology information of the communication flows. For each communication collective, the TOR switch determines, based at least in part on the topology information, groups of communication flows that are correlated in time and in destination. For each respective group, the TOR switch pins each communication flow in the respective group to a corresponding network link connected to the TOR switch so that the communication flows in the respective group are evenly distributed across network links connected to the TOR switch.