Data Center Network Load Balancing via Node Modulo Path Binding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancing technologies in data center networks, particularly in AI cluster training scenarios, fail to uniformly distribute elephant flows, leading to network congestion and packet loss due to the use of static ECMP routing and RDMA over Ethernet, which is inadequate for handling large bandwidth flows.
Innovation Solution
A data center network load balancing method that determines unique forwarding paths for each computing node by calculating remainders from node numbers divided by the quantity of equal-cost paths, ensuring each node is uniformly bound to a distinct path, using BGP route information and IP addresses for forwarding decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If static ECMP routing is used for load balancing, then the routing implementation is simple, but the data flows are not uniformly distributed leading to network congestion and packet loss
Solution Approach 1:
The patent changes the routing parameter from random selection to deterministic selection based on node ID modulo operation. Each computing node's data flows are assigned to specific equal-cost paths using the formula: path_index = node_id % num_paths, where num_paths is the number of equal-cost paths. This parameter change ensures uniform distribution of data flows across all paths while maintaining simple routing implementation through deterministic path selection.
2Adaptability or versatility
If RDMA over Ethernet is used for communication, then the communication protocol is standardized, but it cannot handle large bandwidth flows effectively in AI cluster training
Solution Approach 1:
The patent segments the network traffic into multiple equal-cost paths based on computing node identification. By dividing the data center network traffic into multiple parallel flows assigned to different paths using modulo operation, the system handles large bandwidth AI cluster training workloads effectively while maintaining standardized RDMA over Ethernet communication protocol.
Data Source
AI summary
This application provides a data center network load balancing method, and system, a communication device, and a storage medium. The method includes: obtaining information of a plurality of computing nodes; determining, based on the information of the plurality of computing nodes, forwarding paths corresponding to the plurality of computing nodes, where the forwarding paths corresponding to the plurality of computing nodes are equal-cost forwarding paths, and the forwarding paths corresponding to the plurality of computing nodes are different from each other; and when receiving a data flow, forwarding the data flow based on a forwarding path corresponding to a target computing node of the data flow.