Base Transport Header Load Balancing for Ordered AI/ML Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancing techniques, such as static load balancing (SLB) and dynamic load balancing (DLB), fail to efficiently handle AI/ML workloads due to issues like flow collisions, link saturation, suboptimal bandwidth use, and out-of-order packet reception, leading to increased latency and resource consumption.
Innovation Solution
A network device utilizes base transport headers (BTH) to identify packet characteristics, enabling per-packet load balancing by applying DLB for flows capable of out-of-order reception and SLB for others, optimizing traffic distribution based on opcode values and sequence numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If static load balancing (SLB) is used to distribute traffic across multiple links, then link utilization is improved, but flow collisions and out-of-order packet reception occur leading to increased latency
Solution Approach 1:
The system dynamically switches between SLB and DLB modes based on flow characteristics and network conditions. For AI/ML workloads with large packet sizes, SLB is used to maintain order; for smaller packets, DLB is applied to maximize link utilization, thus dynamically optimizing both productivity and latency
Solution Approach 2:
The invention changes the load balancing parameter selection criteria based on packet characteristics (size, type, opcode). By modifying which header fields are used for hashing (e.g., using 5-tuple for SLB vs. flow ID for DLB), the system adapts to different workload requirements, resolving the contradiction between utilization and latency
2Productivity
If dynamic load balancing (DLB) is used to maximize bandwidth utilization, then link saturation is reduced, but out-of-order packet reception increases causing retransmissions and resource consumption
Solution Approach 1:
The invention segments traffic into different categories based on packet size and type. Large AI/ML packets are routed through SLB paths that guarantee ordering, while smaller packets can use DLB paths for maximum utilization. This segmentation allows simultaneous optimization of reliability and productivity for different traffic classes
Solution Approach 2:
Different load balancing strategies are applied to different parts of the traffic flow based on local characteristics. Each packet is evaluated individually, and the appropriate balancing mode (SLB or DLB) is selected based on its specific requirements, ensuring that ordering-critical packets receive ordered delivery while other packets maximize bandwidth
3Productivity
If per-packet load balancing is applied to all traffic, then link saturation is minimized, but hardware resource consumption increases due to complex processing
Solution Approach 1:
Instead of applying complex DLB to all packets, the invention applies it only partially to packets that can benefit from it (smaller packets, non-AI/ML traffic). Large AI/ML packets use simpler SLB processing, reducing overall hardware resource consumption while still achieving good traffic distribution for the majority of traffic that requires it
Data Source
AI summary
In some implementations, a network device may receive a packet containing a base transport header (BTH). The network device may identify a value in the BTH. The network device may load-balancing, by the network device, the packet using the value in the BTH.