Commodity Switch Load Balancer with ASIC Flow State Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancing systems face inefficiencies and bottlenecks when handling changes in server configurations, such as additions, updates, or removals, due to the use of consistent hashing and limited memory resources in hardware load balancers, leading to potential data errors and system integrity issues.
Innovation Solution
Implementing a Top of Rack (TOR) switch with a Software for Open Networking in the Cloud (SONiC) platform on commodity switch hardware, which maintains or retrieves flow state information using an application-specific integrated circuit (ASIC) and remote direct memory access (RDMA) to manage flow state transformations and network address translation, thereby maintaining system integrity and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If consistent hashing is used to organize data on servers, then load balancing is enabled, but system integrity deteriorates when servers are added, updated, or removed
Solution Approach 1:
The system performs preliminary actions by proactively detecting configuration changes (server additions, updates, or removals) and preemptively updating flow state information before it affects data routing. This prevents integrity issues before they occur rather than reacting after problems arise.
Solution Approach 2:
The system implements feedback mechanisms where the load balancer continuously monitors server configuration changes and adjusts flow state information accordingly. This closed-loop approach ensures that the load balancing system adapts to changes while maintaining system integrity and data consistency.
2Device complexity
If hardware load balancers use limited memory resources, then device complexity is reduced, but productivity deteriorates due to bottlenecks at the multiplexer
Solution Approach 1:
The patent replaces traditional mechanical/CPU-based multiplexing operations with hardware-accelerated flow state transformations. By using dedicated hardware circuits to handle flow state management and packet routing decisions, the system eliminates software bottlenecks while maintaining hardware simplicity.
Solution Approach 2:
The system segments the load balancing function into distinct hardware components: flow state information storage, flow state transformation logic, and packet routing decisions. This segmentation allows each component to be optimized independently, improving overall productivity without increasing overall device complexity.
3Measurement precision
If flow state information is maintained in hardware load balancers, then routing accuracy is improved, but device complexity increases
Solution Approach 1:
The system addresses memory constraints by utilizing network dimension - specifically, leveraging the existing network infrastructure and communication protocols to store and share flow state information. This distributes the memory burden across the network rather than concentrating it in a single hardware device.
Solution Approach 2:
The hardware load balancer is designed with multi-functionality, serving both as a traditional load balancing device and as a flow state management system. By integrating multiple functions into a single device, the system avoids the complexity of separate dedicated components while maintaining routing accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A Top of Rack (TOR) switch operating with a Software for Open Networking in the Cloud (SONiC) platform is implemented using commodity switch hardware and is configured to maintain or retrieve flow state information for incoming data flows in a load balancer. In one embodiment, an application-specific integrated circuit (ASIC) informs a user mode container flow state information for each incoming data flow. The user mode container informs the ASIC of any affected flows that may result pursuant to a modified distributed system (e.g., added, updated, or removed servers). In other embodiments, the ASIC may utilize remote direct memory access (RDMA) to retrieve flow state information maintained by a remote device or may utilize the RDMA to retrieve network address translation (NAT) information for incoming traffic. In each of the implementations, the integrity of the load balancing system is maintained when the distributed system of servers changes.