History-Based Connection Affinity for Network Load Balancers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network load balancers face memory constraints due to resilient hashing and stateful load balancing, making it impractical to build ASIC-based solutions as they require significant memory for hash table replication and per-connection state information, exceeding the capacity of existing ASICs.
Innovation Solution
Implementing history-based connection-server affinity and improved resilient hashing schemes such as resilient hashing with multiple hashes and compression, which reduce memory usage by maintaining server history information on a per-bucket basis and using hierarchical or compressed hash tables, allowing for efficient load balancing without extensive memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional resilient hashing and stateful load balancing are implemented, then connection affinity and traffic distribution are achieved, but memory usage exceeds ASIC capacity
Solution Approach 1:
The patent segments the connection-server affinity mechanism into two parts: (1) a hash table that maps connection identifiers to server identifiers, and (2) a history table that stores only the previous server identifier for each bucket. This segmentation eliminates the need to store complete per-connection state information while maintaining affinity, reducing memory usage from O(N) to O(M) where M is the number of buckets and N is the number of connections.
Solution Approach 2:
Instead of storing complete connection state information for all connections (excessive action), the patent stores only the necessary historical server mapping information (partial action) in the history table. This partial information is sufficient to restore connection affinity when server mappings change, achieving the required reliability with minimal memory overhead suitable for ASIC implementation.
2Quantity of substance
If software-based load balancers are used, then memory capacity requirements are met, but network bandwidth is orders of magnitude less than ASIC-based devices
Solution Approach 1:
The patent changes the fundamental parameters of the data structure from storing complete connection states (software approach) to storing only bucket-to-server mappings and historical server identifiers (hardware-friendly approach). This parameter change reduces memory requirements to fit within ASIC capacity while enabling hardware-accelerated network bandwidth performance.
3Reliability
If per-connection state information is stored for all connections, then connection affinity is maintained, but memory usage becomes impractical for hardware implementation
Solution Approach 1:
The patent extracts only the essential information needed for connection affinity from the complete per-connection state. By taking out just the bucket identifier and previous server identifier pairs, the system maintains connection affinity functionality while eliminating the complexity of storing and managing complete connection states, making the memory structure practical for hardware implementation.
Data Source
AI summary
Techniques for implementing history-based connection-server affinity on a network load balancer are provided. In one set of embodiments, the network load balancer can receive a network packet destined for a service, where the service is associated with a plurality of servers, and where the packet is part of a network connection between a client device and one of the plurality of servers. The network load balancer can further compute, using a portion of the packet, a bucket identifier of a bucket for the network connection, identify a first server in the plurality of servers that is currently mapped to the bucket identifier in a hash table, and send the packet to the first server. If the network load balancer receives the packet back from the first server, the network load balancer can determine, based on local history information, a second server that was previously mapped to the bucket identifier in the hash table and send the packet to that second server.