Stateless Load Balancer Using Consistent Hashing for Session Affinity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load-balancing systems face challenges in scaling out due to the need for maintaining consistent state among load balancers, leading to increased costs and complexity, with a single load balancer being a point of failure and bottleneck, and the cost of scaling up being non-linear with throughput.
Innovation Solution
A software architecture for stateless, affinity-preserving load balancing using replicated packet forwarders that share no state, employing routing functions to distribute packets among hosts based on static protocols, allowing for flexible scaling and constant per-packet processing costs, and rapidly responding to changing load conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single monolithic load-balancer is used to monitor host status and maintain session state, then session affinity and load balancing are achieved, but the system becomes a single point of failure and bottleneck
Solution Approach 1:
The patent segments the monolithic load-balancer into multiple distributed forwarders that each independently perform load balancing without sharing state. Each forwarder maintains no session state, eliminating the single point of failure while preserving session affinity through consistent hashing of session identifiers across the distributed forwarder group.
Solution Approach 2:
The patent extracts the session state management function from the load-balancing function. Forwarders perform load balancing without maintaining session state, instead using session identifiers to consistently route packets belonging to the same session to the same host through coordinated forwarding among multiple forwarders.
2Productivity
If the session table size increases to handle more throughput, then more sessions can be tracked, but the per-packet processing cost increases due to larger table lookups
Solution Approach 1:
The patent extracts session state management from the forwarders, allowing them to perform simple stateless packet forwarding without maintaining large session tables. Session affinity is achieved through consistent hashing of session identifiers rather than through table lookups, maintaining constant per-packet processing cost regardless of system scale.
Solution Approach 2:
The patent uses consistent hashing functions that can be replicated across multiple forwarders without requiring shared state. Each forwarder independently computes the same routing decision for a given session identifier, enabling horizontal scaling without increasing per-packet processing complexity.
3Reliability
If multiple load-balancers are deployed to scale out and eliminate single point of failure, then reliability improves, but state consistency maintenance becomes complicated
Solution Approach 1:
The patent removes session state management from the distributed forwarders, allowing them to operate independently without requiring state consistency protocols. Each forwarder uses session identifiers with consistent hashing to independently determine routing decisions, eliminating the complexity of maintaining shared state while achieving fault tolerance through redundancy.
Solution Approach 2:
The patent deploys identical copies of the stateless forwarder software across multiple nodes. Each forwarder independently implements the same consistent hashing function and packet forwarding logic, enabling simple linear scaling without state synchronization overhead while maintaining reliability through redundancy.
4Productivity
If larger and more powerful load-balancers are used to scale up throughput, then processing capacity increases, but cost increases non-linearly
Solution Approach 1:
The patent segments the load-balancing function across multiple standard-sized forwarder nodes rather than using a single large load-balancer. This enables linear scaling where adding capacity simply involves adding more identical forwarder nodes, avoiding the non-linear cost increases associated with scaling up individual devices.
Solution Approach 2:
The patent uses identical copies of the forwarder software running on standard hardware nodes to scale throughput. Each node performs the same stateless packet forwarding function, allowing capacity to be scaled linearly by adding more nodes without the diminishing returns and non-linear cost increases characteristic of scaling up individual devices.
Data Source
AI summary
The invention relates to an architecture that facilitates load balancing among a plurality of hosts and preserve session affinity to a given host. An incoming stream of data packets that include packet sessions is input to one or more forwarding mechanisms for forwarding to one or more hosts. The forwarders generate a routing function that takes into consideration host availability, and distributes session packets according to the routing function. A session is distributed to the same host to preserve session affinity. When host availability changes, a new routing function is generated, such that any new session is routed according to the new routing function and existing sessions are routed according to the old routing function. When the old routing function becomes irrelevant, it is phased out. An optimization utilizes a maximally backward compatible hash function to minimize the differences between the old and new routing functions.