Anycast Load Balancing via Two-Layer Hash Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data center load balancing systems face challenges in efficiently distributing traffic across multiple application instances and geographical locations, leading to latency and capacity issues, especially when demand exceeds server capacity.
Innovation Solution
A method and system for load balancing anycast traffic using a set of load balancing devices that maintain data structures reflecting weight values associated with application instances and load balancers, generating hash values from packet headers to route data packets to appropriate instances based on capacity and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single layer of load balancers is used to distribute traffic directly to application instances, then the system structure is simple, but the system cannot efficiently handle excess demand and lacks capacity optimization across multiple locations
Solution Approach 1:
The load balancing system is divided into two distinct layers: first-layer load balancers that receive anycast traffic and make initial routing decisions, and second-layer load balancers that provide fine-grained load balancing to application instances. This segmentation allows each layer to specialize in specific functions, improving overall traffic distribution efficiency while maintaining manageable complexity through clear separation of concerns
Solution Approach 2:
The patent introduces a vertical dimension to the load balancing architecture by adding a second layer of load balancers above the first layer. Instead of a single flat layer, the system now has multiple hierarchical levels that process traffic in stages, enabling more sophisticated load distribution strategies and capacity optimization across geographical locations
2Productivity
If load balancers use simple hashing of packet headers for load distribution, then the implementation is straightforward, but the system cannot account for server capacity and load conditions leading to uneven distribution
Solution Approach 1:
The load balancers maintain dynamic data structures that are continuously updated based on real-time server capacity and load conditions. Weight values associated with each application instance are dynamically adjusted according to current system state, allowing the hashing function to distribute traffic more effectively across servers with appropriate capacity while avoiding overloaded instances
Solution Approach 2:
The system pre-computes and maintains data structures containing weight values for all application instances before traffic arrives. This preliminary preparation of load information allows the load balancers to make informed routing decisions without complex real-time calculations, balancing effectiveness with implementation feasibility
3Speed
If anycast traffic is routed to the nearest data center without capacity consideration, then routing speed is fast, but the nearest location may be overloaded causing latency and service degradation
Solution Approach 1:
The first-layer load balancers act as intermediaries between the anycast routing system and the second-layer load balancers. They receive anycast traffic and use their data structures to identify appropriate second-layer load balancers based on capacity information, mediating between the speed advantage of anycast routing and the reliability needs of capacity-aware distribution
Solution Approach 2:
The system implements feedback mechanisms where load balancers continuously update their data structures with current capacity and load information from application instances. This feedback loop ensures that routing decisions are based on up-to-date system state, preventing overload conditions while maintaining fast routing through cached information
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In some implementations, a system and corresponding method for load balancing data traffic addressed to an anycast address include maintaining, by each of a plurality of load balancing (LB) devices a data structure including entries associated with application instances in a group of application instances served by the LB device. The frequency with which each served application instance is included in the data structure is indicative of a weight value associated with a capacity of the corresponding served application instance. Upon receiving a data packet, originally addressed to an anycast address, the LB device uses one or more header fields of the received data packet to identify a virtual Internet protocol (IP) address of one of the served application instances in the data structure maintained by the LB device. The LB device then forwards the data packet to the identified application instance.