Distributed Load Balancer Connection Publishing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancers face scalability issues in large network environments, such as cloud computing services, and typically act as proxies that terminate TCP connections, preventing direct communication between clients and servers, which can lead to inefficiencies in load balancing decisions.
Innovation Solution
A distributed load balancing system that routes TCP packets from clients to servers without termination, using load balancer nodes to randomly select servers and allowing local decision-making based on server metrics, employing techniques like multipath routing, consistent hashing, and health checking to manage packet flows and connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional load balancers are used as dedicated boxes with multiple NICs, then connection management and load distribution can be achieved, but scalability is limited in large network environments with hundreds or thousands of host devices
Solution Approach 1:
The patent segments the monolithic load balancer into multiple distributed load balancer nodes that operate independently across the network. Each node handles a portion of the total traffic, enabling the system to scale horizontally by adding more nodes rather than relying on a single dedicated box with limited capacity.
2Ease of operation
If conventional load balancers terminate TCP connections and act as proxies, then connection management is simplified, but direct communication between clients and servers is prevented, leading to inefficiencies
Solution Approach 1:
The patent extracts the TCP connection termination function from the load balancer nodes. Instead of terminating connections, the load balancer nodes only perform routing decisions based on connection publishing information, allowing direct TCP connections between clients and servers while maintaining simplified load distribution.
3Productivity
If load balancers use techniques like max connections, round robin, and least connections, then load distribution can be achieved, but the decisions are made based on collected data which may not reflect real-time server status
Solution Approach 1:
The patent implements connection publishing where servers proactively publish their connection status and capacity information to load balancer nodes before requests arrive. This preliminary action ensures that load balancing decisions are based on current, accurate server status rather than potentially stale collected data, improving the reliability of load distribution.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A connection publishing method for a distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. A load balancer (LB) module on each server node determines and publishes a list of active connections between client(s) and the server (a connection publishing packet) back to the LB nodes. The LB module may randomly select a LB node to receive the packet. Upon receiving the packet, the LB node processes the connection information in the packet, distributing the connection information among appropriate LB nodes. Receiving the connection publishing information from the server nodes refreshes leases for the connections on the LB nodes.