Distributed Load Balancer Using Per-Flow Hashed Multipath Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancers face scalability issues in large network environments, such as cloud computing services, and rely on outdated techniques like max connections and round robin for load balancing, which are inefficient and difficult to manage in distributed systems.
Innovation Solution
A distributed load balancing system that employs multipath routing, consistent hashing, and health checking to route TCP packets directly between clients and servers, allowing server nodes to make local decisions on connection acceptance based on current status, and uses load balancer nodes to randomly select servers for connection requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional single dedicated load balancers are used, then load balancing function is provided, but scalability is poor in large network environments with hundreds or thousands of host devices
Solution Approach 1:
The patent divides the conventional single load balancer into multiple distributed load balancer nodes that operate independently across the network. Each node handles a portion of the load balancing traffic, enabling the system to scale horizontally by adding more nodes as needed to support hundreds or thousands of host devices.
Solution Approach 2:
The patent transitions from a centralized load balancer architecture to a distributed architecture that operates across multiple network dimensions. Load balancing decisions are made locally at each node based on real-time conditions, rather than through a single centralized point, enabling better scalability and reduced bottlenecks.
2Speed
If conventional load balancers terminate TCP connections and act as proxies, then connection management is centralized, but latency increases and direct communication between clients and servers is blocked
Solution Approach 1:
The patent extracts the TCP connection termination function from the load balancer nodes and transfers it to the server nodes themselves. Load balancer nodes only perform routing decisions and forward packets, while servers establish and manage their own TCP connections directly with clients, eliminating the proxy overhead and reducing latency.
Solution Approach 2:
Servers are empowered to manage their own TCP connections directly with clients without requiring the load balancer to terminate and re-establish connections. Each server independently handles connection establishment, maintenance, and termination, reducing the operational complexity and latency associated with centralized connection management.
3Productivity
If conventional load balancers use max connections, round robin, or least connections techniques, then load distribution is achieved, but management becomes difficult in distributed systems and efficiency decreases
Solution Approach 1:
The patent implements dynamic load balancing where each load balancer node makes real-time routing decisions based on current server conditions such as available capacity, response time, and health status. This dynamic approach replaces static algorithms like round robin with adaptive local decision-making that automatically optimizes load distribution without complex centralized management.
Solution Approach 2:
The system incorporates feedback mechanisms where load balancer nodes continuously monitor server performance metrics and use this information to make informed routing decisions. Servers provide feedback about their current state, and load balancer nodes adjust their routing behavior accordingly, creating an efficient self-regulating system that is easy to manage in distributed environments.
Data Source
AI summary
A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes according to a per-flow hashed multipath routing technique. For a given packet flow, the LB nodes randomly select a server node as a target for the packet flow from among multiple server nodes and send a connection request to the server node. A load balancer module on the server node makes the decision on whether to accept or reject the connection based on one or more metrics indicating a respective server's current load. If the module accepts the connection request, a connection is established between the server and the client. Otherwise, the load balancer nodes may select another server node and try again. The connections established between clients and servers pass through the load balancer nodes, but are not terminated at the load balancer nodes.