Distributed Load Balancer Node Architecture for Scalable Cloud Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancers face scalability issues in large network environments, such as cloud computing services, and rely on outdated techniques like max connections and round robin for load distribution, which are inefficient and difficult to manage in distributed systems.
Innovation Solution
A distributed load balancing system that uses multipath routing, consistent hashing, and health checking to route TCP packets directly between clients and servers, allowing server nodes to make local decisions on connection acceptance based on their current status, and employs load balancer nodes to randomly select servers for connection requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional single load balancer architecture is used, then load balancing function is provided, but scalability is poor in large network environments with hundreds or thousands of host devices
Solution Approach 1:
The patent divides the single load balancer into multiple load balancer nodes forming a distributed cluster. Each node handles a subset of host devices, enabling the system to scale to hundreds or thousands of hosts. The cluster architecture segments the load balancing function across multiple independent nodes that can be added or removed without affecting the entire system.
2Loss of time
If conventional load balancers terminate TCP connections and act as proxies, then connection management is centralized, but direct communication between clients and servers is blocked increasing latency
Solution Approach 1:
The patent extracts the TCP connection termination function from the load balancer nodes. Instead of load balancers acting as proxies that terminate and recreate connections, the system enables direct TCP connections between clients and servers. The load balancer cluster only performs network address translation and routing, allowing end-to-end communication while still providing load distribution.
3Productivity
If conventional load balancers use max connections or round robin techniques, then load distribution is simple to implement, but resource utilization is inefficient and difficult to manage in distributed systems
Solution Approach 1:
The patent implements health checking mechanisms where load balancer nodes continuously monitor the status of host devices and adjust load distribution based on real-time conditions. Nodes can detect when servers are overloaded or unavailable and dynamically reroute traffic, providing feedback-driven load management that optimizes resource utilization while maintaining operational simplicity through automated decision-making.
Data Source
AI summary
A distributed load balancer in which a router receives packets from at least one client and routes packet flows to multiple load balancer (LB) nodes, which in turn distribute the packet flows among multiple server nodes. Each LB node may serve in ingress, egress, and/or flow tracker roles. Each LB node may include a first network interface controller (NIC) that faces the router and a second NIC that faces the server nodes. Each LB node may implement a core packet processing architecture in which packets received at the NICs are distributed among non-blocking input queues of a set of worker cores by receiver cores for the NICs, processed from the input queues by the worker cores, and placed on non-blocking input queues of transmit cores for the NICs.