Capacity-Aware Layer-4 Load Balancer Latency Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancers do not effectively balance traffic across backend instances based on their capacities, leading to increased latency for over-utilized instances, especially in virtualized clusters.
Innovation Solution
A method for load balancing that identifies subsets of data compute nodes with higher and lower latencies, assigns weight values based on target latency, and redistributes excess weight across nodes to optimize traffic distribution, using a load balancer like the L4 AVI Load Balancer from VMware, which measures latency directly and adjusts weights dynamically to ensure uniform latency across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If existing load balancers distribute traffic uniformly across backend instances, then the load balancing mechanism is simple to implement, but the latency increases significantly for over-utilized instances
Solution Approach 1:
The patent changes the parameter distribution from uniform to capacity-proportional by introducing weight values that reflect the actual processing capacity of each backend instance. The load balancer dynamically adjusts these weights based on measured performance metrics, allowing traffic distribution to match the varying capacities of different instances rather than treating all instances equally.
Solution Approach 2:
The load balancing mechanism transitions from a static uniform distribution to a dynamic capacity-aware distribution. The system continuously monitors backend instance performance and adjusts weight values in real-time, enabling the load balancer to adapt to changing capacities and workloads, thereby optimizing latency performance under varying conditions.
2Ease of operation
If load balancers assume uniform capacity across backend instances, then the configuration and operation is simplified, but the system performance deteriorates in virtualized clusters with varying instance capacities
Solution Approach 1:
The load balancer implements self-service by automatically discovering and measuring the capacity of each backend instance through performance testing. Rather than requiring manual configuration of capacity parameters, the system autonomously probes instances, measures their response times and throughput, and derives weight values from these measurements, eliminating the need for manual capacity assessment while optimizing system throughput.
Solution Approach 2:
The system incorporates feedback mechanisms where the load balancer continuously monitors backend instance performance metrics and uses this feedback to adjust weight values. This closed-loop control enables the system to automatically adapt to changes in instance capacity, maintaining optimal throughput without requiring manual reconfiguration or intervention.
3Device complexity
If traffic is distributed without considering instance capacity, then the load balancing algorithm is computationally simple, but the latency for requests to over-utilized instances doubles
Solution Approach 1:
The load balancer performs preliminary capacity assessment by conducting performance tests on backend instances before and during traffic distribution. These preliminary measurements establish baseline capacity values that are used to calculate initial weight assignments, allowing the system to proactively prevent over-utilization rather than reacting to latency problems after they occur.
Data Source
AI summary
Some embodiments provide a method of implementing capacity-aware load balancing across a set of data compute nodes (DCNs) by reducing latency for the set of DCNs. From the set of DCNs, the method identifies (1) a first subset of DCNs including DCNs that have a latency that is higher than an average latency computed for the set of DCNs and (2) a second subset of DCNs including DCNs that have a latency that is lower than the average latency computed for the set of DCNs. For each DCN in the first subset of DCNs, the method assigns to the DCN a weight value that corresponds to a target latency computed for the set of DCNs. Based on the assigned weight values for the first subset of DCNs, the method computes an excess weight value to be redistributed across the second subset of DCNs. The method redistributes the computed excess weight value across the second subset of DCNs.