Layer-4 Load Balancing With Capacity-Aware DIP Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing layer-4 load balancers assume uniform and static capacities of backend instances (DIPs), failing to adapt to dynamic capacity changes due to noisy neighbors, resource competition, and varying DIP generations, leading to CPU utilization imbalances and increased latencies.
Innovation Solution
A capacity-aware load balancer system using an Integer Linear Program (ILP) calculates weights for DIPs based on latency measurements, decoupling weight calculations from individual LB instances to a central controller, and employing curve-fitting and multi-step ILP to adapt to dynamic changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If existing layer-4 load balancers assume uniform and static capacities of backend instances, then the load balancing configuration is simple, but CPU utilization imbalances occur and latencies increase
Solution Approach 1:
The patent changes the parameter of DIP capacity from static/uniform to dynamic/varied by introducing capacity measurements that detect actual performance characteristics of each DIP. The system measures capacity parameters (such as processing speed, latency) and uses these measured parameters to configure load balancing weights, thereby adapting to heterogeneous DIP capabilities without requiring manual configuration complexity
Solution Approach 2:
The load balancer automatically discovers and adapts to DIP capacities through self-measurement and self-configuration. The system performs capacity measurements on DIPs, computes optimal weights based on measured capacities, and applies these weights automatically without external intervention. This self-service approach resolves the contradiction by eliminating manual configuration while achieving balanced CPU utilization
2Ease of operation
If existing layer-4 load balancers assume uniform and static capacities of backend instances, then the system is easy to operate, but latencies increase due to CPU utilization imbalances
Solution Approach 1:
The patent implements feedback by continuously measuring DIP capacities and using these measurements to adjust load balancing weights. The system periodically re-measures DIP capacities and recomputes weights based on current performance, creating a closed-loop feedback mechanism that automatically optimizes latency without requiring operator intervention. This feedback-driven approach maintains ease of operation while reducing latencies through adaptive weight adjustment
3Measurement precision
If capacity measurements are performed for all possible weights, then accurate capacity information is obtained, but the measurement process becomes computationally expensive and time-consuming
Solution Approach 1:
The patent applies partial action by measuring capacity at selectively chosen weight points rather than exhaustively measuring all possible weights. The system identifies a sufficient subset of weight values that provide adequate measurement precision for computing optimal weights, avoiding the computational expense of complete measurement while maintaining sufficient accuracy for load balancing decisions
Solution Approach 2:
The system performs preliminary capacity measurements to establish baseline performance characteristics of DIPs before computing optimal weights. These preliminary measurements provide sufficient information for initial weight configuration, and the system can later refine weights based on actual traffic patterns without requiring exhaustive measurements of all weight combinations
Data Source
AI summary
The present disclosure relates to methods and systems for load balancing traffic per the capacities of direct IPs (DIPs). The methods and systems use latency measurements from each DIP to determine the capacity of each DIP. The methods and systems use the latency measurements to determine the weights for each DIP using an Integer Linear Program (ILP). The weights identify an amount of traffic to provide to each DIP. The methods and systems provide the weights for each DIP to a load balancer controller to program the load balancer dataplane with the weights.