Rate-Based Load Balancing for Server Congestion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Metric-based load balancing techniques face delays in updating server processing capabilities information, leading to inefficient request distribution and potential server overload due to outdated data and the Poisson process nature of incoming requests, causing congestion and performance issues in high-volume data traffic environments.

Innovation Solution

Implementing a rate-based load balancing approach that focuses on timing factors, using rate controllers to determine when servers are ready to receive requests based on the last transmission time rather than CPU utilization, thereby ensuring a coordinated and interspersed distribution of requests across servers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If metric-based load balancing is used to distribute requests based on server processing capabilities, then resource allocation can be optimized, but delays in updating server capability information cause outdated data to be used, leading to inefficient request distribution and potential server overload

Engineering Contradiction:

Improverequest distribution efficiencyVSAvoidinformation update delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-calculating and storing server capability metrics before requests arrive. The load balancer maintains an updated view of server capabilities through periodic polling or push notifications, so that when requests arrive, the most current information is already available for immediate decision-making, eliminating delays in real-time load balancing decisions

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where servers continuously report their current processing capabilities and workload status to the load balancer. This feedback loop ensures that the load balancer has real-time or near-real-time information about server states, allowing it to make accurate load balancing decisions without relying on outdated metrics

Inventive Principle:

Principle #23Feedback

2Productivity

If requests are distributed based on current server metrics, then optimal resource usage can be achieved, but the Poisson process nature of incoming requests causes congestion and backlog when metrics are outdated

Engineering Contradiction:

Improveresource usage optimizationVSAvoidsystem performance stability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system employs periodic action by continuously polling servers at regular intervals to update capability metrics, rather than relying on periodic server push or event-driven updates. This ensures that the load balancer has fresh information about server states before making load balancing decisions, preventing congestion caused by outdated metrics during high-volume request periods

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The system implements dynamics by making the load balancing strategy adaptive and flexible. The load balancer dynamically adjusts its decision-making based on current server conditions, request patterns, and measured performance. This dynamic approach allows the system to respond to changing conditions in real-time, maintaining reliability even when request arrival patterns vary significantly

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If traditional load balancing methods are used to manage high-volume data traffic, then basic request routing can be provided, but congestion and workload backlog occur due to inefficient distribution

Engineering Contradiction:

Improverequest routing simplicityVSAvoidworkload processing efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The system applies parameter changes by considering multiple server attributes simultaneously when making load balancing decisions, including current workload, processing capability, response time, and queue depth. By changing from simple round-robin or random distribution to multi-parameter optimization, the system achieves both ease of operation through automated decision-making and high productivity through efficient workload distribution

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11368524B1Systems and methods for rate-based load balancing

Publication Date: 2022.06.21 META PLATFORMS INC

AI summary

According to examples, a system for rate-based load balancing may include a processor and a memory storing instructions. The processor may, through execution of the instructions, cause the system to receive a request for processing. The system may further identify a target server to transmit the request using a rate-based load balancing technique. In some examples, the rate-based load balancing technique may include: selecting a server, from a plurality of servers, as a potential target; receiving a readiness indicator for the selected server; and designating the selected server as the target server based on the readiness indicator. The system may transmit the request to the target server for processing.