Utilization-Based Load Balancing for Server Clusters

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing load balancing methods in information retrieval systems, such as search engines, fail to efficiently distribute queries across network hardware clusters due to their reliance on static metrics like queries per second (QPS) or CPU usage, which do not account for varying workloads and lead to unpredictable CPU capacity utilization, resulting in potential overloading of hardware.

Innovation Solution

A utilization-based load balancing system that allocates computational resources among tasks within server clusters, calculates a utilization metric to determine available capacity, and redirects queries when a user-defined threshold is reached, ensuring that no cluster is overloaded by using a re-allocation processor to dynamically adjust network traffic based on real-time resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If QPS-based load balancing is used, then the load distribution is simple to implement, but it does not account for varying query costs and leads to hardware overloading

Engineering Contradiction:

Improveload balancing implementationVSAvoidhardware overload prevention

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent changes the load balancing parameter from static QPS to dynamic cost metrics. The system monitors actual resource consumption (CPU usage, memory, I/O) and adjusts query routing based on real-time cost variations, allowing the load balancer to adapt to changing query complexities and system states while maintaining simple operation

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent implements feedback mechanisms where the load balancer continuously monitors resource consumption metrics from server clusters and uses this information to dynamically adjust query routing decisions. This closed-loop control ensures that load distribution responds to actual system conditions, preventing hardware overload while maintaining operational simplicity

Inventive Principle:

Principle #23Feedback

2Stability of the object's composition

If cost-based load balancing is used, then the stability of load balancing increases, but the correlation between items processed and work done becomes unpredictable

Engineering Contradiction:

Improveload balancing stabilityVSAvoidwork capacity correlation

Core Design Contradiction:

Stability of the object's compositionVSMeasurement precision

Solution Approach 1:

The patent uses a composite metric that combines multiple resource consumption measurements (CPU usage, memory consumption, I/O operations) into a unified cost metric. This composite approach captures the multidimensional nature of query work more accurately than single metrics, improving both stability and measurement precision by considering the full spectrum of resource utilization

Inventive Principle:

Principle #40Composite materials

Solution Approach 2:

The patent creates a universal cost metric that can evaluate different types of queries and workloads through a common framework. This multi-functional metric adapts to various query complexities and system states, providing consistent and precise work capacity measurement across diverse operational conditions

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Measurement precision

If CPU usage-based load balancing is used, then the precision of load balancing increases, but system optimization leads to unpredictable capacity changes and requires recalibration

Engineering Contradiction:

Improveload balancing precisionVSAvoidsystem optimization flexibility

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic load balancing where the system continuously adapts to changing conditions rather than relying on static thresholds. The load balancer monitors real-time resource consumption and automatically adjusts routing decisions, allowing the system to accommodate optimizations and capacity changes without manual recalibration while maintaining high precision

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent enables the load balancing system to self-adjust and self-optimize by automatically monitoring resource metrics and making routing decisions based on current system state. This self-service capability eliminates the need for manual recalibration when system optimizations occur, as the system adapts autonomously to new capacity conditions

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS9641431B1System and methods for utilization-based balancing of traffic to an information retrieval system

Publication Date: 2017.05.02 GOOGLE LLC

AI summary

Systems and methods for, among other things, a utilization based load balancing system which controls the distribution of queries to an information retrieval system made up of a network of server clusters. In one embodiment, a server cluster allocates computational resources among computational tasks. These computational tasks are replicated across a given server cluster, typically such that those computational tasks requested more frequently have more replicas, and more resources allocated to them to fulfill the requests. The system applies a utilization metric to determine how much capacity a given task has available and uses this determination to determine the capacity available for the cluster as a whole. Load balancing is achieved by re-directing queries to another cluster in response to the utilization value for a given cluster reaching a threshold.