Distributed Load Balancing via Shared Metrics Array
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional load balancers in distributed services become bottlenecks due to their limited number, increased latency, and resource consumption, as they act as intermediaries between service requesters and servers, leading to inefficiencies in request handling.
Innovation Solution
A distributed load-balancing method that allows service requestors to directly access a shared metrics array across multiple servers, using hardware-accelerated RDMA operations to determine the best server for service requests based on metrics such as capacity and performance, eliminating the need for a conventional intermediary load balancer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional load balancers are used as intermediaries between service requesters and servers, then request dispatching can be performed, but the load balancers become a bottleneck in the system due to their limited number
Solution Approach 1:
The patent extracts the load-balancing logic from the centralized load balancer and embeds it directly into the service requester. The service requester now directly accesses the shared metrics array stored in server memory to determine server selection, eliminating the intermediary load balancer and its associated bottlenecks while maintaining the load-balancing functionality
Solution Approach 2:
The patent introduces a shared metrics array as a new intermediary mechanism that enables direct communication between service requesters and servers. This shared array, accessible through hardware-accelerated RDMA operations, serves as the mediator that provides server status information to requesters without requiring a traditional load balancer intermediary process
2Ease of operation
If conventional load balancers are situated between requesters and servers, then request routing is enabled, but the latency of service calls is at least doubled
Solution Approach 1:
The patent removes the load balancer intermediary that causes latency, extracting only the essential functionality of accessing server metrics. Service requesters directly read from the shared metrics array in server memory and make routing decisions themselves, eliminating the round-trip time to and from a separate load balancer process
Solution Approach 2:
The patent implements preliminary action by having servers continuously maintain and update their metrics in the shared metrics array before any load-balancing decision is needed. Service requesters can directly access this pre-prepared information through RDMA operations, eliminating the need for real-time queries to an intermediary load balancer during the service call
3Productivity
If conventional load balancers are used as live processes, then request management is performed, but CPU time and memory are consumed
Solution Approach 1:
The patent implements self-service by enabling service requesters to independently perform load-balancing decisions without relying on external load balancer processes. Each requester directly accesses the shared metrics array and makes routing decisions autonomously, eliminating the need for dedicated load balancer processes that consume CPU and memory resources
Solution Approach 2:
The patent replaces the mechanical system of live load balancer processes with a hardware-accelerated memory access mechanism. By using RDMA (Remote Direct Memory Access) operations, the system substitutes software-based load balancing with hardware-level memory access, dramatically reducing processor resource consumption while maintaining request management capability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces latency and resource consumption by distributing the load-balancing function across service requestors and servers, enabling more efficient and fair distribution of service requests without the bottlenecks associated with traditional load balancers.
Implementation Method 1
The shared metrics array is directly accessed by the service requestor without involving a processor of the at least one server
Data Source
AI summary
Various embodiments load balance service requests across one or more servers. In one embodiment, a service requestor directly accesses a shared metrics array stored in at least one server of a plurality of servers in a service cluster. Each of these servers includes one or more services. The shared metrics array is accessible by each of the plurality of servers, and includes a set of metrics for each of the plurality of servers. A determination is made based on the set of metrics associated with at least one server in the plurality of servers if a service request is to be sent to the at least one server. The service request is sent to the at least one server based on determining that the service request is to be sent to the at least one sever.