Shuffle Sharded Load Balancer for Request Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In network-based computing environments, load balancers face challenges in distributing service requests efficiently to prevent overload on individual computers, leading to potential failures that can affect all users if a poorly behaved requestor overwhelms the system resources.
Innovation Solution
The implementation of a shuffle sharded load balancer that randomly selects a subset of available hosts to service requests based on source identification values, using a pseudo-random number generator to ensure isolation of request processing and limit the impact of issues in one shard on others, thereby promoting resource uniqueness and redundancy across shards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a load balancer distributes service requests across multiple computers to optimize resource utilization and prevent overload, then resource utilization is improved, but if a poorly behaved requestor overwhelms the system, then system reliability deteriorates because all users are affected by failures
Solution Approach 1:
The patent divides the host pool into multiple shards, where each shard is a subset of hosts that services requests from specific sources. This segmentation isolates request processing so that issues in one shard do not affect other shards, thereby maintaining system availability while distributing resource utilization across segmented groups.
2Productivity
If a load balancer distributes requests evenly across all service computers to prevent overload, then resource utilization is improved, but the complexity of managing and isolating request processing increases
Solution Approach 1:
The patent segments hosts into shards based on source identification values, creating manageable subsets that simplify request routing logic while maintaining even distribution across the overall host pool.
Solution Approach 2:
The patent uses source identification values as parameters to determine shard assignment, changing the routing parameter from simple round-robin to source-based shard selection, which maintains distribution efficiency while adding isolation capabilities.
Data Source
AI summary
A system includes a plurality of computing devices configurable to implement a compute instance of a provider network. A computing device is configured to implement a load balancer that allocates the computing devices to service requests received by the load balancer. The load balancer configured to receive service requests from a plurality of sources, to extract a parameter from each received service request, to select, based on the parameter, a first subset of the plurality of computing devices to be used to service the request, and to forward the request to a selected computing device of the first subset of the computing devices for service.