Multiple Hit Load Balancing for Encrypted Request Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancer technologies face bottlenecks with secure connections and HTTP/2 protocols, leading to cache pollution and intra-PoP cross traffic, as they struggle to efficiently distribute encrypted requests and multiple requests over a single connection without creating bottlenecks.
Innovation Solution
The implementation of multiple hit load balancing decentralizes request distribution by shifting it from the load balancer to object distribution servers, which dynamically scale based on request counts to cache and serve frequently requested objects, eliminating intra-PoP cross traffic and reducing bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the load balancer performs persistent request distribution by inspecting encrypted URLs, then request distribution accuracy is improved, but the load balancer becomes a bottleneck due to computationally expensive decryption operations
Solution Approach 1:
The patent extracts the request distribution function from the load balancer and transfers it to the servers. The load balancer no longer needs to inspect encrypted URLs or perform decryption operations. Instead, servers independently determine their own request distribution based on encrypted URLs, eliminating the bottleneck at the load balancer while maintaining accurate request distribution.
2Adaptability or versatility
If the load balancer performs repeated hand-off and hand-back of connections for HTTP/2 requests, then request distribution flexibility is improved, but the system becomes time-consuming and resource intensive
Solution Approach 1:
Instead of the load balancer actively managing connection hand-offs for HTTP/2 requests, the patent inverts the approach by allowing servers to independently handle their own request distribution. Each server uses its own connection to serve multiple HTTP/2 requests without requiring load balancer intervention, eliminating the time-consuming hand-off process while maintaining distribution flexibility.
3Ease of operation
If persistent request distribution is lost due to secure connections, then connection handling simplicity is improved, but cache pollution and intra-PoP cross traffic increase
Solution Approach 1:
The patent implements self-service by enabling each server to independently determine its own request distribution based on encrypted URLs. Servers use their own connections and resources to serve requests without requiring load balancer coordination, thereby maintaining connection handling simplicity while preserving effective cache distribution and reducing unnecessary cross-traffic.
Data Source
AI summary
Multiple hit load balancing provides a quasi-persistent request distribution for encrypted requests passing over secure connections as well as for multiple requests passing over the same connection. The multiple hit load balancing involves tracking object demand at each server of a set of servers. The multiple hit load balancing further involves dynamically scaling the servers that cache and directly serve frequently requested objects based on the demand that is tracked by each of the servers. For infrequently requested objects, the servers perform a peer retrieval of the objects so to limit the number of the same object being redundantly cached by multiple servers of the set of servers.