Connection Cache for Proxy Servers Reducing TCP Setup Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Proxy servers face inefficiencies in establishing and maintaining TCP connections due to high setup times and initial low data throughput, especially when opening new connections for each HTTP transaction, which limits resource utilization and increases latency.
Innovation Solution
A method for maintaining a connection cache at an intermediate server by monitoring resource requests, generating statistics to identify frequently requested resources, and caching connections to these resources, allowing subsequent requests to use existing connections, thereby reducing latency and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a new TCP connection is opened for each HTTP transaction, then connection freshness is ensured, but setup time and latency increase significantly
Solution Approach 1:
The system performs preliminary actions by establishing TCP connections to remote servers in advance based on predicted future requests. The connection cache pre-opens connections to popular remote servers before they are actually needed, using statistics about frequently requested resources. This eliminates the need for time-consuming connection setup when requests arrive, directly resolving the contradiction between ensuring connection freshness and reducing setup time.
2Reliability
If congestion control strategies like slow start are used, then network stability is maintained, but initial data throughput is limited
Solution Approach 1:
The system performs preliminary data transfer by sending data during the connection establishment phase, before the HTTP transaction formally begins. When a connection is opened to a remote server, the system proactively transfers data that is likely to be requested soon, utilizing the connection before congestion control would normally limit throughput. This allows the system to benefit from higher initial throughput while maintaining network stability through standard congestion control mechanisms.
3Loss of time
If connections to all remote servers are cached, then request latency is minimized, but processing and memory resources are wasted
Solution Approach 1:
The system applies local quality by differentiating between popular and unpopular remote servers. Instead of uniformly caching connections to all remote servers, the system uses statistics about frequently requested resources to identify popular servers and caches connections only to those. This selective approach ensures that memory and processing resources are allocated efficiently to the subset of servers that actually benefit from connection caching, while minimizing resource waste on servers with low request frequency.
Solution Approach 2:
The system dynamically changes parameters by adjusting the connection cache based on changing request patterns. The statistics about frequently requested resources are continuously updated, and the set of cached connections is modified accordingly. When a remote server transitions from unpopular to popular (or vice versa), the system adds or removes connections from the cache, optimizing resource utilization while maintaining low latency for current popular servers.
4Productivity
If connection caching is implemented, then resource utilization is improved, but connection management complexity increases
Solution Approach 1:
The system implements self-service by automatically managing the connection cache without requiring manual intervention. The system monitors resource requests, generates statistics about popular remote servers, automatically adds or removes connections from the cache based on these statistics, and handles connection lifecycle management. This automated approach improves resource utilization while keeping connection management complexity manageable by eliminating the need for manual configuration and monitoring.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, apparatus and computer program product for maintaining a connection cache at an intermediate server, wherein the connection cache relating to resource requests from a plurality of devices to a plurality of servers remote therefrom. The method comprises monitoring resource requests addressed to a plurality of said remote servers during a first time period; generating statistics data on the basis of the monitored resource requests; establishing a plurality of connections from the intermediate server to a subset of the plurality of remote servers, said subset being determined on the basis of the generated statistics data; and storing data indicative of the plurality of established connections in a connection cache. Caching of connections in this manner ensures efficient use of proxy server resources by only caching connections to "popular" remote servers.