Overlay Network TCP Connection Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
TCP performance deteriorates with increasing round trip times, leading to longer connection establishment times, reduced throughput, and limitations due to receive window constraints, especially when window scaling is not enabled.
Innovation Solution
Breaking a single TCP connection into three separate connections (edge-to-client, edge-to-edge, and edge-to-origin) and replicating TCP state across two servers for fault tolerance, while accelerating connection establishment by generating a SYN/ACK packet and sending it back to the edge-to-edge connection to allow data flow before receiving the SYN/ACK from the receiving region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single TCP connection is used between client and origin server, then the connection structure is simple, but the connection establishment time is long and throughput is limited due to round trip time
Solution Approach 1:
The patent segments a single TCP connection into three separate connections: edge-to-client, edge-to-edge, and edge-to-origin. This segmentation allows parallel connection establishment and data transfer across multiple paths, reducing overall connection establishment time and improving throughput by eliminating the bottleneck of a single sequential connection.
Solution Approach 2:
The patent implements preliminary action by sending data across the edge-to-edge connection before waiting for the SYN/ACK from the receiving region. This is achieved by generating a SYN/ACK packet and feeding it back to the edge-to-edge connection, causing TCP to treat the connection as established in advance, thereby allowing data to flow before formal acknowledgment is received.
2Reliability
If TCP state is replicated across two servers, then fault tolerance is improved, but system complexity increases
Solution Approach 1:
The patent applies copying by replicating TCP state from a primary server to a backup server. When a packet is received by the primary server, its state is updated and then passed to the backup server. This creates a redundant copy of the connection state, ensuring fault tolerance while maintaining relatively simple implementation through state replication rather than full system duplication.
Solution Approach 2:
The backup server serves multiple functions: it acts as a standby for fault tolerance, can take over if the primary fails, and maintains synchronization with the primary. This multi-functionality justifies the added complexity by providing both reliability and disaster recovery capabilities within the same architectural framework.
3Loss of time
If data is sent before receiving SYN/ACK from receiving region, then connection establishment latency is reduced, but TCP protocol compliance may be compromised
Solution Approach 1:
The patent introduces an intermediary mechanism by generating a synthetic SYN/ACK packet and feeding it back to the edge-to-edge connection. This intermediary SYN/ACK acts as a mediator that allows TCP to treat the connection as established, enabling data flow before the actual SYN/ACK from the receiving region is received, thus reducing latency while maintaining protocol compliance through the intermediary acknowledgment.
Data Source
AI summary
Applications that run on an overlay network-based managed service achieve high performance gains using a set of TCP optimizations. In a first optimization, a typical single TCP connection between a client and an origin server is broken into preferably three (3) separate TCP connections. These connections are: an edge-to-client connection, an edge-to-edge connection, and edge-to-origin connection. A second optimization replicates TCP state along the connection to increase fault tolerance. In this approach, preferably a given TCP connection is maintained on two servers. When a packet is received by one server, called the primary, its state is updated and then passed to a second server, called the backup. Only when the backup sends an acknowledgement back to the primary can it then send a TCP acknowledgement back to the host that originally sent the packet. Another optimization reduces connection establishment latency.