TCP Connection Splicing with Split Datapaths for One-to-Many Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional one-to-many TCP communication systems incur significant performance overhead and scalability bottlenecks as the number of recipients increases, leading to inefficient memory copy operations and CPU utilization imbalances.
Innovation Solution
Implement a split cross-layer datapath with a fast path for common-case one-to-many data splicing and a slow path for uncommon events, utilizing shallow memory copies and NIC-assisted scatter-gather capabilities, along with dynamic load rebalancing to optimize CPU core assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional TCP data splicing is used to replicate incoming data streams to multiple receiving entities, then one-to-many communication functionality is achieved, but performance overhead increases significantly as the number of recipients increases
Solution Approach 1:
The patent segments the data splicing operation into distinct phases: connection establishment phase (slow path) and data transmission phase (fast path). This segmentation allows the system to handle connection setup with full TCP protocol processing while optimizing the high-volume data transmission phase through reduced copying operations, thereby maintaining functionality while improving throughput as N increases
Solution Approach 2:
The patent uses shallow copying (incrementing reference counters) instead of deep copying (duplicating memory content) for data payload replication. This copying strategy reduces memory copy overhead from O(N) to O(1) per recipient, enabling the system to maintain high message throughput while supporting one-to-many communication to multiple recipients
2Ease of operation
If multiple memory copies are performed for each data packet between TCP stack socket buffer and application memory space, then data is made available to applications, but memory copy overhead increases
Solution Approach 1:
The patent replaces traditional deep copying operations with shallow copying using reference counters. When data is spliced to multiple TCP connections, the system increments reference counters instead of copying memory content, reducing memory copy time from linear O(N) to constant O(1) while still making data available to all applications
Solution Approach 2:
The patent substitutes the mechanical memory copying process with a reference counting mechanism. Instead of physically copying data bytes between memory spaces, the system uses metadata (reference counters) to track data sharing, eliminating the time-consuming mechanical copy operation while maintaining data accessibility
3Reliability
If TCP stack operates in privileged kernel mode with frequent user-to-kernel mode changes during data splicing, then protocol processing is performed, but system call overhead increases
Solution Approach 1:
The patent segments TCP processing into kernel-mode operations (connection establishment, protocol validation) and user-mode operations (data splicing, transmission). By separating these functions, the system maintains reliable protocol processing in the kernel while improving processing efficiency by performing high-volume data operations in user mode, reducing system call overhead
Data Source
AI summary
A computing device includes at least one network interface controller (NIC), a memory, and at least one processor including one or more cores. The NIC(s) receive inbound Transmission Control Protocol (TCP) packets and transmit outbound TCP packets. The memory stores a program of instructions, which the processor executes to split a datapath including both receive endpoints and transmit endpoints within the computing device into a first path and a second path. The processor assigns first inbound TCP packets to the first path in response to determining that the first inbound TCP packets are associated with an established TCP connection and a data splicing group, and assigns second inbound TCP packets not assigned to the first path to the second path. The processor uses the first path to perform one-to-many data splicing of payload data from the first inbound TCP packets into the outbound TCP packets.