TCP Connection Splicing With Split Fast Path for One-to-Many Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional one-to-many TCP data splicing systems incur significant performance overhead and scalability bottlenecks as the number of recipients increases, leading to inefficiencies in memory copies and CPU utilization.
Innovation Solution
Implementing a split cross-layer datapath with a fast path for common-case one-to-many data splicing and a slow path for uncommon events, utilizing shallow memory copies and NIC-assisted scatter-gather capabilities, along with dynamic load rebalancing to optimize CPU core assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional TCP data splicing is used to replicate incoming data streams to multiple receiving entities, then one-to-many communication functionality is achieved, but performance overhead increases significantly as the number of recipients increases
Solution Approach 1:
The patent segments the datapath into a fast path for common-case data splicing operations and a slow path for uncommon events such as TCP connection setup and application-level session creation. This segmentation allows the system to handle the majority of data replication operations with minimal overhead while dedicating resources for complex event handling, thereby resolving the performance overhead issue as the number of recipients increases.
Solution Approach 2:
The patent implements shallow memory copy instead of deep memory copy for data splicing operations. Rather than duplicating the actual data content, the system creates lightweight references to the same data buffer, which dramatically reduces the computational overhead of replicating data streams to multiple recipients while maintaining the functional capability of one-to-many communication.
2Ease of operation
If multiple memory copies are performed for each data packet between TCP socket buffer and application memory space, then data splicing functionality is achieved, but memory copy overhead increases
Solution Approach 1:
The patent replaces traditional deep memory copy operations with shallow memory copy, which creates references to data buffers instead of duplicating actual data content. This approach maintains the data splicing functionality where packets can be replicated to multiple destinations, but eliminates the excessive memory copy overhead by operating on lightweight references rather than full data copies.
Solution Approach 2:
The patent extracts the essential data splicing functionality from the overhead of multiple memory copies by implementing a reference-based copying mechanism. The system separates the functional requirement (replicating data to multiple recipients) from the expensive operation (deep memory copies), keeping only the necessary reference creation operations while eliminating redundant data duplication.
3Reliability
If TCP stack operates in the privileged kernel, then protocol processing is achieved, but system call overhead increases due to frequent user-to-kernel mode changes
Solution Approach 1:
The patent segments the system into user-space and kernel-space components, moving the TCP stack and data splicing operations to user space where possible. This segmentation allows frequent data splicing operations to occur in user space without triggering system calls, while the kernel retains essential protocol processing functions, thereby reducing the overhead of frequent user-to-kernel mode changes.
Solution Approach 2:
The patent introduces a user-space TCP stack that acts as an intermediary between the application and the kernel TCP stack. This intermediary handles data splicing operations in user space, eliminating the need for frequent system calls to the kernel for data replication, while still maintaining reliable protocol processing through the kernel's TCP implementation.
4Use of energy by moving object
If CPU cores are assigned to handle both fast path and slow path activities, then system resources are utilized, but load imbalance occurs between paths
Solution Approach 1:
The patent segments CPU core assignment by dedicating specific cores to the fast path and other cores to the slow path. This segmentation ensures that the fast path, which handles the majority of data splicing operations, has dedicated resources and does not contend with the slow path for CPU time, thereby achieving load balance while maintaining efficient resource utilization.
Solution Approach 2:
The patent applies local quality optimization by assigning different CPU core characteristics to different paths. The fast path is assigned cores optimized for high-throughput data processing, while the slow path is assigned cores that can handle the intermittent, complex operations of connection setup and session management, thereby optimizing overall system performance and load distribution.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computing device includes at least one network interface controller (NIC), a memory, and at least one processor including one or more cores. The NIC(s) receive inbound Transmission Control Protocol (TCP) packets and transmit outbound TCP packets. The memory stores a program of instructions, which the processor executes to split a datapath including both receive endpoints and transmit endpoints within the computing device into a first path and a second path. The processor assigns first inbound TCP packets to the first path in response to determining that the first inbound TCP packets are associated with an established TCP connection and a data splicing group, and assigns second inbound TCP packets not assigned to the first path to the second path. The processor uses the first path to perform one-to-many data splicing of payload data from the first inbound TCP packets into the outbound TCP packets.