One-to-many TCP connection splicing

Abstract

The invention relates to one-to-many TCP connection splicing. A computing device includes at least one network interface controller (NIC), a memory, and at least one processor including one or more cores. The NIC (s) receive inbound transmission control protocol (TCP) data packets and transmit outbound TCP data packets. The memory stores a program of instructions that the processor executes to divide a data path within the computing device that includes both a receiving endpoint and a transmitting endpoint into a first path and a second path. In response to determining that the first inbound TCP data packet is associated with the established TCP connection and the data stitching group, the processor assigns the first inbound TCP data packet to the first path and assigns a second inbound TCP data packet that is not assigned to the first path to the second path. The processor performs a one-to-many data stitching of payload data from the first inbound TCP data packet to the outbound TCP data packet using the first path.

Description

TECHNICAL FIELD

[0001] The present application relates generally to one-to-many connection splicing, and more specifically to one-to-many Transmission Control Protocol (TCP) connection splicing. BACKGROUND

[0002] One-to-many communication is used for various data center applications. For example, commercial internet-scale services often rely on distributed event stream services that dynamically route data from a single source to multiple consuming entities. In distributed storage systems, to be fault-tolerant, write operations on files are replicated to other nodes. One-to-many communication is also used in content delivery networks (CDNs) for distributing media to a wider audience, in mixed and virtual reality applications for view and state synchronization between immersive experience participants, and in video conferencing systems that support real-time multi-party communication. Each of these applications replicates input data streams to multiple receiving entities at various scales. SUMMARY

[0003] The scope of some example embodiments for which protection is sought is presented by the independent claims. Example embodiments and / or features described in this specification that do not fall within the scope of the independent claims will be interpreted as examples useful for understanding some embodiments, if any.

[0004] In various conventional implementations of one-to-N data splicing, as N increases, a significant increase in performance overhead is incurred, leading to potential scalability bottlenecks in conventional Transmission Control Protocol (TCP) application deployment environments. Various example embodiments disclosed herein alleviate this problem and other potential performance problems by dividing the data path into a fast path and a slow path. To reduce storage copy overhead on the fast path, shallow copies can be used in conjunction with the scatter-gather capabilities of certain network interface controllers (NICs) to achieve "zero-copy one-to-many packet transmission." In some example embodiments, the CPU utilization of fast path and slow path activities can be independently scaled by adjusting the respective CPU core assignments. This scaling can result in overall reductions in message latency and increases in maximum information throughput.

[0005] In some conventional systems, when a message is received via TCP, the message is saved in the socket buffer of the TCP stack, then copied to the application's memory space, and finally processed by the application via socket APIs. An outgoing message goes through similar steps in reverse. In conventional systems implementing such a system, for each received packet, there will be (N+l) memory copies of the payload between the socket buffer of the protocol stack and the application, and N additional copies when N full packets with IP / TCP headers are generated for transmission. If the TCP stack is operated in a privileged kernel, the TCP stack will incur additional system call overhead due to frequent user-to-kernel mode changes during data reassembly.

[0006] Various example embodiments disclosed herein reduce or eliminate the need for multiple copies to the application's memory space, as well as other scalability bottlenecks associated with conventional systems. In some example embodiments, the efficiency and scalability improvements in one-to-many TCP data reassembly within a local server can be achieved by using a split cross-layer data path; zero-copy, NIC-assisted data reassembly; dynamic load rebalancing; or some combination of these techniques.

[0007] In implementing a split cross-layer data path, some example embodiments use a unified application-network data path, where TCP protocol stack processing is integrated into application-level data reassembly. To optimize the performance of the unified cross-layer data path, the data path can be divided into a fast path and a slow path. The fast path is dedicated to performing one-to-many data reassembly for general cases across fully established TCP connections, while the slow path handles any remaining, rare TCP-level and application-level events, such as TCP connection establishment and application-level session creation.

[0008] With respect to zero-copy, NIC-assisted data reassembly on the fast path, one-to-N data reassembly requires some method for inserting multiple copies of the payload from an inbound (RX) packet into multiple outbound (TX) packets. To minimize memory copy overhead, some example embodiments use a shallow copy that avoids copying the payload across TCP connections. The terms "shallow memory copy," "shallow copy," "reference copy," and similar terms used herein refer to a process that increments a reference counter to a shared memory buffer. This is in contrast to a deep copy, which copies the contents of a given memory buffer to another memory location. The shallow memory copy can be used in conjunction with the scatter-gather capability of the NIC to achieve zero-copy one-to-N packet transmission. The scatter-gather capability of the NIC allows the NIC to obtain the payload data from a shared memory buffer using a reference counter.

[0009] With respect to dynamic load rebalancing, note that different traffic patterns can cause the processing overhead of the fast path and the slow path to be disproportionate. Accordingly, some example embodiments monitor the CPU utilization of the fast path and the slow path, and independently scale the fast path and the slow path by adjusting their respective CPU core assignments. Additionally, due to the skewed nature of data splice rates and splice group sizes, the receive side scaling (RSS)-driven data distribution of the NIC can cause an uneven load distribution across different CPU cores assigned to the fast path. To address load imbalance within the fast path, some example embodiments incrementally redistribute flows among the available CPU cores upon detecting any imbalanced utilization. In one or more example embodiments, the flow redistribution includes installing flow steering rules in the NIC, moving the processing of the top K largest flows from the most congested CPU core to the least busy core, and migrating the corresponding flow state.

[0010] In some example embodiments, a computing device includes at least one network interface controller configured to receive a plurality of inbound Transmission Control Protocol (TCP) packets and to send a plurality of outbound TCP packets, a memory configured to store a program of instructions, and at least one processor including one or more cores, the at least one processor coupled to the at least one network interface controller and the memory, the at least one processor configured to execute the program of instructions to partition a data path within the computing device that includes both a receive endpoint and a send endpoint into a first path (e.g., a fast path) and a second path (e.g., a slow path), assign a first inbound TCP packet to the first path in response to determining that the first inbound TCP packet is associated with an established TCP connection and a data splice group, assign a second inbound TCP packet that is not assigned to the first path to the second path, and perform one-to-many data splicing of payload data from the first inbound TCP packet to the plurality of outbound TCP packets using the first path.

[0011] In some example embodiments, a computing device includes means for receiving a plurality of inbound Transmission Control Protocol (TCP) packets, means for partitioning a data path that includes both a receive endpoint and a send endpoint into a first path and a second path, means for assigning a first inbound TCP packet of the plurality of inbound TCP packets to the first path in response to determining that the first inbound TCP packet is associated with an established TCP connection and a data splice group, means for assigning any inbound TCP packet that is not assigned to the first path to the second path, means for performing one-to-many data splicing of payload from the first inbound TCP packet to a plurality of outbound TCP packets using the first path, and means for sending the plurality of outbound TCP packets.

[0012] In some example embodiments, a method for use in a computing device includes receiving a plurality of inbound Transmission Control Protocol (TCP) packets, partitioning a data path including both a receiving endpoint and a sending endpoint into a first path and a second path, assigning a first inbound TCP packet of the plurality of inbound TCP packets to the first path in response to determining that the first inbound TCP packet is associated with an established TCP connection and a data splice group, assigning any inbound TCP packets that are not assigned to the first path to the second path, performing one-to-many data splicing of payloads from the first inbound TCP packet to a plurality of outbound TCP packets using the first path; and sending the plurality of outbound TCP packets.

[0013] In some example embodiments, a non-transitory computer readable medium tangibly embodying a program of instructions configured to be stored in a memory and executed by a processor, the computer readable medium comprising: at least one instruction to receive a plurality of inbound Transmission Control Protocol (TCP) packets, at least one instruction to partition a data path including both a receiving endpoint and a sending endpoint into a first path and a second path, at least one instruction to assign a first inbound TCP packet of the plurality of inbound TCP packets to the first path in response to determining that the first inbound TCP packet is associated with an established TCP connection and a data splice group, at least one instruction to assign any inbound TCP packets that are not assigned to the first path to the second path, at least one instruction to perform one-to-many data splicing of payloads from the first inbound TCP packet to a plurality of outbound TCP packets using the first path; and at least one instruction to send the plurality of outbound TCP packets.

[0014] Any or all of the example embodiments described above, and other example embodiments disclosed herein, can be used in some combinations. BRIEF DESCRIPTION OF DRAWINGS

[0015] Example embodiments will be more fully understood from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements in which:

[0016] Figure 1 is a block diagram of a network configured to perform one-to-many Transmission Control Protocol (TCP) connection splicing according to example embodiments;

[0017] Figure 2 is a thread level representation of a fast path according to example embodiments;

[0018] Figure 3 is a thread level representation of a slow path according to example embodiments;

[0019] Figure 4 This is a block diagram illustrating zero-copy data packet copying and transmission according to an example embodiment;

[0020] Figure 5 This is a block diagram illustrating out-of-order packet processing with zero copy, based on an example embodiment; and

[0021] Figure 6 This is a flowchart illustrating the method according to an example embodiment.

[0022] It should be noted that these figures are intended to illustrate general features of the methods, structures, and / or materials used in some exemplary embodiments and are intended to supplement the textual description provided below. However, these figures are not drawn to scale and may not precisely reflect the exact structural or performance characteristics of any of the given embodiments, and should not be construed as limiting or restricting the range of values ​​or characteristics included by the exemplary embodiments. The use of similar or identical reference numerals in the figures is intended to indicate the presence of similar or identical elements or features. Detailed Implementation

[0023] Some exemplary embodiments will now be described more fully with reference to the accompanying drawings in which some embodiments are illustrated.

[0024] This document discloses detailed illustrative embodiments. However, the specific structural and functional details disclosed herein are representative only for the purpose of describing exemplary embodiments. These exemplary embodiments may be implemented in various alternative forms and combinations and should not be construed as being limited to the embodiments set forth herein.

[0025] Furthermore, it should be understood that there is no intention to limit the exemplary embodiments to the specific forms disclosed. Rather, the exemplary embodiments encompass all modifications, equivalents, and alternatives falling within the scope of this disclosure. Throughout the description of the drawings, the same reference numerals refer to the same elements. One or more exemplary embodiments described herein may be combined.

[0026] The Transmission Control Protocol (TCP) stack is designed for general purposes, adaptable to various TCP applications sharing the underlying TCP stack. In contrast, some of the example embodiments disclosed herein can, but do not necessarily, be implemented in a tightly controlled server environment, such as in a dedicated device where only one type of data splicing application is deployed and operational. An example is a distributed-subscribe (PubSub) broker device. A typical PubSub broker maintains TCP connections with publisher and subscriber clients, communicating via PubSub protocols (e.g., Message Queuing Telemetry Transport (MQTT) protocols). Generally, publisher clients send messages to the broker under specific topics, while one or more subscriber clients subscribe to the topics and receive messages published under those topics through the broker. Thus, the broker's role is to facilitate one-to-many data splicing across TCP connections established using publisher and subscriber clients. In at least one example embodiment, performance and scalability in such one-to-many TCP data splicing within a local server where the data splicing application is deployed can be enhanced. In example embodiments implemented in such a specialized setup, the data splicing application and the TCP stack can be tightly integrated and co-designed to improve the performance and scalability of the data splicing. Below, in the context of various example embodiments, some example embodiments of such implementations are described.

[0027] First refer to Figure 1 A block diagram of network 100 configured to perform one-to-many Transmission Control Protocol (TCP) connection splicing will be discussed according to an example embodiment. Network 100 includes processing device 110, data senders, and multiple data receivers. Processing device 110 can be used to provide business internet-grade services that dynamically route data from a single source to multiple consumer entities, distributed storage systems, content delivery networks (CDNs), video conferencing systems, etc.

[0028] In various example embodiments, processing device 110 is illustrated as including two network interface controllers (NICs), an inbound NIC 105 and an outbound NIC 198. However, the illustration of two NICs is for ease of explanation only, as a single NIC can perform both receive (RX) and transmit (TX) functions. Therefore, any reference to inbound NIC 105 or outbound NIC 198 can be understood as a reference to a single NIC performing the corresponding RX or TX function. The processor(s) / processor(s) / processor core(s) 114 may include an internal cache / cache 162 and memory 116. Note that the use of multiple NICs is not excluded in some example embodiments. The processor(s) / processor core(s) 114 may be implemented as a multi-core processor configured to perform arithmetic, logical, and input / output operations of the system by processing multiple threads simultaneously to execute instructions of a computer program. Instructions may be provided to the processor(s) / processor core(s) 114 by memory 116.

[0029] Memory 116 may be a computer-readable storage medium, which typically includes random access memory (RAM), read-only memory (ROM), and / or permanent / long-term mass storage devices, such as hard disk drives. Memory 116 may store an operating system and any other routines / modules / applications for providing functionality to the processing device 110 executed by the processor(s) / processor(s) / processor core(s) 114. These software components may be loaded from separate computer-readable storage media, such as disks, tapes, DVD / CD-ROM drives, memory cards, other memory, etc. In some example embodiments, software components may be loaded into memory 116 via a communication interface, such as an inbound NIC 105.

[0030] Data sender 103 provides data for delivery to multiple data receivers, including data receiver C191 and data receiver D192, through data receiver Y193. Inbound A data packet 121 includes A-data packet data 102, inbound B data packet 122 includes B-data packet data 106, and inbound N data packet 123, each of which includes N-data packet data 108. The processing core sends the data packet data 102, 106, and 108 obtained from inbound data packets 121, 122, and 123 to each of the multiple data receivers.

[0031] Processing device 110 receives inbound data packets sent by data sender 103 at inbound NIC 105 and uses processor(s) / processing core(s) 114 to insert the various data items included in the inbound data packets into each of a plurality of output data packets. Processing device 110 uses outbound NIC 198 to send output data packets to each of a plurality of data receivers. In the illustrated example embodiment, each outbound data packet including outbound C data packet 181, outbound D data packet 182, and outbound Y data packet 183 includes A+B+N data packet 171.

[0032] Despite Figure 1 Although not shown in the figure, the processing device 110 may also include various input / output devices, such as a keyboard, keypad, mouse, touch screen display, speaker, etc.

[0033] refer to Figure 2 and Figure 3 Based on various example embodiments, the separation of cross-layer data path processing will be discussed. At least some example embodiments employ a unified application-network data path, where TCP protocol stack processing is integrated into application-level data concatenation. In example embodiments of this cross-layer design, there is no strict isolation between the protocol stack and the application process, which allows for the avoidance of the overhead associated with strict layering.

[0034] In some example implementations, to allow the unified data path to execute more efficiently on hardware with high CPU core density, the unified data path is divided into fast and slow paths. The fast path is optimized for low-overhead, routine handling, while the slow path handles less frequent, more complex events within the unified data path. Specifically, the fast path is dedicated to performing one-to-many data concatenation across fully established TCP connections. The slow path handles rare events occurring at both the TCP and application layers. As used herein, the term "control function" refers to both TCP layer events and application layer events handled by the slow path. TCP-level slow path events include the TCP three-way handshake, packet retransmission upon timeout, and flow control. The application-layer slow path involves handling specialized control messages that affect data concatenation on active TCP connections, such as handling connection requests, disconnection requests, subscription requests, and unsubscription requests in the case of the MQTT application protocol.

[0035] Next reference Figure 2 The thread-level representation of fast path 200 will be discussed based on the example embodiment.

[0036] In the example embodiment, the primary function of the fast path is to perform pair-N data concatenation over TCP connections. The fast path operates under two assumptions: (1) active TCP connections have been established with (N+1) network endpoints, and (2) a data concatenation group has been formed among these endpoints, with one endpoint acting as the data sender and the other N endpoints acting as data receivers. When the fast path receives a packet from a flow that does not meet either condition, it delegates the processing of that packet to the slow path, which will subsequently refer to... Figure 3 discuss.

[0037] like Figure 2 As shown, one-to-many data concatenation is performed in (multiple) processors / (multiple) processing cores 114. In an example embodiment of the fast path implementation, there are four types of threads operating within the fast path 200, including: (i) fast path RX i Thread 215 (e.g., RX1 220, RX2 225, RX3 230, and RX4 235), (ii) fast path TX i Threads 275 (e.g., TX1 280, TX2 285, and TX3 290), (iii) fast path monitoring thread 295, and (iv) fast path flow bootstrapping thread 297. In the example embodiment, each thread operates under a "run-to-complete" model, continuously looping through tasks on a dedicated CPU core without preemption (e.g., Data Plane Development Kit Extreme Mode Driver (DPDK PMD) thread). Fast path RX i Thread 215 and fast path TX i Thread 275 acts as the main worker thread in the fast path, concatenating received data packets onto the appropriate TCP connection. There are K fast paths RX. i Thread 215 and L fast paths TX i Thread 275, where K and L are adjusted based on traffic requirements and the availability of central processing unit (CPU) cores (K=4 and L=3 in the depicted scenario). Each RX i Thread filling is called ConnState i The table that stores data generated by Thread i The connection states of all TCP streams are processed. Each connection state has its own created packet buffer. Each packet buffer retains received packets or packets to be sent on the corresponding connection.

[0038] Connection state can include TCP stream state (e.g., received TCP sequence number, last acknowledgment sequence number, receive window size, round-trip time (RTT) estimate, etc.) and application-level state (e.g., whether the connection is associated with the data sender or the receiver, and in the case of the data sender, whether it is a receiver connection group). In the K fast paths RX i The distribution of TCP streams in thread 215 is determined by the stream hash value calculated by the NIC's receive-side scaling (RSS) 211. L fast paths TX i Thread 275 uses the same stream hash value to assign TX-related stream processing among them.

[0039] When receiving data packets, RX i Threads in their own ConnState i The table queries the connection status of data packets and updates their TCP flow state as part of TCP layer processing. Additionally, RX i The thread copies data packets based on data concatenation group information and stores the copied packets as part of the application layer processing in the destination connection's packet buffer (e.g., packet buffers 251, 252, 253, or 254). Depending on the specific implementation, the packet copying logic can be applied to TX. i Threads, and RX i The thread simply stores the received data packets in the packet buffer of the packet sender. Each TX j Threads from K ConnStates i The data packets for storage used in the stream are retrieved from the table (i = 1, ..., K), and packets are sent according to the TCP flow control rate for each stream. TX j The thread is also responsible for sending TCP ACK packets, and for K RXs. i The thread responds to the input data packet and generates an optional application-level acknowledgment message 245.

[0040] In various example embodiments, the fast path issues data concatenation acknowledgments for data received via a TCP connection on the sender's side. Operating across the TCP and application layers, the fast path processes acknowledgments at both layers. At the TCP layer, the fast path can generate an ACK packet for each received data packet. Optionally, the fast path can combine multiple acknowledgments into a single ACK packet to improve performance. At the application layer, depending on the configuration of the data sender in the connection, the fast path can generate application-level acknowledgments for each received message. For example, in the MQTT protocol with Quality of Service (QoS) level 1, a 4-byte PUBACK message is sent as a response to each PUBLISH message received from the publisher client. When an acknowledgment message is generated, the TCP ACK flag can be marked along with the message's TCP packet, eliminating the need for a separate ACK packet. In the example embodiments described herein, the RX in the fast path... i Threads are responsible for generating the code at both layers, which is then handled by TX. i The confirmation sent by the thread.

[0041] Next reference Figure 3 The slow path 305, included in the separate data path 300, will be discussed according to an example embodiment. In the example embodiment, the slow path 305 handles rare events at both the TCP layer and the application layer. TCP-level slow path events may include TCP three-way handshakes, packet retransmissions upon retransmission timeouts, and flow control, while application-layer slow path events involve handling application-specific control messages that affect data concatenation on active TCP connections, such as handling connection requests, disconnection requests, subscription requests, and unsubscribe requests in the case of the MQTT application protocol. The following description provides additional details on how these rare events are offloaded to the slow path 305 and how they are handled by the slow path 305.

[0042] The separate data path 300 includes a slow path 305, which includes multiple slow path threads 310, and a fast path 200 (which has been previously referenced). Figure 2 (To be discussed). The start and end points of both slow path 305 and fast path 200 are inbound NIC 105 and outbound NIC 198.

[0043] Continue to refer to Figure 3 The establishment and termination of data splicing will be discussed based on various example embodiments. At the TCP layer, on the server side (e.g.) Figure 1Each TCP stream processed by the processing device 110 undergoes a series of state transitions from stream creation to stream termination: LISTEN → SYN_RECEIVED → ESTABLISHED for stream creation; ESTABLISHED → FIN_WAIT1 → FIN_WAIT2 → TIME_WAIT → CLOSED for stream termination, where termination is initiated on the server side when a disconnect request message is received from the data sender or receiver. In some example embodiments, any TCP packets that trigger state transitions before and / or after the ESTABLISHED state are handled by the slow path. After a given TCP stream has reached the ESTABLISHED state, the two connected network endpoints exchange initial control messages via TCP to establish an application-level session for data concatenation. In an example embodiment using the MQTT protocol, the MQTT client undergoes a connection establishment phase, where it sends a CONNECT message to the broker, which then responds with a CONNACK message. Subsequently, if the client is a publisher (e.g., such as...), Figure 1 (In the case of data sender 103), the client begins sending PUBLISH messages carrying the published data to the agent. Upon receiving the first PUBLISH message, the agent is able to identify the client as the publisher. On the other hand, if the client is a subscriber (e.g., such as...) Figure 1 If the target of the data receiver C 191 is determined, the client sends a SUBSCRIBE message to the broker indicating the topics(s) the client wants to subscribe to. The broker can then identify the client as a subscriber from this SUBSCRIBE message. Given this workflow, TCP packets continue to be processed by the slow path until the role of the remote network endpoint in the data concatenation is identified as either a data sender or a data receiver.

[0044] Conversely, on active application-level sessions, clients can send control messages to deactivate the session. For example, an MQTT client can send an UNSUBSCRIBE message to stop receiving messages, and a DISCONNECT message for graceful sessions. These control messages are typically identified in a fixed-size message header (e.g., the message type in a 2-byte MQTT header), and therefore the control message can be detected by the fast path from a fixed offset of the packet payload and is also sent to the slow path.

[0045] In various embodiments, to support the slow path processing discussed above, the RX in the fast path... i The thread examines the following types of packets received at the inbound NIC 105 and forwards them to the slow path:

[0046] (1) TCP packets with any of the flag bits in the SYN / FIN / RST flags;

[0047] (2) TCP packets for streams whose TCP state is not ESTABLISHED;

[0048] (3) TCP packets used for streams that have not yet been identified as senders or receivers in the data concatenation process; and

[0049] (4) A TCP packet carrying a control message that deactivates a splicing session for a specific sender or receiver.

[0050] Continue to refer to Figure 3 TCP congestion control and retransmission will be discussed based on example embodiments. According to some example embodiments, TCP congestion control includes adjusting the transmission rate of the congestion window or stream based on congestion signals such as duplicate ACK packets, packet loss, ECN-tagged packets, variations in RTT, and bandwidth estimates.

[0051] In some example implementations, TCP congestion control is delegated to the slow path 305, rather than being processed on a per-ACK basis as part of the main packet processing logic. More specifically, the slow path 305 periodically iterates through all active TCP connections associated with the data sender and adjusts the congestion control state for each flow. The frequency of congestion control execution for each flow is adjusted based on the current RTT estimate of the flow, which has been determined by the fast path RX. i Thread 215 calculates and stores the TCP ACK packet from the receiving side of the TCP connection in ConnState. i In the process of traversing connections, the slow path also checks for retransmission timeouts and notifies the fast path (TX) of the timeout. i Thread 275 triggers packet retransmission. Note that the retransmission timeout (RTO) value is typically set based on an RTT range.

[0052] As part of slow path processing, slow path 305 can generate TCP control packets (e.g., SYN / ACK packets) or data packets carrying application-level messages (e.g., CONNACK messages for the MQTT protocol). Once these packets are generated by the slow path, they are then processed by the fast path TX. i Thread 275 sends the data. Depending on the implementation, the transmission of these packets can be handled by the slow path itself.

[0053] Various example embodiments employ dynamic load rebalancing to dynamically adjust the CPU core assignment between the fast path 200 and the slow path 305 based on varying traffic patterns. For example, if the data stitching rate is high but the data stitching pattern remains stable (e.g., the receiver remains the same), the overhead on the slow path will be relatively low. In this case, more CPU cores can be allocated to the fast path to maintain synchronization with the data stitching rate. On the other hand, if there is a high level of dynamism in the data stitching pattern while the overall data rate is low, more CPU cores can be allocated to the slow path to manage the changes in the data stitching group.

[0054] Another aspect of load rebalancing involves the distribution of TCP flows within the fast path. The RSS-driven flow distribution of the NIC can result in an unbalanced load across different CPU cores during fast path processing. Example embodiments address the following load imbalance problem. First, each RX i thread monitors its own CPU utilization and maintains a list of the top K flows in terms of the flow rate between the flows assigned to it. The RX i thread can use space-saving algorithms known to those of ordinary skill in the art to maintain the set of the top K flows with minimal memory / processing overhead. Using the information about the top K flows of traffic, the main thread (such as the fast path monitoring thread 295 ( Figure 2 )) periodically checks for any unbalanced CPU utilization among the RX i threads and performs flow reallocation. This reallocation involves installing specialized flow steering rules in the inbound NIC 105 to override its RSS-based steering and moving the top K flows along with their flow states from the most overloaded CPU core to the least busy core. Here, k (k < K) represents the number of flows to be migrated in each monitoring cycle. In the most conservative case, k can be set to 1, minimizing the overhead of flow reallocation within the fast path. In at least one example embodiment, the installation of the flow rules is delegated to a dedicated offload thread, such as the fast path flow steering thread 297, as Figure 3 depicted in

[0055] Next, referring to Figure 4 , a block diagram illustrating packet copy and transmission with zero copy will be discussed according to example embodiments. Specifically, Figure 4The diagram illustrates an example scenario where data splicing occurs across four TCP connections. Data received from the sender 192.168.1.10:50000 is spliced ​​onto three destination connections representing three receivers: 10.10.1.100:20000, 10.20.1.100:2000, and 10.30.1.100:20000. The Fast Path maintains the splicing state of the sender / receiver connections, including the IP address and port number, and the last TCP sequence number used for splicing. When receiving data packets from the sender, RX... i The thread removes the TCP / IP header from the packet and creates three shallow copies of the packet payload without performing an actual memory copy (i.e., the RX). i The thread increments the reference counter used for the payload memory cache by 3. Then, the RX i The thread generates three separate TCP / IP headers for three destinations by querying the data concatenation status, and appends a duplicate payload to each TCP / IP header. This process produces three packets, each containing a cloned payload appended to a different packet header. These linked packets are stored in the packet buffer of the destination connection and are processed by the corresponding (multiple) TX(s). i The data is retrieved by the thread and ultimately sent to the outbound NIC 198 for transmission. In at least one example embodiment, the outbound NIC 198 is equipped with a scatter-aggregate capability, allowing the outbound NIC 198 to combine the header and payload stored in a non-contiguous storage buffer and transmit them as a complete data packet. In case retransmission is required due to packet loss, an additional shallow copy can be used to retain the data packet in the packet buffer after transmission.

[0056] Next reference Figure 5 The block diagram 500 illustrating out-of-order packet processing with zero-copy will be discussed according to an example embodiment. The inherent responsibility of the TCP protocol stack includes handling out-of-order packet arrivals. In the context of splicing, even when packets arrive out of order on the sending side's TCP connection (e.g., a packet with sequence number N+δ arrives before a packet with sequence number N), the packets must be spliced ​​in the correct order onto the corresponding receiving side's connection. This ensures that congestion or packet loss on any sending side does not propagate to all receiving side connections.

[0057] To allow out-of-order packets to be appended in an ordered sequence to the connection on the corresponding receiver side while maintaining zero copy overhead, various example implementations manage the arrival of out-of-order packets, such as... Figure 5As shown. The range of numbers displayed in each arriving packet (e.g., [1001, 1200]) represents the range of TCP sequence numbers used for the corresponding packet payload 560. This is typical when packets arrive in sequence (e.g., [1001, 1200], [1201, 1400]), and the pointers to these packets are stored consecutively in the packet buffer. When the first out-of-order packet (e.g., [1801, 1200]) arrives, it is normally placed in the next consecutive position, but this action triggers the packet [1201, 1400] in the packet buffer to be marked as anchor packet 525 for the missing sequence number range [1401, 1800]. This anchor packet 525 acts as a placeholder packet, under which the payloads 560 of subsequent late-arriving packets for [1401, 1800] are chained together. Figure 5 As shown, when packets [1401, 1600] and [1601, 1800] arrive later, their payloads 560 are linked under anchor packet 525 [1201, 1400]. After the missing sequence range is completed, the marker for anchor packet 525 [1201, 1400] is removed. If out-of-order packets are frequent, multiple anchor packets can be active simultaneously. For example, in this scenario, if packet [2601, 2800] arrives between [1401, 1600] and [1601, 1800], packet [2601, 2800] will trigger another anchor packet [2001, 2200] for the new missing range [2201, 2600]. Essentially, each anchor packet 525 is responsible for collecting packets for different consecutive missing sequence number ranges. Once the packets are organized in this way within the packet buffer, they are copied and sent as described previously. The payload 560 of the link can be combined for transmission via the NIC's distribute-aggregate offloading. If the combined packets exceed the maximum transmission unit (MTU) size, the packets can also be fragmented for transmission via the NIC's transmit segmentation offloading (TSO).

[0058] Next reference Figure 6 Method 600 will be discussed according to an example embodiment. As shown in block 603, processing device 110 receives inbound data packets at a data path, wherein the data path includes a receiving endpoint (e.g., at inbound NIC 105) and terminates both using a sending endpoint (e.g., at outbound NIC 198), as... Figure 1 As shown. In some embodiments, the data path and the separate data path 300 ( Figure 3 Correspondingly, each inbound packet in the inbound data packet includes payload data.

[0059] As shown in box 605, the processing device 110 divides the data path into a first path (e.g., fast path 200). Figure 2 and 3 ), and the second path (e.g., slow path 305). Figure 3 )).

[0060] As shown in box 607, processing device 110 examines each inbound packet to determine whether the inbound packet is associated with an established connection. In at least one example embodiment, the check at box 607 is a TCP-level check, wherein the RX in the first (e.g., fast) path... i The thread examines each received packet to determine whether the received packet has an "established" flow state (sometimes referred to as a connection state in this document). If the result of the examination at box 607 indicates that the inbound packet is associated with an established connection, method 600 proceeds to box 609.

[0061] As shown in box 609, processing device 110 performs a check to determine whether inbound data packets are also associated with an already established data splice group. In some example embodiments, the check at box 609 is performed by the fast path RX. i The thread performs an application-level check, and the data splice group refers to the association between the data sender and one or more data receivers. See [link to documentation] for additional details. Figure 3 The discussion.

[0062] If the check at boxes 607 and 609 indicates that the inbound data packet is associated with a data splice group and an established connection, then the processing device 110 will assign the assigned data packet to a first path (e.g., a fast path), as shown in box 611.

[0063] As shown in box 613, if an inbound packet is assigned to the first path, the processing device performs a one-to-many data concatenation, in which the payload to be included in the inbound packet is inserted into multiple output packets.

[0064] As shown in box 615, the processing device 110 then sends multiple output data packets to their destinations.

[0065] Returning to boxes 607 and 609, if the check at box 607 indicates that the inbound packet is not associated with an established connection, or if the check at box 609 indicates that the inbound packet is not associated with an established data concatenation group, then as shown in box 617, processing device 110 assigns the inbound packet to the second path. In various example embodiments, if the packet includes state transition information, the inbound packet is also assigned to the second path. See [link to documentation] for additional details. Figure 3 The discussion.

[0066] In at least some example embodiments, the data packets assigned to the second path at block 617 are considered to include control information, such as state transition information. As shown in block 619, the second (e.g., slow) path is used to perform control functions related to data concatenation using the payload of the data packets assigned to the second path. See [link to documentation] for additional details. Figure 3 The discussion.

[0067] Various non-limiting illustrative embodiments will be discussed herein. Illustrative embodiment 1 includes: at least one network interface controller configured to: receive a plurality of inbound Transmission Control Protocol (TCP) packets and send a plurality of outbound TCP packets; a memory configured to store a program of instructions; and at least one processor including one or more cores coupled to the at least one network interface controller and the memory, the at least one processor being configured to execute a program of instructions to: divide a data path within a computing device, including both a receiving endpoint and a sending endpoint, into a first path and a second path; in response to determining that a first inbound TCP packet is associated with an established TCP connection and a data concatenation group, assign the first inbound TCP packet to the first path, assign a second inbound TCP packet not assigned to the first path to the second path, and use the first path to perform a one-to-many data concatenation of the payload from the first inbound TCP packet to the plurality of outbound TCP packets.

[0068] Illustrative embodiment 2 includes a computing device as in illustrative embodiment 1, wherein at least one processor is further configured to execute an instruction program to perform one-to-many data concatenation using zero-copy, network interface controller-assisted data concatenation.

[0069] Illustrative embodiment 3 includes a computing device as described in any one of illustrative embodiments 1 or 2, wherein at least one processor is further configured to execute instructions to: maintain a connection state cache including information indicating a sender-receiver connection; use a receive thread included in a first path to remove headers from a first inbound TCP packet; store a shallow copy of the payload of the first inbound TCP packet in a packet cache; generate different headers for multiple outbound TCP packets based on the information indicating a sender-receiver connection; append different headers to the shallow copy in the packet cache; use a send thread included in the first path to retrieve the different headers and the shallow copy from the packet cache; and wherein at least one network interface controller is configured to assemble the different headers and the shallow copy into multiple complete TCP packets and send the multiple complete TCP packets.

[0070] Illustrative embodiment 4 includes a computing device of any one of illustrative embodiments 1-3, wherein at least one processor is further configured to execute a program of instructions to implement a cross-layer application-network data path that integrates TCP protocol stack processing into application-level data concatenation.

[0071] Illustrative embodiment 5 includes a computing device as described in any of illustrative embodiments 1-4, wherein at least one processor is further configured to execute a program of instructions to, in response to determining that a particular inbound TCP packet satisfies any of the following conditions, use a receive thread included in a first path to transfer the processing of the particular inbound TCP packet to a second path: including an active TCP SYN flag, FIN flag, or TCP RST flag; associated with a flow that has not yet been established; associated with a flow that has not yet been identified as a data splicing sender or receiver; or carrying a control message that triggers the addition or removal of a data splicing sender or receiver in an existing data splicing session.

[0072] Illustrative embodiment 6 includes a computing device as claimed in any one of claims 1-5, wherein at least one processor is further configured to execute a program of instructions to: monitor the utilization of the second path and the first path; and independently scale the utilization of the second path and the first path by adjusting the corresponding processor core assignments.

[0073] Illustrative embodiment 7 includes a computing device as described in any one of illustrative embodiments 1-6, wherein at least one processor is further configured to execute a program of instructions to:

[0074] In response to detecting unbalanced utilization across multiple processing cores, the flow assigned to the first path is incrementally redistributed among the multiple processing cores.

[0075] Illustrative embodiment 8 includes a computing device as described in any one of illustrative embodiments 1-7, wherein at least one processor is further configured to execute a program of instructions to: implement flow guidance rules in a network interface controller to move a first number of maximum TCP flows from a more congested processing core to a less busy core; and migrate the flow state corresponding to the first number of maximum TCP flows from a first connection state cache to a second connection state cache.

[0076] Illustrative embodiment 9 includes a computing device as described in any of illustrative embodiments 1-8, wherein at least one processor is further configured to execute a program of instructions to: perform a plurality of control functions using a second path, the plurality of control functions being associated with data concatenation performed by a first path across the network layer and the application layer.

[0077] Illustrative embodiment 10 includes a computing device as described in any of illustrative embodiments 1-9, wherein at least one processor is further configured to execute a program of instructions to: traverse established TCP connections and adjust the congestion control state of each flow, wherein the frequency of the congestion control state of each particular flow is determined based on the current round-trip time estimate of the particular flow.

[0078] Illustrative embodiment 11 includes the computing device of illustrative embodiment 10, wherein at least one processor is further configured to execute a program of instructions to: detect a retransmission timeout using a second path; and trigger packet retransmission by notifying a sending thread included in a first path.

[0079] Illustrative embodiment 12 includes a computing device as described in any one of illustrative embodiments 1-11, wherein at least one processor is further configured to execute a program of instructions to: process out-of-order arrival of one or more inbound TCP packets by: using out-of-order received TCP packets as anchor packets for late TCP packets; linking subsequently arriving TCP packets to the anchor packets; storing shallow copies of multiple different payloads of the first inbound TCP packet in non-contiguous locations in the packet buffer; appending shallow copies of multiple different payloads to a single different header; using a sending thread included in a first path to retrieve the single different header and shallow copies of multiple different payloads from the packet buffer; and at least one network interface controller is configured to: retrieve TCP packet headers from the packet buffer, retrieve shallow copies of payloads associated with TCP packet headers from non-contiguous locations in the packet buffer; and assemble and send complete TCP packets.

[0080] Illustrative embodiment 13 includes a method comprising: receiving a plurality of inbound Transmission Control Protocol (TCP) packets; dividing a data path including both a receiving endpoint and a sending endpoint into a first path and a second path; in response to determining that a first inbound TCP packet among the plurality of inbound TCP packets is associated with an established TCP connection and a data concatenation group; assigning the first inbound TCP packet to the first path; assigning any inbound TCP packets not assigned to the first path to the second path; using the first path, performing a one-to-many data concatenation of payloads from the first inbound TCP packet to the plurality of outbound TCP packets; and sending the plurality of outbound TCP packets.

[0081] Illustrative embodiment 14 includes the method as in illustrative embodiment 13, further comprising: maintaining a connection state cache including information indicating a sender-receiver connection; storing a shallow copy of data extracted from a first inbound TCP packet in a packet cache; using a receive thread included in a first path to store outbound TCP packet header information and information associating the outbound TCP packet header information with a specific shallow copy of data extracted from the first inbound TCP packet; using a send thread included in the first path to obtain the outbound TCP packet header information and the specific shallow copy; and assembling the outbound TCP packet header information and the specific shallow copy to generate a plurality of outbound TCP packets.

[0082] Illustrative embodiment 15 includes a method as described in any of illustrative embodiments 13 or 14, further comprising: processing out-of-order arrival of one or more inbound TCP packets by: designating out-of-order received TCP packets as anchor packets for late TCP packets; and linking subsequently arriving TCP packets to the anchor packets.

[0083] One or more functions associated with the methods and / or processes described herein can be implemented as a large-scale system operable to receive, transmit, and / or process data on a massive scale. As used herein, massive refers to large amounts of data, such as one or more kilobytes, megabytes, gigabytes, terabytes, or more of data being received, transmitted, and / or processed. The reception, transmission, and / or processing of such data cannot be actually performed on a massive scale by the human brain within a reasonable timeframe (such as in seconds, milliseconds, microseconds, real-time based, or other high-speed requirements of machines that generate, receive, transmit, store, and / or use data).

[0084] One or more functions associated with the methods and / or processes described herein can require data to be manipulated in different ways over overlapping time spans. The human brain does not possess the capacity to perform such different data manipulations independently, simultaneously, in parallel, and / or collaboratively within reasonable timeframes (such as seconds, milliseconds, microseconds, real-time, or other high speeds required by machines that generate, receive, transmit, store, and / or use data).

[0085] One or more functions associated with the methods and / or processes described herein can be implemented in a system operable to electronically receive and / or transmit digital data via a wired or wireless communication network. Such reception and transmission cannot be actually performed by the human brain, as the human brain is not capable of electronically sending or receiving digital data, let alone sending and receiving digital data via a wired or wireless communication network.

[0086] One or more functions associated with the methods and / or processes described herein may operate to cause a processing module to act directly in response to a triggering event, without any human interaction between the triggering event and the action. Any such operation may be identified as being performed "automatically," "automatically based on such triggering events," and / or "automatically in response to such triggering events." Furthermore, any such action identified in this manner explicitly excludes any human activity relating to these actions, even if the triggering event itself may be causally linked to some human activity.

[0087] One or more functions associated with the methods and / or processes described herein can be implemented in a system operable to electronically store digital data in a memory device. This storage cannot be actually performed by the human brain, as the human brain is not capable of electronically storing digital data.

[0088] As discussed in this article, the terms “one or more” and “at least one” may be used interchangeably.

[0089] Although the terms first, second, etc., may be used herein to describe certain elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the scope of this disclosure.

[0090] When a component is described as being "connected to" or "coupled to" another component, the component is able to be directly connected to or coupled to the other component, or there may be intermediate components. In contrast, when a component is described as being "directly connected to" or "directly coupled to" another component, no intermediate components are present. Other terms used to describe relationships between components should be interpreted in a similar manner (e.g., "between" and "directly between," "adjacent" and "directly adjacent," etc.).

[0091] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. Further understanding, the terms “comprising,” “including,” “comprises,” and / or “containing,” when used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or groups thereof.

[0092] It should also be noted that in some alternative implementations, the indicated functions / actions may not occur in the order shown in the figures. For example, depending on the functions / actions involved, two consecutive figures may actually be executed substantially simultaneously, or sometimes in reverse order.

[0093] Specific details have been provided in the preceding description to provide a full understanding of the exemplary embodiments. However, those skilled in the art will understand that the exemplary embodiments can be practiced without these specific details. For example, a system may be shown in block diagrams to avoid obscuring the exemplary embodiments with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the exemplary embodiments.

[0094] As discussed herein, illustrative embodiments are described by reference to symbolic representations of actions and operations that can be implemented as program modules or functional processes (e.g., in the form of flowcharts, diagrams, data flow charts, structural diagrams, block diagrams, etc.). These program modules or functional processes include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types, and can be implemented using existing hardware, such as existing user equipment or other network elements and / or hardware. This existing hardware can be a processing or control circuitry system, such as, but not limited to, one or more processors, one or more central processing units (CPUs), one or more controllers, one or more arithmetic logic units (ALUs), one or more digital signal processors (DSPs), one or more microcomputers, one or more field-programmable gate arrays (FPGAs), one or more system-on-a-chip (SoCs), one or more programmable logic units (PLUs), one or more microprocessors, one or more application-specific integrated circuits (ASICs), or any other device or device capable of responding to and executing instructions in a defined manner.

[0095] Although a flowchart can describe operations as a sequential process, many operations within an operation can be executed in parallel, concurrently, or simultaneously. Furthermore, the order of operations can be rearranged. A process can terminate when its operations are complete, but it may also have additional steps not included in the diagram. A process can correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, its termination can correspond to the function to which the function returns or the main function.

[0096] As disclosed herein, the terms "storage medium," "computer-readable storage medium," or "non-transitory computer-readable storage medium" can refer to one or more devices for storing data, including: read-only memory (ROM), random access memory (RAM), magnetic RAM, core memory, disk storage media, optical storage media, flash memory devices, and / or other tangible machine-readable media for storing information. The term "computer-readable medium" can include, but is not limited to, portable or fixed storage devices, optical storage devices, and other media capable of storing, containing, or carrying instructions and / or data.

[0097] Furthermore, the example embodiments can be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments that perform the necessary tasks can be stored in a machine or computer-readable medium (such as a computer-readable storage medium). When implemented in software, a processor or multiple processors will perform the necessary tasks. For example, as described above, according to one or more example embodiments, at least one memory may include or store computer program code, and at least one memory and computer program code may be configured, together with at least one processor, to cause a network element or network device to perform the necessary tasks. Additionally, the processor, memory, and example algorithms encoded as computer program code act as components for providing or causing the execution of the operations discussed herein.

[0098] The hardware used to implement the various example embodiments may include processing or control circuitry, such as, but not limited to, one or more processors, one or more CPUs, one or more controllers, one or more ALUs, one or more DSPs, one or more microcomputers, one or more FPGAs, one or more SOCs, one or more PLUs, one or more microprocessors, one or more ASICs, or any other device or device capable of responding to and executing instructions in a defined manner.

[0099] A code segment of computer program code can represent a procedure, function, subroutine, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment can be coupled to another code segment or hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or sent via any suitable technology, including memory sharing, message passing, token passing, network transmission, etc.

[0100] The terms “comprising” and / or “having” as used herein are defined as including (i.e., open-ended statements). The term “coupling” as used herein is defined as a connection, although not necessarily a direct connection and not necessarily a mechanical connection. Terms derived from the word “indication” (e.g., “indicates” and “indication”) are intended to encompass all suitable techniques that can be used to convey or reference the indicated object / information. Some, but not all, examples of techniques that can be used to convey or reference the indicated object / information include: the transmission of the indicated object / information, the transmission of an identifier of the indicated object / information, the transmission of information used to generate the indicated object / information, the transmission of some part or portion of the indicated object / information, the transmission of some derivative of the indicated object / information, and the transmission of some symbols representing the indicated object / information.

[0101] The benefits, other advantages, and solutions to problems have been described above with reference to specific embodiments of the invention. However, the benefits, advantages, solutions to problems, and any (multiple) elements are not to be construed as key, essential, or necessary features or elements of any or all claims that may cause or result in such benefits, advantages, or solutions, or make such benefits, advantages, or solutions more significant.

Claims

1. A computing device for one-to-many connection splicing, comprising: at least one network interface controller configured to: receive a plurality of inbound Transmission Control Protocol (TCP) packets, and send a plurality of outbound TCP packets; a memory configured to store a program of instructions; and at least one processor comprising one or more cores, the at least one processor coupled to the at least one network interface controller and the memory, the at least one processor configured to execute the program of instructions to: divide a data path within the computing device comprising both a receiving endpoint and a sending endpoint into a first path and a second path, assign a first inbound TCP packet to the first path in response to determining that the first inbound TCP packet is associated with an established TCP connection and a data splice group, assign a second inbound TCP packet that is not assigned to the first path to the second path, and perform one-to-many data splicing of payload data from the first inbound TCP packet to the plurality of outbound TCP packets using the first path.

2. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: perform the one-to-many data splicing using zero-copy, network interface controller- assisted data splicing.

3. The computing device of claim 2, wherein the at least one processor is further configured to execute the program of instructions to: maintain a connection state cache comprising information indicative of sender-receiver connections.

4. The computing device of claim 3, wherein the at least one processor is further configured to execute the program of instructions to: use a receiving thread included in the first path to remove a header from the first inbound TCP packet; store a shallow copy of a payload of the first inbound TCP packet in a packet cache; generate different headers for the plurality of outbound TCP packets based on the information indicative of sender-receiver connections; and append the different headers to the shallow copy in the packet cache.

5. The computing device of claim 4, wherein: the at least one processor is further configured to use a sending thread included in the first path to retrieve the different headers and the shallow copy from the packet cache; and the at least one network interface controller is configured to assemble the different headers and the shallow copy into a plurality of complete TCP packets and send the plurality of complete TCP packets.

6. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: implement a cross-layer application-network data path that integrates TCP protocol stack processing into application-level data splicing.

7. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: ​ in response to determining that a particular inbound TCP packet satisfies any one of the following conditions, using a receive thread included in the first path to transfer processing of the particular inbound TCP packet to the second path: includes an active TCP SYN flag, FIN flag, or TCP RST flag, is associated with a flow that has not been established, is associated with a flow that has not been identified as a data stitching sender or receiver, or carries a control message that triggers adding or removing a data stitching sender or receiver in an existing data stitching session.

8. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: monitor utilization of the second path and the first path; and independently scale the utilization of the second path and the first path by adjusting respective processor core assignments.

9. The computing device of claim 8, wherein the at least one processor is further configured to execute the program of instructions to: in response to detecting unbalanced utilization of a plurality of processing cores, incrementally redistribute flows assigned to the first path among the plurality of processing cores.

10. The computing device of claim 9, wherein the at least one processor is further configured to execute the program of instructions to: implement flow steering rules in the network interface controller to move a first number of maximum TCP flows from a more congested processing core to a less busy core; and migrate flow states corresponding to the first number of maximum TCP flows from a first connection state cache to a second connection state cache.

11. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: perform a plurality of control functions using the second path, the plurality of control functions related to data stitching performed by the first path across network and application layers.

12. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: walk through established TCP connections and adjust a congestion control state of each flow, wherein a frequency of adjusting the congestion control state of each particular flow is determined based on a current round-trip time estimate of the particular flow.

13. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: detect a retransmission timeout using the second path; and trigger a packet retransmission by notifying a send thread included in the first path.

14. The computing device of claim 1, wherein the at least one processor is further configured to execute the program of instructions to: handle out-of-order arrival of one or more inbound TCP packets by: using an out-of-order received TCP packet as an anchor packet for a late-arriving TCP packet; and linking a subsequently arriving TCP packet to the anchor packet.

15. The computing device of claim 14, wherein the at least one processor is further configured to: storing shallow copies of a plurality of different payloads of the first inbound TCP packet in non-contiguous locations of a data cache; appending the shallow copies of the plurality of different payloads to a single different header; using a send thread included in the first path to retrieve the single different header and the shallow copies of the plurality of different payloads from the data cache; and the at least one network interface controller is configured to: retrieve a TCP packet header from the data packet cache, retrieve a shallow copy of a payload associated with the TCP packet header from the non-contiguous locations in the data packet cache; and assemble and send a complete TCP packet.

16. A method for one-to-many connection stitching for use in a computing device, comprising: receiving a plurality of inbound Transmission Control Protocol (TCP) packets, dividing a data path including both a receiving endpoint and a sending endpoint into a first path and a second path, responsive to determining that a first inbound TCP packet of the plurality of inbound TCP packets is associated with an established TCP connection and a data stitching group, assigning the first inbound TCP packet to the first path, assigning any inbound TCP packets not assigned to the first path to the second path, using the first path, performing one-to-many data stitching of payloads from the first inbound TCP packet to a plurality of outbound TCP packets; and sending the plurality of outbound TCP packets.

17. The method of claim 16, further comprising: maintaining a connection state cache including information indicative of sender-receiver connections.

18. The method of claim 17, further comprising: storing a shallow copy of data extracted from the first inbound TCP packet in a data packet cache; using a receive thread included in the first path to store outbound TCP packet header information and information associating the outbound TCP packet header information with a particular shallow copy of the data extracted from the first inbound TCP packet; using a send thread included in the first path to obtain the outbound TCP packet header information and the particular shallow copy; and assembling the outbound TCP packet header information and the particular shallow copy to generate the plurality of outbound TCP packets.

19. The method of claim 16, further comprising: handling out-of-order arrival of one or more inbound TCP packets by: designating an out-of-order received TCP packet as an anchor packet for a late- arriving TCP packet; and linking a subsequently arriving TCP packet to the anchor packet.

20. A computer-readable medium tangibly embodying a program of instructions configured to be stored in a memory and executed by a processor, the computer-readable medium comprising: at least one instruction to receive a plurality of inbound Transmission Control Protocol (TCP) packets, at least one instruction to divide a data path including both a receiving endpoint and a sending endpoint into a first path and a second path, ​ ​ ​ at least one instruction to assign a first inbound TCP packet of the plurality of inbound TCP packets to the first path in response to determining that the first inbound TCP packet is associated with the established TCP connection and a data plaiting group, at least one instruction to assign any inbound TCP packets that are not assigned to the first path to the second path, at least one instruction to perform a one-to-many data plaiting of payload data from the first inbound TCP packet to a plurality of outbound TCP packets using the first path; and at least one instruction to send the plurality of outbound TCP packets.

Images