Effective packet reordering using hints

Abstract

The present disclosure relates to efficient data packet reordering using hints. A peripheral device coupled to a host includes a network interface, a packet processor, and a data processing unit (DPU). The packet processor receives data packets from a communication network via the network interface, the data packets being emitted from a source in an original order and received at the peripheral device in an order different from the original order. The packet processor splits the received data packets into headers and payloads, sends the payloads for storage in a host memory, and sends the headers without the payloads for storage in a DPU memory, and generates a hint based on the headers, the hint indicating processing to be applied to the headers by the DPU for identifying the original order. Based on the hint, the DPU identifies the original order of the data packets by applying the processing indicated by the hint to the respective headers in the DPU memory, and notifies the host of the original order.

Description

Technical Field

[0001] The embodiments described herein generally relate to data communication, and more particularly to methods and systems for reordering valid data packets using prompts. Background Technology

[0002] In various computing systems, source devices send data packets to destination devices via communication networks. Data packets originating from the host are sent in a certain original order, but may be received at the destination in a different order. Based on transport layer processing, the destination can recover the original order of the received data packets. Summary of the Invention

[0003] The embodiments described herein provide a peripheral device including a network interface, a host interface, a packet processor, and a data processing unit (DPU). The network interface transmits data packets over a communication network. The host interface communicates with a host coupled to the peripheral device. The packet processor receives multiple data packets from a remote source via the communication network. These data packets are sent from the remote source in their original order and are received at the peripheral device in an arrival order different from the original order. The packet processor splits the received data packets into corresponding headers and payloads, transmits the payload via the host interface for storage in the host's host memory, and transmits the headers without payload for storage in the DPU memory. Based on the headers of the received data packets, it generates a prompt indicating the processing to be applied by the DPU to the headers in the DPU memory, for identifying the original order of at least some of the multiple data packets. The DPU receives the prompt from the packet processor, identifies the original order of at least some of the data packets by applying the processing indicated by the prompt to the corresponding headers in the DPU memory, and notifies the host of the original order.

[0004] In some embodiments, the packet processor identifies two or more packets received in their original order among a plurality of packets and reports the corresponding identifiers of the two or more packets received in the original order to the DPU in a prompt instructing the DPU to avoid identifying the original order of the two or more identified packets. In other embodiments, the DPU notifies the host of a merged payload, which includes the payloads of the two or more identified packets. In other embodiments, the DPU identifies the original order of packets received in an order different from the original order.

[0005] In one embodiment, the packet processor sends the payload to be stored in the host memory by translating a virtual address associated with the address space allocated to a hypervisor or virtual machine (VM) running on the host into a corresponding physical address in the host memory. In another embodiment, the DPU accesses the header in one or more header buffers in the DPU memory by executing a work queue element (WQE) in the DPU memory. In yet another embodiment, the DPU processes the header in the DPU memory according to a transport layer protocol used for communication between the host and a remote source. In yet another embodiment, the transport layer protocol belongs to a list including Transmission Control Protocol (TCP) and Datagram Congestion Control Protocol (DCCP).

[0006] According to the embodiments described herein, a method for communication is also provided, comprising: in a peripheral device mediating communication between a host and a remote device via a communication network, a packet processor receives a plurality of data packets from a remote source via the communication network, the data packets being sent from the remote source in their original order and received at the peripheral device in an arrival order different from the original order. The received data packets are split into corresponding headers and payloads. The payload is sent to be stored in the host memory of the host, and the header is sent without the payload to be stored in the DPU memory of a data processing unit (DPU). Based on the headers of the received data packets, a prompt is generated indicating processing to be applied by the DPU to the headers in the DPU memory for identifying the original order of at least some of the plurality of data packets. The prompt from the packet processor is received by the DPU. The original order of the at least some data packets is identified by applying the processing indicated by the prompt to the corresponding headers in the DPU memory. The original order is notified to the host.

[0007] According to the embodiments described herein, a method for packet reordering is also provided, comprising: receiving packets destined for a host in an order different from the original order in which the packets were sent from a remote source. The payload of the packets is stored in the host's host memory, and the header of the packets is stored in the DPU memory of a data processing unit (DPU) without the payload. The original order of the packets is identified by processing the header in the DPU memory, as indicated by a prompt generated based on the header of the received packets. The original order is then communicated to the host. Attached Figure Description

[0008] These and other embodiments will be more fully understood from the following detailed description of their embodiments, together with the accompanying drawings, in which:

[0009] Figure 1The block diagram of the computer system is schematically shown according to the embodiments described herein, with hints that facilitate efficient packet reordering;

[0010] Figure 2 This is a flowchart illustrating, schematically, a method for reordering valid data packets using prompts, according to embodiments described herein; and

[0011] Figure 3 This is a diagram schematically illustrating the sequence of data packets according to the embodiments described herein, some of which do not require reordering as indicated in the prompt. Detailed Implementation

[0012] Overview

[0013] The embodiments described herein provide improved methods and systems for effectively identifying the original order of sent data packets using prompts.

[0014] In various computing systems, the host communicates with remote devices through a communication network that includes underlying transport layer protocols such as Transmission Control Protocol (TCP). Transport layer protocols can require significant host resources, and therefore are sometimes offloaded to another processor outside the host. This external processor may include, for example, a Data Processing Unit (DPU), which resides in peripheral devices that mediate between the host and the communication network. Peripheral devices typically include network adapters (such as network interface controllers (NICs) for accessing the communication network) and memory access interfaces for accessing host memory.

[0015] In the computing system described above, a remote device can send a sequence of data packets to the host in a certain original order. However, due to various reasons, the data packets may be received at the peripheral device in an arrival order different from the original order. For example, different data packets may traverse different paths in the communication network with different corresponding delays. As another example, some data packets may be delayed within the network switch because higher-priority data packets pass through it.

[0016] In some embodiments, the NIC splits the received data packets into headers and payloads, sending the payload to be stored in host memory and the header to be stored in DPU memory. In principle, the DPU can process the header using the underlying transport layer protocol, including reordering all data packets according to their original order. However, processing the entire received data packet to restore its original order, rather than just processing out-of-order packets, can result in long latency, reduced receive throughput, and increased DPU computational resources.

[0017] Techniques for transport layer packet reordering are known in the art. For example, U.S. Patent Application No. 17 / 488,362 by Eran et al. describes a network interface controller device including a network interface, a memory access interface, a data processing unit, and a packet processing unit. The network interface is used to connect to and receive packets through a packet data network. The memory access interface is used to share data with a host device. The packet processing circuitry is used to split the data of a packet into corresponding headers and corresponding payloads, storing the corresponding payloads at corresponding locations in the host memory of the host device, and storing the corresponding headers in the memory of the data processing unit when no corresponding payload is present, with a descriptor indicating the corresponding location in the host memory where the corresponding payload is stored. The data processing unit is used to process the corresponding headers, wherein the packet processing circuitry is configured to send a corresponding completion message to the host device through the memory access interface when the data processing unit completes processing the corresponding headers.

[0018] In the disclosed embodiments, the NIC identifies subsequences of received data packets received in their original order, thus eliminating the need for reordering. The NIC generates a cue to the DPU indicating such subsequences, and the DPU then skips reordering the reported data packets based on the cue.

[0019] Consider an embodiment of a peripheral device including a network interface, a host interface, a packet processor, and a data processing unit (DPU). The network interface transmits data packets over a communication network. The host interface communicates with a host coupled to the peripheral device. The packet processor receives multiple data packets from a remote source via the communication network, wherein the data packets are sent from the remote source in their original order and received at the peripheral device in an arrival order different from the original order. The packet processor splits the received data packets into corresponding headers and payloads, and (i) transmits the payload via the host interface for storage in the host's host memory, and (ii) transmits the headers without payloads for storage in the data processing unit (DPU) memory. Based on the headers of the received data packets, the packet processor generates a prompt indicating the processing to be applied by the DPU to the headers in the DPU memory, for identifying the original order of at least some of the multiple data packets. The DPU receives the prompt from the packet processor, identifies the original order of at least some data packets by applying the processing indicated by the prompt to the corresponding headers in the DPU memory, and informs the host of the original order.

[0020] In some embodiments, the packet processor identifies two or more packets received in their original order among a plurality of packets and reports the respective identifiers of the two or more packets received in their original order to the DPU in a prompt, wherein the prompt instructs the DPU to avoid identifying the original order of the two or more identified packets. In some embodiments, the DPU notifies the host of a merged payload comprising the payloads of the two or more identified packets. The DPU further identifies the original order of packets received in an order different from the original order.

[0021] In some embodiments, the DPU processes the header in its memory according to a transport layer protocol used for communication between the host and the remote source. The transport layer protocol may be one of a list including Transmission Control Protocol (TCP) and Datagram Congestion Control Protocol (DCCP).

[0022] In the disclosed technology, the DPU, which offloads transport layer processing from the host, uses a cue generated by the packet processor to perform effective packet reordering. The cue indicates which packets to apply processing to in their original order, thus allowing their reordering to be skipped. Therefore, the DPU can report such packets with low latency and reduce the computational resources and power consumed by the DPU. Furthermore, only a small amount of data is copied from the DPU to the host because the DPU only sends a notification specifying the original order to the host, not the ordered packets themselves.

[0023] Compared to traditional packet reordering methods, using the disclosed embodiments results in an increase in throughput for processing incoming traffic.

[0024] System Description

[0025] Figure 1 The block diagram of computer system 20 is schematically shown according to the embodiments described herein, which illustrates how efficient packet reordering can be facilitated.

[0026] Computer system 20 includes a host 24 that communicates with remote device 28 via a communication network 36 and peripheral device 32. Computer system 20 can be used in various applications, such as data centers, high-performance computing (HPC) systems, etc.

[0027] Generally, the disclosed embodiments are applicable to applications and / or protocols that benefit from receiving large blocks of data assembled from multiple payloads of received data packets, rather than receiving individual small payloads. Example related applications include storage applications and protocols such as the TCP-based NVM Express (NVMe) protocol, the Internet Small Computer System Interface (iSCSI) protocol, the Server Message Block (SMB) protocol, and the Network File System (NFS) protocol. The disclosed embodiments are also applicable to protocols using graphics processing unit (GPU) direct technology, and Remote Procedure Call (RPC) applications that require the transfer of large blocks of data.

[0028] The communication network 36 may include any suitable packet network operating according to any suitable communication protocol. In some embodiments, the communication network 36 includes an Ethernet network through which packets are communicated according to a suitable transport layer protocol. The transport layer protocol may include, for example, Transmission Control Protocol (TCP), Datagram Congestion Control Protocol (DCCP), or other suitable reliable protocols.

[0029] In this example, host 24 includes a central processing unit (CPU) 40 and host memory 44. CPU 40 can run any suitable program and / or application, virtualized or non-virtualized. Figure 1 In this example, CPU 40 runs a hypervisor 48 that controls the virtualization of one or more virtual machines (VMs) 52. Host 24 also includes a host interface 56 through which peripheral devices 32 access host memory 44. Host interface 56 may include any suitable interface, such as, for example, a peripheral component interconnect (PCIe) bus.

[0030] Peripheral device 32 mediates between host 24 and communication network 36. For this purpose, the peripheral device includes a host memory access interface 60 for accessing host memory 44, and a network adapter 64 for accessing communication network 36. Network adapter 64 may include, for example, a network interface controller (NIC) or any other suitable type of network adapter. NIC 64 includes a network interface 68 for interfacing with communication network 36, and a packet processor 72 for processing incoming and outgoing data packets.

[0031] Peripheral device 32 also includes a data processing unit (DPU) 74 coupled to DPU memory 76. The DPU may include any suitable type of processing unit, such as a processor with a Reduced Instruction Set Computing (RISC) architecture, such as an ARM processor.

[0032] In other tasks, DPU 74 runs transport layer processing 78, for example, implemented as a TCP stack, to apply to incoming (and outgoing) packets. Therefore, DPU offloads transport layer processing from host 24. Transport layer processing 78 typically involves reordering received packets, as described below.

[0033] Packet processor 72 receives packets 80 originating from remote device 28 via network interface 68, each packet including a header 82 and a payload 84. Packet 80 is sent to host 24, meaning the packet is sent by the remote device to a destination target on host 24, such as VM 52 or an application (not shown) running on the host, via the communication network. The packets are sent by the remote device in their original order, but may be received in a different order of arrival at the peripheral device. The packets arrive out of order due to, for example, being sent via multiple different paths on the communication network, which may have different delays. The packets may also be queued and delayed differently within switches in the communication network based on, for example, the priority of other packets traversing a switch.

[0034] In some embodiments, the packet processor 72 splits each received packet 80 into its header 82 and payload 84, and (i) sends the payload (84) to be stored in a data buffer 86 in the host memory 44, and (ii) sends the header (82) to be stored in a header buffer 88 in the DPU memory 76. In some embodiments, the packet processor sends descriptors 90 to the DPU, each descriptor indicating a corresponding location in the host memory 44 where the corresponding payload 84 is stored. Based on the headers of the received packets, the packet processor generates a hint 92 indicating the processing to be applied to the headers in the DPU memory to identify the original order.

[0035] In some embodiments, packet processor 72 sends a payload to be stored in a selected physical address space in host memory. The physical address space is associated with hypervisor 48 and / or one or more VMs 52. In embodiments, packet processor maintains address translation 94, each address translation being identified by a processor referred to herein as a memory key (MKey). Packet processor uses address translation 94 to translate between virtual addresses and corresponding physical addresses in host memory. In some embodiments, packet processor also uses address translation 94 to translate virtual addresses to corresponding physical addresses in DPU memory.

[0036] In some embodiments, DPU 74 receives cues 92 from packet processor 72 and uses these cues to perform effective packet reordering. Based on cues 92, DPU 74 processes headers 82 in DPU memory to effectively identify the original order of the received packets. DPU notifies host 24 of the identified original order, enabling host 24 to process payloads 84 in host memory according to the original order. For example, DPU sends a notification 95 to host 24 reporting the position of the payload in host memory 44 according to the original order.

[0037] In some embodiments, the DPU accesses the header in the DPU memory by executing a work queue element (WQE) 96 in the DPU memory. In some embodiments, the WQE points to a corresponding header buffer 88 in the DPU memory.

[0038] In some embodiments, the DPU includes a shared receive queue (SRQ) 98. The SRQ includes a descriptor (e.g., descriptor 90) pointing to a data buffer 86 in host memory. In some embodiments, upon receiving a data packet from a communication network, the packet processor sends an instruction to the DPU specifying (i) an index in the header buffer 88 used to store the header of the data packet, and (ii) the header size. In embodiments, the DPU uses the SRQ to share the header buffer 88 among multiple communication connections established by host 24.

[0039] The configuration of the communication computer system 20, host 24, peripheral device 32, and communication network 36 is an example configuration, chosen purely for clarity of concept. In alternative embodiments, any other suitable computer system, host, peripheral device, and communication network configuration may be used. For clarity, elements unnecessary for understanding the principles of the invention, such as various interfaces, addressing circuits, timing and sequencing circuits, and debugging circuits, are omitted from the figures.

[0040] Some components of host 24 (such as CPU 40 and host interface 56), and some components of peripheral device 32 (such as packet processor 72 and network interface 68 of NIC 64, host memory access interface 60, and DPU 74), may be implemented in hardware, such as in one or more application-specific integrated circuits (ASICs) or FPGAs. Alternatively or additionally, packet processor 72, network interface 68, host memory access interface 60, and DPU 74 may be implemented in software, or using a combination of hardware and software components. Host memory 44 of host 24 and DPU memory 76 of DPU 74 may include any suitable storage element, such as random access memory (RAM) or non-volatile (NVM) memory (such as flash memory devices).

[0041] In some embodiments, certain functions of the packet processor 72, CPU 40, and DPU 74 may be performed by a general-purpose processor, which is programmed in software to perform the functions described herein. For example, the software may be downloaded electronically to the relevant processor via a network, or may alternatively or additionally be provided and / or stored on a non-transitory tangible medium, such as magnetic, optical, or electronic memory.

[0042] Method for reordering valid data packets using prompts

[0043] Figure 2 This is a flowchart illustrating, according to the embodiments described herein, a method for reordering valid data packets using prompts.

[0044] This method can be derived from Figure 1 The peripheral devices 32 of the computer system 20 are executed by the components.

[0045] The method begins with packet receiving step 150, where packet processor 72 receives packets 80 sent in their original order by a remote source (e.g., remote device 28) via communication network 36. The sequence of received packets may include subsequences of packets not in their original order, as well as other subsequences of packets in their original order.

[0046] In packet splitting step 152, the packet processor splits each packet 80 into a header 82 and a payload 84. In storage step 154, the packet processor sends the payload 84 to be stored in a data buffer 86 in host memory 44, and sends the header 82 (without payload 84) to be stored in a header buffer 88 in DPU memory 76. In some embodiments, the packet processor generates a descriptor for the received packet and sends the descriptor to be stored in DPU memory in association with the packet header. In an embodiment, the packet descriptor indicates the location of the corresponding payload in host memory.

[0047] In prompt generation step 158, the packet processor processes the header 82 of the received packet 80 and, based on the header of the received packet, generates a prompt 92 indicating the processing of the header to be applied by the DPU to the DPU memory, for identifying the original order of at least some of the received packets. In an example embodiment, the prompt indicates that two or more packets were received in their original order and therefore do not need to be reordered by the DPU. In this case, the prompt instructs the DPU to avoid reordering these two or more packets.

[0048] In one embodiment, the host communicates with the remote device using a suitable transport layer protocol (e.g., TCP). In such an embodiment, the packet processor identifies two or more packets received in their original order by parsing the headers of the received packets according to the underlying transport layer protocol. This header parsing allows for the retrieval of sequence numbers carried in the headers of the received packets and allows for the detection of two or more consecutive packets among received packets with consecutive sequence numbers. In some embodiments, the packet processor reports the respective identifiers of the two or more packets received in their original order to the DPU in a prompt, wherein the prompt instructs the DPU to avoid recognizing the original order of the two or more identified packets.

[0049] In the prompt reception step 162, DPU 74 receives prompt 92 from the packet processor. In the non-prompt reordering step 166, DPU identifies the original order of some received packets, whose arrival order differs from the original order. For example, DPU identifies the original order of these packets based on their corresponding sequence numbers.

[0050] In prompt-based processing step 170, the DPU applies the prompt-indicated processing to the corresponding headers of two or more data packets (received in their original order) in the DPU memory. In some embodiments, based on the prompt, the DPU avoids recognizing the original order of the received data packets received in their original order, as indicated by prompt 92.

[0051] In notification step 174, the DPU notifies the original order of the packets reordered in step 166 and the packets that did not require reordering in step 170. After step 174, the method terminates. In some embodiments, the notification message from the DPU to the host includes an ordered list of payload locations in the host memory, consistent with the original order identified by the DPU.

[0052] In some embodiments, instead of notifying the host individually of each packet received in the original order, the DPU notifies the host of a merged payload that includes the payloads of the packets received in the original order.

[0053] exist Figure 2 In this method, by skipping (host-based) reordering of packets received in the original order, the delay in informing the host of the original order is reduced, and the overall ingress throughput is significantly improved.

[0054] Figure 3 This is a diagram schematically illustrating the sequence of data packets according to the embodiments described herein, some of which do not require reordering as indicated in the prompt.

[0055] Figure 3Sequence 200 depicts a sequence 200 of data packets 80 sent by a remote source in the original order, denoted as PCKT1...PCKT8. Sequence 204 depicts data packets of sequence 200 arriving at peripheral device 32 in an order different from the original order. As shown, the data packets arrive in the order PCKT2, PCKT1, PCKT3, PCKT4, PCKT5, PCKT7, PCKT6, PCKT8.

[0056] Sequence 212 depicts the header 82 that the packet processor has separated from the received packet 80. In this example, each subsequence (PCKT1, PCKT2) and (PCKT6, PCKT7, PCKT8) is received in a different order than the original order, thus requiring the DPU to identify the original order of the packets within these subsequences. On the other hand, packets PCKT3, PCKT4, and PCKT5 are received in the original order and are therefore reported to the DPU in prompt 92, as described above. Based on this prompt, the DPU can effectively skip the reordering of packets PCKT3, PCKT4, and PCKT5 and notify the host accordingly. Skipping the reordering of PCKT3, PCKT4, and PCKT5 reduces the latency in notifying the host of the original order and reduces the computational resources and power consumption required by the DPU.

[0057] The embodiments described above are given as examples, and other suitable embodiments may also be used.

[0058] Although the embodiments described herein primarily address efficient reordering of received packets in TCP, the methods and systems described herein can also be used in a variety of applications, such as network storage, applications using GPU direct technology, and remote procedure calls (RPC).

[0059] It is understood that the embodiments described above are cited by way of example, and the following claims are not limited to what has been specifically shown and described above. Rather, the scope includes combinations and sub-combinations of the various features described above, as well as variations and modifications thereof that would occur to a person skilled in the art upon reading the foregoing description and which are not disclosed in the prior art. Documents incorporated herein by reference shall be considered part of the application, unless such incorporated documents define any terms in a manner that conflicts with the express or implied definitions in this specification, in which case only the definitions in this specification shall be considered.

Claims

1. A peripheral device, comprising: A network interface is used to transmit data packets over a communication network. A host interface for communicating with a host coupled to the peripheral device; Data processing unit (DPU), the DPU being coupled to DPU memory; Packet processor, used for: Multiple data packets are received from a remote source via the communication network, wherein the data packets are sent from the remote source in their original order and received at the peripheral device in an arrival order different from the original order. The received data packets are split into corresponding headers and payloads; The payload is sent via the host interface to be stored in the host memory of the host in the order of arrival, and the header without the payload is sent to be stored in the DPU memory in the order of arrival; as well as Based on the header of the received data packets, a prompt is generated indicating that two or more data packets were received in the original order; The DPU is used for: Receive the prompt from the packet processor; By detecting some of the headers, the original order of the data packets received in an arrival order different from the original order is identified, but the original order of the two or more data packets reported in the prompt is avoided. as well as According to the original order, the host is notified of the location of the payload of the received data packets in the host memory.

2. The peripheral device of claim 1, wherein the DPU is configured to notify the host of a merged payload, the merged payload including the payloads of the two or more data packets reported in the notification.

3. The peripheral device of claim 1, wherein the packet processor is configured to send the payload to be stored in the host memory by translating a virtual address associated with an address space allocated to a hypervisor or virtual machine (VM) running on the host into a corresponding physical address in the host memory.

4. The peripheral device of claim 1, wherein the DPU is configured to access the header in one or more header buffers in the DPU memory by executing a work queue element (WQE) in the DPU memory.

5. The peripheral device of claim 1, wherein the DPU is configured to process the header in the DPU memory according to a transport layer protocol for communication between the host and the remote source.

6. The peripheral device according to claim 5, wherein the transport layer protocol belongs to a list including Transmission Control Protocol TCP and Datagram Congestion Control Protocol DCCP.

7. A method for communication, comprising: In peripheral devices that mediate communication between a host and remote devices via a communication network, A packet processor receives multiple data packets from a remote source via the communication network, wherein the data packets are sent from the remote source in their original order and are received at the peripheral device in an arrival order different from the original order. The received data packets are split into corresponding headers and payloads; The payload is sent to be stored in the host memory of the host in the order of arrival, and the header without the payload is sent to be stored in the DPU memory of the data processing unit (DPU) in the order of arrival. Based on the header of the received data packet, the data packet processor generates a prompt indicating that two or more data packets were received in the original order; The DPU receives the prompt from the packet processor; By detecting some of the headers, the original order of the data packets received in an arrival order different from the original order is identified, but the original order of the two or more data packets reported in the prompt is avoided. as well as According to the original order, the host is notified of the location of the payload of the received data packets in the host memory.

8. The method of claim 7, further comprising the DPU notifying the host of a merged payload, the merged payload including the payloads of the two or more packets reported in the notification.

9. The method of claim 7, wherein sending the payload to be stored in the host memory comprises: The virtual address associated with the address space allocated to the hypervisor or virtual machine (VM) running on the host is translated into the corresponding physical address in the host memory.

10. The method of claim 7, further comprising accessing the header in one or more header buffers of the DPU memory by performing a work queue element (WQE) in the DPU memory.

11. The method of claim 7, further comprising processing the header in the DPU memory according to a transport layer protocol for communication between the host and the remote source.

12. The method of claim 11, wherein the transport layer protocol belongs to a list including Transmission Control Protocol TCP and Datagram Congestion Control Protocol DCCP.

13. A method for reordering data packets, comprising: Receive data packets destined for the host in an arrival order that differs from the original order in which the packets were sent from the remote source; The payload of the data packet is stored in the host memory of the host in the order of arrival, and the header of the data packet without the payload is stored in the DPU memory of the data processing unit (DPU) in the order of arrival. The original order of the data packets whose arrival order differs from the original order is identified by processing the headers in the DPU memory, but avoiding processing the headers in the DPU memory of two or more data packets received in the original order as indicated by a prompt generated based on the headers of the received data packets. as well as According to the original order, the host is notified of the location of the payload of the received data packets in the host memory.

Images