Reception buffer based on strided messages

Abstract

The invention relates to a receive buffer based on strided messages. Systems and methods for processing data packets are described herein. An example network adapter may include a network interface operably coupled to a communication network and packet processing circuitry operably coupled to the network interface. The packet processing circuitry is configured to: receive, via the network interface, a plurality of data packets associated with a message; for each data packet, determining at least one corresponding reserved stride in the stride buffer; storing each data packet in at least one corresponding reserved stride; processing the stride buffer after storing the plurality of data packets in the corresponding plurality of reserved strides; and generating a completion notification indicating that the plurality of data packets in the stride buffer have been processed.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to data communications, and in particular to apparatus for interfacing between computing devices and packet data networks. Background Art

[0002] InfiniBand (IB) is a high-speed interconnect technology commonly used in high-performance computing (HPC) and data center environments. Computing devices (host processors and peripherals) are connected to the IB fabric through network adapters (also known as host channel adapters). To send and receive messages over the network, the network adapter uses work queue elements (WQEs) posted by the driver in the appropriate work queue (e.g., send queue or receive queue). In IB, the order in which data packets arrive affects the processing efficiency of receive operations. Handling out-of-order data packets consumes a lot of resources and increases processing latency. To address these challenges, various techniques have been proposed to optimize receive queues and work queue elements (WQEs). One such technique involves using a strided WQE buffer, where each stride in the strided WQE buffer stores data packets. When processing a strided WQE, each stride results in the generation of a unique completion queue entry (CQE). This approach can help improve the efficiency of managing out-of-order data packets, but still requires more complex processes to manage message assembly.

[0003] Applicants have identified a number of deficiencies and problems associated with processing data packets in an HPC data center environment. Many of these identified problems have been addressed by developing solutions included in embodiments of the present disclosure, many examples of which are described in detail herein. Summary of the invention

[0004] Systems, methods, and computer program products for providing a strided message based receive buffer are described.

[0005] In one aspect, a network adapter is provided. The network adapter includes: a network interface operably coupled to a communication network; a packet processing circuit operably coupled to the network interface, wherein the packet processing circuit is configured to: receive a plurality of data packets associated with a message through the network interface; for each data packet, determine at least one corresponding reserved stride in a stride buffer; store each data packet in the at least one corresponding reserved stride; after storing the plurality of data packets in the corresponding plurality of reserved strides, process the stride buffer; and generate a completion notification indicating that the plurality of data packets in the stride buffer have been processed.

[0006] In some embodiments, the packet processing circuit is configured to: determine a message sequence number (MSN) for each data packet, wherein the MSN includes information identifying the corresponding reserved stride in the stride buffer; and store the message in the stride buffer based on the MSN.

[0007] In some embodiments, the packet processing circuit is further configured to: determine an offset associated with each data packet, wherein the offset identifies the position of each data packet in the message composition order; and store each data packet in the at least one corresponding reserved stride based at least on the MSN and the offset.

[0008] In some embodiments, processing the message further comprises writing the plurality of data packets from the stride buffer to one or more memory indexes associated with a memory.

[0009] In some embodiments, the packet processing circuitry is configured to determine the plurality of reservation strides based at least on a maximum message size supported by one or more connections on the communications network.

[0010] In some embodiments, the stride buffer is a work queue element (WQE).

[0011] In another aspect, a method for processing data packets is provided. The method includes: receiving a plurality of data packets associated with a message via a network interface; for each data packet, determining at least one corresponding reserved stride in a stride buffer; storing each data packet in the at least one corresponding reserved stride; after storing the plurality of data packets in the corresponding plurality of reserved strides, processing the stride buffer; and generating a completion notification indicating that the plurality of data packets in the stride buffer have been processed.

[0012] In another aspect, a network adapter is provided. The network adapter includes: a network interface operably coupled to a communication network; a packet processing circuit operably coupled to the network interface, wherein the packet processing circuit is configured to: receive a subset of a plurality of data packets via the network interface, wherein the subset includes in-order data packets; aggregate the subset of the plurality of data packets to store in a first stride buffer; subsequently receive out-of-order data packets; cause the first stride buffer to be processed in response to receiving the out-of-order data packets; and generate a first completion notification indicating that the subset of the plurality of data packets in the first stride buffer has been processed.

[0013] In some embodiments, out-of-order data packets are associated with the plurality of data packets.

[0014] In some embodiments, the packet processing circuit is configured to: determine whether the multiple data packets are received within a predetermined time period; determine that only a first subset of the multiple data packets are received within the predetermined time period; aggregate the first subset of the multiple data packets to be stored in a second stride buffer; process the second stride buffer; and generate a second completion notification, the second completion notification indicating that the first subset of the multiple data packets in the second stride buffer has been processed.

[0015] In another aspect, a method for processing data packets is provided. The method includes: receiving a subset of a plurality of data packets via a network interface, wherein the subset includes in-order data packets; aggregating the subset of the plurality of data packets for storage in a first stride buffer; subsequently receiving out-of-order data packets; causing the first stride buffer to be processed in response to receiving the out-of-order data packets; and generating a first completion notification, the first completion notification indicating that the subset of the plurality of data packets in the first stride buffer has been processed.

[0016] The above content of the invention is provided only to summarize some example embodiments to provide a basic understanding of certain aspects of the present disclosure. Therefore, it should be understood that the above embodiments are only examples and should not be interpreted as narrowing the scope or spirit of the present disclosure in any way. It should be understood that the scope of the present disclosure also covers many potential embodiments in addition to the embodiments summarized here, some of which will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above generally describes the embodiments of the present disclosure, and now reference will be made to the drawings. The components shown in the drawings may or may not be present in certain embodiments described herein. Some embodiments may include fewer (or more) components than those shown in the drawings.

[0018] Figure 1An example data communication system according to an embodiment of the present invention is shown;

[0019] Figure 2 shows a strided structure of a work queue element (WQE) according to an embodiment of the present invention;

[0020] Figure 3 A method for processing data packets using a strided buffer according to an embodiment of the present invention is shown; and

[0021] Figure 4 A method for processing data packets using data packet aggregation according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] Overview

[0023] Computing devices (host processors and peripherals) connect to the IB fabric through network adapters, which are called channel adapters in IB terminology. The host processor (or host) uses a host channel adapter (HCA), while peripherals connect to the IB fabric using a target channel adapter (TCA).

[0024] A client process (e.g., a software application process) running on a host processor communicates with the transport layer of the IB structure by manipulating a transport service instance (referred to as a "queue pair" (QP)), which consists of a send work queue and a receive work queue (often referred to simply as a send queue and a receive queue). To send and receive messages over the network, the network adapter uses a work queue element (WQE) posted by a driver (e.g., a user application) in an appropriate work queue (e.g., a send queue or a receive queue). For example, when the network adapter is to receive a message, the driver posts a receive descriptor by creating a WQE, and then adds the WQE to a receive queue (e.g., a shared receive queue (SRQ)) to prepare for the network adapter to receive the message. In the receive queue, each WQE may include processing parameters, such as a memory descriptor, a pointer to the location in memory where each message is to be dispersed, and metadata that can be used to assemble a data packet containing the message. For each processed WQE, a completion queue element (CQE) may be generated, indicating that the message has been processed according to the processing parameters. Once the CQE is posted, the WQE is considered "consumed" and the driver should repost another WQE to the receive queue to receive another message. In some cases, a single WQE (e.g., a receive WQE) may be used multiple times to receive / distribute multiple data packets or messages before being considered "consumed".

[0025] Typically, two types of receive queues are used in IBs: (i) circular and (ii) linked lists. Circular receive queues are generally preferred over linked lists because of lower memory requirements, better load balancing, and faster message extraction. However, circular receive queues require data packets to arrive in order. If data packets arrive out of order, WQEs can still be created to receive the data packets. However, if there is no complete message, the WQE may not be successfully processed to issue a CQE because the message may not be completed. As more and more data packets arrive out of order, there may be a risk that there are not enough WQEs left for in-order data packets, resulting in in-order data packets being discarded, which may lead to deadlock. Although linked list receive queues can be used to process out-of-order data packets to resolve any potential deadlock (e.g., by using a predefined threshold for in-order data packets), processing WQEs in linked list receive queues consumes a lot of memory, and the generation and consumption of these WQEs increases the latency of data packet processing, as well as consumes resources of the host computing device. To address these challenges, a stride WQE buffer is proposed. In the current implementation, each stride in the stride WQE buffer is configured to store data packets. When dealing with strided WQEs, each stride results in a unique CQE being generated. Since data packets arrive out of order, there is a risk that data packets associated with a particular message may be spread across multiple WQEs, requiring complex processes to manage message assembly.

[0026] Thus, the present disclosure strikes a balance between the need to manage potential deadlocks and improving overall message retrieval rates.

[0027] For IB implementation, embodiments of the present invention allocate or reserve contiguous memory blocks that are organized into a series of "strides" or steps in a stride buffer (e.g., WQE) to store data packets associated with a message. Each data packet may include a message sequence number (MSN) that identifies a specific reserved stride in the stride buffer for storage. The data packet may also include an offset that identifies the position of the data packet in the message composition sequence. Based on the MSN and the offset, each data packet may be stored in a corresponding reserved stride in the stride buffer. Once all data packets associated with the message are received and all previous messages have been processed, the message can be transferred from the stride buffer to a memory index in the memory. In response, a completion notification (e.g., a completion report or a completion queue element (CQE)) is generated, indicating that the message has been received and stored in the memory index. By performing stride allocation in advance and storing the data packets in the corresponding reserved stride, the stride buffer may only need to issue one completion notification per message.

[0028] For Large Receive Offload (LRO) implementation, embodiments of the present invention aggregate data packets associated with a particular message as they are received. In some implementations, when data packets arrive in order, embodiments of the present invention aggregate the data packets and store them in a stride buffer until the entire message is received. The aggregated data packets may then be transferred from the stride buffer to a memory index. In response, a completion notification or report is generated indicating that the message has been received and stored in the memory index. In some other implementations, when the order of arrival of data packets is affected, embodiments of the present invention aggregate only the data packets that have been received in order in the stride buffer for transfer to the memory index. In response, a completion notification is generated indicating that only a portion of the message has been received and that the portion of the message has been stored in the memory index. In some cases, embodiments of the present invention may establish a specific time period for all data packets of a message to be received to be aggregated in a stride buffer. If all data packets of the message do not arrive within the time period, only the data packets that arrive within the time period will be aggregated in the stride buffer and subsequently transferred to the memory index. In response, a completion notification is generated indicating that only a portion of the message has been received within the time period and that the portion of the message has been transferred to the memory index. Any remaining data packets (associated with the same message) arriving outside the time period are aggregated independently in different stride buffers.

[0029] Although the above terminology and some embodiments described below are based on features of the IB architecture and use vocabulary taken from the IB specification, similar mechanisms also exist in networks and I / O devices that operate according to other protocols (such as Ethernet and Fibre Channel). For convenience and clarity, IB terminology and features are used herein by way of example, not limitation.

[0030] Embodiments of the present invention will now be described more fully with reference to the accompanying drawings, some but not all of which are shown in the accompanying drawings. In fact, the present disclosure may be embodied in a variety of different forms and should not be construed as being limited to the embodiments described herein; on the contrary, these embodiments are provided so that the present disclosure meets applicable legal requirements. Where possible, any term expressed in the singular herein is also meant to include the plural form, and vice versa, unless expressly stated otherwise. In addition, as used herein, the terms "one" and / or "one" shall mean "one or more", even if the phrase "one or more" is also used herein. In addition, when it is said herein that something is "based on" something else, it may also be based on one or more other things. In other words, unless expressly stated otherwise, as used herein, "based on" means "based at least in part on" or "based at least in part on". The same number always represents the same element.

[0031] As used herein, "operably coupled" may refer to a functional connection between network components to facilitate the transmission of data packets between connected components, ensuring that they are accurately and timely transmitted, received and processed. The connection may be achieved by direct or indirect means, such as physical wiring, wireless communication, intermediate equipment, software-based protocols, etc., and may be combined with mechanisms for handling data packet transmission, including routing, error detection and flow control. It should also be understood that, as used herein, "operably coupled" means that components may be formed integrally with each other, or may be formed separately and coupled together. In addition, "operably coupled" means that components may be directly connected to each other, or connected to each other, wherein one or more components are located between components that are operably coupled together. In addition, "operably coupled" may mean that components may be separated from each other, or they are permanently coupled together. In addition, "operably coupled" may mean that components may be electronically connected and / or fluidically connected to each other.

[0032] As used herein, "data," "content," "information," and similar terms are used interchangeably to refer to data that can be transmitted, received, and / or stored in accordance with embodiments of the present disclosure. Therefore, the use of any such terms should not be viewed as limiting the spirit and scope of embodiments of the present disclosure. In addition, when a computing device is described herein as receiving data from another computing device, it should be understood that the data can be received directly from the other computing device or indirectly through one or more intermediate computing devices, such as one or more servers, repeaters, routers, network access points, base stations, hosts, etc., sometimes referred to herein as a "network." Similarly, when a computing device is described herein as sending data to another computing device, it should be understood that the data can be sent directly to the other computing device or indirectly through one or more intermediate computing devices, such as one or more servers, repeaters, routers, network access points, base stations, hosts, etc.

[0033] As used herein, "illustrative," "exemplary," and "example" are not provided to convey any qualitative assessment, but only to convey an illustration of an example. Thus, the use of any such terms should not be taken to limit the spirit and scope of embodiments of the present invention.

[0034] As used herein, "in one embodiment," "according to one embodiment," "in some embodiments," etc. generally mean that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of the present disclosure and may be included in more than one embodiment of the present disclosure (importantly, such phrases do not necessarily refer to the same embodiment).

[0035] Example Data Communication System

[0036] Figure 1 FIG. 1 shows a data communication system 100 according to an embodiment of the present invention. Figure 1 As shown, data communication system 100 may include a host processor in the form of a central processing unit (CPU) 102 and a host memory 104 (also referred to as system memory) connected to a suitable bus 110, such as a PCI bus as is known in the art. (PCIe) bus. CPU 102 may include multiple processing cores (not shown) and may perform the functions described herein under the control of program instructions in software. The software may be downloaded to system 100 in electronic form, such as over a network. Additionally or alternatively, the software may be stored in a tangible, non-transitory computer-readable storage medium, such as an optical, magnetic, or electronic memory.

[0037] A network adapter 108, such as an IB host channel adapter (HCA), can connect the data communication system 100 to the network 106. The network adapter 108 may include a network interface 114 coupled to the network 106 and a host interface 112 coupled to the CPU 102 and the memory 104 via a bus 110. A packet processing circuit 116 coupled between the network interface 114 and the host interface 112 can generate outgoing packets for transmission over the network 106 (as described below), and can also process incoming packets received from the network. The host interface 112, the network interface 114, and the packet processing circuit 116 may typically include dedicated hardware logic, the details of which will be clear to those skilled in the art after reading this specification. Alternatively or in addition, at least some of the functions of the packet processing circuit 116 may be implemented in software on a suitable programmable processor.

[0038] In some embodiments, a client process running on CPU 102 may communicate with a peer (e.g., a source host) on network 106 via a queue pair (QP) via network adapter 108. Each QP may include a send work queue and a receive work queue. To send a message over network 106 using network adapter 108, a client process (e.g., a process generated by application software) running on CPU 102 may submit a work request for execution by network adapter 108. The work request may specify a message to be transmitted via a pointer to a buffer location in data area 128 of memory 104. A driver running on CPU 102 may process the work request to generate a WQE 122 and place the WQE 122 in a send queue (SQ) 120 in memory 104, as shown in FIG. Figure 1As shown. Each WQE 122 may include metadata 124, such as addressing and protocol information, for the network adapter 108 to use in constructing a packet header and transmitting the corresponding packet. In addition, each WQE 122 may include a descriptor 126 indicating the actual message to be included in the packet payload. The descriptor 126 may include a pointer to an appropriate buffer in a data area 128 of the memory 104. Alternatively, for an inline WQE in which the actual message is stored within the WQE 122 itself, the driver may copy the message from the data area 128 directly into the descriptor 126 so that the WQE 122 may include the actual message instead of a pointer to the message. As described above and further described below, the driver may dynamically and automatically decide under software control when to generate an inline WQE 122 and when to use a pointer-based WQE. To this end, the driver may set a flag (typically a single bit) in the WQE 122 to indicate whether the data following the bit includes a pointer or actual inline data.

[0039] In some embodiments, the network adapter 108 may read the WQE 122 from the appropriate queue via a direct memory access (DMA) transaction on the bus 110, which may be performed by the host interface 112. The packet processing circuit 116 may parse the WQE 122 and specifically determine whether each WQE 122 is a pointer-based type or an inline type. In the former case, the host interface 112 may perform a further bus transaction to read the message from the location in the memory 104 indicated by the pointer. In the latter case, this additional bus transaction may not be required when the WQE 122 is an inline type. The packet processing circuit 116 may then construct one or more packets containing the message and transmit the packets to the network 106 via the network interface 114.

[0040] Upon completion of execution of the WQE 122, the packet processing circuitry 116 may write a completion report (e.g., a completion queue element (CQE) 132) to a designated completion queue (CQ) 130 in the memory 104. Depending on the reliability requirements of the transport protocol, the packet processing circuitry 116 may write the CQE 132 immediately after sending the message required by the WQE 122, or may wait until an acknowledgement is received from the message destination. The client process running on the CPU 102 may read the CQE 132 from its assigned CQ 130 and may then determine that its work request has been satisfied.

[0041] In order to receive messages over the network 106 using the network adapter 108, a driver running on the CPU 102 may prepare to receive incoming messages by issuing receive work requests. These work requests are processed to generate WQEs 122, which are then placed in appropriate receive queues (not shown). Each WQE 122 may include metadata (e.g., addressing and protocol information) for use by the network adapter 108 to interpret the packet header and reassemble the corresponding one or more packets. In addition, each WQE 122 may include a descriptor pointing to a location in memory (e.g., a memory index) where the incoming message is to be stored. When the network adapter 108 detects an incoming message, it may read the WQE 122 from the receive queue using a DMA transaction on the bus 110, which may be performed by the host interface 112. The packet processing circuit 116 may parse the received packet and use the information in the WQE 122 to determine an appropriate location in the data area 128 of the memory 104 for storing the incoming message. If WQE 122 contains a pointer, host interface 112 performs a bus transaction to write the incoming message to the location indicated by the pointer in memory 104. Packet processing circuitry 116 may also generate a CQE 132 indicating that the message has been received and processed according to the parameters specified in WQE 122.

[0042] Once CQE 132 is posted, WQE 122 is considered "consumed" and the driver running on CPU 102 is expected to repost another WQE 122 to the receive queue in preparation for receiving another message. This ensures that network adapter 108 is ready to receive new incoming messages over network 106 and process them.

[0043] Example stride buffer structure for WQE

[0044] Figure 2 FIG. 2 shows a stride buffer structure of a work queue element (WQE) 122 according to an embodiment of the present invention. Figure 2As shown, the WQE can be a stride buffer containing multiple strides 134. The stride 134 can refer to a common fixed-size memory segment whose length can be as small as a single byte or multiple bytes. For example, the size of the stride 134 can be advantageously set to correspond to the expected packet size on a given transport service instance (e.g., equal to or slightly larger than the expected packet size). Therefore, the size of the stride can be configurable and can be set to different values ​​based on the connection. The total number of strides 134 can be determined based on at least the maximum message size that the transport service instance can support. Therefore, in one embodiment, a message with a small payload can consume a single stride, while a message with a large payload can consume multiple strides. In an example embodiment, the number of reserved strides for each multi-packet message can be the same. If the multi-packet message does not use all the reserved strides, any remaining reserved strides will remain unused. Multiple strides 134 can be arranged in the memory address space in the form of continuous or discontinuous memory segments, depending on the application. For example, to accommodate a sequence of packets having headers and payloads of fixed respective sizes, each stride 134 may include a first portion mapped to one memory region to receive the header and a second portion mapped to another region to receive the payload.

[0045] In an example embodiment, the network adapter 108 may receive a message in the form of a sequence of data packets. As described herein, the data packets may be received out of order. However, each data packet may include a field, such as a packet offset, which may indicate the position of the data packet in the message. In addition, each data packet may include a message sequence number (MSN), which may identify the reserved stride of the message in the stride buffer (e.g., WQE 122). According to some embodiments, when the network adapter 108 detects an incoming message MSG1 136 or MSG2 138, it reads the WQE 122 from the receive queue using a direct memory access (DMA) transaction. The packet processing circuit 116 may parse the message into data packets and use the MSN to identify a specific reserved stride in the WQE 122 for storing the message. In addition, the packet processing circuit 116 may determine an offset associated with each data packet to identify the position of each data packet in the message composition sequence. Based on the MSN and the offset, the packet processing circuit 116 may store each data packet in a corresponding reserved stride of an appropriate number within a continuous allocation. For example, Figure 2As shown, the components of MSG1 136, MSG1_FIRST (11), and MSG1_LAST (12) may be stored in the corresponding reserved stride 134A, while the components of MSG2 138, MSG2_FIRST (9), and MSG2_LAST (10) may be stored in the corresponding reserved stride 134B. Here, each component may include one or more data packets associated with the message. When the reserved allocation of stride 134 is full, or when all data packets of the message have been written to its corresponding stride, the packet processing circuit 116 may process the WQE 122 by writing the data packets from the stride buffer to one or more memory indexes (e.g., data areas) in the memory. In response, the packet processing circuit 116 may generate a CQE indicating that the message has been received and processed according to the parameters specified in the WQE 122. Because the CQE is generated for the entire message rather than for each individual packet, the computing resources required by the CPU to process the CQE are greatly reduced.

[0046] Example method for processing data packets using strided buffers

[0047] Figure 3 A method 200 for processing data packets using a stride buffer structure according to an embodiment of the present invention is shown. The method 200 may be associated with an IB implementation of the present invention. The method may begin at step 202, where the network adapter 108 may receive a message including a plurality of data packets via the network interface 114. The method may then proceed to step 204, where the packet processing circuit 116 of the network adapter 108 may determine at least one corresponding reserved stride 134 in the stride buffer for each data packet of the message. In some embodiments, the stride buffer may be a work queue element (WQE) 122. In order to determine the corresponding reserved stride for each data packet, the packet processing circuit 116 may first determine a message sequence number (MSN) for each data packet. The MSN may include information identifying the reserved stride 134 in the WQE 122. For example, for a particular data packet, the MSN may identify a particular index of the reserved stride 134 within the WQE 122. The packet processing circuit 116 may also determine an offset associated with each data packet, the offset identifying the position of each data packet in the message composition order. The packet processing circuitry 116 may then store the message in the WQE 122 based on the MSN and the offset, as described with respect to Figure 2 In some embodiments, the packet processing circuit 116 may also determine the total number of strides 134 to reserve based on the maximum message size supported by one or more connections on the communication network 106, such as Figure 2 Described in more detail.

[0048] The method may then proceed to step 206, where the packet processing circuitry 116 may store each data packet in at least one corresponding reserved stride 134 of the WQE 122. The method may then proceed to step 208, where, after storing (or in response to) the data packets, the packet processing circuitry 116 may process the WQE 122. Specifically, the WQE may be processed in response to determining that all data packets associated with the incoming message have been received. In some embodiments, processing the data packets may also include writing the data packets from the WQE 122 to a memory index associated with the memory 104.

[0049] The method may then end at step 210, where the packet processing circuitry 116 may generate a completion notification indicating that the plurality of data packets in the WQE 122 have been processed. In some embodiments, the completion notification may be a completion queue element (CQE) 132, and the packet processing circuitry 116 may write the CQE 132 to a designated completion queue 130 in the memory 104. The CQE 132 may indicate that a complete message has been received and stored in a memory index.

[0050] Example method for processing data packets using data packet aggregation

[0051] Figure 4 A method 300 for processing data packets using data packet aggregation according to an embodiment of the present invention is shown. The method may be associated with a large receive offload (LRO) implementation of the present invention. The method may begin at step 302, where the network adapter 108 may receive a subset of an incoming plurality of data packets via the network interface 114. In some embodiments, the packet processing circuitry 116 of the network adapter 108 may determine that the incoming plurality of data packets are in order, i.e., are all packets of the same incoming message and arrive in the same order as they were originally sent. If the data packets are in order, the method may proceed to step 304, where the packet processing circuitry 116 may aggregate the subset of the ordered data packets for storage in a first stride buffer or WQE 122. In some embodiments, in response to determining that all data packets associated with the incoming message have been received, the packet processing circuitry 116 may process a first WQE 122 and write the data packets from the first WQE 122 to a memory index associated with the memory 104.

[0052] Additionally or alternatively, before processing the first WQE 122, the method may continue to step 306, where the network adapter 108 may receive an out-of-order data packet via the network interface 114. The out-of-order data packet may be associated with a second incoming message and / or a second plurality of data packets. In some embodiments, the method may continue to step 308, where the packet processing circuitry 116 may cause the first WQE 122 to be processed in response to receiving the out-of-order data packet. The method may then end at step 310, where the packet processing circuitry 116 may generate a first completion notification indicating that a subset of the in-order data packets in the first WQE 122 have been processed. In some embodiments, the first completion notification may be a completion queue element (CQE) 132. The method may then repeat step 304 using the second plurality of data packets, where the second plurality of data packets are aggregated in the second WQE 122 until a data packet associated with a third message is received, thereby triggering the processing of the second WQE 122 and the generation of the second CQE 132.

[0053] In some embodiments, step 306 may also include establishing, by the packet processing circuitry 116, a specific time period (e.g., setting a timer) during which the first WQE 122 is not processed. In the described embodiment, if additional data packets associated with the first message are received within the predetermined time period, the packet processing circuitry 116 may write the data packets to the remaining reserved stride of the first WQE 122. However, if the predetermined time period has elapsed before the additional data packets are received, the packet processing circuitry 116 may process the first WQE as is. If additional data packets associated with the first message are received after the predetermined time period, the packet processing circuitry 116 may aggregate the packets into different WQEs 122.

[0054] Embodiments of the present disclosure will be described below with reference to block diagrams and flowchart illustrations. Therefore, it should be understood that each box in the block diagram and flowchart illustrations can be implemented in the form of a computer program product; a completely hardware embodiment; a completely firmware embodiment; a combination of hardware, computer program products and / or firmware; and / or executing instructions, operations, steps and similar terms (e.g., executable instructions, instructions for execution, program codes, etc.) on a computer-readable storage medium in the form of an apparatus, system, computing device, computing entity, etc. for execution. For example, the retrieval, loading and execution of the code can be performed sequentially so that one instruction is retrieved, loaded and executed at a time. In some exemplary embodiments, the retrieval, loading and / or execution can be performed in parallel so that multiple instructions are retrieved, loaded and / or executed together. Therefore, such an embodiment can produce a machine with a specific configuration for executing the steps or operations specified in the block diagram and flowchart illustrations. Therefore, the block diagram and flowchart illustrations support a combination of various embodiments for executing specified instructions, operations or steps.

[0055] Many modifications and other embodiments of the invention described herein will occur to those skilled in the art to which the invention relates, with the benefit of the teachings presented in the foregoing description and the associated drawings. Although the drawings show only certain components of the methods and systems described herein, it is understood that various other components may also be part of the disclosure herein. In addition, the above methods may include fewer steps in some cases and additional steps in other cases. In some cases, modifications to the steps of the above methods may be performed in any order and in any combination.

[0056] Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0057] Incorporated by Reference

[0058] To supplement the present disclosure, this application also incorporates by reference the following commonly assigned patent applications in their entirety:

[0059]

Claims

1. A network adapter, comprising: a network interface operatively coupled to a communications network; a packet processing circuit operatively coupled to the network interface, wherein the packet processing circuit is configured to: receiving, via the network interface, a plurality of data packets associated with a message; For each data packet, determining at least one corresponding reserved stride in the stride buffer; storing each data packet in the at least one corresponding reserved stride; processing the stride buffer after storing the plurality of data packets in the corresponding plurality of reserved strides; as well as A completion notification is generated, the completion notification indicating that the plurality of data packets in the stride buffer have been processed.

2. The network adapter of claim 1 , wherein the packet processing circuit is configured to: determining a message sequence number MSN for each data packet, wherein the MSN comprises information identifying the corresponding reserved stride in the stride buffer; and The message is stored in the stride buffer based on the MSN.

3. The network adapter of claim 2, wherein the packet processing circuit is further configured to: determining an offset associated with each data packet, wherein the offset identifies the position of each data packet in the message composition sequence; and Each data packet is stored in the at least one corresponding reserved stride based at least on the MSN and the offset.

4. The network adapter of claim 1 , wherein processing the message further comprises: The plurality of data packets from the stride buffer are written to one or more memory indexes associated with a memory.

5. The network adapter of claim 1, wherein the packet processing circuit is configured to determine the plurality of reservation strides based at least on a maximum message size supported by one or more connections on the communications network. The network adapter of claim 1 , wherein the stride buffer is a work queue element (WQE).

7. A method for processing a data packet, the method comprising: receiving, via a network interface, a plurality of data packets associated with a message; For each data packet, determining at least one corresponding reserved stride in the stride buffer; storing each data packet in the at least one corresponding reserved stride; processing the stride buffer after storing the plurality of data packets in the corresponding plurality of reserved strides; as well as A completion notification is generated, the completion notification indicating that the plurality of data packets in the stride buffer have been processed.

8. The method according to claim 7, wherein the method further comprises: determining a message sequence number MSN for each data packet, wherein the MSN includes information identifying a corresponding reserved stride in the stride buffer; and The message is stored in the stride buffer based on the MSN.

9. The method according to claim 8, wherein the method further comprises: determining an offset associated with each data packet, wherein the offset identifies the position of each data packet in the composition order of the message; and Each data packet is stored in the at least one corresponding reserved stride based at least on the MSN and the offset.

10. The method according to claim 7, wherein the method further comprises: The plurality of data packets from the stride buffer are written to one or more memory indexes associated with a memory.

11. The method according to claim 7, wherein the method further comprises: The plurality of reservation steps are determined based at least on a maximum message size supported by one or more connections on the communication network.

12. The method of claim 7, wherein the stride buffer is a work queue element (WQE).

13. A network adapter, comprising: a network interface operatively coupled to a communications network; a packet processing circuit operatively coupled to the network interface, wherein the packet processing circuit is configured to: receiving, via the network interface, a subset of a plurality of data packets, wherein the subset comprises ordered data packets; aggregating the subset of the plurality of data packets for storage in a first stride buffer; subsequently receiving out-of-order data packets; causing the first stride buffer to be processed in response to receiving the out-of-order data packet; as well as A first completion notification is generated, the first completion notification indicating that the subset of the plurality of data packets in the first stride buffer has been processed.

14. The network adapter of claim 13, wherein the out-of-order data packet is associated with the plurality of data packets.

15. The network adapter of claim 13, wherein the packet processing circuit is configured to: determining whether the plurality of data packets are received within a predetermined time period; determining that only a first subset of the plurality of data packets are received within the predetermined time period; aggregating the first subset of the plurality of data packets for storage in a second stride buffer; processing the second stride buffer; as well as A second completion notification is generated, the second completion notification indicating that the first subset of the plurality of data packets in the second stride buffer has been processed.

16. A method for processing a data packet, the method comprising: receiving, via a network interface, a subset of a plurality of data packets, wherein the subset comprises ordered data packets; aggregating the subset of the plurality of data packets for storage in a first stride buffer; subsequently receiving out-of-order data packets; causing the first stride buffer to be processed in response to receiving the out-of-order data packet; and A first completion notification is generated, the first completion notification indicating that the subset of the plurality of data packets in the first stride buffer has been processed.

17. The method of claim 16, wherein the out-of-order data packet is associated with the plurality of data packets.

18. The method according to claim 16, wherein the method further comprises: determining whether the plurality of data packets are received within a predetermined time period; determining that only a first subset of the plurality of data packets are received within the predetermined time period; aggregating the first subset of the plurality of data packets for storage in a second stride buffer; processing the second stride buffer; as well as A second completion notification is generated, the second completion notification indicating that the first subset of the plurality of data packets in the second stride buffer has been processed.

Images