Method and network interface card for implementing out-of-order packet processing in a data type engine and storage medium

Abstract

The present disclosure relates to mechanisms for enabling out-of-order packet processing in a data type engine. A network interface card (NIC) receives packets corresponding to read or write requests, the packets being associated with a data type descriptor stored in a data type engine of the NIC, and each packet being associated with a pre-computed context indicating values for each dimension of a multi-dimensional array and a starting location of the respective packet within a host memory block. The NIC generates, for the respective packet, a data type handle corresponding to the data type descriptor and an offset indicating a location of the respective packet within the packet. The NIC determines, based on the data type handle and the offset, a cached context for the respective packet, and initializes the data type engine based on the cached context. The data type engine generates, based on the cached context, a read or write request including an address and a length, thereby allowing the NIC to process out-of-order packets based on the pre-computed and cached context.

Description

Technical Field

[0001] This disclosure relates in general to mechanisms for implementing out-of-order packet handling in data type engines. Background Technology

[0002] User applications can send and receive data residing in non-contiguous buffers. A typical process might involve: packing data into a contiguous send buffer; transmitting the data; and unpacking the data from a contiguous receive buffer to the appropriate application location. These data copying operations (i.e., packing and unpacking) occur in software and are therefore serial with data transmission communication, which can significantly increase communication time. In hardware solutions, a network interface card (NIC) can incorporate a data type engine that allows users to describe complex data layouts for sending and receiving data, including multidimensional arrays with unique strides in each of multiple dimensions. Adaptive routing can be used to optimize the overall network throughput. However, adaptive routing can result in out-of-order packets arriving over the network. The arrival of out-of-order packets may require a non-linear progression through the description of data types, which can be challenging or expensive due to the varying dimensional sizes of the multidimensional arrays or the varying element sizes as substructures of larger data structures. Summary of the Invention

[0003] According to some embodiments of this disclosure, a computer-implemented method executed by one or more processing elements is provided, comprising: receiving, via a network interface card (NIC), a plurality of data packets corresponding to a read request or a write request, wherein the plurality of data packets are associated with data type descriptors stored in a data type engine of the NIC, wherein each data packet is associated with a pre-computed context indicating the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory, wherein the data type descriptor includes at least one of 1) and 2) below: 1) a representation of the multidimensional array, the multidimensional array including the number of elements in each dimension, the size of the block to be transmitted, and the stride in each dimension, wherein the representation of the multidimensional array indicates an old data type name and new data. A handle to a type name; or 2) a reference to an input / output vector IOWEC, which indicates the address and length of data to be read from or written to the host memory; generating a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the plurality of data packets for the corresponding data packet; determining the context in which the corresponding data packet is cached based on the data type handle and the offset; initializing the data type engine based on the cached context; generating a plurality of read or write requests, including addresses and lengths, by the data type engine based on the pre-computed and cached context, thereby allowing the NIC to process out-of-order data packets based on the pre-computed and cached context; and caching the current context when processing of the corresponding data packet is complete.

[0004] According to some embodiments of this disclosure, a non-transitory computer-readable storage medium is also provided, the non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0005] According to some embodiments of this disclosure, a network interface card is also provided, comprising: a command unit configured to receive, via the network interface card, a plurality of data packets corresponding to a read request or a write request, wherein the plurality of data packets are associated with data type descriptors stored in a data type engine of the network interface card, wherein each data packet is associated with a pre-computed context, the pre-computed context indicating the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory, wherein the data type descriptor includes at least one of the following 1) and 2): 1) a representation of the multidimensional array, the multidimensional array including the number of elements in each dimension, the size of the block to be transmitted, and the stride in each dimension, wherein the representation of the multidimensional array indicates handles to an old data type name and a new data type name; or 2) a reference to an input / output vector IOWEC, the IOWEC indicating to be read from or to be written from the host memory. The network interface card (NIC) is configured to: specify the address and length of data written to the host memory; generate a context generation unit for generating a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the plurality of data packets; a cache management unit for determining the context in which the corresponding data packet is cached based on the data type handle and the offset; a data type engine management unit for initializing the data type engine based on the cached context; and a request generation unit for generating a plurality of read or write requests, including an address and a length, based on a pre-calculated and cached context using the data type engine, thereby allowing the NIC to process out-of-order data packets based on the pre-calculated and cached context; and wherein the cache management unit is further configured to cache the current context when processing of the corresponding data packet is completed. Attached Figure Description

[0006] Figure 1 The diagram illustrates an architecture that facilitates high-performance computing through a data type engine, according to one aspect of this application.

[0007] Figure 2 An exemplary code excerpt of an application using data types according to one aspect of this application is illustrated.

[0008] Figure 3 The diagram illustrates an outbound packet engine (OXE) according to one aspect of this application, which has a data type engine as part of the transmission path.

[0009] Figure 4 The diagram illustrates an inbound delivery engine (IXE) that facilitates out-of-order packet processing according to one aspect of this application.

[0010] Figure 5 The diagram illustrates an inbound delivery engine (IXE) (including a data type engine) that facilitates out-of-order packet processing according to one aspect of this application.

[0011] Figure 6 A flowchart illustrating a method for facilitating out-of-order packet processing in a data type engine according to one aspect of this application is presented.

[0012] Figure 7 The illustration shows an apparatus for facilitating out-of-order packet processing in a data type engine according to one aspect of this application.

[0013] In these accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0014] The following description is presented to enable any person skilled in the art to make and use the aspects and examples, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects and applications without departing from the spirit and scope of this disclosure. Therefore, the aspects described herein are not limited to those shown, but are intended to be consistent with the maximum scope of the principles and features disclosed herein.

[0015] The described aspects provide a solution to the challenges of out-of-order packet handling in a data type engine by pre-computing the initial state or context of each packet and generating read or write requests based on said pre-computed state or context. Out-of-order packet handling may be required when Put messages are transmitted from the initiator node to the target node as a series of Put packets. Allowing out-of-order packet transmission can potentially improve performance. Out-of-order handling may also be required when transmitting or receiving Get response messages as a series of packets. In this case, the data type at the target node can define how data is collected from host memory to construct the overall Get response. The data type at the initiator node can define a distribution operation to determine how data in the overall Get response is placed in the initiator's host memory. The initiator initially processes Get messages by sending a series of Get request packets to the target. At the target, the Get request packets may be received out of order, and the corresponding Get response packets are also constructed and sent out of order, and thus received back out of order at the initiator.

[0016] Current software interfaces allow users to represent data layouts in their native form. A commonly used software interface is the Message Passing Interface (MPI), where users can provide a base address to initiate a transmission and the data type applied to that transmission. MPI also provides a mechanism called "derived data types," which can describe the data layout in host memory and use that description for certain MPI calls (e.g., MPI_Send and MPI_Recv). Historically, using derived data types in MPI can have resulted in poor performance because the library must interpret the description to perform copying to and from internal buffers for the MPI_Send and MPI_Recv commands, respectively. Therefore, packing and unpacking operations are still performed in software and are still serial with communication, which can lead to overall performance inefficiency for data-related operations such as data collection / distribution.

[0017] In hardware solutions, a network interface card (NIC) can incorporate a data type engine that allows users to describe complex data layouts for sending and receiving data. These complex layouts include multidimensional arrays with unique strides in each of multiple dimensions. The data type engine can perform the aforementioned collection / distribution operations, eliminating the inefficient data copying used in previous software solutions. The data type engine can reside in the NIC or its application-specific integrated circuit (ASIC), as part of both the outbound and inbound packet engines. The data type engine can generate a series of addresses and lengths based on data type descriptors. The data type descriptor can be an input / output vector (IOVEC) structure with relative addressing and can describe a multidimensional array defined by counts, block sizes, and strides. By using data type descriptors as part of the data path, data type engines in both the transmit path (e.g., read operations from host memory handled by the outbound packet engine) and the receive path (e.g., write operations to host memory handled by the inbound packet engine) can potentially reduce communication time involved in data transmission over the network. This could lead to more efficient message delivery between network entities, which in turn could lead to more efficient use of resources, such as to support high-performance computing (HPC).

[0018] The aspects described by the data type engine can perform collect / distribute operations while addressing several challenges. The first challenge is ensuring that the representation of the data type covers common use cases while remaining compact. This can be difficult when dealing with code that constructs arrays of structures. The described aspects can represent data types using a representation of three strides, three counts, and basic blocks. The system can use length, number of valid bytes, and a bitmask to represent basic blocks to indicate which bytes are valid. This data type can represent arbitrary three-dimensional (3D) arrays of C structures (up to 256 bytes per structure), with each data type using less than 64 bytes of storage.

[0019] The second challenge involves using redundant memory resources as part of compaction. Reading data from memory and placing it into a temporary local buffer before compacting the valid bytes may require redundant memory resources. The described aspect addresses this challenge by compressing 256 bytes with an arbitrary bitmask to capture only the valid bytes in the eight stages of a 2-to-1 multiplexer. The system can further leverage the POPCNT instruction in the x86 processor to quickly and efficiently encode additional information, reducing the amount of logic required for each stage of compaction.

[0020] The third challenge (as discussed in this article and below) Figures 4 to 7 The above relates to handling out-of-order packets. Because modern HPC networks can utilize adaptive routing to optimize overall throughput, some packets may arrive out of order. The arrival of out-of-order packets may require a non-linear progression through the description of data types, which can be challenging or expensive due to the varying dimensional sizes of multidimensional arrays or the varying element sizes of substructures of larger data structures. These out-of-order packets can be of three types: packets received as "Put" (e.g., a write operation); request packets received as "Get" (e.g., a read operation), to which a response must be generated; and packets received in response to a "Get". For a Put operation, a first node can send data to a second node, and the packet may arrive at the second node out of order. For a Get operation, a first node can request data from a second node, and the request packet may be received out of order (and responded to) at the second node, and the packet may be received out of order as part of the response at the first node.

[0021] To process Get operations at the initiator, aspects of the described Data Type Engine (DTE) can pre-compute the initial state or context of each packet when forming the Get request, for example, 8 bytes of context for each 4-kilobyte (KB) packet. The system can store this initial state or context along with other information needed to process the Get response, allowing out-of-order responses to initialize the DTE using the stored state. To process Get operations at the destination, when the data type defining the collection operation is defined, the software can pre-compute the context of each response packet, for example, 8 bytes of context for each 4-KB packet. This context can be stored in the destination host memory. Information, along with each Get request packet, can be provided to the DTE at the destination responsible for constructing the Get response packets, allowing the DTE to read the appropriate context from host memory as needed. The DTE at the destination can also cache the context when it finishes processing the Get response, which can be used to process subsequent Get responses of the same overall Get.

[0022] To handle Put operations, the receiving side can provide the data type used to receive Put operations. Software associated with the system (e.g., a NIC ASIC) can pre-compute a small amount of DTE state or context (e.g., 8 bytes) corresponding to the start of each packet. To handle out-of-order packets, the system can combine a lookup from host memory with a data type descriptor. The system can further optimize the target or destination of the Put operation to reuse the state in the DTE from one packet to the next. Even with adaptive routing, packets can arrive mostly in order. Reusing or maintaining the state in the DTE eliminates host memory lookups. The following section combines... Figures 4 to 7 Describes the context for pre-compiling, caching, and retrieving packets, as well as the initialization of the data type engine to handle out-of-order packets.

[0023] In addition to addressing these challenges to improve the performance of data transmission over the network and support high-performance computing, the described system may also include other aspects. One aspect may involve the ordering of data type commands (e.g., installing a data type descriptor in the NIC's DTE) relative to payload commands (e.g., executing a user read / write command). The data type must be created and set up or installed before being used by the payload or user command. Commands may, for example, be based on a series of parallel queues traversing the NIC via multiple paths. The system can select queues by hashing various fields in the executed command or associated data packet. Thus, both the command for installing the data type descriptor into the NIC's DTE and the user command can have stream hashes that allow both commands to take the same path. Furthermore, the system may include events indicating that a data type command has been completed, as follows regarding... Figure 1 The description illustrates the communication with the event engine. Once the data type command completes, the installed data type descriptor can be used by many independent subsequent commands.

[0024] Another aspect of the described system may relate to data type selection and storage efficiency. A command used to initiate a new message must indicate the data type it wants to use. However, including the full data type descriptor in every command may not be efficient or practical. Furthermore, the structure of data types may often be identical between processes on a given node (as part of the same user job). The described aspect may include a data type allocation mechanism that can lead to improved storage efficiency and the efficiency of new message commands. Software (e.g., an operating system (OS)) can allocate data type storage space and provide data type keys to the user. The system may use a single key space for data type descriptors, which can be shared between processes. Hardware may provide a enforcement mechanism that ensures that only processes that are part of the same application can use a given key. The user can include a given key in a new message to specify the data type to use. The system may use empty entries for data type keys to indicate that a command or transmission does not include a data type.

[0025] Another aspect of the described system may involve using both absolute and relative addressing within the IOWEC structure used by the data type engine. Previous solutions could use absolute addressing in the collection / distribution list to support use cases involving OS kernel modes. However, user-level software interfaces typically use relative addressing, which can impose limitations when MPI uses the collection / distribution engine. When using the IOWEC structure in the described aspect, the DTE can retrieve data from host memory using two different addressing modes based on a list of addresses and lengths. The first mode is absolute addressing, where the address retrieved from host memory is the virtual address to be used, i.e., using a set of absolute address / length pairs from the IOWEC. The second mode is relative addressing, where the address retrieved from host memory is used as an offset relative to a virtual base address provided by the user in a new message, where the offset can be obtained from the IOWEC (an offset and length list) or from the data type descriptor.

[0026] Another aspect of the described system involves data type storage, IOWEC, and command efficiency. Previous solutions, to initiate a new message, could add a pointer to an IOWEC for each command, potentially increasing command size and reducing the efficiency of the command interface. The system can encode and set up or install IOWEC in the same way as other data types. However, some IOWEC use cases may be used only once; that is, the system creates an IOWEC, uses it for a single command (“use once”), and then discards it. Because command ordering works in conjunction with the IOWEC approach, the command ordering logic described above allows the system to maintain existing command formats while still ensuring efficient setup paths for these “use once” IOWECs.

[0027] Therefore, the various aspects of the described system can address the aforementioned challenges by providing a Data Type Engine (DTE) that generates a series of addresses and lengths based on data type descriptors. Data type descriptors can represent multidimensional arrays defined by counts, block sizes, and strides, as shown below regarding... Figure 2 As described. Data type descriptors can also reference IOWEC structures with relative addressing. The following is about... Figure 1 The description includes the overall architecture of the system (e.g., NIC) encompassing both outbound and inbound transport engines, while the following section discusses... Figure 3 A detailed diagram describing the outbound delivery engine of the DTE module that communicates with other modules.

[0028] Furthermore, aspects of the described system can address the challenges of out-of-order packet processing by pre-compiling a small (e.g., 8-byte) state or context for each packet to initialize the DTE, and further by caching the pre-computed context for later retrieval or use, which facilitates out-of-order packet processing. The following section discusses... Figure 4 A diagram describing the inbound transfer engine (IXE), and below about Figure 5 Describes the DTE module of IXE that facilitates out-of-order packet processing.

[0029] Exemplary high-level architecture

[0030] Figure 1 The diagram illustrates an architecture that facilitates high-performance computing through a data type engine, according to one aspect of this application. Figure 1The diagram depicts a network interface card (NIC) 100, which includes various components or modules, interfaces, and communications. At one end of the NIC 100, a host interface 110 exists between a component of the NIC 100 and a host. The host interface 110 can handle communication with the host (via communication 142). At the other end of the NIC 100, a port macro 132 exists between a component of the NIC and a network, and can handle communication with the network (via communication 186). Typically, the components on the left side of Figure 100 belong to the outbound path and can handle transmit operations related to reading operations from host memory (indicated by the thick lines of communications 146, 172, and 184 described below), while the components on the right side of Figure 100 belong to the inbound path and can handle receive operations related to writing operations to host memory (indicated by the thick lines of communications 188, 182, and 154).

[0031] Host interface 110 can receive data from various NIC components via system bus arbitration / host interface 112. Host interface 110 can also (via communication 140) send command list pointers or commands with data to command queue 114, which may include metadata (MD) table 115. Command queue 114 can (via communication 156) send portal and Ethernet operations (e.g., related to read operations in the transmit path) to outbound transmission engine 116, which may include outbound buffer 117 for storing data related to operations performed by outbound transmission engine 116. Command queue 114 can also receive commands related to reading data, such as direct memory access (DMA) acquire commands (via communication 144) and events from event engine 118 (via communication 158). Command queue 114 can also (via communication 148) issue additional commands to list processing engine 122.

[0032] Outbound transmission engine 116 may include a data type engine ( Figure 1 An instance (not shown) is provided, and various operations can be performed to generate requests or commands related to read DMA operations. Outbound transfer engine 116 can receive outbound DMA data from host memory (via communication 146 from host interface 112), which can be stored in output buffer 117. The following is about... Figure 3 Provides a detailed description of outbound transport engine 116.

[0033] After the data type engine of outbound transmission engine 116 uses IOWEC with relative addressing and describes the multidimensional array in the manner described herein, outbound transmission engine 116 can output a request or command to a pending command table (emit (TX)) 126 (via communication 172). Pending command table 126 may include a retry buffer 127. Pending command table 126 may send a completion notification to event engine 118 upon completion of the corresponding command (via communication 174). Pending command table 126 may emit outbound data packets corresponding to the read command (via communication 184 to port macro 132).

[0034] NIC 100 can receive inbound data packets to be written to host memory (via communication 188). Inbound data packets can be sent to a pending command table (receive (RX)) 130. Pending command table 130 can then send information (via communication 180) to message status table 124 and (via communication 178) to list processing engine 122, which may include list entries 123. List processing engine 122 can send information (via communication 168) to message status table 124, which can send information (via communication 170) to inbound delivery engine 120. Inbound delivery engine 120 may include input buffer 121. List processing engine 122 can also send memory or data type descriptors (via communication 166) to inbound delivery engine 120 for storage in its data type engine (not shown). List processing engine 122 can further send events (via communication 164) to event engine 118. The pending command list 130 can also (via communication 182) transmit payloads associated with inbound data packets to the inbound delivery engine 120, the payloads of which can be stored in the input buffer 121. The inbound delivery engine 120 may include a data type engine (…). Figure 1 An instance (not shown) can be generated and various operations can be performed to generate requests or commands related to write DMA operations. After the data type engine of the inbound transfer engine 120 uses IOWEC with relative addressing and describes the multidimensional array in the manner described herein, the inbound transfer engine 120 can (via communication 154 to the host interface 112) send inbound DMA data to the host memory.

[0035] Inbound transmission engine 120 can receive data from host interface 112 (via communication 150) and can also send a completion notification to event engine 118 (via communication 162) upon completion of a corresponding command. In addition to receiving completion notifications from pending command table 126 and inbound transmission engine 120 (via communication 174 and 162 respectively), event engine 118 can also send event-related information to host interface 112 (via communication 160). Pending command tables in the transmit and receive paths can convey response status (via tables 126 and 130 of communication 176), and inbound transmission engine 120 can also send responses to outbound transmission engine 116 (via communication 152). Similarly, outbound transmission engine 116 can form data packets, and its DTE can form multiple reads; list processing engine 122 can form an address for writing data packets and pass that address to the inbound transmission engine 120's DTE. The DTE of the inbound delivery engine 120 can form multiple write requests based on a given data type descriptor, wherein the multiple write requests include an offset relative to that address and a corresponding length. It will be apparent to those skilled in the art that this mechanism can also be used for packet input processing.

[0036] Data type descriptor overview

[0037] In a typical NIC implementation, the user software can provide commands indicating the virtual address and length of the data to be transmitted. The NIC can receive the commands and translate the virtual addresses. Subsequently, the NIC can issue read (for transmitting data, e.g., in a Put operation at the initiator) or write (for receiving data, e.g., in a Put operation at the destination) orders to host memory. In conventional solutions, these reads and writes to host memory are typically for large blocks of continuous data.

[0038] The described aspect provides a data type engine that generates a series of addresses and lengths based on data types or "data type descriptors." Data types can natively represent data layouts in application memory, which can lead to the elimination of resource-intensive copying. That is, a data type is a way of describing a native data layout in memory. Without the data type engine described herein, the described data copying is performed using a central processing unit (CPU), a general-purpose processing unit (GPU), and advanced memory systems. However, using these systems can be inefficient because operations using them can be expensive (i.e., CPUs, GPUs, and advanced memory systems are typically better suited for other tasks).

[0039] An example of a data type descriptor could be a multidimensional array descriptor, where the data type engine uses parameters to describe an array in host memory, as shown below. Figure 2 The matrix transpose example is described below. Multidimensional arrays can include: the number of elements in each dimension; the size of the block to be transposed; and the stride in each dimension. Data type descriptors can also include handles to the old data type name and the new data type name, as shown below regarding... Figure 2 and Figure 4 As described. Another example of a data type descriptor form could be an Input / Output (I / O) Vector (IOVEC), where the DTE retrieves a list of addresses from host memory. IOVEC can use relative or absolute addressing. In relative addressing, the DTE can retrieve data based on an address “offset” relative to a user-provided virtual base address, while in absolute addressing, the DTE can retrieve data based on an address that serves as the virtual address to be used.

[0040] Example code for applications using data types

[0041] Figure 2 An exemplary code excerpt 200 of an application using data types according to one aspect of this application is illustrated. Section 210 may indicate a data type “MPI_Type_contiguous” that allows the data type to be copied to consecutive locations. Sections 212, 214, and 216 may indicate a data type “MPI_Type_vector” that allows the data type to be copied to locations with equidistant blocks, where each block can be obtained by concatenating the same number of copies of an older data type. The spacing between blocks may be a multiple of the range of the older data type. “MPI_Type_vector” may indicate a structure comprising a multidimensional array including: a count corresponding to the number of blocks or elements in each dimension; a block size or length corresponding to the number of elements in each block; a stride in each dimension corresponding to the number of elements between the beginnings of each block; an indicator of the existing or older data type; and a handle to the new data type.

[0042] Section 210 can create floating-point ternary vectors and be used to construct descriptions of multiple faces (e.g., three) of a matrix. Using the data types indicated in sections 214 and 216 can result in data being emitted in large chunks or many large chunks, which can be aggregated into messages, and can result in efficient delivery. However, using the data types indicated in section 212 can result in 12-byte collection / distribution from many locations, which may be less efficient than using the data types in sections 214 and 216. When comparing NIC performance to CPU and GPU performance, the data type engine in a NIC with data type descriptors can perform more efficiently when delivering data in larger chunks (e.g., 128-byte segments) rather than smaller chunks (e.g., 8-byte or 12-byte segments).

[0043] Outbound packet engine and transmission path

[0044] As mentioned above Figure 1 As described, the data type engine can be integrated with the transmit and receive logic in the outbound packet engine and the inbound packet engine, respectively. Figure 3 Figure 300 illustrates an outbound packet engine 301 according to one aspect of this application, which has a data type engine as part of the transmission path. The outbound packet engine (OXE) 301 may include: a packet forming module 302; a data type engine (DTE) module 304; a direct memory access (DMA) module 306; and a post-processing module 308. The OXE 301 can take data from command 320 as input and can output packets to reliability level 322. Additionally, the OXE 301 can communicate with an address translation unit (ATU) 310 and host memory 312.

[0045] During operation, OXE 301 can receive commands (via communication 330), for example, from the command queue of command 320. These commands can be received via a message passing interface (MPI) and can be commands for reading data from host memory. In the standard path (i.e., where the data type descriptor is absent, thus bypassing DTE), packet forming module 302 can form a DMA instruction, referred to as a "non-DTE DMA instruction," indicating the header, address, and length associated with the data to be read from host memory. This standard non-DTE DMA instruction can be sent (via communication 334) to DMA module 306. DMA module 306 can obtain the corresponding address translation from ATU 310, for example, by sending a virtual address 338 to ATU 310 and receiving a physical address 340 from ATU 310. DMA module 306 can then issue one or more read requests to host memory 312 (via communication 342) and receive one or more corresponding read responses from host memory 312 (via communication 344). DMA module 306 can transmit a read response to post-processing module 308 (via communication 346), and the post-processing module can transmit the processed data as output to reliability level 322 (via communication 348).

[0046] When a data type descriptor (e.g., a handle indicating a previously stored data type descriptor) is present in the command, packet forming module 302 can form a DMA instruction, referred to as a "DTE instruction," that indicates or references the data type descriptor and associates it with a base address and length of data to be read from host memory. This DTE instruction can be sent (via communication 332) to DTE module 304. DTE module 304 can identify or determine that the data type descriptor indicated in the DTE instruction matches a data type descriptor stored in DTE module 304 (e.g., as previously configured and stored by the user). Using the identified matching data type descriptor, DTE module 304 can generate multiple DMA read instructions having an offset relative to the base address and a corresponding length. DTE module 304 can then send these generated DMA read instructions to DMA module 306 (via communication 336). DMA module 306 can translate DMA read instructions into read requests and perform the address translation (if necessary) and read operations described above with respect to communications 338, 340, 342 and 344, as well as subsequent operations related to post-processing module 308 and reliability level 322.

[0047] Exemplary IXE and DTE: Using pre-computed and cached contexts to facilitate the use of data type engines Out-of-order packet processing

[0048] Figure 4Figure 400 illustrates an inbound delivery engine (IXE) 402 for facilitating out-of-order packet processing according to one aspect of this application. IXE 402 can receive a request 432 or a response 434 (via communications 442 and 444, respectively). Figure 400 focuses on the request path. A packet buffer 420 can receive the request via communication 442 (after necessary decoding of the request, not shown). The packet buffer 420 can send necessary information, such as information about the corresponding packet for the request or message, to a context determination unit 430 (via communication 446). The corresponding packet can be associated with a data type descriptor stored in the data type engine of the NIC. The context determination unit 430 can determine whether the corresponding packet is associated with an already received message or a new message. Based on this determination, the context determination unit 430 can generate a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding packet within multiple packets. The context determination unit 430 can (via communication 448) send the generated data type handle and offset to the write dispatcher or write multiplexer (WrMux) 412 of IXE 402. The write dispatcher 412 can (via communication 450) send the generated data type handle and offset as DMA write instructions to the data type engine (DTE) 414. The DTE 414 can determine whether the context for the corresponding data type handle and offset is cached. If the context is cached, the DTE 414 can initialize based on the cached context and can further generate multiple DMA write instructions including address and length.

[0049] DTE 414 can output multiple generated DMA write commands to a write dispatcher (write request queue (WRQ)) 416, which can convert the DMA commands into multiple write requests, for example, by obtaining addresses from the address translation unit (ATU) 438 as needed via the address translator (ATR) 418 of IXE 402 (via communications 454 and 456 respectively). Write dispatcher 416 can (via communication 464) issue the multiple write requests to a direct memory access (DMA) write controller 420, which can then issue the requests to host memory via, for example, a fast peripheral component interconnect (PCIe) interface 440 (via communication 466).

[0050] If no cached context is available, DTE 414 can obtain the context from host memory, which may include obtaining the appropriate address information from ATU 438 via ATR 418 of IXE 402 (via communications 454 and 452, respectively). After obtaining the appropriate address information, DTE 414 can obtain the context by sending a request to host memory via PCIe interface 440 (via communication 460) and receiving a response from host memory.

[0051] Figure 5 Figure 500 illustrates an inbound delivery engine (IXE) (including IXE data type engine 502) that facilitates out-of-order packet processing according to one aspect of this application. The IXE may include an IXE DTE 502 (indicated by dashed boundaries) and: a write dispatcher (WrMux) 412; an ATR 522; a write request queue 530; and a write request engine 532 (indicated by thick boundaries). The IXE 502 may communicate with an ATU 540 and a host memory 542, which are not part of the IXE or the IXE DTE 502 (as indicated by dashed boundaries). The host memory 542 may represent a PCIe arbiter (or PARB) communicating with the host memory.

[0052] During operation, write dispatcher 412 can (via communication 550) send a data packet with a DTE handle and offset to IXE DTE 502, which is received by allocation / bypass unit 502. If the received data packet does not indicate a data type handle (e.g., indicates "none"), allocation / bypass unit 502 can (via communication 554) send the data packet directly to write request queue 530 for immediate processing, as described above. Figure 3 As described in communication 334. If a data type handle is indicated in the received data packet, the allocation / bypass unit 502 can allocate the data packet in instruction RAM 508 (via communication 552) and indicate the command in command tracker 504. Command tracker 504 can track received commands and can (via communication 558) send the data packet to DTE pipeline unit 506, which can be responsible for collecting information for processing the data packet. DTE pipeline 506 can determine whether a previous data packet with the same message has been received recently, or whether the context of the data packet has been cached.

[0053] Based on the data type handle, the DTE pipeline 506 can read the corresponding data type from the descriptor storage device 516 (via communication 560). The descriptor storage device 516 may include non-byte mask descriptor storage devices and byte mask descriptor storage devices. The byte mask portion of the descriptor may refer to a value with multiple bits, which can be used as a byte mask for data having up to a predetermined number of bytes. For example, during the compaction process, each bit in an X-byte mask may correspond to an X-byte data block or a byte within a block. The non-byte mask portion of the descriptor may include other information, such as loop counts and strides (for the obtained data type) or pointers to IOWEC in host memory (for IOWEC data types), which is created by the user and stored in the descriptor storage device 516 for subsequent use or retrieval by the DTE.

[0054] If a context is cached, the DTE pipeline 506 can retrieve the context (via communications 564 and 566) from either hot context random access memory (RAM) 512 or cold context RAM 514. Typically, hot context RAM 512 can temporarily store the context during packet processing. Cold context RAM 514 can store the context when packet processing is complete, and this context can be used to begin processing subsequent packets of the same overall message. For example, the DTE pipeline 506 can obtain the current value (e.g., current loop value) of each dimension of a multidimensional array and the starting position of the corresponding packet within a block of host memory. The DTE pipeline 506 can then immediately begin processing the packet. The DTE pipeline 506 can use the context retrieved from the cache to initialize the state of the packet and can further (via communication 568) send the packet to queue 518 for processing by data type (DT) processor 520. The DT processor 520 can generate multiple instructions based on the pre-computed and cached context, such as multiple write requests including addresses and lengths.

[0055] If the context is not cached, the DTE pipeline 506 can retrieve the context from host memory using the obtained data type descriptor. DMA instructions and information from the descriptor travel to the DT processor 520 via queue 518 (via communication 568). Unlike normal processing (i.e., generating a DMA write), this generates a DMA read transaction to retrieve the context from host memory.

[0056] The DT processor 520 can obtain the appropriate address information from the ATU 540 via the ATR 522 and the Address Translation Queue (ATQ) 421 (via communications 574 and 572, respectively). After obtaining the appropriate address information, the DTE pipeline 506 can retrieve the context from the host memory 542 via the Memory Data Read (MDR) 524 (via, for example, communications 576 and 584). The MDR 524 can (via communication 582) transmit the obtained context information (including the current loop counter value) to the IVEC / Context_Fast Forward (FF) buffer (ICB) 510. For example, given a three-dimensional array of dimensions [X, Y, Z], the obtained context can be a fast-forward entry of the obtained data type and can include 8 bytes (sorted from bits 63:0), which can include: unused bits (bits 63:56); the current byte position with the block (bits 55:48); the current element in the Z dimension (bits 47:32); the current element in the Y dimension (bits 31:16); and the current element in the X dimension (bits 15:0).

[0057] When ICB 510 receives this information, command tracker 504 can manage (via communications 556 and 558) sending a wait instruction to DTE pipeline 506. DTE pipeline 506 can (via communication 562) directly read the obtained context from ICB 510 and begin processing the packet, i.e., initialize the state of the packet using the context obtained from ICB 510, and send the packet to queue 518 for processing by DT processor 520, wherein DT processor 520 can generate the plurality of instructions.

[0058] When the DT processor 520 has generated multiple instructions including addresses and lengths (whether using a cached context or a context obtained from host memory), the DT processor 520 can issue the multiple instructions to the write request queue 530 (which corresponds to) via communication 586. Figure 4 The write dispatcher (WRQ) 416. The write request queue 530 can also obtain information related to address and packet information via communications 570, 572, 574, 576, 578, 580, and 586. The write request engine 532 can (via communication 588) issue multiple instructions (i.e., write requests) to the host memory 542 and receive appropriate responses.

[0059] Methods for improving out-of-order packet handling in data type engines

[0060] Figure 6 A flowchart 600 is presented, which illustrates one aspect of the present application and as described above regarding Figure 4 and Figure 5 The described method facilitates out-of-order packet processing in a data type engine (in the system). During operation, the system receives multiple packets corresponding to write requests via a network interface card (NIC), wherein the packets are associated with data type descriptors stored in the data type engine of the NIC, and each packet is associated with a pre-computed context indicating the value of each dimension of a multidimensional array and the starting position of the corresponding packet within a block of host memory (operation 602, as described above regarding...). Figure 4 (As described in request 432 and communication 442). The system generates a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within multiple data packets for the corresponding data packet (operation 604, as above regarding...). Figure 4 (As described in context determination unit 430). The system determines whether the context of the corresponding packet is cached based on the data type handle and offset (operation 606). If the context is cached (decision 608), the system initializes the data type engine based on the cached context (operation 610). The system generates multiple write requests, including addresses and lengths, based on the pre-computed and cached context using the data type engine, thereby allowing the NIC to process out-of-order (OOO) packets based on the pre-computed and cached context (operation 612, as described above regarding...). Figure 4 DTE 414 and Figure 5 (As described in the DT processor 520). When processing of the corresponding packet is complete, the system caches the current context (operation 614). The operation returns.

[0061] If the context is cached (decision 608), the system retrieves the context from host memory (operation 620). The system initializes the data type engine based on the retrieved context (operation 622). The system uses the data type engine to generate multiple write requests, including addresses and lengths, based on the retrieved context, thereby allowing the NIC to process out-of-order (OOO) packets based on pre-computed and retrieved context (operation 624, as mentioned above). Figure 4 DTE 414 and Figure 5 (As described in the DT processor 520). When processing of the corresponding packet is complete, the system caches the current context (operation 614). The operation returns.

[0062] Flowchart 600 depicts a method for processing write requests (e.g., a Put operation performed at the destination by a DTE of an inbound delivery engine). As described herein, a similar method can be used to process read requests (e.g., a Get operation performed at the destination by a DTE of an outbound delivery engine). The key difference is that the IXE DTE generates write instructions for writing to host memory, while the OXE DTE generates read instructions for reading from host memory. Therefore, in flowchart 600, received data packets may correspond to read requests (e.g., similar to operation 602), and the DTE may generate multiple read requests including addresses and lengths based on pre-computed and cached context (e.g., similar to operations 612 and 624), which are sent to the DMA module for reading from or retrieving from host memory (as described above). Figure 3 (as described).

[0063] Devices that enable data type engines to support high-performance computing

[0064] Figure 7 The illustration shows an apparatus 700 for facilitating out-of-order packet processing in a data type engine according to one aspect of this application. Apparatus 700 may represent a network interface card (NIC) and may include: a command unit 702 for receiving a command for writing data to host memory and determining whether the command instructs a data type descriptor stored in the data type engine of the NIC; a buffer management unit 704 for storing incoming data packets and information associated with corresponding instructions; a context generation unit 706 for generating a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within a plurality of data packets for the corresponding data packet; and a data type engine management unit 708 for... The system determines whether the context of the corresponding data packet is cached based on the data type handle and offset, and further initializes the data type engine based on the pre-computed, cached, or acquired context, and if the context is not found in the cache, it further acquires the context from the host memory; a cache management unit 710 is used to store pre-computed contexts of up to a predetermined number of messages; and a request generation unit 712 is used to generate multiple write requests including addresses and lengths based on the pre-computed, cached, or acquired context, thereby allowing the NIC to process out-of-order data packets based on the pre-computed, cached, or acquired context.

[0065] Command unit 702 may correspond to, for example Figure 5 Command tracker 504. Buffer management unit 704 can correspond to, for example... Figure 4The packet buffer 420. The context generation unit 706 can correspond to, for example... Figure 4 The context determination unit 430. The data type engine management unit 708 can correspond to Figure 4 DTE 414 and Figure 5 The IXE DTE 502. The cache management unit 710 can correspond to... Figure 5 The hot context RAM 512 and cold context RAM 514. The request generation unit 712 can correspond to... Figure 5 The DT processor 520. The unit described as part of device 700 can perform the above-mentioned... Figure 1 , Figure 4 , Figure 5 and Figure 6 The described operation.

[0066] Typically, the disclosed aspects provide a system, method, apparatus, and nontransitory storage medium for facilitating out-of-order packet processing in a data type engine. In one aspect, the system receives multiple data packets corresponding to read or write requests via a network interface card (NIC), wherein the data packets are associated with data type descriptors stored in a data type engine of the NIC, and each data packet is associated with a pre-computed context indicating the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory (e.g., as described above regarding...). Figure 4 (As described in request 432 and communication 442). The system generates a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within multiple data packets (e.g., as mentioned above regarding...). Figure 4 (As described in context determination unit 430). The system determines the context in which the corresponding data packet is cached based on the data type handle and offset. The system initializes the data type engine based on the cached context. The system generates multiple read or write requests, including addresses and lengths, based on the pre-computed and cached context using the data type engine (e.g., as described above regarding...). Figure 4 DTE 414 and Figure 5 As described in the DT processor 520, this allows the NIC to process out-of-order packets based on the pre-computed and cached context. When processing of the corresponding packet is complete, the system caches the current context.

[0067] In this regard, in response to determining that the context of a corresponding data packet is not cached, the system performs the following operations: retrieves the context from host memory; caches the retrieved context; initializes the data type engine based on the retrieved context; and generates multiple write requests, including addresses and lengths, based on the retrieved context. When processing of the corresponding data packet is complete, the system caches the current context.

[0068] In another change in this regard, the system determines whether a corresponding data packet is associated with an already received message or with a new message. The system generates an offset based on whether the data packet is associated with an already received message or a new message.

[0069] In another variation, the generated offset is further based on information in the corresponding data packet.

[0070] In another variation, the data type descriptor includes at least one of the following: a representation of a multidimensional array; and a reference to an input / output vector (IOVEC), which indicates the address and length of data to be read from or written to host memory. The representation of the multidimensional array includes: the number of elements in each dimension; the size of the block to be transferred; and the stride in each dimension.

[0071] In another variation, the corresponding data packets are received out of order within multiple packets, and the data type engine is initialized based on the cached context so that the out-of-order data packets are handled correctly based on the generated data type handles and offsets.

[0072] In another variation, the corresponding context for a predetermined number of messages can be cached at once.

[0073] In other variations, the pre-computed context is stored in host memory.

[0074] On the other hand, a non-transitory computer-readable storage medium is provided, which stores instructions that, when executed by a computer, cause the computer to perform the methods described herein.

[0075] On the other hand, a network interface card (NIC) is also provided (for example, as mentioned above). Figure 1 NIC 100 Figure 4 IXE 402 and Figure 5As described in IXE DTE 502, the NIC includes a command unit for receiving multiple data packets corresponding to read or write requests via the network interface card (NIC). Each data packet is associated with a data type descriptor stored in the NIC's data type engine. Each data packet is associated with a pre-computed context indicating the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory. The NIC further includes a context generation unit for generating a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the multiple data packets. The NIC further includes a cache management unit for determining the context in which the corresponding data packet is cached based on the data type handle and the offset. The NIC further includes a data type engine management unit for initializing the data type engine based on the cached context. The NIC further includes a request generation unit configured to generate multiple read or write requests, including addresses and lengths, based on the pre-computed and cached context using the data type engine, thereby allowing the NIC to process out-of-order packets based on the pre-computed and cached context. The cache management unit is further configured to cache the current context when processing of the corresponding packet is complete. In a variation of this, the pre-computed context is stored in at least one of the host memory, hot context random access memory (RAM), and cold context random access memory (RAM).

[0076] The foregoing descriptions of the various aspects have been presented solely for illustrative and descriptive purposes. These descriptions are not intended to be exhaustive or to limit the aspects described herein to the disclosed forms. Accordingly, many modifications and variations will be apparent to those skilled in the art. Furthermore, the foregoing disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.

Claims

1. A computer-implemented method executed by one or more processing elements, comprising: The network interface card (NIC) receives multiple data packets corresponding to read or write requests, wherein the multiple data packets are associated with data type descriptors stored in the NIC's data type engine. Each data packet is associated with a pre-computed context, which indicates the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory. The data type descriptor includes at least one of the following: The representation of the multidimensional array includes the number of elements in each dimension, the size of the block to be transmitted, and the stride in each dimension, wherein the representation of the multidimensional array indicates handles to the old data type name and the new data type name; or A reference to the input / output vector IOWEC, which indicates the address and length of data to be read from or written to the host memory; Generate a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the plurality of data packets for the corresponding data packet; Based on the data type handle and the offset, the context in which the corresponding data packet was cached is determined; The data type engine is initialized based on the cached context; The data type engine generates multiple read or write requests, including addresses and lengths, based on a pre-computed and cached context, thereby allowing the NIC to process out-of-order packets based on the pre-computed and cached context; and The current context is cached when the processing of the corresponding data packet is complete.

2. The method of claim 1, further comprising: In response to determining that the context of the corresponding data packet is not cached: The context is obtained from the host memory; The context obtained from the cache; The data type engine is initialized based on the obtained context; The multiple write requests, including address and length, are generated based on the obtained context. as well as The current context is cached when the processing of the corresponding data packet is complete.

3. The method of claim 1, further comprising: Determine whether the corresponding data packet is associated with a message that has already been received or with a new message; as well as The offset is generated based on whether the data packet is associated with a previously received message or with a new message.

4. The method as described in claim 3, in, The offset is generated based on information in the corresponding data packet.

5. The method as described in claim 1, in, The corresponding data packets are received out of order within the plurality of data packets, and Specifically, the data type engine is initialized based on the cached context so that the out-of-order data packets are processed correctly based on the generated data type handles and offsets.

6. The method as described in claim 1, in, It can cache the corresponding context of a predetermined number of messages at once.

7. The method of claim 1, wherein, The pre-computed context is stored in the host memory.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method, the method comprising: The network interface card (NIC) receives multiple data packets corresponding to read or write requests, wherein the multiple data packets are associated with data type descriptors stored in the NIC's data type engine. Each data packet is associated with a pre-computed context, which indicates the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory. The data type descriptor includes at least one of the following: The representation of the multidimensional array includes the number of elements in each dimension, the size of the block to be transmitted, and the stride in each dimension, wherein the representation of the multidimensional array indicates handles to the old data type name and the new data type name; or A reference to the input / output vector IOWEC, which indicates the address and length of data to be read from or written to the host memory; Generate a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the plurality of data packets for the corresponding data packet; Based on the data type handle and the offset, the context in which the corresponding data packet was cached is determined; The data type engine is initialized based on the cached context; The data type engine generates multiple read or write requests, including addresses and lengths, based on a pre-computed and cached context, thereby allowing the NIC to process out-of-order packets based on the pre-computed and cached context; and The current context is cached when the processing of the corresponding data packet is complete.

9. The non-transitory computer-readable storage medium as claimed in claim 8, wherein, The method further includes: In response to determining that the context of the corresponding data packet is not cached: The context is obtained from the host memory; The context obtained from the cache; The data type engine is initialized based on the obtained context; Generate the plurality of write requests, including address and length, based on the obtained context; and The current context is cached when the processing of the corresponding data packet is complete.

10. The non-transitory computer-readable storage medium of claim 8, wherein, The method further includes: Determine whether the corresponding data packet is associated with an already received message or a new message; and The offset is generated based on whether the data packet is associated with a previously received message or with a new message.

11. The non-transitory computer-readable storage medium as claimed in claim 10, in, The offset is generated based on information in the corresponding data packet.

12. The non-transitory computer-readable storage medium as described in claim 8, in, The corresponding data packets are received out of order within the plurality of data packets, and Specifically, the data type engine is initialized based on the cached context so that the out-of-order data packets are processed correctly based on the generated data type handles and offsets.

13. The non-transitory computer-readable storage medium as claimed in claim 8, wherein, The pre-computed context is stored in the host memory.

14. The non-transitory computer-readable storage medium as described in claim 8, in, It can cache the corresponding context of a predetermined number of messages at once.

15. A network interface card, comprising: The command unit is configured to receive multiple data packets corresponding to read or write requests via the network interface card, wherein the multiple data packets are associated with data type descriptors stored in the data type engine of the network interface card. Each data packet is associated with a pre-computed context, which indicates the value of each dimension of a multidimensional array and the starting position of the corresponding data packet within a block of host memory. The data type descriptor includes at least one of the following: The representation of the multidimensional array includes the number of elements in each dimension, the size of the block to be transmitted, and the stride in each dimension, wherein the representation of the multidimensional array indicates handles to the old data type name and the new data type name; or A reference to the input / output vector IOWEC, which indicates the address and length of data to be read from or written to the host memory; A context generation unit is used to generate a data type handle corresponding to the data type descriptor and an offset indicating the position of the corresponding data packet within the plurality of data packets; A cache management unit, configured to determine the context in which the corresponding data packet is cached based on the data type handle and the offset; A data type engine management unit, configured to initialize the data type engine based on the cached context; and A request generation unit is configured to generate multiple read or write requests, including addresses and lengths, based on a pre-calculated and cached context using the data type engine, thereby allowing the network interface card to process out-of-order packets based on the pre-calculated and cached context; and The cache management unit is further configured to cache the current context when the processing of the corresponding data packet is completed.

16. The network interface card as described in claim 15, in, In response to the cache management unit determining that the context of the corresponding data packet is not cached: The data type engine management unit is further configured to obtain the context from the host memory; The cache management unit is further used to cache the acquired context; The data type engine management unit is further used to initialize the data type engine based on the obtained context; The request generation unit is further configured to generate the plurality of write requests, including address and length, based on the obtained context; and The cache management unit is further configured to cache the current context when the processing of the corresponding data packet is completed.

17. The network interface card as described in claim 15, in, The command unit is further configured to determine whether the corresponding data packet is associated with a previously received message or with a new message; and The context generation unit is further configured to generate the offset based on whether the data packet is associated with the already received message or with the new message, and further based on the information in the corresponding data packet.

18. The network interface card as described in claim 15, in, The corresponding data packets are received out of order within the plurality of data packets, and Specifically, the data type engine is initialized based on the cached context so that the out-of-order data packets are processed correctly based on the generated data type handles and offsets.

19. The network interface card as described in claim 15, wherein, The pre-computed context is stored in at least one of the host memory, hot context random access memory (RAM), and cold context random access memory (RAM).

20. The network interface card as described in claim 15, in, It can cache the corresponding context of a predetermined number of messages at once.

Images