MECHANISM FOR ENCHANTING THE PROCESSING OF OUT-OF-SEQUENCE PACKETS IN A DATA TYPE ENGINE
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- HEWLETT PACKARD ENTERPRISE DEV LP
- Filing Date
- 2023-09-15
- Publication Date
- 2026-07-09
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
BACKGROUND area
[0001] User applications can send and receive data located in non-contiguous buffers. A typical flow might involve packing the data into a contiguous send buffer, transmitting the data, and unpacking the data from a contiguous receive buffer into the appropriate application locations. These data copying operations (i.e., packing and unpacking) are performed in software and are therefore serialized with the data transmission communication, which can significantly increase communication time. In a hardware solution, a network interface card (NIC) may include a data type engine that allows users to describe complex data layouts for sending and receiving data, including a multidimensional array with unique steps in each of the multiple dimensions. Adaptive routing can be used to optimize the overall throughput of a network.However, adaptive routing can cause packets to arrive on the network out of order. The arrival of out-of-order packets may require a nonlinear progression through the data type description, which can be difficult or expensive given the varying dimensions of the multidimensional array or the varying sizes of the elements that are substructures of a larger data structure. BRIEF DESCRIPTION OF THE CHARACTERS Fig. Figure 1 shows a diagram of an architecture that enables a data type engine to support high-performance computing, in accordance with one aspect of the present application. Fig. Figure 2 shows an example code excerpt for an application that uses data types in accordance with one aspect of the present application. Fig. 3 shows a diagram of an Outbound Packet Engine (OXE) with a data type engine as part of the send path, according to one aspect of the present application. Fig. Figure 4 shows a diagram of an inbound transfer engine (IXE) that facilitates out-of-order packet processing according to one aspect of the present application. Fig. Figure 5 shows a diagram of an inbound transfer engine (IXE) including a data type engine that facilitates out-of-order packet processing, according to one aspect of the present application. Fig. 6 shows a flowchart illustrating a method that facilitates out-of-order packet processing in a data type engine in accordance with an aspect of the present application. Fig. Figure 7 shows an apparatus that facilitates out-of-order packet processing in a data type engine, in accordance with one aspect of the present application.
[0002] In the figures, like numbers refer to the same elements of the figure. DETAILED DESCRIPTION
[0003] The following description is intended to enable one skilled in the art to make and use the aspects and examples, and is provided in the context of a specific application and its requirements. Various modifications to the disclosed aspects will be readily apparent to one skilled in the art, and the general principles defined herein may be applied to other aspects and applications without departing from the spirit and scope of the present disclosure. Therefore, the aspects described herein are not limited to the aspects shown, but are to be as broadly as possible consistent with the principles and features disclosed herein.
[0004] The described aspects provide a solution to the problem of out-of-order packet processing in a data type engine by pre-computing a starting state or context for each packet and generating read or write requests based on the pre-computed state or context. Out-of-order processing of packets may be necessary in the case of a Put message transmitted from an initiator node to a destination node as a series of Put packets. It may be beneficial for performance if the packets can be delivered out of order. Out-of-order processing may also be required when sending or receiving a Get response message transmitted as a series of packets. In this case, a data type at the destination node can specify how the data is collected from host memory to create the overall response to the query.A data type at the initiator node can define a scatter operation that determines how data from the entire Fetch response is stored in the initiator's host memory. The initiator first processes the Get message by sending a series of Get request packets to the target. The Get request packets may be received out of order at the target, and the corresponding Get response packets may also be constructed and transmitted out of order, thus being received out of order at the initiator.
[0005] Current software interfaces can allow the user to express the data layout in its native form. One commonly used software interface is the Message Passing Interface (MPI), where the user can specify a base address to start a transfer and a data type to apply to that transfer. MPI also provides a mechanism called "derived data types" that can describe the data layout in host memory and use that description for certain MPI calls (e.g., MPI_Send and MPI_Recv). Historically, the use of derived data types in MPI can result in poor performance because the library must interpret the description to perform copies to and from an internal buffer for the MPI_Send and MPI_Recv commands.As a result, the packing and unpacking operations are still performed in software and serialized with communication, which can lead to inefficiencies in the overall performance of data-related operations, such as data gathering / scattering operations.
[0006] In a hardware solution, a network interface card (NIC) may contain a data type engine that allows users to describe complex data layouts for sending and receiving data, including a multidimensional array with unique steps in each of the multiple dimensions. The data type engine can perform the gather / scatter operations described above, thereby avoiding the inefficient data copies used in previous software solutions. The data type engine can be located in the network interface card (NIC) or an application-specific integrated circuit (ASIC) of the NIC, including as part of an outbound packet engine and an inbound packet engine. The data type engine can generate a range of addresses and lengths based on a data type descriptor.The data type descriptor can be an input / output vector structure (IOVEC) with relative addressing, describing a multidimensional array defined by a count, a block size, and a stride. By using the data type descriptor as part of the data path, the data type engine can reduce communication time during data transmission over a network, both in the send path (e.g., when the outbound packet engine handles reads from host memory) and the receive path (e.g., when the inbound packet engine handles writes to host memory). This can lead to more efficient transmission of messages between network devices, which in turn can lead to more efficient use of resources, e.g., to support high-performance computing (HPC).
[0007] The described aspects of the data type engine can perform gather / scatter operations, addressing several challenges. A first challenge is ensuring that the representation of a data type covers common use cases while remaining compact. This can be difficult when dealing with code that forms arrays of structures. The described aspects can represent a data type with three strides, three counts, and a representation of a basic block. The system can represent the basic block with a length, a valid number of bytes, and a bitmask to indicate which bytes are valid. This data type can represent any three-dimensional (3D) array of C structures (up to 256 bytes per structure) and requires less than 64 bytes of storage for each data type.
[0008] A second challenge is the use of redundant memory resources during compression. Reading data from memory and storing the read data in a temporary local buffer before compressing the valid bytes can require redundant memory resources. The described aspects address this challenge by compressing 256 bytes with an arbitrary bitmask to capture only the valid bytes in only eight stages of 2-to-1 multiplexers. The system can also leverage the POPCNT instruction in x86 processors to quickly and efficiently encode additional information, thus reducing the logic required for each stage of compression.
[0009] A third challenge (as discussed here and below with regard to the Fig. described) is dealing with packets that arrive out of order. Since modern HPC networks can use adaptive routing to optimize overall throughput, it is possible that some packets arrive out of order. The arrival of out-of-order packets may require a non-linear progression through the data type description, which can be difficult or expensive given the varying size of the multidimensional array or the differently sized elements that are substructures of a larger data structure. These out-of-order packets can be of three types: packets received as a "put" (e.g., a write operation); request packets received as a "get" (e.g., a read operation) and for which a response must be generated; and packets received in response to a "get."In "put" operations, a first node may send data to a second node, and the data packets may arrive at the second node out of order. In "get" operations, a first node may request data from a second node, and the request packets may be received (and responded to) out of order at the second node, and the data packets as part of the response may be received out of order at the first node.
[0010] To handle the "Get" operations at the initiator, aspects of the described Data Type Engine (DTE) can pre-compute the starting state or context for each packet when the "Get" requests are formed, e.g., an 8-byte context for each 4-kilobyte (KB) packet. The system can store this initial state or context along with the other information required to process the "Get" response, so that the out-of-order response can use the stored state to initialize the DTE. To process the Get operations at the target, the software can pre-compute the context per response packet, e.g., an 8-byte context for each 4-KB packet if the data type defining the gather operation is specified. This context can be stored in the memory of the target host.The DTE at the destination, which is responsible for creating GET response packets, can be provided with information along with each GET request packet, allowing it to read the correct context from host memory when needed. The DTE at the destination can also cache the context after it has completed processing a GET response and use it to process a subsequent GET response of the same total GET.
[0011] To process put operations, the receiving side can provide the data type for receiving the put operation. Software associated with the system (e.g., the NIC ASIC) can precompute a small portion of DTE state or context (e.g., 8 bytes) corresponding to the beginning of each packet. To process an out-of-order packet, the system can combine a host memory lookup with the data type descriptor. The system can also optimize the target of the put operation to reuse state in the DTE from one packet to the next. Even with adaptive routing, packets usually arrive in the correct order. Reusing or storing state in the DTE can eliminate the need for a host memory lookup.Precalculating, caching, and retrieving the context for packets, as well as initializing the data type engine to handle out-of-order packets, is described below with reference to Figures 4-7.
[0012] In addition to addressing these challenges to improve the performance of data transfers over a network and support high-performance computing, the described system may also include other aspects. One aspect may concern the ordering of data type instructions (e.g., the installation of a data type descriptor in a DTE of a NIC) relative to payload instructions (e.g., the execution of a user read / write command). A data type must be created and set up, or installed, before it can be used by a payload or user instruction. Instructions may traverse the NIC via multiple paths, e.g., based on a series of parallel queues. The system may select the queue by hashing various fields in a instruction or an associated packet.As a result, both the command to install a data type descriptor in the NIC's DTE and the user command can have a flow hash, allowing the two commands to take the same path. Furthermore, the system can include an event indicating that a data type command has completed, as described below with reference to . Fig. The communication with the event engine shown in Figure 1 is shown. Once the data type command is complete, the installed data type descriptor can be used by many independent subsequent commands.
[0013] Another aspect of the described system may concern data type selection and memory efficiency. A command to begin a new message must specify the data type it wishes to use. However, specifying a complete data type descriptor with each command may not be efficient or practical. Furthermore, data types are often structured the same between processes on a given node that are part of the same user job. The described aspects may include a data type allocation mechanism that can improve both memory efficiency and the efficiency of new message commands. The software, e.g., the operating system (OS), may allocate data type space and provide the user with a data type key. The system may use a single key space for data type descriptors that can be shared among different processes.The hardware can provide an enforcement mechanism that ensures that only processes that are part of the same application can use a given key. The user can include the given key in a new message to specify which data type to use. The system can use a null entry for the data type key to indicate that a command or transfer does not contain a data type.
[0014] Another aspect of the described system may relate to the use of both absolute and relative addressing in the IOVEC structure used by the data type engine. Previous solutions may use absolute addressing in a gathering / scatter list to support use cases involving the kernel mode of the operating system. However, the interface for user-level software typically uses relative addressing, which can lead to limitations when using gathering / scatter engines through MPI. When using the IOVEC structure in the described aspects, the DTE can retrieve data from host memory based on a list of addresses and lengths using two different addressing modes. The first mode is absolute addressing, in which the addresses retrieved from host memory are the virtual addresses to be used, i.e.using a set of absolute address / length pairs in an IOVEC. The second mode is relative addressing, in which addresses retrieved from host memory are used as offsets from the virtual base address specified by the user in a new message. The offset can be obtained from an IOVEC (list of offsets and lengths) or from a data type descriptor.
[0015] Another aspect of the described system concerns the efficiency of storing data types, IOVECs, and instructions. To begin a new message, previous solutions add a pointer to an IOVEC to each instruction, which can increase the size of the instruction and reduce the efficiency of the command interface. The system can encode and set up or install IOVECs in the same way as other data types. However, some IOVEC use cases can only be used once; that is, the system creates the IOVEC, uses it for one instruction ("use once"), and then discards it. Because instruction ordering works in conjunction with the IOVEC approach, the instruction ordering logic described above can allow the system to maintain existing instruction formats while ensuring an efficient setup path for these one-time use IOVECs.
[0016] Thus, aspects of the described system can address the challenges described above by providing a data type engine (DTE) that generates a series of addresses and lengths based on a data type descriptor. The data type descriptor can represent a multidimensional array defined with a count, a block size, and a step size, as described below with respect to Fig. 2. The data type descriptor can also refer to an IOVEC structure with relative addressing. An overall architecture of a system (e.g., a NIC) that includes an outgoing transfer engine and an incoming transfer engine is described below with reference to Fig. 1, while a detailed diagram of an outgoing transfer machine with a DTE module in communication with other modules is described below with reference to Fig. 3 is described.
[0017] In addition, aspects of the described system can address the challenges of processing out-of-order packets by pre-computing a small (e.g., 8 bytes) state or context for each packet to initialize the DTE, and by caching the pre-computed context for later retrieval or use, which can facilitate out-of-order packet processing. A diagram of an Inbound Transfer Engine (IXE) is presented below in relation to Fig. 4, and a DTE module of an IXE that facilitates the processing of out-of-order packets is described below with respect to Fig. 5 described. Exemplary high-level architecture
[0018] Fig. Figure 1 shows a diagram of an architecture enabling a data type engine to support high-performance computing according to one aspect of the present application. The diagram of Fig. 1 may depict a network interface card (NIC) 100, which includes various components or modules, as well as interfaces and communications. At one end of the NIC 100, a host interface 110 is located between the components of the NIC 100 and a host. The host interface 110 may handle communications with the host (via a communications link 142). At the other end of the NIC 100, a port macro 132 is located between the NIC components and a network, through which communications with the network (via a communications link 186) may be handled.In general, the components on the left side of the diagram 100 belong to an outgoing path and can process send operations related to performing read operations from the host memory (indicated by the bold underlined communications 146, 172 and 184, which are described further below), while the components on the right side of the diagram 100 belong to an incoming path and can process receive operations related to performing write operations to the host memory (indicated by the bold underlined communications 188, 182 and 154).
[0019] The host interface 110 may receive data from the various NIC components via a system bus arbitration / host interface 112. The host interface 110 may also send a command list pointer or a command with data to a command queue 114 (via a communication link 140), which may include a metadata table (MD) 115. The command queue 114 may send portal and Ethernet operations (e.g., related to read operations in the transfer path) to an outbound transfer engine 116 (via a communication link 156), which may include an outbound buffer 117 for storing data related to the operations performed by the outbound transfer engine 116. The command queue 114 may also receive commands related to reading data, such as a direct memory access (DMA) command (via communication 144) as well as events from an event engine 118 (via communication 158).The command queue 114 may also transmit append commands to a list processing engine 122 (via communication 148).
[0020] The Outbound Transfer Engine 116 may use an instance of a data type engine (in Fig. 1 not shown) and can perform various operations to generate requests or commands related to DMA reads. The outbound transfer engine 116 can receive outgoing DMA data from the host memory (via communication 146 from the host interface 112), which data can be stored in the output buffer 117. A detailed description of the outbound transfer engine 116 is provided below with reference to Fig. 3 given.
[0021] Following the data type engine of the output transmission engine 116, which uses an IOVEC with relative addressing and describes a multidimensional array in the manner described herein, the output transmission engine 116 may issue the requests or commands to a pending command table (Transmit (TX)) 126 (via communication 172). The pending command table 126 may include a replay buffer 127. The pending command table 126 may send a completion message to the event program 118 (via communication 174) upon completion of a corresponding command. The pending command table 126 may transmit outgoing packets corresponding to read commands (via communication 184 to the port macro 132).
[0022] NIC 100 may receive incoming packets to be written to host memory (via communication 188). The incoming packets may be transferred 130 to a pending command table (Receive (RX)). Pending command table 130 may then send information to a message state table 124 (via communication 180) and a message to list processing engine 122 (via communication 178), which may include list entries 123. List processing engine 122 may send information to message state table 124 (via communication 168), which in turn may send information to an incoming transfer engine 120 (via communication 170). Incoming transfer engine 120 may include an input buffer 121.The list processing engine 122 may also send a storage or data type descriptor to the input transfer engine 120 (via communication 166) which is stored in a data type engine (not shown) of the input transfer engine 120.
[0023] The list processing engine 122 may further send events to the event engine 118 (via communication 164). The pending command table 130 may also transmit a payload associated with the incoming packets to the inbound transfer engine 120 (via communication 182), where the payload may be stored in the input buffer 121. The inbound transfer engine 120 may include an instance of a data type engine (in Fig. 1 not shown) and can perform various operations to generate requests or commands related to DMA write operations. Following the data type engine of the input transfer module 120, which uses an IOVEC with relative addressing and describes a multidimensional array in the manner described herein, the input transfer module 120 can transfer incoming DMA data to the host memory (via communication 154 to the host interface 112).
[0024] The inbound transfer system 120 may receive data from the host interface 112 (via communication 150) and may also send a completion message to the event engine 118 (via communication 162) when a corresponding command is completed. In addition to receiving completions from the pending command table 126 and the inbound transfer module 120 (via communications 174 and 162, respectively), the event module 118 may send event-related information to the host interface 112 (via communication 160). The pending command tables in the send and receive paths may communicate response status (tables 126 and 130 via communication 176), and the inbound transfer engine 120 may also communicate responses to the outbound transfer engine 116 (via communication 152).Just as the outbound transfer engine 116 can form a packet and the DEE of the outbound transfer engine 116 can form a plurality of reads, the list processing engine 122 can form an address to write a packet and pass that address to the DEE of the inbound transfer engine 120. The DEE of the inbound transfer engine 120 can form a plurality of write requests that include offsets from that address and corresponding lengths based on a given data type descriptor. It should be apparent to one of ordinary skill in the art that this mechanism can also be used for inbound processing of a packet. Overview of data type descriptors
[0025] In a typical NIC implementation, user software can issue a command specifying a virtual address and the length of data to be transferred. The NIC can receive the command and translate the virtual address. The NIC can then perform read accesses to the host memory (to transfer data, e.g., in a put operation at the initiator) or write accesses to the host memory (to receive data, e.g., in a put operation at the target). In conventional solutions, these read and write operations in host memory typically refer to large, contiguous blocks of data.
[0026] The described aspects provide a data type engine that generates a range of addresses and lengths based on a data type or a "data type descriptor." Data types can natively express data layouts in application memory, which can eliminate the need for resource-intensive copies. That is, a data type is a way to describe the native data layout in memory. Without a data type engine as described here, central processing units (CPUs), general purpose processing units (GPUs), and extended memory systems are used to perform the described data copies. However, because operations with these systems can be expensive (i.e., CPUs, GPUs, and extended memory systems are generally better suited for other tasks), using these systems can be inefficient.
[0027] An example of a data type descriptor form can be a multidimensional array descriptor, where the data type engine uses parameters to describe an array in host memory, as described below with respect to the matrix transposition example of Fig. 2. The multidimensional array can contain the following: a number of elements in each dimension, a size of a block to be transferred, and a step size in each dimension. The data type descriptor can also contain a handle to the old data type name and the new data type name, as described below with respect to the Fig. 2 and Fig. 4. Another example of a data type descriptor form can be an input / output (I / O) vector (IOVEC), where the DTE retrieves a list of addresses from host memory. The IOVEC can use either relative addressing or absolute addressing. With relative addressing, the DTE can retrieve data based on addresses that are offsets from a user-specified virtual base address, whereas with absolute addressing, the DTE can retrieve data based on addresses that are the virtual addresses to be used. Example code for an application with data types
[0028] Fig. 2 shows an example code excerpt 200 for an application using data types in accordance with one aspect of the present application. A section 210 may specify an "MPI_Type_contiguous" data type, which enables replication of a data type in contiguous locations. Sections 212, 214, and 216 may specify an "MPI_Type_vector" data type, which enables replication of a data type into memory locations with evenly distributed blocks, where each block can be obtained by concatenating the same number of copies of the old data type. The spacing between the blocks may be a multiple of the extent of the old data type.The "MPl_Type_vector" may denote a structure comprising a multidimensional array containing: a count corresponding to a number of blocks or elements in each dimension; a block size or length corresponding to a number of elements in each block; a stride in each dimension corresponding to a number of elements between the beginning of each block; an indicator of an existing or old data type; and a handle to a new data type.
[0029] Section 210 can create a three-element vector of floats and can be used to create descriptions of multiple pages (e.g., three) of a matrix. Using the data type specified in sections 214 and 216 can result in data being transferred in large chunks or many large chunks, where these chunks can be combined into one message, which can result in efficient transfer. However, using the data type specified in section 212 can result in 12 bytes being gathered / scattered from many locations, which may not be as efficient as using the data type of sections 214 and 216. When comparing NIC performance to the performance of CPUs and GPUs, the data type engine in the NIC can operate more efficiently with data type descriptors when data is transferred in larger chunks (e.g., 128-byte segments) rather than smaller chunks (e.g., 8-byte or 12-byte segments). Outbound Packet Engine and the transmission path
[0030] As above with regard to Fig. As described in Section 1, the data type engine can be integrated into both the send and receive logic in the outbound packet engine and the inbound packet engine, respectively. Fig. 3 shows a diagram 300 of an outbound packet engine 301 with a data type engine as part of the transmit path, according to one aspect of the present application. The outbound packet engine (OXE) 301 may include: a packet formation module 302; a data type module (DTE) 304; a direct memory access (DMA) module 306; and a post-processing module 308. The OXE 301 may accept data from instructions 320 as input and output data packets to a reliability stage 322. In addition, the OXE 301 may communicate with an address translation unit (ATU) 310 and host memory 312.
[0031] During operation, OXE 301 may receive a command, e.g., from a command queue 320 (via communication 330). The command may be received via a message passing interface (MPI) and may be a command to read data from host memory. In a standard path (i.e., when the data type descriptor is not present, thus bypassing the DTE), packet formation module 302 may form a DMA instruction specifying a header, address, and length associated with the data to be read from host memory, referred to as a "non-DTE DMA instruction." This standard non-DTE DMA instruction may be sent to DMA module 306 (via communication link 334). The DMA module 306 may retrieve the appropriate address translations from the ATU 310, for example, by sending a virtual address 338 to the ATU 310 and receiving a physical address 340 from the ATU 310.The DMA module 306 may then issue one or more read requests to the host memory 312 (via communication 342) and receive the corresponding read response(s) from the host memory 312 (via communication 344). The DMA module 306 may transmit the read responses to the post-processing module 308 (via communication 346), where the processed data may be transmitted as output to the reliability stage 322 (via communication 348).
[0032] If the data type descriptor is present in the instruction (e.g., as a handle to a previously stored data type descriptor), the packet formation module 302 may form a DMA instruction that specifies or references the data type descriptor as well as a base address and a length associated with the data to be read from host memory, referred to as a "DTE instruction." This DTE instruction may be sent to the DTE module 304 (via communication 332). The DTE module 304 may detect or determine that the data type descriptor specified in the DTE instruction matches a data type descriptor stored in the DTE module 304, e.g., as previously configured and stored by a user. Using the identified matching data type descriptor, the DTE module 304 may generate a plurality of DMA read instructions with offsets from the base address and corresponding lengths.The DTE module 304 may then transmit these generated DMA read commands to the DMA module 306 (via communication 336). The DMA module 306 may convert the DMA read commands into read requests and perform the address translation (if necessary) and read fetch operations described above with respect to communications 338, 340, 342, and 344, as well as the subsequent operations with respect to the post-processing module 308 and the reliability stage 322. Example IXE and DTE of IXE: Use of precomputed and cached contexts to facilitate out-of-order packet processing by the data type engine
[0033] Fig. 4 shows a diagram 400 of an inbound transfer engine (IXE) 402 that facilitates out-of-order packet processing according to one aspect of the present application. A request 432 or a response 434 may be received by IXE 402 (via communication paths 442 and 444, respectively). Diagram 400 focuses on the request path. Packet buffer 420 may receive the request (after the required decoding of the request, not shown) via communication 442. Packet buffer 420 may send the required information to context determining unit 430, e.g., for a corresponding packet of the request or message (via communication 446). The respective packet may be associated with a data type descriptor stored in a data type engine of the NIC. Context determination unit 430 may determine whether the respective packet is associated with a previously received message or a new message.Based on this determination, the context determination unit 430 may generate a data type handle corresponding to the data type descriptor for the respective packet and an offset indicating a position of the respective packet within the plurality of packets. The context determination unit 430 may transmit the generated data type handle and offset to a write dispatcher or write multiplexer (WrMux) 412 of IXE 402 (via communication 448). The write dispatcher 412 may transmit the generated data type handle and offset as DMA write instructions to a data type engine (DTE) 414 (via communication 450). DTE 414 may determine whether a context for the respective data type handle and offset is cached. If the context is cached, DTE 414 may initialize based on the cached context and further generate a plurality of DMA write instructions with addresses and lengths.
[0034] The DEE 414 may issue the generated plurality of DMA write commands to a write dispatcher (write request queues (WRQs)) 416, which may convert the DMA commands into a plurality of write requests, e.g., by obtaining the required addresses from an address translation unit (ATU) 438 via an address translator (ATR) 418 of the IXE 402 (via communication links 454 and 456, respectively). The write dispatcher 416 may forward the plurality of write requests to a DMA write controller 420 (via communication 464), which in turn may forward the requests to a host memory, e.g., via a PCIe interface 440 (via communication 466).
[0035] If the context is not cached, the DTE 414 may retrieve the context from host memory, which may involve obtaining the appropriate address information from the ATU 438 via ATR 418 from the IXE 402 (via communication 454 or 452, respectively). After receiving the appropriate address information, the DTE 414 may retrieve the context by sending a request to the host memory via the PCIe interface 440 (via communication 460) and receiving a response from the host memory.
[0036] Fig. 5 shows a diagram 500 of an inbound transfer engine (IXE), including an IXE datatype engine 502 that facilitates out-of-order packet processing, according to one aspect of the present application. The IXE may include IXE DTE 502 (indicated by the dashed outline), as well as: write dispatcher (WrMux) 412; an ATR 522; write request queues 530; and write request engines 532 (indicated by the bold outlines). IXE 502 may communicate with an ATU 540 and host memory 542 that are not part of the IXE or IXE DTE 502 (as indicated by the dashed lines). Host memory 542 may represent a PCIe arbitrator (or PARB) that communicates with host memory.
[0037] During operation, the write dispatcher 412 may send a packet with a DEE handle and an offset to the IXE DEE 502 (via a communication 550), which is received by an allocation / bypass unit 502. If no data type handle is specified in the received packet (e.g., as "none"), the allocation / bypass unit 502 may send the packet directly to the write request queues 530 for immediate processing (via a communication 554), as described above with respect to the communication 334 of Fig. 3. If a data type handle is specified in the received packet, the allocation / bypass unit 502 may allocate the packet in an instruction RAM 508 (via communication 552) and specify the instruction in an instruction tracker 504. The instruction tracker 504 may track received instructions and transfer the packet to a DTE pipeline unit 506 (via communication 558), which may be responsible for gathering the information to process the packet. The DTE pipeline 506 may determine if a previous packet containing the same message was recently received or if the context for the packet has been cached.
[0038] Based on the data type handle, the DEE pipeline 506 can read the corresponding data type from a descriptor memory 516 (via communication 560). The descriptor memory 516 can contain both a non-byte mask descriptor memory and a byte mask descriptor memory. The byte mask portion of a descriptor can refer to a value with a number of bits that can be used as a byte mask for data of up to a predetermined number of bytes. For example, during a compaction process, each bit in an X-byte mask can correspond to a byte in an X-byte chunk or block of data. The non-byte mask portion of a descriptor can contain other information, such as loop counts and strides (for a derived data type) or a pointer to an IOVEC in host memory (for an IOVEC data type), etc.created by a user and stored in descriptor memory 516 for later use or retrieval by the DTE.
[0039] When context is cached, DTE pipeline 506 may retrieve the context from a hot context RAM 512 or a cold context RAM 514 (via communication 564 or 566, respectively). In general, hot context RAM 512 may temporarily store the context while a packet is being processed. The context may be stored in cold context RAM 514 when processing of a packet is complete and processing of a subsequent packet of the same overall message can begin. For example, DTE pipeline 506 may receive a current value for each dimension of a multidimensional array (e.g., the current loop value) and the starting position of the respective packet within a block of host memory. DTE pipeline 506 can then immediately begin processing the packet.The DTE pipeline 506 may use the context retrieved from the cache to initialize the state for the packet and may forward the packet to a queue 518 (via communication 568) to be processed by a data type processor 520. The DT processor 520 may generate the multiple instructions, e.g., the multiple write requests with addresses and lengths, based on the pre-computed and cached context.
[0040] If the context is not cached, the DEE pipeline 506 can retrieve the context from host memory using the received data type descriptor. The DMA command and the information from the descriptor are passed to the DT processor 520 via queue 518 (via communication 568). Unlike usual processing (i.e., generating DMA writes), a DMA read transaction is generated here to retrieve the context from host memory.
[0041] The DT processor 520 may obtain the appropriate address information from the ATU 540 via ATR 522 and an address translation queue (ATQ) 421 (via communication 574 and 572, respectively). After receiving the appropriate address information, the DEE pipeline 506 may retrieve the context from the host memory 542 via a memory data read (MDR) 524 (e.g., via communication 576 and 584). MDR 524 may transfer the obtained context information (including the current loop counter values) to an IOVEC / Context Fast Forward (FF) buffer (ICB) 510 (via communication 582).For example, for a three-dimensional array with dimensions [X, Y, Z], the obtained context may be a derived data type fast-forward entry and may be 8 bytes (ordered from bits 63:0), which may include: unused bits (bits 63:56); a current byte position with a block (bits 55:48); a current element in the Z dimension (bits 47:32); a current element in the Y dimension (bits 31:16); and a current element in the X dimension (bits 15:0).
[0042] When the ICB 510 receives this information, the instruction trace 504 may manage the sending of the waiting instruction to the DTE pipeline 506 (via communication links 556 and 558). The DTE pipeline 506 may read the obtained context directly from the ICB 510 (via communication 562) and begin processing the packet as described above, i.e., by using the context retrieved from the ICB 510 to initialize the state for the packet and sending the packet to the queue 518 for processing by the DT processor 520, where the DT processor 520 may generate the multiple instructions.
[0043] Once the DT processor 520 has generated the multiple instructions with addresses and lengths (either using the cached context or the context obtained from host memory), the DT processor 520 may send the multiple instructions via communication 586 to the write request queues 530 (which are assigned to the write dispatchers (WRQs) 416 of Fig. 4). Write request queues 530 may also receive information related to addresses and packet information via communications 570, 572, 574, 576, 578, 580, and 586. Write request engines 532 may transmit the multiple instructions (i.e., write requests) to host memory 542 and receive the corresponding response (via communication 588). Methods for facilitating out-of-order packet processing in a data type engine
[0044] Fig. 6 shows a flowchart 600 illustrating a method (in a system) that facilitates out-of-order packet processing in a data type engine according to an aspect of the present application and as described above with respect to Fig. 4 and Fig. 5. During operation, the system receives through a network interface card (NIC) a plurality of packets corresponding to a write request, the packets being associated with a data type descriptor stored in a data type engine of the NIC, each packet being associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory (operation 602, as described above with respect to request 432 and communication 442 of Fig. 4). The system generates for a respective packet a data type handle corresponding to the data type descriptor and an offset indicating a position of the respective packet within the plurality of packets (operation 604, as described above with respect to the context determination unit 430 of Fig. 4). The system determines, based on the data type handle and offset, whether a context is cached for the respective packet (operation 606). If the context is cached (decision 608), the system initializes the data type engine based on the cached context (operation 610). The system generates, through the data type engine, a plurality of write requests comprising addresses and lengths based on the pre-computed and cached context, thereby enabling the NIC to process packets out of order (OOO) based on the pre-computed and cached context (operation 612, as described above with respect to DTE 414 of Fig. 4 and DT processor 520 from Fig. 5). The system caches the current context when processing for the respective packet is complete (operation 614). The operation returns.
[0045] If the context is not cached (decision 608), the system retrieves the context from host memory (operation 620). The system initializes the data type engine based on the retrieved context (operation 622). The system generates, through the data type engine, a plurality of write requests comprising addresses and lengths based on the retrieved context, thereby enabling the NIC to process out-of-order (OOO) packets based on the pre-computed and retrieved context (operation 624, as described above with respect to DTE 414 of Fig. 4 and DT processor 520 from Fig. 5). The system caches the current context when processing for the respective packet is complete (operation 614). The operation returns.
[0046] Flowchart 600 illustrates a method for processing a write request (e.g., a put operation at the destination by the DTE of an inbound transfer engine). As described here, a similar method may be used for processing a read request (e.g., a get operation at the destination by the DTE of an outbound transfer engine). The primary difference is that the IXE DTE generates write commands to write to the host memory, while the OXE DTE generates read commands to read from the host memory. Therefore, in flowchart 600, the received packets may correspond to a read request (e.g., similar to operation 602), and the DTE may generate a plurality of read requests comprising addresses and lengths based on the pre-computed and cached context (e.g.,similar to operations 612 and 624), with the read requests being sent to a DMA module to be read or retrieved from the host memory (as described above with respect to . Fig. 3 described). Device that facilitates a data type engine to support high-performance computing
[0047] Fig. 7 shows an apparatus 700 that facilitates out-of-order processing of packets in a data type engine in accordance with one aspect of the present application. The apparatus 700 may represent a network interface card (NIC) and may include: a command unit 702 for receiving a command to write data to host memory and determining whether the command specifies a data type descriptor stored in a data type engine of the NIC; a buffer management unit 704 for storing the incoming packet and information related to the corresponding command; a context generation unit 706 for generating a data type handle for a respective packet corresponding to the data type descriptor and an offset specifying a position of the respective packet within the plurality of packets;a data type engine management unit 708 for determining, based on the data type handle and the offset, whether a context is cached for the respective packet, and for further initializing the data type engine based on a pre-computed cached or obtained context, and further obtaining the context from host memory if the context is not found in the cache; a cache management unit 710 for storing pre-computed contexts for up to a predetermined number of messages; and a request generation unit 712 for generating, based on the pre-computed cached or obtained context, a plurality of write requests including addresses and lengths, thereby enabling the NIC to process packets out of order based on the pre-computed cached or obtained context.
[0048] The command unit 702 can, for example, provide the command follower 504 with Fig. 5. The buffer management unit 704 may, for example, correspond to the packet buffer 420 of Fig. 4. The context generation unit 706 can, for example, correspond to the context determination unit 430 of Fig. 4. The data type engine management unit 708 may be assigned to the DTE 414 of Fig. 4 and the IXE DTE 502 from Fig. 5. The cache management unit 710 may be assigned to the hot context RAM 512 and the cold context RAM 514 of Fig. 5. The request generation unit 712 may be provided to the DT processor 520 of Fig. 5. The units described as part of the device 700 may be the units described above with respect to the Fig. 1, Fig. 4, Fig. 5 and Fig. 6 are described.
[0049] In general, the disclosed aspects provide a system, method, apparatus, and non-transitory storage medium for facilitating out-of-order processing of packets in a data type engine. In one aspect, the system receives, through a network interface card (NIC), a plurality of packets corresponding to a read request or a write request, the packets associated with a data type descriptor stored in a data type engine of the NIC, each packet associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory (e.g., as described above with respect to request 432 and communication 442 of Fig. 4). The system generates for each packet a data type handle corresponding to the data type descriptor and an offset indicating a position of the respective packet within the plurality of packets (e.g., as described above with respect to the context determination unit 430 of Fig. 4). The system determines, based on the data type handle and the offset, that a context is cached for the respective packet. The system initializes the data type engine based on the cached context. The system generates, through the data type engine, a plurality of read or write requests comprising addresses and lengths based on the pre-calculated and cached context (e.g., as described above with respect to DTE 414 of Fig. 4 and DT processor 520 from Fig. 5), which enables the NIC to process out-of-order packets based on the pre-computed and cached context. The system saves the current context when processing of the respective packet is complete.
[0050] In a variation of this aspect, the system responds to determining that the context for the respective packet is not cached by: retrieving the context from host memory; caching the retrieved context; initializing the data type engine based on the retrieved context; and generating the majority of write requests with addresses and lengths based on the retrieved context. The system caches the current context when processing of the respective packet is complete.
[0051] In another variation of this aspect, the system determines whether the respective packet is associated with a previously received message or a new message. Depending on whether the packet belongs to the previously received message or the new message, the system generates the offset.
[0052] In another variant, the generation of the offset is based on information in the respective packet.
[0053] In another variant, the data type descriptor comprises at least one of the following elements: a representation of a multidimensional array and a reference to an input / output vector (IOVEC) that specifies addresses and lengths of data to be read from or written to host memory. The representation of the multidimensional array includes: a number of elements in each dimension; a size of a block to be transferred; and a step size in each dimension.
[0054] In another variant, the respective packet is received out of order within the plurality of packets, and the initialization of the data type engine based on the cached context causes the out of order packet to be processed correctly based on the generated data type handle and the offset.
[0055] In another variant, a corresponding context can be cached for a predetermined number of messages at once.
[0056] In another variant, the pre-computed context is stored in the host memory.
[0057] Another aspect is a non-transitory, computer-readable storage medium that stores instructions that, when executed by a computer, cause the computer to perform the method described herein.
[0058] Another aspect is provided by a network interface card (NIC) (e.g., as described above with reference to NIC 100 of Fig. 1, IXE 402 of Fig. 4 and IXE DTE 502 from Fig.5) comprising a command unit for receiving, by a network interface card (NIC), a plurality of packets corresponding to a read request or a write request, wherein the packets are associated with a data type descriptor stored in a data type engine of the NIC, wherein each packet is associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory. The NIC further comprises a context generation unit for generating, for a respective packet, a data type handle corresponding to the data type descriptor and an offset specifying a position of the respective packet within the plurality of packets. The NIC further comprises a cache management unit for determining, based on the data type handle and the offset, that a context for the respective packet is cached.The NIC further comprises a data type engine management unit to initialize the data type engine based on the cached context. The NIC further comprises a request generation unit to generate, by the data type engine, a plurality of read or write requests including addresses and lengths based on the pre-computed and cached context, thereby enabling the NIC to process out-of-order packets based on the pre-computed and cached context. The cache management unit is further to cache a current context when processing of the respective packet is complete. In a variation of this aspect, the pre-computed context is stored in at least one of the following memories: the host memory, a random access memory (RAM) for the hot context, and a random access memory (RAM) for the cold context.
[0059] The above descriptions of aspects are for illustrative and descriptive purposes only. They are not intended to be exhaustive and do not limit the aspects described herein to the forms disclosed. Accordingly, many modifications and variations will be apparent to those skilled in the art. Furthermore, the above disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.
Claims
[1] A computer-implemented method comprising: Receiving a plurality of packets corresponding to a read request or a write request by a network interface card (NIC), the packets being associated with a data type descriptor stored in a data type engine of the NIC, wherein each packet is associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory; for a respective packet, generating a data type handle corresponding to the data type descriptor and an offset indicating a position of the respective packet within the plurality of packets; Determine that a context is cached for the respective packet based on the data type handle and offset; Initialize the data type engine based on the cached context; Generating a plurality of read requests or write requests comprising addresses and lengths by the data type engine based on the pre-computed and cached context, thereby enabling the NIC to process out-of-order packets based on the pre-computed and cached context; and Caching a current context when processing of the respective packet is complete. [2] The method of claim 1, further comprising: in response to determining that the context for the respective packet is not cached: Retrieve the context from the host memory; Caching the retrieved context; Initialize the data type engine based on the retrieved context; Generating the plurality of write requests with addresses and lengths based on the retrieved context and Caching a current context when processing of the respective packet is complete. [3] The method of claim 1, further comprising: Determine whether the respective packet is associated with an already received message or a new message; and Generate the offset based on whether the packet is associated with the already received message or the new message. [4] The method of claim 3, wherein the generation of the offset is further based on information in the respective packet. [5] The method of claim 1, wherein the data type descriptor comprises at least one of the following: a representation of a multidimensional array that includes: a number of elements in each dimension; a size of a block to be transferred and a step size in each dimension; and a reference to an input / output vector (IOVEC) that specifies addresses and lengths of data to be read from or written to host memory. [6] Method according to claim 1, wherein the respective packet within the plurality of packets is received out of order, and where initializing the data type engine based on the cached context causes the out-of-order packet to be processed correctly based on the generated data type handle and offset. [7] The method of claim 1, wherein a respective context can be cached for a predetermined number of messages at a time. [8] The method of claim 1, wherein the pre-computed context is stored in host memory. [9] A non-transitory, computer-readable storage medium storing instructions which, when executed by a computer, cause the computer to perform a method, the method comprising: Receiving a plurality of packets corresponding to a read request or a write request by a network interface card (NIC), the packets being associated with a data type descriptor stored in a data type engine of the NIC, wherein each packet is associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory; for a respective packet, generating a data type handle corresponding to the data type descriptor and an offset indicating a position of the respective packet within the plurality of packets; Determine that a context is cached for the respective packet based on the data type handle and offset; Initialize the data type engine based on the cached context; Generating a plurality of read requests or write requests comprising addresses and lengths by the data type engine based on the pre-computed and cached context, thereby enabling the NIC to process out-of-order packets based on the pre-computed and cached context; and Caching a current context when processing of the respective packet is complete. [10] The non-transitory computer-readable storage medium of claim 9, wherein the method further comprises: in response to determining that the context for the respective packet is not cached: Retrieve the context from the host memory; Caching the retrieved context; Initialize the data type engine based on the retrieved context; Generating the plurality of write requests with addresses and lengths based on the retrieved context and Caching a current context when processing of the respective packet is complete. [11] The non-transitory computer-readable storage medium of claim 9, wherein the method further comprises: Determine whether the respective packet is associated with an already received message or a new message; and Generate the offset based on whether the packet is associated with the already received message or the new message. [12] Non-transitory computer-readable storage medium according to claim 11, whereby the generation of the offset is still based on information in the respective packet. [13] The non-transitory computer-readable storage medium of claim 9, wherein the data type descriptor comprises at least one of the following: a representation of a multidimensional array that includes: a number of elements in each dimension; a size of a block to be transferred and a step size in each dimension; and a reference to an input / output vector (IOVEC) that specifies addresses and lengths of data to be read from or written to host memory. [14] Non-transitory computer-readable storage medium according to claim 9, wherein the respective packet within the plurality of packets is received out of order, and where initializing the data type engine based on the cached context causes the out-of-order packet to be processed correctly based on the generated data type handle and offset. [15] The non-transitory computer-readable storage medium of claim 9, wherein the pre-computed context is stored in host memory. [16] A network interface card comprising: a command unit for receiving, through a network interface card (NIC), a plurality of packets corresponding to a read request or a write request, the packets being associated with a data type descriptor stored in a data type engine of the NIC, wherein each packet is associated with a pre-computed context specifying a value for each dimension of a multi-dimensional array and a starting position of the respective packet within a block of host memory; a context generation unit for generating, for a respective packet, a data type handle corresponding to the data type descriptor and an offset indicating a position of the respective packet within the plurality of packets; a cache management unit for determining, based on the data type handle and the offset, that a context for the respective packet is stored in the cache; a data type engine management unit for initializing the data type engine based on the cached context and a request generation unit for generating, by the data type engine, a plurality of read requests or write requests comprising addresses and lengths based on the pre-computed and cached context, thereby enabling the NIC to process out-of-order packets based on the pre-computed and cached context; and wherein the cache management unit is further configured to cache a current context when processing for the respective packet is completed. [17] Network interface card according to claim 16, wherein, in response to the cache management unit determining that the context for the respective packet is not cached: the data type engine management unit is further configured to retrieve the context from the host memory; the cache management unit is further configured to cache the retrieved context; the data type engine management unit is further configured to initialize the data type engine based on the retrieved context; the request generation unit is further configured to generate the plurality of write requests comprising addresses and lengths based on the retrieved context; and wherein the cache management unit is further configured to cache a current context when processing for the respective packet is completed. [18] Network interface card according to claim 16, wherein the command unit is further configured to determine whether the respective packet is associated with an already received message or a new message; and wherein the context generation unit is further configured to generate the offset based on whether the packet is associated with the already received message or the new message, and further based on information in the respective packet. [19] Network interface card according to claim 16, wherein the respective packet within the plurality of packets is received out of order, and where initializing the data type engine based on the cached context causes the out-of-order packet to be processed correctly based on the generated data type handle and offset. [20] The network interface card of claim 16, wherein the pre-computed context is stored in the host memory, a hot context random access memory (RAM) and / or a cold context random access memory (RAM).