Computer-implemented methods, network interface cards, and storage media

Abstract

A data type engine for supporting high-performance computing is provided. A method and apparatus for facilitating the data type engine (DTE) to support high-performance computing are also provided. A network interface card (NIC) receives a command via a messaging interface for reading data from host memory. The NIC determines a first data type descriptor stored in the NIC in accordance with the command. The NIC forms a data packet based on the command, indicating a base address and length associated with the data to be read from host memory, and passes the data packet to the DTE. The DTE generates multiple read requests based on the first data type descriptor, the multiple read requests including an offset relative to the base address and a corresponding length. The DTE passes the multiple read requests to a direct memory access module, thereby allowing the NIC to access host memory while clearing copies of the data on the host during command transmission over the network.

Description

Technical Field

[0001] This disclosure generally pertains to data type engines. Background Technology

[0002] User applications can send data residing in a non-contiguous buffer. Similarly, these same user applications can receive data in a non-contiguous buffer. In a traditional system, a typical process might involve: copying or packing data into a continuous transmit buffer; transmitting the data; and copying or unpacking the data from the continuous receive buffer to the appropriate application location. These data copying operations (i.e., packing and unpacking) occur in software and are therefore serial with data transmission communication, which can significantly increase communication time. Summary of the Invention

[0003] According to one aspect of this disclosure, a computer-implemented method is provided, comprising: receiving a command via a messaging interface card (NIC) for reading data from host memory; determining that the command indicates a first data type descriptor stored in a data type engine of the NIC; forming a data packet based on the command indicating a base address and a length associated with the data to be read from the host memory; passing the data packet to the data type engine; generating a plurality of read requests by the data type engine based on the first data type descriptor, the plurality of read requests including an offset relative to the base address and a corresponding length; and passing the plurality of read requests to a direct memory access module of the NIC, thereby allowing the NIC to access the host memory while clearing a copy of the data in the host memory during transmission of the command over a network.

[0004] According to one aspect of this disclosure, a network interface card (NIC) is provided, comprising: a command module configured to receive a command for reading data from host memory via a messaging interface; a packet forming module configured to: determine whether the command indicates a first data type descriptor stored in a first data type engine of the NIC; in response to determining that the command indicates the first data type descriptor, form a packet based on the command indicating a base address and a length associated with the data to be read from the host memory; and pass the packet to the first data type engine; the first data type engine configured to: generate a plurality of read requests based on the first data type descriptor, the plurality of read requests including an offset relative to the base address and a corresponding length; and pass the plurality of read requests to a direct memory access (DMI) module of the NIC, thereby allowing the NIC to access the host memory while clearing a copy of the data in the host memory during transmission of the command over the network; and the DMI module configured to issue the plurality of read requests to the host memory to obtain the data indicated in the command.

[0005] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores instructions, when executed by a computer, cause the computer to perform a method, the method comprising: storing one or more data type descriptors created by a user associated with the computer in a data type engine of a network interface card; receiving, via a messaging interface, a command for reading data from host memory by the network interface card; determining whether the command indicates a first data type descriptor stored in the data type engine of the network interface card; and, in response to determining that the command indicates the first data type descriptor stored in the data type engine of the network interface card, performing the following operations: forming a data packet based on the command indicating a base address and a length associated with the data to be read from the host memory; passing the data packet to the data type engine; generating a plurality of read requests by the data type engine based on the first data type descriptor, the plurality of read requests including an offset relative to the base address and a corresponding length; and passing the plurality of read requests to a direct memory access module of the network interface card, thereby allowing the network interface card to access the host memory while clearing a copy of the data in the host memory during transmission of the command over the network. Attached Figure Description

[0006] Figure 1 The diagram illustrates an architecture for a data type engine that facilitates high-performance computing according to one aspect of this application.

[0007] Figure 2 An exemplary code excerpt of an application using data types according to one aspect of this application is illustrated.

[0008] Figure 3 The diagram illustrates an outbound packet engine according to one aspect of this application, which has a data type engine as part of the transmission path.

[0009] Figure 4 An example of matrix transpose using a Fast Fourier Transform (FFT) with a data type descriptor according to one aspect of this application is illustrated.

[0010] Figure 5 A flowchart illustrating a method for promoting a data type engine to support high-performance computing, according to one aspect of this application, is presented.

[0011] Figure 6 The illustration shows an apparatus for promoting a data type engine to support high-performance computing according to one aspect of this application.

[0012] In these accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0013] The following description is presented to enable any person skilled in the art to make and use the aspects and examples, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects and applications without departing from the spirit and scope of this disclosure. Therefore, the aspects described herein are not limited to those shown, but are intended to be consistent with the maximum scope of the principles and features disclosed herein.

[0014] User applications can send data residing in a non-contiguous buffer. Similarly, these same user applications can receive data in a non-contiguous buffer. In a traditional system, a typical process might involve: copying or packing data into a continuous transmit buffer; transmitting the data; and copying or unpacking the data from the continuous receive buffer to the appropriate application location. These data copying operations (i.e., packing and unpacking) occur in software and are therefore serial with data transmission communication, which can significantly increase communication time.

[0015] Current software interfaces allow users to represent data layouts in their native form. A commonly used software interface is the Message Passing Interface (MPI), where users can provide a base address to initiate a transfer and the data type applied to that transfer. MPI also provides a mechanism called "derived data types," which can describe the data layout in host memory and use that description for certain MPI calls (e.g., MPI_Send and MPI_Recv). Historically, using derived data types in MPI can have resulted in poor performance because the library must interpret the description to perform copying to and from internal buffers for the MPI_Send and MPI_Recv commands, respectively. Therefore, packing and unpacking operations are still performed in software and are still serial with communication, which can lead to overall performance inefficiency for data-related operations such as data collection / distribution.

[0016] The described aspects provide a system for addressing these problems by providing a Data Type Engine (DTE) in hardware that performs collection / distribution operations, thereby eliminating the inefficient data copying used in previous software solutions. The Data Type Engine can reside in a Network Interface Card (NIC) or its Application-Specific Integrated Circuit (ASIC), as part of both the outbound and inbound packet engines. The Data Type Engine can generate a series of addresses and lengths based on a Data Type Descriptor. The Data Type Descriptor can be an Input / Output Vector (IOVEC) structure with relative addressing and can describe a multidimensional array defined by counts, block sizes, and strides. By using a Data Type Descriptor as part of the data path, the Data Type Engine in both the transmit path (e.g., read operations from host memory handled by the outbound packet engine) and the receive path (e.g., write operations to host memory handled by the inbound packet engine) can lead to reduced communication time involved in data transmission over the network. This can result in more efficient message delivery between network entities, which in turn can lead to more efficient resource utilization, such as supporting high-performance computing.

[0017] The aspects described by the data type engine can perform collect / distribute operations while addressing several challenges. The first challenge is ensuring that the representation of the data type covers common use cases while remaining compact. This can be difficult when dealing with code that constructs arrays of structures. The described aspects can represent data types using a representation of three strides, three counts, and basic blocks. The system can use length, number of valid bytes, and a bitmask to represent basic blocks to indicate which bytes are valid. This data type can represent arbitrary three-dimensional (3D) arrays of C structures (up to 256 bytes per structure), with each data type using less than 64 bytes of storage.

[0018] The second challenge involves using redundant memory resources as part of the compression process. Reading data from memory and placing it into a temporary local buffer before compressing the valid bytes may require redundant memory resources. The described aspect addresses this challenge by compressing 256 bytes with an arbitrary bitmask to capture the valid bytes only in the eight stages of a 2-to-1 multiplexer. The system can further leverage the POPCNT instruction in the x86 processor to encode additional information quickly and efficiently, reducing the amount of logic required for each stage of compression.

[0019] The third challenge involves handling out-of-order packets. Because modern HPC networks can utilize adaptive routing to optimize overall throughput, some packets may arrive out of order. These out-of-order packets can be of two types: packets received as a "Put" operation (e.g., a write operation); and packets received in response to a "Get" operation (e.g., a read operation). For a Put operation, a first node can send data to a second node, and the packet may arrive at the second node out of order. For a Get operation, a first node can request data from a second node, and the packet may be received out of order at the first node as part of the response.

[0020] To handle Get operations, the described Data Type Engine (DTE) can pre-calculate the initial state of each packet when a Get request is formed. The system can store this initial state along with other information needed to process the Get response, allowing out-of-order responses to initialize the DTE using the stored state.

[0021] To handle Put operations, the receiving side can provide the data type used to receive Put operations. Software associated with the system (e.g., a NIC ASIC) can pre-compute a small amount of DTE state (e.g., 8 bytes) corresponding to the start of each packet. To handle out-of-order packets, the system can combine lookups from host memory with data type descriptors. The system can further optimize the target or destination of the Put operation to reuse the state in the DTE from one packet to the next. Even with adaptive routing, packets can arrive mostly in order. Reusing or maintaining the state in the DTE eliminates host memory lookups.

[0022] In addition to addressing these challenges to improve the performance of data transmission over the network and support high-performance computing, the described system may also include other aspects. One aspect may involve the ordering of data type commands (e.g., installing a data type descriptor in the NIC's DTE) relative to payload commands (e.g., executing a user read / write command). The data type must be created and set up or installed before being used by the payload or user command. Commands may, for example, be based on a series of parallel queues traversing the NIC via multiple paths. The system can select queues by hashing various fields in the executed command or associated data packet. Thus, both the command for installing the data type descriptor into the NIC's DTE and the user command can have stream hashes that allow both commands to take the same path. Furthermore, the system may include events indicating that a data type command has been completed, as follows regarding... Figure 1 The description illustrates the communication with the event engine. Once the data type command completes, the installed data type descriptor can be used by many independent subsequent commands.

[0023] Another aspect of the described system may relate to data type selection and storage efficiency. A command used to initiate a new message must indicate the data type it wants to use. However, including the full data type descriptor in every command may not be efficient or practical. Furthermore, the structure of data types may often be identical between processes on a given node (as part of the same user job). The described aspect may include a data type allocation mechanism that can lead to improved storage efficiency and the efficiency of new message commands. Software (e.g., an operating system (OS)) can allocate data type storage space and provide data type keys to the user. The system may use a single key space for data type descriptors, which can be shared between processes. Hardware may provide a enforcement mechanism that ensures that only processes as part of the same application can use a given key. The user can include a given key in a new message to specify the data type to use. The system may use empty entries for data type keys to indicate that a command or transmission does not include a data type.

[0024] Another aspect of the described system may involve using both absolute and relative addressing in the IOWEC structure used by the data type engine. Previous solutions could use absolute addressing in the collection / distribution list to support use cases involving OS kernel mode. However, user-level software interfaces typically use relative addressing, which can impose limitations when MPI uses the collection / distribution engine. When using the IOWEC structure in the described aspect, the DTE can retrieve data from host memory using two different addressing modes based on a list of addresses and lengths. The first mode is absolute addressing, in which the address retrieved from host memory is the virtual address to be used, i.e., using a set of absolute address / length pairs in the IOWEC. The second mode is relative addressing, in which the address retrieved from host memory is used as an offset relative to a virtual base address provided by the user in a new message, where the offset can be obtained from the IOWEC (an offset and length list) or from the data type descriptor.

[0025] Another aspect of the described system involves data type storage, IOWEC, and command efficiency. Previous solutions, to initiate a new message, could add a pointer to an IOWEC for each command, potentially increasing command size and reducing the efficiency of the command interface. The system can encode and set up or install IOWEC in the same way as other data types. However, some IOWEC use cases may only be used once; that is, the system creates an IOWEC, uses it for a single command (“use once”), and then discards it. Because command ordering works in conjunction with the IOWEC approach, the command ordering logic described above allows the system to maintain existing command formats while still ensuring efficient setup paths for these “use once” IOWECs.

[0026] Therefore, the various aspects of the described system can address the aforementioned challenges by providing a Data Type Engine (DTE) that generates a series of addresses and lengths based on data type descriptors. The data type descriptor can be an IOWEC structure with relative addressing and can describe a multidimensional array defined by a count, block size, and stride, as shown below. Figure 2 and Figure 4 As described. Below about Figure 1 The description includes the overall architecture of the system (e.g., NIC) including the outbound and inbound transport engines, while the following section discusses... Figure 3 A detailed diagram describing the outbound delivery engine of the DTE module that communicates with other modules.

[0027] Exemplary high-level architecture

[0028] Figure 1Figure 100 illustrates an architecture for facilitating high-performance computing through a data type engine according to one aspect of this application. Figure 100 depicts a network interface card (NIC) comprising various components or modules, interfaces, and communications. On one of the NICs, a host interface 110 exists between the NIC components and a host. The host interface 110 can handle communication with the host (via communication 142). At the other end, a port macro 132 exists between the NIC components and the network and can handle communication with the network (via communication 186). Typically, the components on the left side of Figure 100 belong to the outbound path and can handle transmit operations related to performing read operations from host memory (indicated by the thick lines of communications 146, 172, and 184 described below), while the components on the right side of Figure 100 belong to the inbound path and can handle receive operations related to performing write operations to host memory (indicated by the thick lines of communications 188, 182, and 154).

[0029] Host interface 110 can receive data from various NIC components via system bus arbitration / host interface 112. Host interface 110 can also (via communication 140) send command list pointers or commands with data to command queue 114, which may include metadata (MD) table 115. Command queue 114 can (via communication 156) send portal and Ethernet operations (e.g., related to read operations in the transmit path) to outbound transmission engine 116, which may include outbound buffer 117 for storing data related to operations performed by outbound transmission engine 116. Command queue 114 can also receive commands related to reading data, such as direct memory access (DMA) acquire commands (via communication 144) and events from event engine 118 (via communication 158). Command queue 114 can also (via communication 148) issue additional commands to list processing engine 122.

[0030] Outbound transmission engine 116 may include a data type engine ( Figure 1 An instance (not shown) can be provided, and various operations can be performed to generate requests or commands related to read DMA operations. The outbound transfer engine 116 can receive outbound DMA data from host memory (via communication 146 from host interface 112), which can be stored in output buffer 117. The following is about... Figure 3 Provides a detailed description of outbound transfer engine 116.

[0031] After the data type engine of outbound transmission engine 116 uses IOWEC with relative addressing and describes the multidimensional array in the manner described herein, outbound transmission engine 116 can output a request or command to a pending command table (Send (TX)) 126 (via communication 172). Pending command table 126 may include a retry buffer 127. Pending command table 126 may send a completion notification to event engine 118 upon completion of the corresponding command (via communication 174). Pending command table 126 may transmit outbound data packets corresponding to read commands (via communication 184 to port macro 132).

[0032] The NIC can receive inbound data packets to be written to host memory (via communication 188). The inbound data packets can be sent to a pending command table (receive (RX)) 130. The pending command table 130 can then send information (via communication 180) to a message status table 124 and (via communication 178) to a list processing engine 122, which may include list entries 123. The list processing engine 122 can send information (via communication 168) to the message status table 124, which can send information (via communication 170) to the inbound delivery engine 120. The inbound delivery engine 120 may include an input buffer 121. The list processing engine 122 can also send memory or data type descriptors (via communication 166) to the inbound delivery engine 120 for storage in the data type engine (not shown) of the inbound delivery engine 120. The list processing engine 122 can further send events (via communication 164) to an event engine 118. The pending command list 130 can also (via communication 182) transmit payloads associated with inbound data packets to the inbound delivery engine 120, the payloads of which can be stored in the input buffer 121. The inbound delivery engine 120 may include a data type engine (…). Figure 1 An instance (not shown) can be generated, and various operations can be performed to generate requests or commands related to write DMA operations. After the data type engine of the inbound transfer engine 120 uses IOWEC with relative addressing and describes the multidimensional array in the manner described herein, the inbound transfer engine 120 can (via communication 154 to the host interface 112) send inbound DMA data to the host memory.

[0033] Inbound transmission engine 120 can receive data from host interface 112 (via communication 150) and can also send a completion notification to event engine 118 (via communication 162) upon completion of a corresponding command. In addition to receiving completion notifications from pending command table 126 and inbound transmission engine 120 (via communication 174 and 162 respectively), event engine 118 can also send event-related information to host interface 112 (via communication 160). Pending command tables in the transmit and receive paths can convey response status (via tables 126 and 130 of communication 176), and inbound transmission engine 120 can also send responses to outbound transmission engine 116 (via communication 152). Similarly, outbound transmission engine 116 can form data packets, and its DTE can form multiple reads; list processing engine 122 can form an address for writing data packets and pass that address to the inbound transmission engine 120's DTE. The DTE of the inbound delivery engine 120 can form multiple write requests based on a given data type descriptor, wherein the multiple write requests include an offset relative to that address and a corresponding length. It will be apparent to those skilled in the art that this mechanism can also be used for packet input processing.

[0034] Data type descriptor overview

[0035] In a typical NIC implementation, the user software can provide commands indicating the virtual address and length of the data to be transmitted. The NIC can receive the commands and translate the virtual addresses. Subsequently, the NIC can issue read (for transmitting data, e.g., in a Get operation) or write (for receiving data, e.g., in a Put operation) orders to host memory. In conventional solutions, these reads and writes to host memory are typically for large blocks of continuous data.

[0036] The described aspects provide a data type engine that generates a series of addresses and lengths based on data types or "data type descriptors." Data types can natively represent data layouts in application memory, which can lead to the elimination of resource-intensive copying. That is, a data type is a way of describing a native data layout in memory. Without the data type engine described herein, the described data copying is performed using a central processing unit (CPU), a general-purpose processing unit (GPU), and advanced memory systems. However, using these systems can be inefficient because operation with them can be expensive (i.e., CPUs, GPUs, and advanced memory systems are generally better suited for other tasks).

[0037] An example of a data type descriptor could be a multidimensional array descriptor, where the data type engine uses parameters to describe an array in host memory, as shown below. Figure 4 The matrix transpose example is described below. Multidimensional arrays can include: the number of elements in each dimension; the size of the block to be transposed; and the stride in each dimension. Data type descriptors can also include handles to the old data type name and the new data type name, as shown below regarding... Figure 2 and Figure 4 As described. Another example of a data type descriptor form could be an Input / Output (I / O) Vector (IOVEC), where the DTE retrieves a list of addresses from host memory. IOVEC can use relative or absolute addressing. In relative addressing, the DTE can retrieve data based on an address “offset” relative to a user-provided virtual base address, while in absolute addressing, the DTE can retrieve data based on an address that serves as the virtual address to be used.

[0038] Example code for applications using data types

[0039] Figure 2 An exemplary code excerpt 200 of an application using data types according to one aspect of this application is illustrated. Section 210 may indicate a data type “MPI_Type_contiguous” that allows the data type to be copied to consecutive locations. Sections 212, 214, and 216 may indicate a data type “MPI_Type_vector” that allows the data type to be copied to locations with equidistant blocks, where each block can be obtained by concatenating the same number of copies of an older data type. The spacing between blocks may be a multiple of the range of the older data type. “MPI_Type_vector” may indicate a structure comprising a multidimensional array including: a count corresponding to the number of blocks or elements in each dimension; a block size or length corresponding to the number of elements in each block; a stride in each dimension corresponding to the number of elements between the beginnings of each block; an indicator of the existing or older data type; and a handle to the new data type.

[0040] Section 210 can create floating-point ternary vectors and be used to construct descriptions of multiple faces (e.g., three) of a matrix. Using the data types indicated in sections 214 and 216 can result in data being transmitted in large chunks or many large chunks, which can be aggregated into messages, and can result in efficient delivery. However, using the data types indicated in section 212 can result in 12-byte collection / distribution from many locations, which may be less efficient than using the data types in sections 214 and 216. When comparing NIC performance to CPU and GPU performance, the data type engine in a NIC with data type descriptors can perform more efficiently when transmitting data in larger chunks (e.g., 128-byte segments) rather than smaller chunks (e.g., 8-byte or 12-byte segments).

[0041] Outbound packet engine and transmission path

[0042] As mentioned above Figure 1 The data type engine can be integrated with the transmit and receive logic in the outbound packet engine and the inbound packet engine, respectively. Figure 3 Figure 300 illustrates an outbound packet engine 301 according to one aspect of this application, which has a data type engine as part of the transmission path. The outbound packet engine (OXE) 301 may include: a packet forming module 302; a data type engine (DTE) module 304; a direct memory access (DMA) module 306; and a post-processing module 308. The OXE 301 can take data from command 320 as input and can output packets to reliability level 322. Additionally, the OXE 301 can communicate with an address translation unit (ATU) 310 and host memory 312.

[0043] During operation, OXE 301 can receive commands (via communication 330), for example, from the command queue of command 320. These commands can be received via a message passing interface (MPI) and can be commands for reading data from host memory. In the standard path (i.e., where the data type descriptor is absent, thus bypassing DTE), packet forming module 302 can form a packet indicating the header, address, and length associated with the data to be read from host memory, referred to as a "non-DTE DMA command." This standard non-DTE DMA command can be sent (via communication 334) to DMA module 306. DMA module 306 can obtain the corresponding address translation from ATU 310, for example, by sending a virtual address 338 to ATU 310 and receiving a physical address 340 from ATU 310. DMA module 306 can then issue one or more read requests to host memory 312 (via communication 342) and receive one or more corresponding read responses from host memory 312 (via communication 344). DMA module 306 can (via communication 346) transmit a read response to post-processing module 308, which can (via communication 348) transmit the processed data as output to reliability level 322.

[0044] When a data type descriptor (e.g., a handle indicating a previously stored data type descriptor) is present in the command, packet forming module 302 can form a packet, referred to as a "DTE command," that indicates or references the data type descriptor and includes a base address and length associated with the data to be read from host memory. This DTE command can be sent (via communication 332) to DTE module 304. DTE module 304 can identify or determine that the data type descriptor indicated in the packet matches a data type descriptor stored in DTE module 304 (e.g., as previously configured and stored by the user). Using the identified matching data type descriptor, DTE module 304 can generate multiple read requests having offsets relative to the base address and corresponding lengths. DTE module 304 can then send these generated read requests to DMA module 306 (as DMA commands via communication 336). DMA module 306 can perform the address translation (if needed) and read operations described above with respect to communications 338, 340, 342 and 344, as well as subsequent operations related to post-processing module 308 and reliability level 322.

[0045] Example: Matrix transpose of FFT using data type descriptors

[0046] Figure 4 An example of matrix transpose using a Fast Fourier Transform (FFT) with a data type descriptor according to one aspect of this application is illustrated. Figure 4It may include: a source 410 depicted in various arrangements; a message payload 430 (an exemplary arrangement from the source 410); a temporary target memory 450 corresponding to the message payload 430; and a destination 470, which is also depicted having corresponding arrangements.

[0047] exist Figure 4 Source 410 may include a two-dimensional (2D) array of double complex floating-point numbers using the data type descriptor “MPI_Type_vector(2,2,8,MPI_DOUBLE_COMPLEX,&new_type)”. This data type descriptor indicates that the basic data type is a 16-byte double complex number, the element count or quantity is 2, the block size is 2 elements (32 bytes), and the stride between blocks is 8. “MPI_DOUBLE_COMPLEX” is the standard basic data type in MPI, referring to a complex double-precision floating-point number. Destination 470 may include a 2D array of transposed double complex numbers using the data type descriptor “MPI_Type_vector(2,1,8,MPI_DOUBLE_COMPLEX,&column)” or “MPI_Type_vector(2,1,1,column,&array)”.

[0048] The system can use hardware collection operation 420 based on data type descriptors (the data type engine performs read operations in the emission path, such as...) Figure 1 and Figure 3 The data (as depicted) is transmitted from source 410 to destination 470. The data may be collected into various message payloads (e.g., buffers on the NIC) and subsequently transmitted via wired connection (via network transmission 440). The data may be received in temporary target memory 450 (e.g., host memory at the destination or target side). The target side may use software scattering operation 460 to place the data into a user-provided buffer (e.g., as a transposed two-dimensional array). In some examples, if the target side has a NIC, the NIC includes an inbound transmission engine with a data type engine (as described above). Figure 1 (as described above), then the target side can use hardware distributed operation, that is, the NIC inbound transmission engine can use its DTE to perform write operations.

[0049] For example, source 410 may include four permutations, designated as permutation 0 412, permutation 1 414, permutation 2 416, and permutation 3 418. Each permutation may include two rows, where each square in the row corresponds to 16 bytes representing the basic data type of a double complex number. For example, in permutation 0 412, blocks of two elements can be collected into message payload 430: blocks marked with elements "1, 2" (via communication 421) can be collected into message payload 432; blocks marked with elements "3, 4" (via communication 423) can be collected into message payload 434; blocks marked with elements "5, 6" (via communication 425) can be collected into message payload 436; and blocks marked with elements "7, 8" (via communication 427) can be collected into message payload 436. In message payload 438, blocks marked with elements "9, 10" (via communication 422) can be collected into message payload 432; blocks marked with elements "11, 12" (via communication 424) can be collected into message payload 434; blocks marked with elements "13, 14" (via communication 426) can be collected into message payload 436; and blocks marked with elements "15, 16" (via communication 428) can be collected into message payload 438. The paths of the data represented by blocks "3, 4" and "11, 12" can be followed, as indicated by the thick line boundaries around these blocks. Similarly: data represented by blocks "1, 2" and "9, 10" are indicated by circular dashed line boundaries; data represented by blocks "5, 6" and "13, 14" are indicated by thick dashed line boundaries; and data represented by blocks "7, 8" and "15, 16" are indicated by thin dashed line boundaries.

[0050] Message payloads 432, 434, 436, and 438 can be transmitted over the network (via 440) and are placed in buffers 452, 454, 456, and 458 of a temporary destination memory 450, respectively. Software on the destination side can perform a scatter operation (460) and place the data in a transposed 2D array 470. Destination 470 may include four permutations, designated as permutation 0 472, permutation 1 474, permutation 2 476, and permutation 3 478. For example, Figure 4 The two blocks “3,4” and “11,12” are described as being collected by source hardware (e.g., source NIC via hardware collection operation 420) based on data type descriptors with a count of 2, a block size of 2, and a stride of 8. These two blocks are placed in two separate columns with a permutation of 1,474.

[0051] Similarly, the blocks shown in arrangement 1 414 (indicated by a right-tilted fill pattern) are collected by the source hardware, transmitted over the network, placed in a temporary target memory, and dispersed into columns by the destination software (as indicated by the four blocks with similar right-tilted fill patterns in each of arrangements 472, 474, 476, and 478). Additionally, the blocks shown in arrangement 2 416 (indicated by a vertical cross-line fill pattern) are collected by the source hardware, transmitted over the network, placed in a temporary target memory, and dispersed into columns by the destination software (as indicated by the four blocks with similar vertical cross-line fill patterns in each of arrangements 472, 474, 476, and 478). Furthermore, the blocks shown in arrangement 3 418 (indicated by a diagonal cross-line fill pattern) are collected by the source hardware, transmitted over the network, placed in a temporary target memory, and dispersed into columns by the destination software (as indicated by the four blocks with similar diagonal cross-line fill patterns in each of arrangements 472, 474, 476, and 478).

[0052] although Figure 4 The example depicts hardware collection operations (e.g., operation 420) on the source side and software distribution operations (e.g., operation 460) on the target side, but in some respects, the data type engine can be used by both the source and target sides. That is, both the source-side collection operation and the target-side distribution operation can be performed in hardware by the corresponding data type engine. In other respects, the source side can perform software collection operations, while the target side can use the data type engine to perform hardware distribution operations. These variations are all possible and can depend on the specific circumstances of the application or data emission.

[0053] Methods for enhancing data type engines to support high-performance computing

[0054] Figure 5 A flowchart 500 illustrating a method for promoting high-performance computing using a data type engine according to one aspect of this application is presented. During operation, the system receives a command from a network interface card (NIC) via a message passing interface (MPI) for reading data from host memory (operation 502). The system determines whether the command indicates a first data type descriptor stored in the data type engine (DTE) of the NIC. If the command does not indicate a first data type descriptor stored in the DTE (decision 504), the system forms a data packet based on the command, indicating the header, base address, and length associated with the data to be read from host memory (operation 506). The system passes the data packet to a direct memory access (DMA) module (operation 508). The system issues a read request to host memory based on the data packet through the DMA module (operation 510), and the operation returns.

[0055] If the system determines that the command indicates a first data type descriptor stored in the NIC's data type engine (decision 504), the system forms a data packet based on the command, indicating the base address and length associated with the data to be read from host memory (operation 512). The system passes the data packet to the data type engine (operation 514). The system generates multiple read requests by the data type engine based on the first data type descriptor, the multiple read requests including an offset relative to the base address and a corresponding length (operation 516). The system passes the multiple read requests to the NIC's Direct Memory Access (DMA) module, thereby allowing the NIC to access host memory while clearing copies of the data on the host during command transmission over the network (operation 518). The DMA module then issues multiple read requests to host memory (not shown). The operation returns.

[0056] Devices that enable data type engines to support high-performance computing

[0057] Figure 6 The illustration shows an apparatus 600 that facilitates high-performance computing by promoting a data type engine, according to one aspect of this application. The device 600 may represent a network interface card (NIC) and may include: a command unit 602 for receiving a command to read data from host memory and determining whether the command indicates a first data type descriptor stored in a data type engine of the NIC; a packet forming unit 604 for forming a packet based on the command, indicating a base address and length associated with data to be read from host memory; a direct memory access unit 606 for receiving a read request and issuing a read request to host memory; a first data type engine unit 608 for generating multiple read requests based on the first data type descriptor, including an offset relative to the base address and a corresponding length, and passing the read requests to a DMA module of the NIC, wherein the first data type engine unit 608 is further configured to handle transmit operations related to performing a read operation from host memory; a second data type engine unit 610 for handling receive operations related to performing a write operation to host memory; and a data type descriptor management unit 612 for storing one or more data type descriptors created by the user.

[0058] Command unit 602 may correspond to, for example Figure 1 The command queue 114 and the data packet forming module 302 from Figure 3 Command 320 receives the data packet. The first data type engine 608 can correspond to, for example... Figure 1 Outbound transmission engine 116 and Figure 3The OXE 301, while the second data type engine 610 can correspond to Figure 1 The inbound delivery engine 120. The packet forming unit 604 can correspond to... Figure 3 The data packet forming module 302, and the direct memory access unit 606 can correspond to Figure 3 The DMA module 306. The data type descriptor management unit 612 can reside, for example, in a first data type engine 608 and a second data type engine unit 610. The unit described as part of the device 600 can perform the above-mentioned... Figure 1 , Figure 3 and Figure 5 The described operation.

[0059] Typically, the disclosed aspects provide a system, method, apparatus, and nontransitory storage medium for facilitating data type engines to support high-performance computing. In one aspect, the system receives a command via a network interface card (NIC) for reading data from host memory through a messaging interface. The system determines that the command indicates a first data type descriptor stored in the data type engine of the NIC. The system forms a data packet based on the command, indicating a base address and length associated with the data to be read from host memory. The system passes the data packet to the data type engine. The system generates multiple read requests by the data type engine based on the first data type descriptor, the multiple read requests including an offset relative to the base address and a corresponding length. The system passes the multiple read requests to the NIC's direct memory access (DMA) module, thereby allowing the NIC to access host memory while clearing copies of the data on the host during command transmission over the network.

[0060] In a variant of this aspect, in response to determining that the command does not indicate a data type descriptor stored in the NIC's data type engine, the system passes the data packet to the DMA module, wherein the data packet is formed to indicate a header, base address, and length associated with the data to be read from host memory. The system then issues a read request to host memory based on the data packet via the DMA module.

[0061] In a further variant, the data type descriptor represents a multidimensional array, which includes: the number of elements in each dimension; the size of the block to be transferred; and the stride in each dimension.

[0062] In a further variant, the data type descriptor represents an array of 256-byte masks that indicate which bytes of data to be read from the host memory should be returned in response to the command.

[0063] In a further variant, the data type descriptor references an Input / Output Vector (IOVEC), which indicates the address and length of data to be read from host memory. The IVOEC uses at least one of the following: relative addressing, in which the indicated address to be read from host memory includes an offset relative to the virtual base address indicated in the command; and absolute addressing, in which the indicated address to be read from host memory includes a virtual address corresponding to the data to be read.

[0064] In a further variant, the system allows a user associated with a command to create one or more data type descriptors. The system stores these one or more data type descriptors using a data type engine.

[0065] In a further variant, the data type engine includes: a first instance in an outbound packet engine that handles transmit operations related to performing read operations, including commands for reading data from the host memory; and a second instance in an inbound packet engine that handles receive operations related to performing write operations, including commands for writing data to the host memory.

[0066] In a further variant, the system issues the plurality of read requests to the host memory via a DMA module to obtain the data indicated in the command.

[0067] On the other hand, a non-transitory computer-readable storage medium is provided, which stores instructions that, when executed by a computer, cause the computer to perform the methods described herein.

[0068] Another approach provides a network interface card (NIC) including a command module for receiving a command via a messaging interface for reading data from host memory. The NIC further includes a packet forming module for: determining whether the command indicates a first data type descriptor stored in a first data type engine of the NIC; in response to determining that the command indicates the first data type descriptor, forming a packet based on the command indicating a base address and length associated with data to be read from the host memory; and passing the packet to the first data type engine. The NIC further includes a first data type engine for: generating a plurality of read requests based on the first data type descriptor, the plurality of read requests including an offset relative to the base address and a corresponding length; and passing the plurality of read requests to a direct memory access (DMA) module of the NIC, thereby allowing the NIC to access the host memory while clearing a copy of the data on the host during transmission of the command over the network. The NIC further includes a DMA module for issuing the plurality of read requests to the host memory to obtain the data indicated in the command.

[0069] In a variant of this, the apparatus further includes an inbound packet engine and an outbound packet engine. A first data type engine resides in the outbound packet engine, which processes transmit operations related to performing read operations, including commands for reading data from host memory, and a second data type engine resides in the inbound packet engine, which processes receive operations related to performing write operations, including commands for writing data to host memory.

[0070] The foregoing descriptions of the various aspects have been presented solely for illustrative and descriptive purposes. These descriptions are not intended to be exhaustive or to limit the aspects described herein to the disclosed forms. Accordingly, many modifications and variations will be apparent to those skilled in the art. Furthermore, the foregoing disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.

Claims

1. A computer-implemented method, comprising: The network interface card (NIC) receives commands for reading data from the host memory via the message passing interface. The command indicates the first data type descriptor stored in the data type engine of the NIC; Based on the command, a data packet is generated that indicates the base address and length associated with the data to be read from the host memory; The data packet is passed to the data type engine; The data type engine generates multiple read requests based on the first data type descriptor, and the multiple read requests include an offset relative to the base address and a corresponding length; as well as The multiple read requests are passed to the NIC's direct memory access module, thereby allowing the NIC to access the host memory, while clearing the copy of the data on the host memory during the transmission of the command over the network.

2. The method as described in claim 1, wherein, In response to determining that the command does not indicate a data type descriptor stored in the data type engine of the NIC, the method further includes: The data packet is passed to the direct memory access module, wherein the data packet is configured to indicate a header, a base address, and a length associated with the data to be read from the host memory; and The direct memory access module sends a read request to the host memory based on the data packet.

3. The method as described in claim 1, in, The data type descriptor represents a multidimensional array, which includes: The number of elements in each dimension; The size of the block to be transferred; and Step size in each dimension.

4. The method as described in claim 1, in, The data type descriptor represents an array of 256-byte masks that indicate which bytes of the data to be read from the host memory should be returned in response to the command.

5. The method of claim 1, wherein, The data type descriptor references an input / output vector IOWEC, which indicates the address and length of data to be read from the host memory, and wherein the IOWEC uses at least one of the following: Relative addressing, in which the indicated address to be read from the host memory includes an offset relative to the virtual base address indicated in the command; and Absolute addressing, in which the address indicated to be read from the host memory includes a virtual address corresponding to the data to be read.

6. The method of claim 1, further comprising: One or more data type descriptors are created by the user associated with the command; as well as The data type engine stores the one or more data type descriptors.

7. The method of claim 1, wherein, The data type engine includes: A first instance in the outbound packet engine, the first instance processing transmit operations related to performing read operations, including the command for reading the data from the host memory; and A second instance in the inbound packet engine, which handles receive operations related to performing write operations, including commands for writing data to the host memory.

8. The method of claim 1, further comprising: The direct memory access module issues the plurality of read requests to the host memory to obtain the data indicated in the command.

9. A network interface card, comprising: The command module is configured to receive commands for reading data from host memory via a message passing interface; The data packet forming module is used for: Determine whether the command instructs a first data type descriptor stored in the first data type engine of the network interface card; In response to determining the command indicating the first data type descriptor, a data packet indicating the base address and length associated with the data to be read from the host memory is formed based on the command; as well as The data packet is passed to the first data type engine; The first data type engine is used for: Multiple read requests are generated based on the first data type descriptor, and the multiple read requests include an offset relative to the base address and a corresponding length; as well as The multiple read requests are passed to the direct memory access module of the network interface card, thereby allowing the network interface card to access the host memory, while clearing the copy of the data in the host memory during the transmission of the command over the network; as well as The direct memory access module is used to issue the plurality of read requests to the host memory to obtain the data indicated in the command.

10. The network interface card as described in claim 9, in, The packet forming module is further configured to perform the following operations in response to determining that the command does not indicate the first data type descriptor or any data type descriptor stored in the first data type engine: The data packet is formed to indicate the header, the base address, and the length associated with the data to be read from the host memory; and The data packet is then passed to the direct memory access module; and The direct memory access module is used to send a read request to the host memory based on the data packet.

11. The network interface card as described in claim 9, in, The data type descriptor represents a multidimensional array, which includes: The number of elements in each dimension; The size of the block to be transferred; and Step size in each dimension.

12. The network interface card as described in claim 9, in, The data type descriptor represents an array of 256-byte masks that indicate which bytes of the data to be read from the host memory should be returned in response to the command.

13. The network interface card as described in claim 9, wherein, The data type descriptor references an input / output vector IOWEC, which indicates the address and length of data to be read from the host memory, and wherein the IOWEC uses at least one of the following: Relative addressing, in which the indicated address to be read from the host memory includes an offset relative to the virtual base address indicated in the command; and Absolute addressing, in which the address indicated to be read from the host memory includes a virtual address corresponding to the data to be read.

14. The network interface card as described in claim 9, in, The first data type engine is further used to store one or more data type descriptors created by the user associated with the command.

15. The network interface card as described in claim 9, in, The device further includes an inbound packet engine and an outbound packet engine. The first data type engine resides in the outbound packet engine, which processes transmit operations related to performing read operations, including the command for reading the data from the host memory. The second data type engine resides in the inbound packet engine, which processes receive operations related to performing write operations, including commands for writing data to the host memory.

16. A non-transitory computer-readable storage medium storing instructions, said instructions causing the computer to perform a method when executed by the computer, said method comprising: The network interface card's data type engine stores one or more data type descriptors created by a user associated with the computer; The network interface card receives a command to read data from the host memory via a message passing interface; Determine whether the command instructs a first data type descriptor stored in the data type engine of the network interface card; In response to determining that the command instructs the first data type descriptor stored in the data type engine of the network interface card, the following operations are performed: Based on the command, a data packet is generated that indicates the base address and length associated with the data to be read from the host memory; The data packet is passed to the data type engine; The data type engine generates multiple read requests based on the first data type descriptor, and the multiple read requests include an offset relative to the base address and a corresponding length; as well as The multiple read requests are passed to the Direct Memory Access module of the network interface card, thereby allowing the network interface card to access the host memory, while clearing the copy of the data in the host memory during the transmission of the command over the network.

17. The storage medium of claim 16, wherein, In response to determining that the command does not indicate the first data type descriptor stored in the data type engine of the network interface card, the method further includes: The data packet is passed to the direct memory access module, wherein the data packet is configured to indicate a header, a base address, and a length associated with the data to be read from the host memory; and The direct memory access module sends a read request to the host memory based on the data packet.

18. The storage medium as claimed in claim 16, in, The data type descriptor represents a multidimensional array, which includes: the number of elements in each dimension; the size of the block to be transmitted; and the stride in each dimension.

19. The storage medium of claim 16, wherein, The data type descriptor references an input / output vector IOWEC, which indicates the address and length of data to be read from the host memory, and wherein the IOWEC uses at least one of the following: Relative addressing, in which the indicated address to be read from the host memory includes an offset relative to the virtual base address indicated in the command; and Absolute addressing, in which the address indicated to be read from the host memory includes a virtual address corresponding to the data to be read.

20. The storage medium of claim 16, wherein, The data type engine includes: A first instance in the outbound packet engine, the first instance processing transmit operations related to performing read operations, including the command for reading the data from the host memory; and A second instance in the inbound packet engine, which handles receive operations related to performing write operations, including commands for writing data to the host memory.

Images